How is Graphics Formed?

by Cassie Tarakajian

With the advent of HTML5, the Internet as we knew it completely changed. Gone were the days of Flash and Actionscript, and we welcomed a new era of Web Audio and WebGL. It is in this era that I realized that you can make really, really cool stuff on the Internet, and it is in this era that I decided I wanted to join in on making the Internet.

Now, it's my job as a maker of the Internet to reveal the trick behind the magic. Welcome to WebGL, and here goes.

<Canvas>

It all begins with the introduction in HTML5 of the <canvas> tag. It's a rectangular container that allows for dynamic rendering of 2D shapes and bitmap (a matrix of pixel bits) images. Not only is it dynamic, it is scriptable, as in Javascript can access the drawing area, meaning it can be interactive, or used to create graphs, animations, and games.

The <canvas> tag was originally introduced in 2004 as a WebKit Component by Apple. At first, it was criticized since it forged its own path rather than following the existing SVG standard. Once its benefits were recognized, it was adopted by Opera and Gecko browsers before becoming part of the HTML5 standard.

OpenGL and WebGL

The story of WebGL begins with OpenGL, or Open Graphics Library, which is a API for rendering 2D and 3D graphics. Initially released in 1992, the API is used to communicate with a GPU, and while it can be written in software, it is usually written in hardware for faster rendering. Its syntax is very similar to C, though it is technically language independent.

WebGL allows the power of OpenGL to be harnassed in the browser. It is a Javascript API which renders 2D and 3D graphics on an HTML5 canvas element without the use of a browser plugin. The graphic's control code is executed in Javascript, whereas the shader code is executed on a computer's GPU (I'll get back to those terms in a bit). The parts of graphics processing that are computationally heavy are run in hardware rather than in Javascript, thus making it possible to create complicated graphics in the browser.

In Practice

When actually creating graphics on the web, the WebGL API itself can be cumbersome to use since it is so low level. Most developers use one of the many utility libraries abstract away the unnecessary details. One popular library is Three.js, with extended features in THREEx. Game engines such as Unity and Unreal also allow development for WebGL.

I'm going to talk about Three.js, as I think it's a great, lightweight introduction to the world of graphics.

The Basics of Three.js

In order start making graphics, there are a few different components, and it is important to understand what they are:

  • scene - this holds all components that are to be placed in the "scene", or what's actually in the HTML5 canvas. Creating one is simple:
var scene = new THREE.Scene();  
  • camera - think of this like an actual video camera. It's what the user actually sees, which may not be the entire scene. There are different types of cameras, but to create the most simple one:
var camera =  
  new THREE.PerspectiveCamera(
    VIEWING_ANGLE,
    ASPECT_RATIO,
    NEAR_FRUSTRUM_PLANE_POSITION,
    FAR_FRUSTRUM_PLANE_POSITION);
  • renderer - the component that creates the graphic. There are two different types of renderers: Canvas, which is rendered in browser (and therefore slow), and WebGL, which is rendered on the GPU. Some browsers do not support WebGL, which is why you would use a Canvas renderer. Instantiate one like so:
var renderer = new THREE.WebGLRenderer();  
  • objects - the stuff in the scene. You might call them a mesh which contains information about the object's shape (called geometry) and the object's material (called material). The latter is important when lights are introduced to the scene, but the default is full ambient light.

Code structure

When rendering graphics (and therefore creating an animation), there's some code that only needs to run once, and there's some code that needs to run again and again to update the video frame. In Three.js, this is done by creating two functions, setup and draw. You can name them whatever you want, I just like setup and draw because it's like Processing.

//global variables needed for creating scene
var camera, scene, renderer;  
var geometry, material, mesh;  
function setup() {  
  //create renderer, scene, camera, etc.
}

function draw() {  
  requestAnimationFrame( draw );

  //update renderer and anything else that has changed
  renderer.render( scene, camera );
}

Note requestAnimationFrame being called from within the draw function. Remember that video is actually just a series of frames, and therefore we want to just update the frame a few times a second. requestAnimationFrame also adds some efficiency to the browser by not running the animation if the tab isn't selected. Pretty cool, right? As an argument it takes a callback function, which in our case is the function we want to run over and over again, draw.

Objects

Objects are made up of a few different components. In the most simple case, it's a geometry, material, and mesh.

A geometry is the actual shape of the object. It can be a cube

geometry = new THREE.CubeGeometry(200, 200, 200);  

or a torus

geometry = new THREE.TorusGeometry(100, 40, 40, 40, 6.28);  

or even a bunch of particles, using a for loop

geometry = new THREE.Geometry();  
for ( i = 0; i < 5000; i ++ ) {  
    var vertex = new THREE.Vector3();
    vertex.x = 1000 * Math.random() - 500;
    vertex.y = 1000 * Math.random() - 500;
    vertex.z = 1000 * Math.random() - 500;
    geometry.vertices.push( vertex );
}

The material is the stuff that the object is made off. This will effect the color and pattern on the object, but also how it is affected by the lights in the scene. The material can be normal or basic, and unaffected by light

material = new THREE.MeshNormalMaterial({shading: THREE.FlatShading});  
//or
material = new THREE.MeshBasicMaterial({shading: THREE.FlatShading, color: 0xdcdcdc});  

or it can react to lights. The lambert material is based on Lambertian reflectance, which defines an ideal matte surface

material = new THREE.MeshLambertMaterial({shading: THREE.FlatShading, color: 0xdcdcdc});  

and the phong material uses Phong shading and the Phong reflection model, describing reflection as a combination of the diffuse reflection of matte surfaces and the specular reflection of shiny surfaces.

material = new THREE.MeshPhongMaterial({shading: THREE.FlatShading, color: 0xdcdcdc, ambient: 0xffffff, emissive: 0x000000, specular: 0x111111, shininess: 40.00});  

You can also add a texture to a material, which is basically mapping an image to the surface of the object.

You bring the material and geometry together in a mesh. It's pretty simple.

mesh = new THREE.Mesh(geometry, material);  

Note that there are other ways to create geometries with a material, like particle systems. However, I'll leave that for another post.

Shaders

Shaders are the fun part of this whole operation. This is where you can run custom, intensive graphics processing as it's running on your GPU instead of the browser. They're implemented in GLSL, or OpenGL Shading Language, which is based on ANSI C but is a little bit different. You can go deep into this and make awesome custom stuff, but I don't recommend it as a beginner.

Debugging

As I started to play around with three.js, I quickly discovered that I had no idea how to debug when the animation I thought would be rendered was not.

Built into Chrome Developer Tools is a feature called 'Canvas Inspector'. There's a great tutorial on how to use it over at learnthreejs.com.

Another tip is including the non-minified version of Three.js in your project so that if the library code is throwing an error, the variables and functions actually have readable, intuitive names.

If you find that your code is lagging, it might be helpful to include stats.js. This may help you pinpoint areas of your code that are inefficient or taking a lot of time.

Resources

For more resources on three.js, I recommend checking out a wiki page made by the creator of the library. Check it out here. There's tons of tutorials, inroductions, and documentation there.

To get familiar with three.js without needing to create your own project from scratch, I recommend the Three.js Playground. It's more like a hands-on tutorial, which is always way more fun than watching videos or reading.

Lastly, I found a Udacity Course if you're looking for some structured learning. I haven't gotten a chance to check out its quality but it seems highly recommended.

Ur a pro now, rite?

Okay, you're probably not. And neither am I. But I'm trying to learn more about browser magic, and hopefully you've enjoyed an introduction to the secrets behind it.

Cassie Tarakajian

hardware nerd, software nerd, web nerd, nerd nerd.

Read more from this author