An HTML5 application is obviously written using JavaScript. But compared to other kind of development environments (like native one), JavaScript historically suffers from an important limitation: all its execution process remains inside a unique thread. This could be pretty annoying with today multi-cores processors like the i5/i7 containing up to 8 logical CPUs and even with the latest ARM mobile processors being dual or even quad-cores. Hopefully, we’re going to see that HTML5 offers to the web a way to better handle these new marvelous processors to help you embrace a new generation of web applications.

• Before the workers…
• Web Workers or how to be executed out of the UI Thread
  • My 1st Web Worker
  • Posting messages using JSON
• Browsers support
• Non accessible elements from a worker
• Errors handling & debugging
  • The F12 development bar for a better debugging experience
  • An interesting solution to mimic console.log()
• Use cases and how to identify potential candidates
  • Web Workers for which scenarios?
  • How to identity hot spots in your code
    • Case 1: canvas animation with the Speed Reading demo
    • Case 2: Raytracers inside canvas
• Conclusion

Pour ceux qui pratiquent la langue de Molière, vous trouverez une version française ici : Introduction aux Web Workers d’HTML5 : le multithreading version JavaScript

Before the workers…

This JavaScript limitation implies that a long running processing will freeze the main window. We often say in our developers’ words that we’re blocking the “UI Thread”. This is the main thread in charge of handling all the visual elements and associated tasks: drawing, refreshing, animating, user inputs events, etc. We all know the bad consequences of overloading this thread: the page freezes and the user can’t interact anymore with your application. The user experience is then of course very bad and the user will probably decide to kill the tab or the browser instance. This is probably not something you’d like to see while using your application!

To avoid that, the browsers have implemented a protection mechanism which alerts the user when a long-running suspect script occurs. Unfortunately, this protection mechanism can’t make the difference between a script not written correctly and a script which really needs more time to accomplish its work. Still, as it blocks the UI thread, it’s better to tell you that something wrong is maybe currently occurring. Here are some messages examples (from Firefox 5 & IE9):

Hopefully, up to now, those problems were rarely occuring for 2 main reasons:

1 – HTML and JavaScript weren’t used in the same way and for the same goals as other technologies able to achieve multithreaded tasks. The Websites were offering richless experiences to the users than native applications.
2 – There were some other ways to more or less solve this concurrency problem.

Those ways are well known from all web developers. We were indeed trying to simulate parallel tasks thanks to the setTimeout() and setInterval() methods for instance. HTTP requests can also be done in asynchronous manner thanks to the XMLHttpRequest object which avoids freezing the UI while loading some resources from some remote servers. At last, the DOM Events allow us to write application making the illusion that several things occurs at the same time. Illusion, really? Yes!

To better understand why, let’s have a look to a fake piece of code and let’s then see what happens inside the browser:

<script type="text/javascript">
    function init(){
        { piece of code taking 5ms to be executed } 
        A mouseClickEvent is raised
        { piece of code taking 5ms to be executed }
        setInterval(timerTask,"10");
        { piece of code taking 5ms to be executed }
    }

    function handleMouseClick(){
          piece of code taking 8ms to be executed 
    }

    function timerTask(){
          piece of code taking 2ms to be executed 
    }
</script> 

Let’s take this code to project it on a model. This diagram then shows us what’s happening in the browser on a time scale:

This diagram well illustrates the non-parallel nature of our tasks. Indeed, the browser is only enqueuing the various execution requests:

– from 0 to 5ms: the init() function starts by a 5ms task. After 5ms, the user raises a mouse click event. However, this event can’t be handled right now as we’re still executing the init() function which currently monopolize the main thread. The click event is saved and will be handled later on.

– from 5 to 10ms: the init() function continues its processing during 5ms and then asks to schedule the call to the timerTask() in 10ms. This function should then logically be executed at the 20ms timeframe.

– from 10 to 15ms: 5 new milliseconds are needed to finish the complete run of the init() function. This is then corresponding to the 15ms yellow block. As we’re freeing the main thread, it can now start to dequeue the saved requests.

– from 15 to 23ms: the browser starts by running the handleMouseClock() event which runs during 8ms (the blue block).

– from 23 to 25 ms: as a side effect, the timerTask() function which was scheduled to be run on the 20ms timeframe is slightly shifted of 3ms. The other scheduled frames (30ms, 40ms, etc.) are respected as there is no more code taking some CPU.

Note: this sample and the above diagram (in SVG or PNG via a feature detection mechanism) were inspired by the following article: HTML5 Web Workers Multithreading in JavaScript

In conclusion, all these tips don’t really solve our initial problem: everything keeps being executed inside the main UI thread.

Moreover, even if JavaScript weren’t used in the past for the same kind of applications like the so-called “high level languages”, it starts to change lastly thanks to the new possibilities offered by HTML5 and its friends. It was then more than important to provide to JavaScript some new powers to make it ready building a new generation of applications being able to leverage parallel tasks. This is exactly what the Web Workers were made for.

Web Workers or how to be executed out of the UI Thread

The Web Workers APIs define a way to run script in the background. We will then be able to execute some tasks in threads living outside the main page and thus non-impacting the drawing performance. However, in the same way that we know that not all algorithms can be parallelized, not all JavaScript code can take advantage of workers. Ok, enough blah blah, let’s have a look to those famous workers.

My 1st Web Worker

As Web Workers will be executed on separated threads, you need to host their code into separated files from the main page. Once done, you need to instantiate a Worker object to call them:

var myHelloWorker = new Worker('helloworkers.js');

You’ll then start the worker (and thus a thread under Windows) by sending it a first message:

myHelloWorker.postMessage();

Indeed, the Web Workers and the main page are communicating via messages. Those messages can be formed with normal strings or JSON objects. To illustrate simple messages posting, we’re going to start by reviewing a very basic sample. It will post a string to a worker that will simply concatenate it with something else. To do that, add the following code into the “helloworker.js” file:

function messageHandler(event) {
    // Accessing to the message data sent by the main page
    var messageSent = event.data;
    // Preparing the message that we will send back
    var messageReturned = "Hello " + messageSent + " from a separate thread!";
    // Posting back the message to the main page
    this.postMessage(messageReturned);
}

// Defining the callback function raised when the main page will call us
this.addEventListener('message', messageHandler, false);

We’ve just defined inside “helloworkers.js” a piece of code that will be executed on another thread. It can receive messages from your main page, do some tasks on it and send a message back to your page in return. We then need to write the receiver in the main page. Here is the page that will handle that:

<!DOCTYPE html>
<html>
<head>
    <title>Hello Web Workers</title>
</head>
<body>
    <div id="output"></div>

    <script type="text/javascript">
        // Instantiating the Worker
        var myHelloWorker = new Worker('helloworkers.js');
        // Getting ready to handle the message sent back
        // by the worker
        myHelloWorker.addEventListener("message", function (event) {
            document.getElementById("output").textContent = event.data;
        }, false);

        // Starting the worker by sending a first message
        myHelloWorker.postMessage("David");

        // Stopping the worker via the terminate() command
        myHelloWorker.terminate();
    </script>
</body>
</html>

 

The result will the be: “Hello David from a separate thread!”. You’re now impressed, aren’t you?

The worker will live until you kill it. Beware of that. As they are not automatically garbage collected, it’s up to you to control their states. Moreover, keep in mind that instantiating a worker will have some memory cost and don’t negligate the cold start time neither. To stop a worker, there are 2 possible solutions:

1 – from the main calling page by calling the terminate() command.

2 – from the worker itself via the close() command.

DEMO: You can test this slightly enhanced sample in your browser here -> http://david.blob.core.windows.net/html5/HelloWebWorkers_EN.htm <-

Posting messages using JSON

Of course, most of the time we will send some more structurated data to the workers. By the way, Web Workers can also communicate between each others using Message channels.

But the only way to send some structurated messages to a worker is to use the JSON format. Hopefully, browsers that currently support Web Workers are nice enough to also natively support JSON. How kind they are!

Let’s then take our previous code sample. We’re going to add an object of type WorkerMessage. This type will be used to send some commands with parameters to our Web Workers.

Let’s use now the following simplified HelloWebWorkersJSON_EN.htm web page:

<!DOCTYPE html>
<html>
<head>
    <title>Hello Web Workers JSON</title>
</head>
<body>
    <input id=inputForWorker /><button id=btnSubmit>Send to the worker</button><button id=killWorker>Stop the worker</button>
    <div id="output"></div>

    <script src="HelloWebWorkersJSON.js" type="text/javascript"></script>
</body>
</html>

We’re using here the Unobtrusive JavaScript approach which helps us dissociating the view from the attached logic. The attached logic is then living inside this HelloWebWorkersJSON_EN.js file:

// HelloWebWorkersJSON_EN.js associated to HelloWebWorkersJSON_EN.htm

// Our WorkerMessage object will be automatically
// serialized and de-serialized by the native JSON parser
function WorkerMessage(cmd, parameter) {
    this.cmd = cmd;
    this.parameter = parameter;
}

// Output div where the messages sent back by the worker will be displayed
var _output = document.getElementById("output");

/* Checking if Web Workers are supported by the browser */
if (window.Worker) {
    // Getting references to the 3 other HTML elements
    var _btnSubmit = document.getElementById("btnSubmit");
    var _inputForWorker = document.getElementById("inputForWorker");
    var _killWorker = document.getElementById("killWorker");

    // Instantiating the Worker
    var myHelloWorker = new Worker('helloworkersJSON_EN.js');
    // Getting ready to handle the message sent back
    // by the worker
    myHelloWorker.addEventListener("message", function (event) {
        _output.textContent = event.data;
    }, false);

    // Starting the worker by sending it the 'init' command
    myHelloWorker.postMessage(new WorkerMessage('init', null));

    // Adding the OnClick event to the Submit button
    // which will send some messages to the worker
    _btnSubmit.addEventListener("click", function (event) {
        // We're now sending messages via the 'hello' command 
        myHelloWorker.postMessage(new WorkerMessage('hello', _inputForWorker.value));
    }, false);

    // Adding the OnClick event to the Kill button
    // which will stop the worker. It won't be usable anymore after that.
    _killWorker.addEventListener("click", function (event) {
        // Stopping the worker via the terminate() command
        myHelloWorker.terminate();
        _output.textContent = "The worker has been stopped.";
    }, false);
}
else {
    _output.innerHTML = "Web Workers are not supported by your browser. Try with IE10: <a href="http://ie.microsoft.com/testdrive">download the latest IE10 Platform Preview</a>";
}

At last, here is the code for the web worker contained in helloworkerJSON_EN.js the file:

function messageHandler(event) {
    // Accessing to the message data sent by the main page
    var messageSent = event.data;

    // Testing the command sent by the main page
    switch (messageSent.cmd) {
        case 'init':
            // You can initialize here some of your models/objects
            // that will be used later on in the worker (but pay attention to the scope!)
            break;
        case 'hello':
            // Preparing the message that we will send back
            var messageReturned = "Hello " + messageSent.parameter + " from a separate thread!";
            // Posting back the message to the main page
            this.postMessage(messageReturned);
            break;
    }
}

// Defining the callback function raised when the main page will call us
this.addEventListener('message', messageHandler, false);

Once again, this sample is very basic. Still, it should help you to understand the underlying logic. For instance, nothing prevents you to use the same approach to send to a worker some gaming elements that will be handled by an AI or physics engine.

DEMO: You can test this JSON sample here: -> http://david.blob.core.windows.net/html5/HelloWebWorkersJSON_EN.htm <-

Browsers support

Web Workers have just arrived in the IE10 PP2 (Platform Preview). This is also supported by Firefox (since 3.6), Safari (since 4.0), Chrome & Opera 11. However, this is not supported by the mobile versions of these browsers. If you’d like to have a more detailed support matrix, have a look here: http://caniuse.com/#search=worker

In order to dynamically know in your code that this feature is supported, please use the a feature detection mechanism. You shouldn’t use anymore some user-agent sniffing!

To help you, there are 2 available solutions. The first one is to simply test the feature yourself using this very simple piece of code:

/* Checking if Web Workers are supported by the browser */
if (window.Worker) {
    // Code using the Web Workers
}

The second one is to use the famous Modernizr library (now natively shipped with the ASP.NET MVC3 project templates). Then, simply use a code like that:

<script type="text/javascript">
    var divWebWorker = document.getElementById("webWorkers");
    if (Modernizr.webworkers) {
        divWebWorker.innerHTML = "Web Workers ARE supported";
    }
    else {
        divWebWorker.innerHTML = "Web Workers ARE NOT supported";
    }
</script>

Here is for instance the current support in your browser:

This will allow you to expose 2 versions of your application. If Web Workers are not supported, you will simply execute your JavaScript code as usual. If Web Workers are supported, you will be able to push some of the JavaScript code to the workers to enhance the performance of your applications for the most recent browsers. You won’t then break anything or build a specific version only for the very latest browsers. It will work for all browsers with some performance differences.

Non accessible elements from a worker

Rather than looking to what you don’t have access from workers, let’s rather take a look to what you’ve only have access to. For that, please have check the table from our MSDN Documentation: HTML5 Web Worker

But the short version is: you don’t have access to the DOM.

The Web Workers in IE10: Background JavaScript Makes Web Apps Faster article from our IE Blog has a very good diagram summarizing that:

For instance, as you don’t have access to the window object from a worker, you won’t be able to access to the Local Storage (which any way doesn’t seem to be thread-safe). Those limitations may look too constraint for developers used to multithreaded operations in other environments. However, the big advantage is not to fall into the same problems we usually encounter: lock, races conditions, etc. We won’t have to think about that with web workers. This then makes the web workers something very accessible while allowing some interesting performance boosts in specific scenarios.

Errors handling & debugging

It is very easy to handle errors raised from your Web Workers.  You simply have to subscribe to the OnError event in the same way we’ve done it with the OnMessage event:

myWorker.addEventListener("error", function (event) {
    _output.textContent = event.data;
}, false);

This is the best Web Workers can give you natively to help you debugging their code… This is very limited, isn’t it?

The F12 development bar for a better debugging experience

To go beyond that, IE10 offers you to directly debug the code of your Web Workers inside its script debugger like any other script.

For that, you need to launch the development bar via the F12 key and navigate to the “Script” tab. You shouldn’t see the JS file associated to your worker yet. But right after pressing the “Start debugging” button, it should magically be displayed:

Next step is then to simply debug your worker like you’re used to debug your classic JavaScript code!

IE10 is currently the only browser offering you that. If you want to know more about this feature, you can read this detailed article: Debugging Web Workers in IE10

An interesting solution to mimic console.log()

At last, you need to know that the console object is not available within a worker. Thus, if you need to trace what’s going on inside the worker via the .log() method, it won’t work as the console object won’t be defined. Hopefully, I’ve found an interesting sample that mimic the console.log() behavior by using the MessageChannel: console.log() for Web Workers. This works well inside IE10, Chrome & Opera but not in Firefox as it doesn’t support the MessageChannel yet.

Note: in order to make the sample from this link working fine in IE10, you need to change this line of code:

console.log.apply(console, args); // Pass the args to the real log

By this one:

console.log(args); // Pass the args to the real log

Then, you should be able to obtain such results:

DEMO: If you want to try this console.log() simulation: -> http://david.blob.core.windows.net/html5/HelloWebWorkersJSONdebug_EN.htm <-

Use cases and how to identify potential candidates

Web Workers for which scenarios?

When you browse the web looking for sample usages of the Web Workers, you always find the same kind of demos: intensive mathematical/scientific computation. You’ll then find some JavaScript raytracers, fractals, prime numbers, and stuff like that. Nice demos to understand the way workers works but this gives us few concrete perspectives on how to use them in “real world” applications.

It’s true that the limitations we’ve seen above on the resources available inside web workers narrow down the number of interesting scenarios. Still, if you just take some time to think about it, you’ll start to see new interesting usages:

– image processing by using the data extracted from the <canvas> or the <video> elements. You can divide the image into several zones and push them to the different workers that will work in parallel. You’ll then benefit from the new generation of multi-cores CPUs. The more you have, the fastest you’ll go.

– big amount of data retrieved that you need to parse after an XMLHTTPRequest call. If the time needed to process this data is important, you’d better do it in background inside a web worker to avoid freezing the UI Thread. You’ll then keep a reactive application.

– background text analysis: as we have potentially more CPU time available when using the web workers, we can now think about new scenarios in JavaScript. For instance, we could imagine parsing in real-time what the user is currently typing without impacting the UI experience. Think about an application like Word (of our Office Web Apps suite) leveraging such possibility: background search in dictionaries to help the user while typing, automatic correction, etc.
– concurrent requests against a local database. IndexDB will allow what the Local Storage can’t offer us: a thread-safe storage environment for our Web Workers.

Moreover, if you switch to the video game world, you can think about pushing the AI or physics engines to the Web Workers. For instance, I’ve found this experimentation: On Web Workers, GWT, and a New Physics Demo which use the Box2D physic engine with workers. For your Artificial Intelligence engine, this means also that you will be able in the same timeframe to process more data (anticipate more moves in a chess game for instance).

Some of my colleagues may now argue that the only limit is your imagination!

But in a general manner, as long as you don’t need the DOM, any time consuming JavaScript code that may impact the user experience is a good candidate for the web workers. However, you need to pay attention to 3 points while using the workers:

1 – The initializing time and the communication time with the worker shouldn’t be superiors to the processing itself

2 – The memory cost of using several workers

3 – The dependency of the code blocks between them as you may then need some synchronization logic. Parallelization is not something easy my friends!

On our side, we’ve recently published the demo named Web Workers Fountains:

This demo displays some particles effects (the fountains) and uses 1 web worker per fountain to try to compute the particles in the fastest way possible. Each worker result is then aggregated to be displayed inside the <canvas> element. Web Workers can also exchange messages between them via the Message Channels. In this demo, this is used to ask to each of the workers when to change the color of the fountains. We’re then looping through this array of colors: red, orange, yellow, green, blue, purple and pink thanks to the Message Channels. If you’re interesting into the details, jump into the LightManager() function of the Demo3.js file.

Feel free also to launch this demo inside Internet Explorer 10, it’s funny to play with!

How to identity hot spots in your code

To track the bottlenecks and identity which parts of your code you could send to the web workers, you can use the script profiler available with the F12 bar of IE9/10. It will then help you to identify your hot spots. However, identifying a hot spot doesn’t mean you’ve identified a good candidate for web workers. To better understand that, let’s review together 2 different interesting cases.

Case 1: <canvas> animation with the Speed Reading demo

This demo comes from our IE Test Drive center and can be browsed directly here: Speed Reading. It tries to display as fast as possible some characters using the <canvas> element. The goal is to stress the quality of the implementation of the hardware acceleration layer of your browser. But going beyond that, would it be possible to obtain more performance by splitting some operations on threads? We need to achieve some analysis to check that.

If you run this demo inside IE9/10, you can also start the profiler during a couple of seconds. Here is the kind of results you’ll obtain:

If you’re sorting the time consuming functions in decreasing order, you’ll clearly see those functions coming first: DrawLoop(), Draw() and drawImage(). If you’re double-clicking on the Draw line, you’ll jump into the code of this method. You’ll then observe several calls of this type:

surface.drawImage(imgTile, 0, 0, 70, 100, this.left, this.top, this.width, this.height);

Where the surface object is referencing a <canvas> element.

A quick conclusion of this brief analysis is that this demo spends most of its time drawing inside the Canvas through the drawImage() method. As the <canvas> element is not accessible from a web worker, we won’t be able to offload this time consuming task to different threads (we could have imagined some ways of handling the <canvas> element in a concurrency manner for instance). This demo is then not a good candidate for the parallelization possibilities offered by the web workers.

But it’s well illustrating the process you need to put in place. If, after some profiling job, you’re discovering that the major part of the time consuming scripts are deeply linked to DOM objects, the web workers won’t be able to help you boosting the performance of your web app.

Case 2: Raytracers inside <canvas>

Let’s now take another easy example to understand. Let’s take a raytracer like this one: Flog.RayTracer Canvas Demo. A raytracer uses some very CPU intensive mathematical computations in order to simulate the path of light. The idea is to simulate some effects like reflection, refraction, materials, etc.

Let’s render a scene while launching the script profiler, you should obtain something like that:

Again, if we sort the functions in a decreasing order, 2 functions clearly seem to take most of the time: renderScene() and getPixelColor().

The goal of the getPixelColor() method is to compute the current pixel. Indeed, ray-tracing is rendering a scene pixel per pixel. This getPixelColor() method is then calling the rayTrace() method in charge of rendering the shadows, ambient light, etc. This is the core of our application. And if you’re reviewing the code of the rayTrace() function, you’ll see that it’s 100% pure JavaScript juice. This code has no DOM dependency. Well, I think you’ll get it: this sample is a very good candidate to parallelization. Moreover, we can easily split the image rendering on several threads (and thus potentially on several CPUs) as there’s no synchronization needed between each pixel computation. Each pixel operation is independent from its neighborhood as no anti-aliasing is used in this demo.

This is not then a surprise if we can find some raytracers samples using some web workers like this one: http://nerget.com/rayjs-mt/rayjs.html

After profiling this raytracer using IE10, we can see the important differences between using no worker and using 4 workers:

In the first screenshot, the processRenderCommand() method is using almost all of the CPU available and the scene is rendered in 2.854s.

With 4 web workers, the processRenderCommand() method is executed in parallel on 4 different threads. We can even see their Worker Id on the right column. The scene is rendered this time in 1.473s. The benefits were real: the scene has been rendered 2 times faster.

Conclusion

There is no magical or new concepts linked to the web workers in the way to review/architect your JavaScript code for parallel execution. You need to isolate the intensive part of your code. It needs to be relatively independent of the rest of your page’s logic to avoid waiting for synchronization tasks. And the most important part: the code shouldn’t be linked to the DOM. If all these conditions are met, think about the web workers. They could definitely help you boosting the general performance of your web app!

David

PS : All the samples of this article are available in this ZIP file: https://david.blob.core.windows.net/html5/WebWorkersSamples.zip .