If a C++ program utilizes multithreading via pthreads, then compiling it with Emscripten into WebAssembly requires setting certain flags for both compiler and linker. In addition to that, web-server that will be hosting the resulting web-application also requires some configuration.

A couple of years ago I compiled a Qt-based application into WebAssembly using Emscripten, and I did mention pthreads there too, but it was very briefly and without any details. And as it turned out, there are some interesting moments in there which are worth being documented.

An example C++ program

To demonstrate multithreading I needed a C++ program that executes some tasks in parallel. As I am still not a real developer, especially when it comes to parallel programming, especially in C++, the example I’ve managed to come up with is probably isn’t the best example in the world. My only hope is that it is at least a correct one.

The sources of the program are here.

The idea is that there are two executables and one common function. All that function does is it sleeps for 2 seconds and performs some meaningless calculation:

void doShit(int someID, bool isAThread)
{
    sleep(2);

    if(isAThread)
    {
        printf("ololo, inside a thread, batch ID: %d\n", someID);
    }
    else
    {
        std::cout << "ololo, inside the task #" << someID << std::endl;
    }

    double result = 0.0;
    for (int i = 0; i < 11111; i++)
    {
        result = result + sin(i) * tan(i);
    }
}

One executable sequently calls this function 10 times, while another executable spawns 10 threads to call that same function in parallel. Naturally, the latter one is supposed to finish the work faster.

Let’s check that. First the variant without threads:

$ cd /path/to/webassembly-pthreads-example
$ mkdir build && cd $_

$ clang++ ../src/without-pthreads.cpp ../src/some-shit.cpp -o some-without-threads
$ time ./some-without-threads
A program WITHOUT threads

ololo, inside the task #0
ololo, inside the task #1
ololo, inside the task #2
ololo, inside the task #3
ololo, inside the task #4
ololo, inside the task #5
ololo, inside the task #6
ololo, inside the task #7
ololo, inside the task #8
ololo, inside the task #9

real    0m20.306s

and then the variant with threads:

$ pwd
/path/to/webassembly-pthreads-example/build
$ rm -r ./*; ls -lah

$ clang++ ../src/with-pthreads.cpp ../src/some-shit.cpp -o some-with-threads
$ time ./some-with-threads
A program WITH threads

Spawned thread #0
Spawned thread #1
Spawned thread #2
Spawned thread #3
Spawned thread #4
Spawned thread #5
Spawned thread #6
Spawned thread #7
Spawned thread #8
Spawned thread #9

ololo, inside a thread, batch ID: 1
ololo, inside a thread, batch ID: 1
ololo, inside a thread, batch ID: 1
ololo, inside a thread, batch ID: 1
ololo, inside a thread, batch ID: 1
ololo, inside a thread, batch ID: 1
ololo, inside a thread, batch ID: 1
ololo, inside a thread, batch ID: 1
ololo, inside a thread, batch ID: 1
ololo, inside a thread, batch ID: 1

real    0m2.218s

As expected, ~20 seconds of sequential execution without threads versus ~2 seconds of parallel computing with threads.

Compiling into WebAssembly

I have already described installing Emscripten, and the only addition here is that instead of getting exactly the version 1.39.8 (which is rather old by now, after all it’s web we are talking about) you can just set the latest value, and it well get you whichever is the latest one available (I got 3.1.48).

Also don’t forget to source the emsdk_env.sh environment before trying to build the project.

Without pthreads the compilation is the same as before:

$ cd /path/to/webassembly-pthreads-example
$ mkdir build && cd $_

$ emcc ../src/without-pthreads.cpp ../src/some-shit.cpp \
    -o ./some-without-pthreads.html

I didn’t like how it bundles styles and scripts into one .html, so I’ve split it and used -o to bare *.js afterwards. The resulting files are:

$ emcc ../src/without-pthreads.cpp ../src/some-shit.cpp \
    -o ./some-without-pthreads.js

$ du -h ./*
176K    ./some-without-pthreads.js
168K    ./some-without-pthreads.wasm

To compile into WebAssembly with enabled pthreads support, you need to set -pthread compiler flag. Documentation says that you also need to set -pthread linker flag, although I haven’t noticed any problems if it isn’t set. Anyway, the compilation command then becomes the following:

$ pwd
/path/to/webassembly-pthreads-example/build
$ rm -r ./*; ls -lah

$ emcc ../src/with-pthreads.cpp ../src/some-shit.cpp \
    -o ./some-with-pthreads.js \
    -pthread

$ du -h ./*
212K    ./some-with-pthreads.js
184K    ./some-with-pthreads.wasm
8.0K    ./some-with-pthreads.worker.js

So in addition to *.js and *.wasm files now there is also *.worker.js.

If you’ll try to build it without providing -pthread flag, then it will still compile, but there will no *.worker.js file produced, and the program will fail to run in web-browser like this:

some-with-pthreads.js:1 [ERROR] return code from pthread_create() is 6 | Resource temporarily unavailable

thanks to this check in the program sources:

int rez = pthread_create(&thread_ids[i], NULL, threadCallback, &batchID);
if (rez)
{
    std::cerr << "[ERROR] return code from pthread_create() is " << rez
              << " | " << strerror(rez)
              << std::endl;
    return EXIT_FAILURE;
}

There is one more linker flag which you most probably will need to add too: -sPROXY_TO_PTHREAD. There will be some details about it later, but for now let’s continue without it.

With CMake

Of course, there is the fun in compiling everything in just one line, so I made a CMake project for this. Jokes aside, even though it looks more complicated, managing a project with CMake will inevitably pay off going forward, especially when you’ll start bringing in dependencies.

Adding compiler and linker flags can be done like this:

set(EMSCRIPTEN_PTHREADS_COMPILER_FLAGS "-pthread")
set(EMSCRIPTEN_PTHREADS_LINKER_FLAGS "${EMSCRIPTEN_PTHREADS_COMPILER_FLAGS}") # -sPROXY_TO_PTHREAD

if(EMSCRIPTEN)
    string(APPEND CMAKE_C_FLAGS " ${EMSCRIPTEN_PTHREADS_COMPILER_FLAGS}")
    string(APPEND CMAKE_CXX_FLAGS " ${EMSCRIPTEN_PTHREADS_COMPILER_FLAGS}")
    string(APPEND CMAKE_EXE_LINKER_FLAGS " ${EMSCRIPTEN_PTHREADS_LINKER_FLAGS}")
endif()

Alternatively, linker flags can be set via target properties:

if(EMSCRIPTEN)
    set_target_properties(${CMAKE_PROJECT_NAME}
        PROPERTIES
            LINK_FLAGS "${EMSCRIPTEN_PTHREADS_LINKER_FLAGS}"
    )
endif()

Either way, building the project now becomes the following. First, regular CLI program without pthreads:

$ cd /path/to/webassembly-pthreads-example/build
$ rm -r ./*; rm .ninja_*; ls -lah

$ cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DWITH_PTHREADS=0 ..
$ cmake --build . --target install
...
[3/4] Install the project...
-- Install configuration: "Release"
-- Installing: /path/to/webassembly-pthreads-example/install/some-without-pthreads

and with pthreads:

$ pwd
/path/to/webassembly-pthreads-example/build
$ rm -r ./*; rm .ninja_*; ls -lah

$ cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DWITH_PTHREADS=1 ..
$ cmake --build . --target install
...
[3/4] Install the project...
-- Install configuration: "Release"
-- Installing: /path/to/webassembly-pthreads-example/install/some-with-pthreads

Note that I didn’t provide -DCMAKE_INSTALL_PREFIX="../install" for the installation, as one would expect me to. This is because that path is already managed inside the project.

Then WebAssembly variant without pthreads:

$ pwd
/path/to/webassembly-pthreads-example/build
$ rm -r ./*; rm .ninja_*; ls -lah

$ cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DWITH_PTHREADS=0 \
    -DCMAKE_TOOLCHAIN_FILE="$EMSDK/upstream/emscripten/cmake/Modules/Platform/Emscripten.cmake" \
    ..
$ cmake --build . --target install
...
[3/4] Linking CXX executable some-without-pthreads.js
em++: warning: ignoring unsupported linker flag: `-rpath` [-Wlinkflags]
[3/4] Install the project...
-- Install configuration: "Release"
-- Installing: /path/to/webassembly-pthreads-example/web/some-without-pthreads.js
-- Installing: /path/to/webassembly-pthreads-example/web/some-without-pthreads.wasm

$ du -h ../web/some-without*
 68K    ../web/some-without-pthreads.js
140K    ../web/some-without-pthreads.wasm

and with pthreads:

$ pwd
/path/to/webassembly-pthreads-example/build
$ rm -r ./*; rm .ninja_*; ls -lah

$ cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DWITH_PTHREADS=1 \
    -DCMAKE_TOOLCHAIN_FILE="$EMSDK/upstream/emscripten/cmake/Modules/Platform/Emscripten.cmake" \
    ..
$ cmake --build . --target install
...
[3/4] Linking CXX executable some-with-pthreads.js
em++: warning: ignoring unsupported linker flag: `-rpath` [-Wlinkflags]
[3/4] Install the project...
-- Install configuration: "Release"
-- Installing: /path/to/webassembly-pthreads-example/web/some-with-pthreads.js
-- Installing: /path/to/webassembly-pthreads-example/web/some-with-pthreads.wasm
-- Installing: /path/to/webassembly-pthreads-example/web/some-with-pthreads.worker.js

$ du -h ../web/some-with-*
 84K    ../web/some-with-pthreads.js
156K    ../web/some-with-pthreads.wasm
4.0K    ../web/some-with-pthreads.worker.js

No idea why does it warn about -rpath flag and what I can do about it.

Also, it is interesting how the sizes are different between the same files that were built directly with emcc and those built through CMake (the latter ones are smaller). It is probably because Emscripten.cmake toolchain sets some optimization flags, although I didn’t see any at first glance.

Running the results

Most of the information I got from this article. In short, to be able to run a pthreads-enabled WebAssembly binary in a web-browser, you need to fulfil several requirements:

Web-server:
- MIME types for .wasm and .js;
- cross-origin isolation HTTP headers (COEP/COOP);
- secure context (HTTPS);
Web-browser:
- SharedArrayBuffer support.

MIME types and HTTP headers

With a basic Python HTTP server as an example:

class HttpRequestHandler(http.server.SimpleHTTPRequestHandler):
    extensions_map = {
        ".js": "application/javascript",
        ".wasm": "application/wasm",
        # ...
        "": "application/octet-stream"
    }
    
    def send_response(self, *args, **kwargs):
        http.server.SimpleHTTPRequestHandler.send_response(self, *args, **kwargs)
        self.send_header("Cross-Origin-Embedder-Policy", "require-corp")
        self.send_header("Cross-Origin-Opener-Policy", "same-origin")

Of course, this particular web-server implementation isn’t recommended for actual production use. Basically, it’s only good for localhost development environment, and so just in case here’s an example of configuring the same stuff for NGINX:

http {
    # ...

    types {
        # ...
        application/javascript js;
        application/wasm wasm;
    }
    
    server {
        # ...

        add_header Cross-Origin-Embedder-Policy "require-corp";
        add_header Cross-Origin-Opener-Policy "same-origin";
    }
}

Other web-servers have similar configuration options too.

Secure context with HTTPS

This basically means that you need to have an SSL/TLS certificate to serve your web-application via HTTPS. For production environment you most probably already have a certificate for your domain, but how does one get a certificate for localhost?

Turns out, it’s just one command (with a set of arguments you’d never figure out on your own):

$ cd /path/to/webassembly-pthreads-example/certificates
$ openssl req -x509 -newkey rsa:2048 -nodes -sha256 \
    -subj '/CN=localhost' -extensions EXT -config <( \
    printf "[dn]\nCN=localhost\n[req]\ndistinguished_name=dn\n[EXT]\nsubjectAltName=DNS:localhost\nkeyUsage=digitalSignature\nextendedKeyUsage=serverAuth") \
    -out localhost.crt -keyout localhost.key

Resulting certificate (localhost.crt) and key (localhost.key) files can be provided to a web-server, as you would normally do that. Here’s an example for Python HTTP server:

# ...

context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
context.load_cert_chain(
    certfile="/path/to/webassembly-pthreads-example/certificates/localhost.crt",
    keyfile="/path/to/webassembly-pthreads-example/certificates/localhost.key"
)

and here’s an example for NGINX:

server {
    # ...

    ssl_certificate /path/to/webassembly-pthreads-example/certificates/localhost.crt;
    ssl_certificate_key /path/to/webassembly-pthreads-example/certificates/localhost.key;
}

Web-browsers will still show a warning when you’ll try to open the page for the first time, as the certificate is self-signed and isn’t verified by anyone. Here’s a warning you’ll get in Firefox:

Self-signed certificate for localhost, Firefox warning

and here’s a warning from the certificate preview in Safari:

Self-signed certificate for localhost, Safari warning

Anyway, it’s no problem to just ignore these warnings and still open the page, you don’t even need to add this certificate to the trusted ones on system level.

Struggling with OpenSSL on Windows

That OpenSSL command for creating a certificate should succeed on Mac OS and GNU/Linux, but it most likely will fail on Windows.

In regular cmd it will fail with:

The system cannot find the file specified.

If trying to check OpenSSL version fails too, then it might be because OpenSSL executable isn’t in the PATH:

> openssl version
'openssl' is not recognized as an internal or external command,
operable program or batch file.

> c:\programs\git\usr\bin\openssl.exe version
OpenSSL 3.1.2 1 Aug 2023 (Library: OpenSSL 3.1.2 1 Aug 2023)

But still, even with the full path to openssl.exe the command to create a certificate will fail with the same error, and that is apparently because cmd can’t process input redirection (and there is no printf in cmd).

However, trying to execute the command in Git BASH fails too:

Can't open "/proc/1677/fd/63" for reading, No such file or directory
341B0000:error:80000003:system library:BIO_new_file:No such process:../openssl-3.1.2/crypto/bio/bss_file.c:67:calling fopen(/proc/1677/fd/63, r)
341B0000:error:10000080:BIO routines:BIO_new_file:no such file:../openssl-3.1.2/crypto/bio/bss_file.c:75:

And surprisingly that is also because of the issues with input redirection. So instead of providing parameters inline with printf, create a file params.conf and put them there:

[dn]
CN=localhost
[req]
distinguished_name=dn
[EXT]
subjectAltName=DNS:localhost
keyUsage=digitalSignature
extendedKeyUsage=serverAuth

Then the command becomes this:

$ openssl req -x509 -newkey rsa:2048 -nodes -sha256 \
    -subj '/CN=localhost' -extensions EXT -config params.conf \
    -out localhost.crt -keyout localhost.key

But it will still fail in Git BASH, although with a different error:

req: subject name is expected to be in the format /type0=value0/type1=value1/type2=... where characters may be escaped by \. This name is not in that format: 'C:/programs/git/CN=localhost'

A workaround for that one is to add one more / to the -subj argument, so it becomes -subj '//CN=localhost'.

At the same time, it succeeds in regular cmd even without adding this /, so at least you should be able to use that:

> c:\programs\git\usr\bin\openssl.exe req -x509 -newkey rsa:2048 -nodes -sha256 -subj '/CN=localhost' -extensions EXT -config params.conf -out localhost.crt -keyout localhost.key

If nothing helps, then you’ll have to use proper environment on Mac OS or GNU/Linux to create certificate there and copy its files to your Windows machine.

Web-browsers support

Not that long ago you could count only on Chromium-based browsers, because while there was a good chance that simple web-applications would run in any browser, more complex applications were often stumbling upon various issues in non-Chromium browsers.

For example, Firefox (at least in version 99 but probably later too) did not support dynamic loading of modules in workers (here’s a bugreport about that) and was failing with the following error:

Dynamic module import is disabled or not supported in this context

And WebKit-based browsers, such as Safari (at least around version 15.4), were simply crashing, trying to reload the page several times before finally giving up for good.

Fortunately, nowadays all(?) modern web-browsers do support pthreads-enabled WebAssembly. Nevertheless, it is still a good idea to ship your web-application in two variants: with and without pthreads enabled, and load this or that variant based on certain checks.

In particular, to detect whether current web-browser has support for SharedArrayBuffer or not, you can do the following:

let pthreadsSupported = false;

const sharedArrayBufferSupported = typeof SharedArrayBuffer !== "undefined";
console.debug(`SharedArrayBuffer supported: ${sharedArrayBufferSupported}`);
try
{
    if(sharedArrayBufferSupported && typeof MessageChannel !== "undefined")
    {
        console.debug("Testing for MessageChannel/postMessage in case of Firefox...");
        // test for transferability of SharedArrayBuffer
        // https://groups.google.com/forum/#!msg/mozilla.dev.platform/IHkBZlHETpA/dwsMNchWEQAJ
        new MessageChannel().port1.postMessage(new SharedArrayBuffer(1));
        pthreadsSupported = true;
    }
    else
    {
        pthreadsSupported = false;
    }
}
catch(ex)
{
    console.debug(`postMessage failed: ${ex}`);
    pthreadsSupported = false;
}

console.debug(`WASMT (WebAssembly with pthreads) supported: ${pthreadsSupported}`);

// fallback to the variant without pthreads
const wasmURL = pthreadsSupported === true
    ? "./some-with-pthreads.js"
    : "./some-without-pthreads.js";
console.debug(`Variant to load: ${wasmURL}`);

let scrpt = document.createElement("script");
scrpt.src = wasmURL;
document.body.appendChild(scrpt);

Speaking about checking for available WebAssembly features in general, here are some more detectors which you might want to use.

Now, to serve your application on localhost you can run this crude Python script (note that it expects to find certificate files in ./certificates/ and your web-application content in ./web/):

$ cd /path/to/webassembly-pthreads-example/
$ python ./server.py

and open https://localhost:8000/ in your web-browser.

Here’s how the program without pthreads will run: