WebAssembly

How to Cross Compile a C library to WebAssembly for use with JavaScript

In this post I’ll explore how to use an existing C library in a JavaScript with WebAssembly.

This uses information from previous posts for
dataframes,
and
wasi.

You can find the source code
here.

The Toolchain

In this example I used the ready made clang toolchain from
wasi-sdk. I downloaded
wasi-sdk-11.0-linux.tar.gz
and unpacked it in /opt/wasi-sdk as follows.

1
2
3
4
5
6
cd /tmp
wget https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-11/wasi-sdk-11.0-linux.tar.gz
cd /opt
sudo tar xvf /tmp/wasi-sdk-11.0-linux.tar.gz
sudo ln -s wasi-sdk-11.0 wasi-sdk
rm /tmp/wasi-sdk-11.0-linux.tar.gz

I also installed wabt 1.0.16, and
wasmer, and unpacked them in /opt. I then
put the bin directories from all on my path.

Cross Compiling

I decided to use the
GNU Scientific Library
as it is written in plain old C, and I can use it to further the
dataframes
project I have discuessed in previous posts.

Compiling it proved to be remarkably straightforward. After downloading and
unpacking the tarball I did the following.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# Ensure wasi-sdk is on the path
export PATH=/opt/wasi-sdk/bin:$PATH

# Go to the project folder and make a build folder.
cd gsl-2.6
mkdir build
cd build

# Configure the project setting the clang toolchain
CC=clang \
CFLAGS="-fno-trapping-math --target=wasm32-wasi -mthread-model single" \
CPP=clang-cpp \
AR=llvm-ar \
RANLIB=llvm-ranlib \
NM=llvm-nm \
LD=wasm-ld \
../configure \
    --prefix=/opt/gsl-2.6 \
    --host=wasm32-wasi \
    --enable-shared=no

# build and install it (you need to have permissions to /opt/gsl-2.6)
make install

Amazingly that just worked. The result of the install was the folder
/opt/gsl-2.6 with three sub-folders: include, lib, and share.

Testing the Static Library

I created an example.c with the following contents taken from the
gsl documentation.

1
2
3
4
5
6
7
8
9
10
#include <stdio.h>
#include <gsl/gsl_sf_bessel.h>

int main (int argc, char** argv)
{
  double x = 5.0;
  double y = gsl_sf_bessel_J0 (x);
  printf ("J0(%g) = %.18e\n", x, y);
  return 0;
}

I compiled this in the following manner (using the wasi-sdk clang).

1
clang example.c -o example -O2 --target=wasm32-wasi -I/opt/gsl-2.6/include -L/opt/gsl-2.6/lib -lgsl -lm -lc

No errors! But is it really a wasm file? We can test with the file utility.

1
2
$ file ./example
example: WebAssembly (wasm) binary module version 0x1 (MVP)

This is all very cool, but what’s in the static library? Let’s take a look.

1
2
3
4
5
6
7
8
9
10
11
12
# Make sure we're using the llvm archive function.
$ which ar
/opt/wasi-sdk/bin/ar
# Find the first file in the library.
$ ar t /opt/gsl-2.6/lib/libgsl.a | head -1
version.o
# Extract the file
$ ar xv /opt/gsl-2.6/lib/libgsl.a version.o
x - version.o
# What kind of object file is it?
$ file version.o
version.o: WebAssembly (wasm) binary module version 0x1 (MVP)

It seems that the “object” files are themselves wasm, which get linked in to the
finaly module, which is itself wasm. Interesting!

We can run the “executable” file with wasmer. Wasmer is a runtime engine for WebAssembly
modules.

1
2
$ /opt/wasmer/bin/wasmer example1
J0(5) = -1.775967713143382642e-01

It works :) Let’s see how to use the library in our JavaScript code.

Create the JavaScript

This is definately TL;DR

Making a concise example of this would have been much shorted, but I’m exploring
the data science possibilities of WebAssembly, so what I’ve done is wire the
functions in to my growing dataframe/series code.

First I need to create a helper to call aggregate functions. I do this in
the file wasi-memorymanager.js.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
class WasmMemoryManager {

  ...

  invokeAggregateFunction(func, array, typedArrayType) {
    let input = null

    try {
      input = this.createTypedArray(typedArrayType, array.length)
      input.set(array)

      return func(input.byteOffset, array.length)
    } finally {
      this.free(input.byteOffset)
    }
  }

  ...
}

To use the aggregate function I add it to the setup-wasi.js.

1
2
3
4
5
6
7
8
9
10
11
12
13
...

arrayMethods.set(
  'mean',
  makeAggregateOperation(
    wasi.wasiMemoryManager,
    null,
    (array, length) => gsl_stats_mean(array, 1, length),
    (series) => series.array.reduce((a, b) => a + b, 0) / series.array.length
  )
)

...

You can see here I’m calling the raw gsl function gsl_stats_mean (I needed the
arrow function to provide a stride parameter of 1). What I want to show is
that there is little or no requirement for a binding library. We’re using generic
marshalling code.

The Series.js needs a small patch to handle functions that return a single
value, rather than a new series. I added a check for the return value of the
function.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
class Series {
  constructor(...) {
    ...

    return new Proxy(this, {
      get: (obj, prop, receiver) => {
        if (prop in obj) {
          return Reflect.get(obj, prop, receiver)
        } else if (arrayMethods.has(prop)) {
          return (...args) => {
            const [value, type] = arrayMethods.get(prop)(obj, ...args)
            if (value instanceof Array) {
              return new Series('', value, type)
            } else {
              // An aggregated value
              return value
            }
          }
        } else {
          return Reflect.get(obj.array, prop, receiver.array)
        }
      },

      ...

Dude Where’s My Function?

Ideally we would like to call the function without any extra effort, but a
function from a static library will not be included in the final module unless
there is a reference to it. This means we need a C file to create a reference so
it gets exported. I did this as follows.

1
2
3
4
5
6
7
8
#include <gsl/gsl_statistics.h>

void force_gsl_imports()
{
  void* funcs[] = {
      gsl_stats_mean
    };
}

This is all we need to ensure the function is included in the emitted wasm module. A
specific binding function is unnecessary.

The example program

Now we just need to see if it works.

1
2
3
const height = new Series('height', [1.82, 1.72, 1.64, 1.88], 'double')
const avg = height.mean()
console.log('mean', avg)

Success!

Thoughts

The thing that surprised me the most was how simple it was to create the static
library (although it’s possible I just got lucky with the library I chose).

My example was too verbose, But I’m hoping you’re following the journey with me
and have read my previous posts.

The gsl already provides a BLAS library, so we could write a JavaScript matrix
class and do linear algebra. My hope is that the anticipated FORTRAN compiler
(flang) will be just as easy to create a library from.