This is the last post in our Intro to waPC series. Make sure to check out part 1: Building WebAssembly platforms with waPC, and part 2: Getting started with waPC and WebAssembly.

Writing your waPC host​

We're using nodejs as our host platform because we've already dealt with Go, Rust, and web browsers so let's keep the trend going. Why stick to one platform in this crazy new WebAssembly world?

There are also host implementations for Rust and Go and if you are more familiar with those languages. The differences aren't extensive, but because of how rich Rust's WebAssembly scene is, the Rust host abstracts the WebAssembly runtime away behind a WebAssemblyEngineProvider so you can swap out runtimes and ignore their API differences.

New to nodejs?​

I recommend using nvm to install node and npm. nvm makes swapping versions easier and does all its magic without needing root access.

$ curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.38.0/install.sh | bash

After starting a new shell or source-ing the configuration as it suggests, run this command to install the latest version of node:

$ nvm install node # "node" is an alias for the latest version

Verify your installation of node and npm with node -v && npm -v

Create a new nodejs project​

Create a new directory and run npm init to initialize a package.json file. package.json files store metadata and dependencies for nodejs projects.

$ mkdir wapc-host-test && cd wapc-host-test && npm init -y

Use npm to add @wapc/host and @msgpack/msgpack as dependencies:

$ npm install @wapc/host @msgpack/msgpack

Create a file called index.js and add this JavaScript source:

const { instantiate } = require("@wapc/host");
const { encode, decode } = require("@msgpack/msgpack");
const { promises: fs } = require("fs");
const path = require("path");

// Argument as index 0 is the node executable, index 1 is the script filename
// Script arguments start at index 2
const wasmfile = process.argv[2];
const operation = process.argv[3];
const json = process.argv[4];

// If we dont have the basic arguments we need, print usage and exit.
if (!(wasmfile && operation && json)) {
  console.log("Usage: node index.js [wasm file] [waPC operation] [JSON input]");
  process.exit(1);
}

async function main() {
  // Read wasm off the local disk as Uint8Array
  buffer = await fs.readFile(path.join(__dirname, wasmfile));

  // Instantiate a WapcHost with the bytes read off disk
  const host = await instantiate(buffer);

  // Parse the input JSON and encode as msgpack
  const payload = encode(JSON.parse(json));

  // Invoke the operation in the wasm guest
  const result = await host.invoke(operation, payload);

  // Decode the results using msgpack
  const decoded = decode(result);

  // log to the console
  console.log(`Result: ${decoded}`);
}

main().catch((err) => console.error(err));

The first four lines import nodejs standard libraries as well as the waPC JavaScript host library, @wapc/host, and a JavaScript implementation of MessagePack, @msgpack/msgpack

const { instantiate } = require("@wapc/host");
const { encode, decode } = require("@msgpack/msgpack");
const { promises: fs } = require("fs");
const path = require("path");

Why MessagePack? WaPC — the protocol — does not prescribe a serialization algorithm, but wapc — the CLI — generates guests that use MessagePack as a default.

The code that follows grabs positional arguments passed via the command line starting with the filepath to the wasm, the operation as a string, and a JSON string that we'll serialize and pass to the guest as input. If we don't receive all those parameters then print some basic help before exiting.

// Argument as index 0 is the node executable, index 1 is the script filename
// Script arguments start at index 2
const wasmfile = process.argv[2];
const operation = process.argv[3];
const json = process.argv[4];

// If we don't have the basic arguments we need, print usage and exit.
if (!(wasmfile && operation && json)) {
  console.log("Usage: node index.js [wasm file] [waPC operation] [JSON input]");
  process.exit(1);
}

The async main(){} function allows us to use the more intuitive await syntax for Promises. JavaScript doesn't run any functions by default so we have to manually invoke main() immediately after.

async function main() {
  /* ... */
}

main().catch((err) => console.error(err));

Inside our main function we read our bytes off disk:

buffer = await fs.readFile(path.join(__dirname, wasmfile));

Pass the wasm bytes to instantiate() which returns a waPC host:

const host = await instantiate(buffer);

Parse the passed input as JSON and encode it with MessagePack:

const payload = encode(JSON.parse(json));

Invoke the operation we received from the command line arguments with the MessagePack-ed payload.

const result = await host.invoke(operation, payload);

Decode our response (again with MessagePack) and log it to the console.

const decoded = decode(result);
console.log(`Result: ${decoded}`);

In the previous tutorial we built a waPC guest that exposed two operations, toUppercase and toLowercase. If you don't have a waPC guest wasm file handy, download the tutorial result here: wapc_tutorial.wasm

Run our nodejs host with the command node index.js and pass it three arguments: your wasm file (e.g. wapc_tutorial.wasm), the operation to execute (e.g. toUppercase), and a JSON payload (e.g. '{"name":"Samuel Clemens"}').

$ node index.js wapc_tutorial.wasm toUppercase '{"name":"Samuel Clemens"}'
Result: SAMUEL CLEMENS

That's it! You're running logic written in Rust from nodejs! You can use this same experience to build waPC guests in Go, Zig, or AssemblyScript and run them in Rust, Go, or JavaScript like we did here.

Extra credit​

Remember the wapc_guest::prelude I mentioned in the waPC guest tutorial? That included a host_call function which you can use to issue arbitrary calls from a waPC guest to a waPC host.

It looks something like this:

let result = host_call("binding", "namespace", "operation", &serialize("your data")?)?;

You can respond to these by defining a host call handler in your host. In the nodejs script above that would look something like this:

const host = await instantiate(
  buffer,
  (binding, namespace, operation, data) => {
    console.log(
      `I got a call for operation ${operation} with ${data.length} bytes of data`
    );
  }
);

You can respond to these host call functions however you like. You can build something like a stdlib for native functionality, you could broker calls between guests, or you can build a pluggable interface that dynamically loads other wasm and forwards operations along!

Wrap-up​

This is the final post in our waPC introduction and thank you for sticking with it! We've only scratched the surface of what you can do with waPC and WebAssembly. In the future we'll go over how to get started with WasmCloud and soon we'll introduce new tools to get you working with Vino and all WebAssembly.

If you build anything you want to share, let us know on twitter at @vinodotdev and we'll pass it along!

This tutorial gets you hands-on experience building a waPC guest using Rust and compiling down to WebAssembly. For the guest language you can switch to Go, AssemblyScript, or Zig with only light modifications. When we're done we'll run our guest in the browser with the waPC JavaScript host. Head over to Building a waPC Host in Node.js after to build your own host from scratch.

Installing the wapc CLI​

The waPC CLI is a Go program you can install via the command:

$ curl -fsSL https://raw.githubusercontent.com/wapc/cli/master/install/install.sh | /bin/bash

Confirm your installation was successful by running wapc --help

Creating your waPC guest project​

Use the wapc new command followed by your desired language and the project name. I'm writing in rust and I chose the name wapc-tutorial

$ wapc new rust wapc-tutorial
Creating project directory ./wapc-tutorial
Please enter the project description: Just a basic tutorial

Please enter the version (Default is 0.0.1): 0.0.1

New to Rust?​

Use rustup to keep Rust and tools installed and up-to-date. Install it via the command:

$ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

You'll need to add the WebAssembly target so that you can compile down to wasm:

$ rustup target add wasm32-unknown-unknown

These commands will also install cargo, Rust's package manager.

Editing your schema.apexlang​

The default schema generated by wapc is pretty slim:

namespace "greeting"

interface {
  sayHello(name: string): string
}

The namespace helps describe a scope for objects in Apex but note that Apex does nothing on its own. The wapc CLI's code generation uses Apex but the templates are configurable. I point this out to highlight that namespace is a convention and the implementation is undefined. In terms of this tutorial it means that namespace has no effect. WasmCloud — one of the first major consumers of waPC — uses [project name]:[module name] for its namespaces so I'll do the same and use tutorial but you're free to do what you want.

The interface block describes the operations we expose. The default Apex template describes an operation named sayHello that takes a string for the name parameter and returns a string as its output. Let's change that method to toLowercase and add another for toUppercase with the same parameter name. This is what we have so far:

namespace "tutorial"

interface {
  toUppercase(name: string): string
  toLowercase(name: string): string
}

Generate your code!​

Run make codegen to automatically run wapc generate codegen.yaml. You'll eventually want to familiarize yourself with the internals of codegen.yaml but ignore it for now.

$ make codegen
wapc generate codegen.yaml
Generating src/generated.rs...
Generating src/lib.rs...
Formatting src/generated.rs...
Formatting src/lib.rs...

You'll end up with two new Rust files, src/generated.rs and src/lib.rs.

generated.rs is worth looking at but diving into it is another post. Open up lib.rs which should look like this:

mod generated;
extern crate wapc_guest as guest;
pub use generated::*;
use guest::prelude::*;

#[no_mangle]
pub fn wapc_init() {
    Handlers::register_to_uppercase(to_uppercase);
    Handlers::register_to_lowercase(to_lowercase);
}

fn to_uppercase(_name: String) -> HandlerResult<String> {
    Ok("".to_string()) // TODO: Provide implementation.
}

fn to_lowercase(_name: String) -> HandlerResult<String> {
    Ok("".to_string()) // TODO: Provide implementation.
}

The first two lines include our generated module and the wapc_guest Rust crate. The following two import everything from our generated module as well as wapc_guest::prelude:

mod generated;
extern crate wapc_guest as guest;
pub use generated::*;
use guest::prelude::*;

generated.rs exports data types generated from your Apex, register_* functions, and utility functions like deserialize and serialize.

wapc_guest::prelude exports useful functions like console_log and data types like HandlerResult and CallResult.

Next we have the public wapc_init function which is the function a waPC host executes when initializing a guest.

#[no_mangle]
pub fn wapc_init() {
    Handlers::register_to_uppercase(to_uppercase);
    Handlers::register_to_lowercase(to_lowercase);
}

This needs to be exposed as-is so we use #[no_mangle] to prevent the compiler from changing the name and causing initialization problems. Inside the wapc_init function we have two generated lines for each of our exposed operations. We don't need to change these but just note that wapc generates code that suits the style of the destination language. It changed toUppercase to to_uppercase to matches Rust's coding guidelines.

The two helper functions passed to the register_* functions are stubs where you can add your custom logic.

fn to_uppercase(_name: String) -> HandlerResult<String> {
    Ok("".to_string()) // TODO: Provide implementation.
}

fn to_lowercase(_name: String) -> HandlerResult<String> {
    Ok("".to_string()) // TODO: Provide implementation.
}

Here we have two functions that both take a String and return a HandlerResult<String>. If you are not familiar with Rust, Result types capture success and errors into a single return value. This is idiomatic Rust, not waPC. Guest implementations in other languages will match the style and practices of those languages.

To flesh out our uppercase and lowercase functions we just code as normal. If you're following along with this simple tutorial, we're only a couple of methods away from making these functions do as they're expected:

fn to_uppercase(name: String) -> HandlerResult<String> {
    Ok(name.to_ascii_uppercase())
}

fn to_lowercase(name: String) -> HandlerResult<String> {
    Ok(name.to_ascii_lowercase())
}

That's it! The generated waPC code in generated.rs does all the serialization and type conversion for you so the code you write is just normal code. If you have previous Rust experience (or the language you chose), take a look at the generated code to see how it works. It's not magic, it just feels that way.

Compiling to WebAssembly​

The project wapc new creates includes a Makefile that has the basic tasks you need to go from start to finish. To compile our code to a wasm binary, run make. This runs three subtask: make deps which does nothing for this project by default, make codegen which is what we already ran to generate our code, and make build which runs cargo build --target wasm32-unknown-unknown --release and copies the resulting .wasm file to a build/ directory in our project root.

If all goes as planned, then you should have a wasm file in your build/ directory. In my case it's build/wapc_tutorial.wasm. Yours will be named according to what you named your project when you ran wapc new or later edited in your Cargo.toml file.

Whoa that size​

If you're using the default Rust setup, take notice that your wasm file is not very lean. On my machine the wasm file clocks in at a whopping 1.7 megabytes for these two measly string manipulation functions. Keep in mind WaPC started on the server and users are only just starting to bring it to the web. A large chunk of this size can be optimized away and work is already underway to trim this down, but the fact remains that WebAssembly fundamentally changes the game. Many existing libraries and languages start at a disadvantage. Developers didn't need to worry about build size much unless they targeted embedded platforms. Compare the Rust build with WebAssembly-first languages like AssemblyScript or TinyGo which come in at 17k and 11k, respectively. While both are smaller by orders of magnitude, it's a tradeoff. Despite the marketing TinyGo is not actually Go and AssemblyScript is not actually TypeScript. You can't benefit from the developer and open source community as much as you can with Rust. There is no one-size-fits-all WebAssembly language yet.

Some day we'll probably see a single language dominate the WebAssembly landscape but right now you have to optimize for your destination. WaPC really helps out here because you can jump in and out of other languages while retaining the same workflow. The API to every waPC guest and host is identical and Apex is universal. You can use the wapc CLI to generate a lot of the platform-specific code and still only work on your business logic.

Running your wasm file​

Head on over to our waPC loader and load your wasm file from disk. Fill in your operation (toUppercase or toLowercase if you're following along) and the input data (e.g. {"name": "Samuel Clemens"}) and press Run. Hopefully all goes well and you'll have the result show up in the Result: box along with some status logging down below.

You should see something similar to this:

If something did go wrong then hopefully the error messages will help. If not, then drop us a line and we'll figure out what went wrong.

Writing your own host​

WaPC guests are only half the equation, the next step is making your own host that leverages the functionality. Head over to Building a waPC Host in Node.js when you're ready to get started!

While waPC is not the holy grail, it's a satisfying solution to the headache of getting productive with WebAssembly. When WebAssembly's "Hello World" can leave you with more questions than answers, waPC scales with you from starter projects to enterprise application meshes.

What is waPC?​

The WebAssembly Procedure Calls (waPC) project is like a standard module interface for WebAssembly on top of an extensible RPC framework. It irons out the wrinkles between native code and WebAssembly guests to make passing and receiving complex data types trivial. Under the hood, waPC defines an opaque protocol that allows you to broker arbitrary, dynamic calls across native logic, WebAssembly, from WebAssembly to other WebAssembly, or across the internet without knowing anything about the call's data structure or serialization algorithms.

WaPC is great for platform builders but covers common use cases just as well. Even though the goal is broader than some other projects, the waPC experience is so intuitive and powerful that you should keep it in your toolbox no matter what you end up using.

waPC vs wasm-bindgen (et al)?​

Similar looking tools, different audiences​

If you write Rust and want to target the web browser alone, the wasm-bindgen project is mature and tailored specifically to this use case. WaPC is more generic. WaPC is better suited for building cross-platform applications than fast browser code.

WaPC has host implementations in Rust, Go, and JavaScript (nodejs + browser) and guest SDKs for Rust, TinyGo, AssemblyScript, and Zig. If you're looking for the "portable" part of WebAssembly, waPC is for you. If you're looking to make lightning fast web application, wasm-bindgen is forging the path there.

Both wasm-bindgen and waPC are made of layers that could complement each other, but they aren't made to coexist at the moment.

The waPC suite​

Do you learn better by copy/pasting code and running things yourself? Try heading to Getting started with waPC and use this post as a reference when you need it.

The waPC protocol​

The waPC protocol is a handful of complementary host and guest methods that act as the communication contract across the WebAssembly boundary. They provide an interface for calling operations, reading data, communicating errors, and exposing a logger. Check out this TypeScript representation of the waPC protocol from the host's side for a glimpse into the technicals.

waPC Hosts​

A waPC host is the native implementation that loads and initializes a WebAssembly guest and makes requests over the waPC protocol. WaPC has host implementations for Rust, Go, and JavaScript. If you don't see your platform, the protocol is small and you could create a host implementation in a day or two. This is another reason we committed to waPC, each layer of the technology revolves around a dense but understandable core.

waPC Guests​

WaPC guests are WebAssembly modules that implement the guest portion of the waPC protocol. "Operations" are an important part of waPC guests and hosts. Operations are like the exported functions you could expose when compiling to wasm normally, but waPC adds a layer of abstraction that keeps the interface in and out of wasm consistent. This keeps reliable bindings for hosts while maintaining a dynamic interface to internal guest functionality.

Guests can also make host calls, i.e. native function calls. These are operations in the reverse direction that include the input payload and the requested operation name as well as strings representing a namespace and binding. The additional data gives hosts flexibility to define the interface it exposes to its guests. A host could provide native functionality like a custom stdlib, or it could dynamically load and forward those calls to another WebAssembly module. WaPC has guest implementations for Rust, TinyGo, AssemblyScript, and Zig.

The actual process of implementing the guest bindings is abstracted away from you via the wapc CLI and Apex (see below).

Apexlang​

Apex is an interface definition language for describing waPC modules. It's easier to understand with an example:

interface {
  # This defines an add operation that takes
  # two numbers and returns another
  add(left: number, right: number): number

  # This is a more complex example that shows an example
  # HTTP Request operation
  request(url: string): HttpResponse
}

type HttpResponse {
  status:number
  headers: [Header]
  body: string
}

type Header {
  name: string
  value: string
}

The wapc CLI uses Apex like this to generate type definitions and integration code for waPC guests and hosts. For most usage, you define your interface with Apex and wapc generates everything else. You're left writing only your business logic.

The Apex spec defines the types and syntax of additional features. The Apex parser and Apex-codegen are written in JavaScript and available as npm modules or browser bundles.

The wapc CLI​

wapc is a command line interface to helper methods:

• wapc new automatically creates new waPC guests with default templates in the language of your choice.
• wapc generate automatically generates integration code from your Apex schemas.
• wapc update keeps the wapc internals up-to-date.

The officially supported CLI is a Go program and the logic is written as JavaScript and available on npm with bundles for browser environments. Rust & nodejs versions of the CLI exist to show how you can embed wapc functionality like I did with the Apex-validator. The validator does all the parsing and code generation on the client side. No servers were harmed in this demo.

Passing and receiving complex data types​

The waPC protocol doesn't prescribe a data serialization algorithm, but wapc generates code that uses MessagePack as a default. MessagePack is a sensible option while waPC's format-agnostic stance means you can substitute something more efficient for your data or build brokers that pass the data along without deserialization.

How it looks​

Linked here are some sample projects for you to get a feel for what Rust guests look like: jsoverson/wapc-guest-examples. The contained projects were all generated via the wapc new command.

The echo example sends returns the input string value as output and has the following schema Apex:

interface {
  echo(msg: string): string
}

Here is the compiled wasm running live in the browser:

{{< wapc-loader url="/wasm/rust_echo_string.wasm" operation="echo" >}}

Writing your own waPC hosts and guests​

That's it! The next step is to start working with actual code yourself. Check out our guest and host tutorial over at Getting started with waPC.