Apache APISIX embraces the WASM ecosystem

Support for WASM will be added in the upcoming Apache APISIX version (2.11.0)! By reading this article you will learn how Apache APISIX deploys the support and development of this feature from 0 to 1.

In the upcoming release of Apache APISIX 2.11.0, we will add support for WASM! You can develop plugins in WASM, Lua, Java, Go, Python, JavaScript with Apache APISIX 2.11.0.

Click to Preview

WASM, known as WebAssembly, differs from the specific programming language runtimes mentioned above, is a set of bytecode standards specifically designed to be used nested in a host environment. If a programming language provides the ability to compile to WASM bytecode, applications written in that language can be compiled to WASM bytecode and run in some WASM-enabled host environment.

Doesn't it sound like you can run any application like an operating system as long as the host environment supports WASM?

But there is a limitation here. Just like an operating system needs to implement a specific standard syscall, in order to run a specific application, you need to implement the API required for that application.

Take JavaScript for example, although it is also JavaScript code, JS modules written for browsers can't be used directly in npm packages, because the APIs are different.

So just putting WASM into Apache APISIX doesn't work. To allow developers to run WASM on Apache APISIX, we also need to provide a special API.

Why Proxy WASM​

We weighed two options on how to provide this API.

1. Implement the corresponding WASM version of the API by referring to the lua-nginx-module interface
2. Implement Proxy WASM as a set of standards

Proxy WASM is Envoy's WASM API standard. So the above question is really equivalent to, do we make our own API standard or do we reuse Envoy's existing standard?

The WASM API standard can be broken down into two aspects:

1. Host, which is responsible for providing the implementation of the API
2. SDK, which is responsible for implementing a set of glue layers in a different programming language in order to call the provided APIs in that language

Copy

If we follow Envoy's standards, the advantage is that we can reuse Envoy's existing WASM SDK (Proxy WASM SDK), while the disadvantage is that this set of standards is developed by Envoy in conjunction with its own situation, and if we follow the implementation, tailoring it to meet our own demands is difficult.

After some community discussion, we finally decided to adopt the Proxy WASM standard. "Doing the hard and right thing" is naturally the hard thing to do, but we believe it is the right thing to do, and through community collaboration and building together, we can build a more prosperous ecosystem.

How to use WASM in Apache APISIX​

Apache APISIX now has initial support for WASM, which can be used to write parts of the fault-injection plugin. You can try it in Apache APISIX 2.11.0 at the end of this month, so stay tuned!

In the following, we will talk about how to use WASM to inject custom responses in conjunction with proxy-wasm-go-sdk.

Step 1: Write code based on proxy-wasm-go-sdk​

The implementation code (including go.mod and others) can be found at here.

It should be explained that although the proxy-wasm-go-sdk project carries the Go name, it actually uses tinygo instead of native Go, which has some problems supporting WASI (which you can think of as a non-browser WASM runtime interface), see here for more details.

Step 2: Build the corresponding WASM file​

tinygo build -o ./fault-injection/main.go.wasm -scheduler=none -target=wasi ./fault-injection/main.go

Step 3: Refer to this file in config.yaml in Apache APISIX​

apisix:
        ...
wasm:
    plugins:
        - name: wasm_fault_injection
          priority: 7997
          file: t/wasm/fault-injection/main.go.wasm

By doing so, you can use this WASM plugin as if it were a Lua plugin. For example:

---
uri: "/wasm"
plugins:
  wasm_fault_injection:
    conf: '{"body":"hello world", "http_status":200}'
upstream:
  type: roundrobin
  nodes:
    127.0.0.1:1980: 1

Note that the configuration of the WASM plugin is a string under the conf field, which is parsed by the corresponding plugin itself.

Cross-sectional evaluation​

Apache APISIX has evolved to the point where there are three ways to write plugins:

1. Native Lua way, running inside of APISIX.
2. External plugin runner for multiple languages, where the plugin logic runs outside of APISIX.
3. Compile multiple languages into WASM, still running inside of APISIX.

Click to Preview

Step 1: Configure Routing​

The Lua way of fault-injection, naturally, uses the built-in fault-injection plugin.The Runner way fault-injection implementation can be found here.

Let's configure different routes for each of them.

---
uri: "/wasm"
plugins:
  wasm_fault_injection:
    conf: '{"body":"hello world", "http_status":200}'
upstream:
  type: roundrobin
  nodes:
    127.0.0.1:1980: 1
---
plugins:
  ext-plugin-pre-req:
    conf:
    - name: fault-injection
      value: '{"body":"hello world", "http_status":200}'
upstream:
  nodes:
    127.0.0.1:1980: 1
  type: roundrobin
uri: /ext-plugin
---
plugins:
  fault-injection:
    abort:
      body: hello world
      http_status: 200
upstream:
  nodes:
    127.0.0.1:1980: 1
  type: roundrobin
uri: /fault-injection

Step 2: The actual pressure test​

Next, try to use wrk pressure. The specific data comparison is shown in the following chart:

As you can see from the results, the performance of the WASM version is somewhere between that of the external plugin and the native Lua.

The reason why the WASM version performs so much better than the external plugin is that the fault-injection functionality is simple, so the performance loss caused by the external plugin RPC is too obvious. Considering that we haven't made any optimizations to the WASM implementation, we are satisfied with this situation.

Another benefit of WASM is that we have multilingual support at once (thanks to the Proxy WASM SDK). Details can be found in the following documentation：

• Fault-injection(Rust)
• Fault-injection(AssemblyScript)

Advantages and disadvantages​

With all these benefits of WASM, aren't you a bit excited? But it is not a perfect solution at the moment, WASM/Proxy WASM still has some immature areas. For example：

• Programming language support to be improved: native Go's WASM support is mainly based on the browser environment, so we had to use tinygo to implement it. However, tinygo is a young project and still has a lot of limitations.
• Proxy WASM ecosystem needs to be mature: AssemblyScript version of the fault injection implementation, and there is no JSON decode part. This is because the AssemblyScript SDK is based on AssemblyScript version 0.14.x, and several open-source AssemblyScript JSON libraries are implemented for higher versions of AssemblyScript, and can not be used on the older AssemblyScript 0.14.
• WASM doesn't have a built-in concurrent process: WASM does not have a built-in concurrent process, so it cannot be scheduled by the host's scheduling system.

While there are several shortcomings listed above, we believe the future of this technology stack is bright:

1. Open source projects, including Apache APISIX and Envoy, have a strong interest in WASM, and there are many startups and large enterprises adding to the WASM ecosystem, which means that difficulties such as the stagnant AssemblyScript SDK will only be temporary. In the long run, the Proxy WASM ecosystem will flourish.
2. WASM has a bright future as the darling of serverless and edge computing. The implementation and optimization of many practical scenarios will solve the technical deficiencies more quickly.

Come and join the project​

Apache APISIX is a project that keeps up with the technology trend, Apache APISIX support for WASM is a long-term process.

"A journey of a thousand miles begins with a single step",Apache APISIX has launched the wasm-nginx-module open source project to support WASM.

Interested readers can follow the progress of the project, we look forward to your joining us and creating the world's top projects together.