fault-injection

Description

The fault-injection Plugin can be used to test the resiliency of your application. This Plugin will be executed before the other configured Plugins.

The abort attribute will directly return the specified HTTP code to the client and skips executing the subsequent Plugins.

The delay attribute delays a request and executes of the subsequent Plugins.

Attributes

Name	Type	Requirement	Default	Valid	Description
abort.http_status	integer	required		[200, ...]	HTTP status code of the response to return to the client.
abort.body	string	optional			Body of the response returned to the client. Nginx variables like client addr: $remote_addr\n can be used in the body.
abort.percentage	integer	optional		[0, 100]	Percentage of requests to be aborted.
abort.vars	array[]	optional			Rules which are matched before executing fault injection. See lua-resty-expr for a list of available expressions.
delay.duration	number	required			Duration of the delay. Can be decimal.
delay.percentage	integer	optional		[0, 100]	Percentage of requests to be delayed.
delay.vars	array[]	optional			Rules which are matched before executing fault injection. See lua-resty-expr for a list of available expressions.

IMPORTANT

To use the fault-injection Plugin one of abort or delay must be specified.

tip

vars can have expressions from lua-resty-expr and can flexibly implement AND/OR relationship between rules. For example:

[
    [
        [ "arg_name","==","jack" ],
        [ "arg_age","==",18 ]
    ],
    [
        [ "arg_name2","==","allen" ]
    ]
]

This means that the relationship between the first two expressions is AND, and the relationship between them and the third expression is OR.

Enabling the Plugin

You can enable the fault-injection Plugin on a specific Route as shown below:

curl http://127.0.0.1:9180/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
    "plugins": {
       "fault-injection": {
           "abort": {
              "http_status": 200,
              "body": "Fault Injection!"
           }
       }
    },
    "upstream": {
       "nodes": {
           "127.0.0.1:1980": 1
       },
       "type": "roundrobin"
    },
    "uri": "/hello"
}'

Similarly, to enable a delay fault:

curl http://127.0.0.1:9180/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
    "plugins": {
       "fault-injection": {
           "delay": {
              "duration": 3
           }
       }
    },
    "upstream": {
       "nodes": {
           "127.0.0.1:1980": 1
       },
       "type": "roundrobin"
    },
    "uri": "/hello"
}'

You can also enable the Plugin with both abort and delay which can have vars for matching:

curl http://127.0.0.1:9180/apisix/admin/routes/1  -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
    "plugins": {
        "fault-injection": {
            "abort": {
                "http_status": 403,
                "body": "Fault Injection!\n",
                "vars": [
                    [
                        [ "arg_name","==","jack" ]
                    ]
                ]
            },
            "delay": {
                "duration": 2,
                "vars": [
                    [
                        [ "http_age","==","18" ]
                    ]
                ]
            }
        }
    },
    "upstream": {
        "nodes": {
            "127.0.0.1:1980": 1
        },
        "type": "roundrobin"
    },
    "uri": "/hello"
}'

Example usage

Once you have enabled the Plugin as shown above, you can make a request to the configured Route:

curl http://127.0.0.1:9080/hello -i

HTTP/1.1 200 OK
Date: Mon, 13 Jan 2020 13:50:04 GMT
Content-Type: text/plain
Transfer-Encoding: chunked
Connection: keep-alive
Server: APISIX web server

Fault Injection!

And if we configure the delay fault:

time curl http://127.0.0.1:9080/hello -i

HTTP/1.1 200 OK
Content-Type: application/octet-stream
Content-Length: 6
Connection: keep-alive
Server: APISIX web server
Date: Tue, 14 Jan 2020 14:30:54 GMT
Last-Modified: Sat, 11 Jan 2020 12:46:21 GMT

hello

real    0m3.034s
user    0m0.007s
sys     0m0.010s

Fault injection with criteria matching

You can enable the fault-injection Plugin with the vars attribute to set specific rules:

curl http://127.0.0.1:9180/apisix/admin/routes/1  -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
    "plugins": {
        "fault-injection": {
            "abort": {
                    "http_status": 403,
                    "body": "Fault Injection!\n",
                    "vars": [
                        [
                            [ "arg_name","==","jack" ]
                        ]
                    ]
            }
        }
    },
    "upstream": {
        "nodes": {
            "127.0.0.1:1980": 1
        },
        "type": "roundrobin"
    },
    "uri": "/hello"
}'

Now, we can test the Route. First, we test with a different name argument:

curl "http://127.0.0.1:9080/hello?name=allen" -i

You will get the expected response without the fault injected:

HTTP/1.1 200 OK
Content-Type: application/octet-stream
Transfer-Encoding: chunked
Connection: keep-alive
Date: Wed, 20 Jan 2021 07:21:57 GMT
Server: APISIX/2.2

hello

Now if we set the name to match our configuration, the fault-injection Plugin is executed:

curl "http://127.0.0.1:9080/hello?name=jack" -i

HTTP/1.1 403 Forbidden
Date: Wed, 20 Jan 2021 07:23:37 GMT
Content-Type: text/plain; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Server: APISIX/2.2

Fault Injection!

Disable Plugin

To disable the fault-injection Plugin, you can delete the corresponding JSON configuration from the Plugin configuration. APISIX will automatically reload and you do not have to restart for this to take effect.

curl http://127.0.0.1:9180/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
    "uri": "/hello",
    "plugins": {},
    "upstream": {
        "type": "roundrobin",
        "nodes": {
            "127.0.0.1:1980": 1
        }
    }
}'