ua-restriction

Description

The ua-restriction Plugin allows you to restrict access to a Route or Service based on the User-Agent header with an allowlist and a denylist.

A common scenario is to set crawler rules. User-Agent is the identity of the client when sending requests to the server, and the user can whitelist or blacklist some crawler request headers in the ua-restriction Plugin.

Attributes

Name	Type	Required	Default	Valid values	Description
bypass_missing	boolean	False	false		When set to true, bypasses the check when the User-Agent header is missing.
allowlist	array[string]	False			List of allowed User-Agent headers.
denylist	array[string]	False			List of denied User-Agent headers.
message	string	False	"Not allowed"	[1, 1024]	Message with the reason for denial to be added to the response.

note

Both allowlist and denylist can be used on their own. If they are used together, the allowlist matches before the denylist.

Enabling the Plugin

You can enable the Plugin on a Route or a Service as shown below:

curl http://127.0.0.1:9080/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
    "uri": "/index.html",
    "upstream": {
        "type": "roundrobin",
        "nodes": {
            "127.0.0.1:1980": 1
        }
    },
    "plugins": {
        "ua-restriction": {
             "bypass_missing": true,
             "allowlist": [
                 "my-bot1",
                 "(Baiduspider)/(\\d+)\\.(\\d+)"
             ],
             "denylist": [
                 "my-bot2",
                 "(Twitterspider)/(\\d+)\\.(\\d+)"
             ]
        }
    }
}'

You can also configure the Plugin to respond with a custom rejection message:

"plugins": {
    "ua-restriction": {
        "denylist": [
            "my-bot2",
            "(Twitterspider)/(\\d+)\\.(\\d+)"
        ],
        "message": "Do you want to do something bad?"
    }
}

Example usage

After you have configured the Plugin as shown above, you can make a normal request which will get accepted:

curl http://127.0.0.1:9080/index.html -i

HTTP/1.1 200 OK
...

Now if the User-Agent header is in the denylist i.e the bot User-Agent:

curl http://127.0.0.1:9080/index.html --header 'User-Agent: Twitterspider/2.0'

HTTP/1.1 403 Forbidden
...
{"message":"Not allowed"}

Disable Plugin

To disable the ua-restriction Plugin, you can delete the corresponding JSON configuration from the Plugin configuration. APISIX will automatically reload and you do not have to restart for this to take effect.

curl http://127.0.0.1:9080/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
    "uri": "/index.html",
    "plugins": {},
    "upstream": {
        "type": "roundrobin",
        "nodes": {
            "127.0.0.1:1980": 1
        }
    }
}'