file-logger

Description

The file-logger Plugin is used to push log streams to a specific location.

tip

• file-logger plugin can count request and response data for individual routes locally, which is useful for debugging.
• file-logger plugin can get APISIX variables and NGINX variables, while access.log can only use NGINX variables.
• file-logger plugin support hot-loaded so that we can change its configuration at any time with immediate effect.
• file-logger plugin saves every data in JSON format.
• The user can modify the functions executed by the file-logger during the log phase to collect the information they want.

Attributes

Name	Type	Required	Description
path	string	True	Log file path.
log_format	object	False	Log format declared as key value pairs in JSON format. Values only support strings. APISIX or Nginx variables can be used by prefixing the string with $.
include_req_body	boolean	False	When set to true includes the request body in the log. If the request body is too big to be kept in the memory, it can't be logged due to Nginx's limitations.
include_req_body_expr	array	False	Filter for when the include_req_body attribute is set to true. Request body is only logged when the expression set here evaluates to true. See lua-resty-expr for more.
include_resp_body	boolean	False	When set to true includes the response body in the log file.
include_resp_body_expr	array	False	When the include_resp_body attribute is set to true, use this to filter based on lua-resty-expr. If present, only logs the response into file if the expression evaluates to true.
match	array[]	False	Logs will be recorded when the rule matching is successful if the option is set. See lua-resty-expr for a list of available expressions.

Example of default log format

{
  "service_id": "",
  "apisix_latency": 100.99999809265,
  "start_time": 1703907485819,
  "latency": 101.99999809265,
  "upstream_latency": 1,
  "client_ip": "127.0.0.1",
  "route_id": "1",
  "server": {
      "version": "3.7.0",
      "hostname": "localhost"
  },
  "request": {
      "headers": {
          "host": "127.0.0.1:1984",
          "content-type": "application/x-www-form-urlencoded",
          "user-agent": "lua-resty-http/0.16.1 (Lua) ngx_lua/10025",
          "content-length": "12"
      },
      "method": "POST",
      "size": 194,
      "url": "http://127.0.0.1:1984/hello?log_body=no",
      "uri": "/hello?log_body=no",
      "querystring": {
          "log_body": "no"
      }
  },
  "response": {
      "headers": {
          "content-type": "text/plain",
          "connection": "close",
          "content-length": "12",
          "server": "APISIX/3.7.0"
      },
      "status": 200,
      "size": 123
  },
  "upstream": "127.0.0.1:1982"
}

Metadata

You can also set the format of the logs by configuring the Plugin metadata. The following configurations are available:

Name	Type	Required	Default	Description
log_format	object	False		Log format declared as key value pairs in JSON format. Values only support strings. APISIX or Nginx variables can be used by prefixing the string with $.

The example below shows how you can configure through the Admin API:

note

You can fetch the admin_key from config.yaml and save to an environment variable with the following command:

admin_key=$(yq '.deployment.admin.admin_key[0].key' conf/config.yaml | sed 's/"//g')

curl http://127.0.0.1:9180/apisix/admin/plugin_metadata/file-logger -H "X-API-KEY: $admin_key" -X PUT -d '
{
  "log_format": {
    "host": "$host",
    "@timestamp": "$time_iso8601",
    "client_ip": "$remote_addr"
  }
}'

With this configuration, your logs would be formatted as shown below:

{"host":"localhost","@timestamp":"2020-09-23T19:05:05-04:00","client_ip":"127.0.0.1","route_id":"1"}
{"host":"localhost","@timestamp":"2020-09-23T19:05:05-04:00","client_ip":"127.0.0.1","route_id":"1"}

Enable Plugin

The example below shows how you can enable the Plugin on a specific Route:

curl http://127.0.0.1:9180/apisix/admin/routes/1 -H "X-API-KEY: $admin_key" -X PUT -d '
{
  "plugins": {
    "file-logger": {
      "path": "logs/file.log"
    }
  },
  "upstream": {
    "type": "roundrobin",
    "nodes": {
      "127.0.0.1:9001": 1
    }
  },
  "uri": "/hello"
}'

Example usage

Now, if you make a request, it will be logged in the path you specified:

curl -i http://127.0.0.1:9080/hello

You will be able to find the file.log file in the configured logs directory.

Filter logs

curl http://127.0.0.1:9180/apisix/admin/routes/1 \
-H "X-API-KEY: $admin_key" -X PUT -d '
{
  "plugins": {
    "file-logger": {
      "path": "logs/file.log",
      "match": {
        {
          { "arg_name","==","jack" }
        }
      }
    }
  },
  "upstream": {
    "type": "roundrobin",
    "nodes": {
      "127.0.0.1:9001": 1
    }
  },
  "uri": "/hello"
}'

Test:

curl -i http://127.0.0.1:9080/hello?name=jack

Log records can be seen in logs/file.log.

curl -i http://127.0.0.1:9080/hello?name=rose

Log records cannot be seen in logs/file.log.

Delete Plugin

To remove the file-logger Plugin, you can delete the corresponding JSON configuration from the Plugin configuration. APISIX will automatically reload and you do not have to restart for this to take effect.

curl http://127.0.0.1:9180/apisix/admin/routes/1  -H "X-API-KEY: $admin_key" -X PUT -d '
{
  "methods": ["GET"],
  "uri": "/hello",
  "plugins": {},
  "upstream": {
    "type": "roundrobin",
    "nodes": {
      "127.0.0.1:9001": 1
    }
  }
}'