What is istio-proxy access log mean?

Question

I am trying to troubleshoot my service by looking at the istio-proxy access log (it logs every access). However, I can\'t find any documentation that explains the meaning of

Answer 1

Istio proxy access log's configuration is defined as part of envoy.http_connection_manager or envoy.tcp_proxy filters. To see it's configuration, run:

istioctl proxy-config listeners <your pod> -n <your namespace> -o json

Search for access_log of envoy.http_connection_manager for HTTP and access_log of envoy.tcp_proxy for TCP.

You will see something like this:

 "filters": [
                {
                    "name": "envoy.http_connection_manager",
                    "config": {
                        "access_log": [
                            {
                                "config": {
                                    "format": "[%START_TIME%] \"%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%\" %RESPONSE_CODE% %RESPONSE_FLAGS% %BYTES_RECEIVED% %BYTES_SENT% %DURATION% %RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)% \"%REQ(X-FORWARDED-FOR)%\" \"%REQ(USER-AGENT)%\" \"%REQ(X-REQUEST-ID)%\" \"%REQ(:AUTHORITY)%\" \"%UPSTREAM_HOST%\" %UPSTREAM_CLUSTER% %UPSTREAM_LOCAL_ADDRESS% %DOWNSTREAM_LOCAL_ADDRESS% %DOWNSTREAM_REMOTE_ADDRESS% %REQUESTED_SERVER_NAME%\n",
                                    "path": "/dev/stdout"

Check the log attributes definitions here

If access_log's format is not specified in the output above, the default format is used.

Answer 2

Istio/Envoy access logs comes with a default format. Here is the default format

[%START_TIME%] "%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%" %RESPONSE_CODE% %RESPONSE_FLAGS% %BYTES_RECEIVED% %BYTES_SENT% %DURATION% %RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)% "%REQ(X-FORWARDED-FOR)%" "%REQ(USER-AGENT)%" "%REQ(X-REQUEST-ID)%" "%REQ(:AUTHORITY)%" "%UPSTREAM_HOST%"\n

It matches with the sample log entry that you have given. You can find more details about the fields and generally about envoy's access logs here

Answer 3

Here is the format of log:

\[%{TIMESTAMP_ISO8601:timestamp}\] \"%{DATA:method} (?:%{URIPATH:uri_path}(?:%{URIPARAM:uri_param})?|%{DATA:}) %{DATA:protocol}\" %{NUMBER:status_code} %{DATA:response_flags} \"%{**DATA:mixer_status**}\" %{NUMBER:bytes_received} %{NUMBER:bytes_sent} %{NUMBER:duration} (?:%{NUMBER:upstream_service_time}|%{DATA:tcp_service_time}) \"%{DATA:forwarded_for}\" \"%{DATA:user_agent}\" \"%{DATA:request_id}\" \"%{DATA:authority}\" \"%{DATA:upstream_service}\" %{DATA:upstream_cluster} %{DATA:upstream_local} %{DATA:downstream_local} %{DATA:downstream_remote} %{**DATA:requested_server**}

Here is real log:

[2019-09-05T06:55:32.008Z] "GET /solutionprofile/api/v1/health HTTP/1.1" 200 - "-" 0 16 10 10 "-" "kube-probe/1.12" "dc9ac3b2-2ee4-4a4b-967e-8f0cc3953e80" "10.228.69.15:3000" "127.0.0.1:3000" inbound|80|http|hp-solutionprofile-service.hp.svc.cluster.local - 10.228.69.15:3000 10.228.69.1:59692 -