Nginx - Logging Request/Responses - "Character codes"

rezaee.hadi · 2019-05-17T19:23:53+00:00

Hi folks! I am happy to announce the new formal release, 1.15.8.1, of the OpenResty web platform based on NGINX and LuaJIT: https://openresty.org/en/an...

Nginx - Logging Request/Responses - "Character codes"

rezaee.hadi

Dears,

I've already logged the nginx request and responses (header / body) via Openresty/Lua (https://gist.github.com/morhekil/1ff0e902ed4de2adcb7a)

my log format is:

log_format log_req_resp '$remote_addr - $remote_user [$time_local] "$request" '

'$status $body_bytes_sent "$http_referer" '

'"$http_user_agent" "$http_x_forwarded_for"'

'\nReq_Header: "$req_header"'

'\nReq_Body: "$request_body"'

'\nRes_Header: "$resp_header"'

'\nRes_Body: "$resp_body"'

and the Request and Responses are getting logged without any problem.

My problem is that when you look at the content, you'll find some "character codes" instead of their "actual character" :

127.0.0.1 - - [17/May/2019:15:41:30 +0430] "POST /luatest HTTP/1.1" 200 538 "-" "Apache-HttpClient/4.1.1 (java 1.5)" "-"

Req_Header: "host=127.0.0.1:8080 content-type=application/soap+xml;charset=UTF-8;action="" connection=Keep-Alive content-length=540 user-agent=Apache-HttpClient/4.1.1 (java 1.5) accept-encoding=gzip,deflate "

Req_Body: "<soap:Envelope xmlns:soap=\x22http://www.w3.org/2003/05/soap-envelope\x22 xmlns:web=\x22http://webservice.smsrelay.armaghan.net/\x22>\x0A <soap:Header/>\x0A <soap:Body>\x0A <web:sendMessageOneToMany>\x0A <username>XXXXX</username>\x0A <password>XXXXXX</password>\x0A <originator>XXXXXX</originator>\x0A <destination>XXXXXX</destination>\x0A <destination>XXXXXXX</destination>\x0A <content><#> my code: 11111 </content>\x0A </web:sendMessageOneToMany>\x0A </soap:Body>\x0A</soap:Envelope>"

Res_Header: "content-type=application/octet-stream connection=keep-alive "

Res_Body: "<soap:Envelope xmlns:soap=\x22http://www.w3.org/2003/05/soap-envelope\x22 xmlns:web=\x22http://webservice.smsrelay.armaghan.net/\x22>\x0A <soap:Header/>\x0A <soap:Body>\x0A <web:sendMessageOneToMany>\x0A <username>XXXXX</username>\x0A <password>XXXXXX</password>\x0A <originator>XXXXXX</originator>\x0A <destination>XXXXXX</destination>\x0A <destination>XXXXXX</destination>\x0A <content>Your code: 11111</content>\x0A </web:sendMessageOneToMany>\x0A </soap:Body>\x0A</soap:Envelope>\x0A"

In this particular case:

\x0A = ' \n '

\x22 = ' " '

Is there anyway to fix this issue ? it'd help me to get more clear content.

Thanks.

nishant.kumar_3c1d

Hi Hadi,

Nginx by default uses character escaping in logs. If you are using the latest version of Openresty (1.15.8.1) then you can use in .

http://nginx.org/en/docs/http/ngx_http_log_module.html#log_format

On Friday, May 17, 2019 at 4:53:53 PM UTC+5:30, Hadi Rezaee wrote:

Dears,

I've already logged the nginx request and responses (header / body) via Openresty/Lua (https://gist.github.com/morhekil/1ff0e902ed4de2adcb7a)

my log format is:

log_format log_req_resp '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"'
'\nReq_Header: "$req_header"'
'\nReq_Body: "$request_body"'
'\nRes_Header: "$resp_header"'
'\nRes_Body: "$resp_body"'

and the Request and Responses are getting logged without any problem.
My problem is that when you look at the content, you'll find some "character codes" instead of their "actual character" :

127.0.0.1 - - [17/May/2019:15:41:30 +0430] "POST /luatest HTTP/1.1" 200 538 "-" "Apache-HttpClient/4.1.1 (java 1.5)" "-"
Req_Header: "host=127.0.0.1:8080 content-type=application/soap+xml;charset=UTF-8;action="" href="http://webservice.smsrelay.armaghan.net/MessageRelay/sendMessageOneToManyRequest%5Cx22" target="_blank" rel="nofollow" _onmousedown_="this.href="" true;" _onclick_="this.href='http://www.google.com/url?q\x3dhttp%3A%2F%2Fwebservice.smsrelay.armaghan.net%2FMessageRelay%2FsendMessageOneToManyRequest%255Cx22\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHbYseAstxi7mKOOmqaAqQlwmkaXg';return true;">http://webservice.smsrelay.armaghan.net/MessageRelay/sendMessageOneToManyRequest\x22 connection=Keep-Alive content-length=540 user-agent=Apache-HttpClient/4.1.1 (java 1.5) accept-encoding=gzip,deflate "
Req_Body: "<soap:Envelope xmlns:soap=\x22http://www.w3.org/2003/05/soap-envelope\x22 xmlns:web=\x22http://webservice.smsrelay.armaghan.net/\x22>\x0A <soap:Header/>\x0A <soap:Body>\x0A <web:sendMessageOneToMany>\x0A <username>XXXXX</username>\x0A <password>XXXXXX</password>\x0A <originator>XXXXXX</originator>\x0A <destination>XXXXXX</destination>\x0A <destination>XXXXXXX</destination>\x0A <content><#> my code: 11111 </content>\x0A </web:sendMessageOneToMany>\x0A </soap:Body>\x0A</soap:Envelope>"
Res_Header: "content-type=application/octet-stream connection=keep-alive "
Res_Body: "<soap:Envelope xmlns:soap=\x22http://www.w3.org/2003/05/soap-envelope\x22 xmlns:web=\x22http://webservice.smsrelay.armaghan.net/\x22>\x0A <soap:Header/>\x0A <soap:Body>\x0A <web:sendMessageOneToMany>\x0A <username>XXXXX</username>\x0A <password>XXXXXX</password>\x0A <originator>XXXXXX</originator>\x0A <destination>XXXXXX</destination>\x0A <destination>XXXXXX</destination>\x0A <content>Your code: 11111</content>\x0A </web:sendMessageOneToMany>\x0A </soap:Body>\x0A</soap:Envelope>\x0A"

In this particular case:
\x0A = ' \n '
\x22 = ' " '

Is there anyway to fix this issue ? it'd help me to get more clear content.

Thanks.