Hi,
Following this topic, I made the following stap file, but for some reason the
lj_trace_exit it's only called 10 times, after that, is never called again,
example:
Sample stap: https://pastebin.com/reyr2gP6
Stap output:
```
[root@centos-s-1vcpu-2gb-fra1-01 stapxx]# stap lj.stp
Start tracing process 0 (/usr/local/openresty-debug/nginx/sbin/nginx)...
Please wait for 10 seconds...
----INIT REQUEST----:: 0
TRACE_START:: 35004832
LJ_TRACE_EXIT:: 35004832 PID: 6126
----END REQUEST----:: 0
----INIT REQUEST----:: 1
LJ_TRACE_EXIT:: 35004832 PID: 6126
----END REQUEST----:: 1
----INIT REQUEST----:: 2
LJ_TRACE_EXIT:: 35004832 PID: 6126
----END REQUEST----:: 2
----INIT REQUEST----:: 3
LJ_TRACE_EXIT:: 35004832 PID: 6126
----END REQUEST----:: 3
----INIT REQUEST----:: 4
LJ_TRACE_EXIT:: 35004832 PID: 6126
----END REQUEST----:: 4
----INIT REQUEST----:: 5
LJ_TRACE_EXIT:: 35004832 PID: 6126
----END REQUEST----:: 5
----INIT REQUEST----:: 6
LJ_TRACE_EXIT:: 35004832 PID: 6126
----END REQUEST----:: 6
----INIT REQUEST----:: 7
LJ_TRACE_EXIT:: 35004832 PID: 6126
----END REQUEST----:: 7
----INIT REQUEST----:: 8
LJ_TRACE_EXIT:: 35004832 PID: 6126
----END REQUEST----:: 8
----INIT REQUEST----:: 9
LJ_TRACE_EXIT:: 35004832 PID: 6126
TRACE_START:: 35004832
----END REQUEST----:: 9
----INIT REQUEST----:: 10
TRACE_START:: 35004832
----END REQUEST----:: 10
----INIT REQUEST----:: 11
----END REQUEST----:: 11
```
This does not happen in the 1.13 version series, only happens with 1.15
version, is there any issue or a new feature that am I missing?
The ngix config is the following:
```
worker_processes 1;
daemon off;
pid /tmp/nginx.pid;
error_log logs/error.log debug;
events {
worker_connections 1024;
}
http {
init_by_lua_block {
require "resty.core"
-- require("jit.v").start("dump.txt")
require('jit.p').start('vl')
require('jit.dump').start('bsx', 'jit.log')
}
server {
listen 8080;
access_log off;
location / {
content_by_lua_block {
function main()
local j = 0
for i=0,10000 do
j = i
end
end
main()
ngx.say("Hello world")
}
}
}
}
```
Many thanks
On Tuesday, July 16, 2019 at 3:10:18 PM UTC+2, Eloy Coto Pereiro wrote:
Hi,
I'm using stapxx project to profile my application, but looks like that
Systemtap is not been able to probe in specific functions and systemstap scripts
are not working correctly:
For example, ngx-rps always return 0, but I'm running wrk in background and a
few hundred of request happens in that time. Also tried with
`ngx-lj-trace-exits` and no samples were found.
I'm using the following resty config:
```
worker_processes 1;
daemon off;
pid /tmp/nginx.pid;
error_log logs/error.log;
events {
worker_connections 1024;
}
http {
server {
listen 8080;
access_log off;
location / {
content_by_lua_block {
function main()
local i = 0
while true do
i = i+1
end
end
main()
ngx.say("Hello world")
}
}
}
}
```
The output of `ngx-lj-trace-exits` is the following:
```
[root@centos-s-1vcpu-2gb-fra1-
01 stapxx]# ./samples/ngx-lj-trace-exits.sxx -x $(cat /tmp/nginx.pid) --arg time=30
Found exact match for libluajit: /usr/local/openresty-debug/luajit/lib/libluajit-5.1.so.2.1.0
Start tracing process 9286 (/usr/local/openresty-debug/nginx/sbin/nginx)...
Please wait for 30 seconds...
No samples found so far.
[root@centos-s-1vcpu-2gb-fra1-01 stapxx]#
```
I'm running with openresty-debug script, that has dtrace handlers:
```
^C[root@centos-s-1vcpu-2gb-fra1-01 stapxx]# openresty-debug -V
nginx version: openresty/1.15.8.1
built by gcc 7.3.1 20180303 (Red Hat 7.3.1-5) (GCC)
built with OpenSSL 1.1.0h 27 Mar 2018 (running with OpenSSL 1.1.0k 28 May 2019)
TLS SNI support enabled
configure arguments: --prefix=/usr/local/openresty-debug/nginx --with-debug --with-cc-opt='-DNGX_LUA_USE_ASSERT -DNGX_LUA_ABORT_AT_PANIC -O2 -I/usr/local/openresty/zlib/include -I/usr/local/openresty/pcre/include -I/usr/local/openresty-debug/openssl/include -O0' --add-module=../ngx_devel_kit-0.3.1rc1 --add-module=../echo-nginx-module-0.61 --add-module=../xss-nginx-module-0.06 --add-module=../ngx_coolkit-0.2 --add-module=../set-misc-nginx-module-0.32 --add-module=../form-input-nginx-module-0.12 --add-module=../encrypted-session-nginx-module-0.08 --add-module=../srcache-nginx-module-0.31 --add-module=../ngx_lua-0.10.15 --add-module=../ngx_lua_upstream-0.07 --add-module=../headers-more-nginx-module-0.33 --add-module=../array-var-nginx-module-0.05 --add-module=../memc-nginx-module-0.19 --add-module=../redis2-nginx-module-0.15 --add-module=../redis-nginx-module-0.3.7 --add-module=../ngx_stream_lua-0.0.7 --with-ld-opt='-Wl,-rpath,/usr/local/openresty-debug/luajit/lib -L/usr/local/openresty/zlib/lib -L/usr/local/openresty/pcre/lib -L/usr/local/openresty-debug/openssl/lib -Wl,-rpath,/usr/local/openresty/zlib/lib:/usr/local/openresty/pcre/lib:/usr/local/openresty-debug/openssl/lib' --with-pcre-jit --with-stream --with-stream_ssl_module --with-stream_ssl_preread_module --with-http_v2_module --without-mail_pop3_module --without-mail_imap_module --without-mail_smtp_module --with-http_stub_status_module --with-http_realip_module --with-http_addition_module --with-http_auth_request_module --with-http_secure_link_module --with-http_random_index_module --with-http_gzip_static_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-threads --with-poll_module --with-dtrace-probes --with-stream --with-stream_ssl_preread_module --with-http_ssl_module
[root@centos-s-1vcpu-2gb-fra1-01 stapxx]#
```
The statp command that it's running is the following:
```
root 9291 34.6 9.2 376268 175020 pts/2 S+ 12:54 0:02 stap -DMAXACTION=100000 -DMAXMAPENTRIES=5000 -DMAXBACKTRACE=200 -x 9286 -d /usr/local/openresty-debug/nginx/sbin/nginx -d /usr/lib64/ld-2.17.so -d /usr/lib64/libc-2.17.so -d /usr/lib64/libcrypt-2.17.so -d /usr/lib64/libdl-2.17.so -d /usr/lib64/libfreebl3.so -d /usr/lib64/libgcc_s-4.8.5-20150702.so.1 -d /usr/lib64/libm-2.17.so -d /usr/lib64/libnss_files-2.17.so -d /usr/lib64/libpthread-2.17.so -d /usr/local/openresty-debug/luajit/lib/libluajit-5.1.so.2.1.0 -d /usr/local/openresty-debug/openssl/lib/libcrypto.so.1.1 -d /usr/local/openresty-debug/openssl/lib/libssl.so.1.1 -d /usr/local/openresty/pcre/lib/libpcre.so.1.2.10 -d /usr/local/openresty/zlib/lib/libz.so.1.2.11 -I stapxx-xzDGNxdb -
```
And stap version is the following:
```
Try '--help' for more information.
[root@centos-s-1vcpu-2gb-fra1-01 stapxx]# stap -V
Systemtap translator/driver (version 3.3/0.172, rpm 3.3-3.el7)
Copyright (C) 2005-2018 Red Hat, Inc. and others
This is free software; see the source for copying conditions.
tested kernel versions: 2.6.18 ... 4.18-rc0
enabled features: AVAHI BOOST_STRING_REF DYNINST BPF JAVA PYTHON2 LIBRPM LIBSQLITE3 LIBVIRT LIBXML2 NLS NSS READLINE
[root@centos-s-1vcpu-2gb-fra1-01 stapxx]#
```
The way that I installed openresty is the following:
```
yum install yum-utils
yum-config-manager --add-repo https://openresty.org/package/centos/openresty.repo
yum install openresty openresty-debug-debuginfo openresty-debug openresty-pcre-debuginfo openresty-pcre-devel openresty-zlib-debuginfo
yum install -y kernel-devel kernel-debuginfo kernel-debuginfo-common
debuginfo-install kernel-3.10.0-957.12.2.el7.x86_64
yum install -y systemtap systemtap-runtime
git clone https://github.com/openresty/stapxx.git
cd stapxx
export PATH=$PATH:$(pwd)
```
Even if I try to do something really simple, print some information on custom
functions I can't see that:
```
[root@centos-s-1vcpu-2gb-fra1-01 stapxx]# stap++ -e 'probe @pfunc(ngx_http_lua_run_thread), @pfunc(ngx_http_lua_log_handler), @pfunc(ngx_http_lua_set_by_chunk), @pfunc(ngx_http_lua_header_filter_by_chunk), @pfunc(ngx_http_lua_body_filter_by_chunk) { println("new request")}' -x $(cat /tmp/nginx.pid )
```
Or when the request if free: (without the while true in the nginx.conf)
```
[root@centos-s-1vcpu-2gb-fra1-01 stapxx]# stap++ -e 'probe @pfunc(ngx_http_free_request) {println("FREE request") }' -x $(cat /tmp/nginx.pid)
```
I'm using latest Centos 7 version in 3.10 Kernel. Any idea why tracing is not
available? Can anyone help me?
