使用 ssl_session 的发现的 bug

nomyself.boy

问题描述：

1. 使用 https://github.com/openresty/openresty-packaging/tree/master/rpm/SPECS 打包的版本

2. 在多个 ssl server 片段中，后面的 server 不启用 ssl_session_store 则会出现 ssl 握手失败，nginx worker coredump

3. coredump 的位置 ngx_http_lua_ssl_sess_store_handler() 函数

以下配置可以稳定复现问题 nginx.conf

server {

listen 80;

listen 443 ssl;

server_name abc.haha.com;

ssl_certificate /etc/nginx/HTTPS/haha.com.crt;

ssl_certificate_key /etc/nginx/HTTPS/haha.com.key;

include /etc/nginx/conf.d/ssl.conf;

location / {

return 200 "abc\n";

}

server {

listen 80;

listen 443 ssl;

ssl_certificate /etc/nginx/HTTPS/haha.com.crt;

ssl_certificate_key /etc/nginx/HTTPS/haha.com.key;

server_name 123.haha.com;

#include /etc/nginx/conf.d/ssl.conf;

location / {

return 200 "123\n";

}

/etc/nginx/conf.d/ssl.conf 就是简单的使用 ssl_session

ssl_session_fetch_by_lua_block {

local ssl_sess = require "ngx.ssl.session"

local sess_id, err = ssl_sess.get_session_id()

ngx.log(ngx.ERR, "session ID: ", sess_id)

if not sess_id then

ngx.log(ngx.ERR, "failed to get session ID: ", err)

-- considered a cache miss, and just return...

return

end

}

ssl_session_store_by_lua_block {

local ssl_sess = require "ngx.ssl.session"

local sess_id, err = ssl_sess.get_session_id()

ngx.log(ngx.ERR, "session ID: ", sess_id)

if not sess_id then

ngx.log(ngx.ERR, "failed to get session ID: ", err)

-- just give up

return

end

}

doujiang24

Hello

我使用你提供的配置，并不能复现出来问题

curl "https://127.0.0.1/foo" -k

abc

curl "https://127.0.0.1/foo" -k -H "abc.haha.com"

abc

还请你提供详细的复现步骤，多谢配合 :)

2016-09-09 15:16 GMT+08:00 nomyself <nomys...@gmail.com>:

问题描述：

1. 使用 https://github.com/openresty/openresty-packaging/tree/master/rpm/SPECS 打包的版本

比如这个，最好提供版本号，因为 master 总是最新的，只能根据时间来推断版本号了

2. 在多个 ssl server 片段中，后面的 server 不启用 ssl_session_store 则会出现 ssl 握手失败，nginx worker coredump

这个对应的是发起什么样的请求呢？

3. coredump 的位置 ngx_http_lua_ssl_sess_store_handler() 函数

最好能提供完整的 gdb backtrace，这样这容易定位问题

以下配置可以稳定复现问题 nginx.conf

server {
listen 80;
listen 443 ssl;
server_name abc.haha.com;
ssl_certificate /etc/nginx/HTTPS/haha.com.crt;
ssl_certificate_key /etc/nginx/HTTPS/haha.com.key;
include /etc/nginx/conf.d/ssl.conf;

location / {
return 200 "abc\n";
}
}

server {
listen 80;
listen 443 ssl;
ssl_certificate /etc/nginx/HTTPS/haha.com.crt;
ssl_certificate_key /etc/nginx/HTTPS/haha.com.key;
server_name 123.haha.com;
#include /etc/nginx/conf.d/ssl.conf;

location / {
return 200 "123\n";
}
}

/etc/nginx/conf.d/ssl.conf 就是简单的使用 ssl_session

ssl_session_fetch_by_lua_block {
local ssl_sess = require "ngx.ssl.session"

local sess_id, err = ssl_sess.get_session_id()
ngx.log(ngx.ERR, "session ID: ", sess_id)
if not sess_id then
ngx.log(ngx.ERR, "failed to get session ID: ", err)
-- considered a cache miss, and just return...
return
end
}

ssl_session_store_by_lua_block {
local ssl_sess = require "ngx.ssl.session"

local sess_id, err = ssl_sess.get_session_id()
ngx.log(ngx.ERR, "session ID: ", sess_id)
if not sess_id then
ngx.log(ngx.ERR, "failed to get session ID: ", err)
-- just give up
return
end
}

--

nomyself.boy

软件环境：

1. openresty 1.11.2.1 spec 的 commitid c00bf81351149fdca0888fbd6c49f966e6f6cca0 （目前的master最新版本）

2. luajit 2.0 2.1 都测试过

我这边复现的方法：

1. 两个 vhost，前面开启 ssl_session_store_by_lua，后面的不开启，http 段的配置不能有 ssl_session_store_by_lua，不然也不能复现。最小复现配置如下

http {

server_tokens off;

include /etc/nginx/mime.types;

default_type text/html;

access_log off;

ssl_session_tickets off;

server {

listen 80;

listen 443 ssl;

server_name abc.haha.com;

ssl_certificate /etc/nginx/HTTPS/haha.com.crt;

ssl_certificate_key /etc/nginx/HTTPS/haha.com.key;

ssl_session_store_by_lua_block {

local ssl_sess = require "resty.ngx.ssl.session"

local sess_id, err = ssl_sess.get_session_id()

ngx.log(ngx.ERR, "session ID: ", sess_id)

if not sess_id then

ngx.log(ngx.ERR, "failed to get session ID: ", err)

-- just give up

return

end

}

location / {

return 200 "abc\n";

}

server {

listen 80;

listen 443 ssl;

ssl_certificate /etc/nginx/HTTPS/haha.com.crt;

ssl_certificate_key /etc/nginx/HTTPS/haha.com.key;

server_name 123.haha.com;

location / {

return 200 "123\n";

}

2. 请求 123.haha.com 则会 core，curl 123.haha.com。报错信息 curl: (56) SSLRead() return error -9806

3. gdb backtrace

Core was generated by `nginx: worker process '.

Program terminated with signal 11, Segmentation fault.

#0 0x0000000000000000 in ?? ()

Missing separate debuginfos, use: debuginfo-install libgcc-4.4.7-17.el6.x86_64 nss-softokn-freebl-3.14.3-17.el6.x86_64 sssd-client-1.13.3-22.el6_8.4.x86_64

(gdb) bt

#0 0x0000000000000000 in ?? ()

#1 0x000000000050fb5d in ngx_http_lua_ssl_sess_store_handler ()

#2 0x00007f160825d1e0 in ssl_update_cache () from /usr/local/openresty/openssl/lib/libssl.so.1.0.0

#3 0x00007f16082373ad in ssl3_accept () from /usr/local/openresty/openssl/lib/libssl.so.1.0.0

#4 0x000000000044ba36 in ngx_ssl_handshake ()

#5 0x000000000044bdd3 in ngx_ssl_handshake_handler ()

#6 0x0000000000446dd7 in ngx_epoll_process_events ()

#7 0x000000000043ccc7 in ngx_process_events_and_timers ()

#8 0x0000000000444785 in ngx_worker_process_cycle ()

#9 0x000000000044320c in ngx_spawn_process ()

#10 0x00000000004449e4 in ngx_start_worker_processes ()

#11 0x0000000000445751 in ngx_master_process_cycle ()

#12 0x000000000041dae7 in main () at src/core/nginx.c:367

On Sunday, September 11, 2016 at 12:11:42 PM UTC+8, doujiang wrote:

Hello

我使用你提供的配置，并不能复现出来问题

curl "https://127.0.0.1/foo" -k
abc

curl "https://127.0.0.1/foo" -k -H "abc.haha.com"
abc

还请你提供详细的复现步骤，多谢配合 :)

2016-09-09 15:16 GMT+08:00 nomyself <nomyse...@gmail.com>:
问题描述：

1. 使用 https://github.com/openresty/openresty-packaging/tree/master/rpm/SPECS 打包的版本

比如这个，最好提供版本号，因为 master 总是最新的，只能根据时间来推断版本号了

2. 在多个 ssl server 片段中，后面的 server 不启用 ssl_session_store 则会出现 ssl 握手失败，nginx worker coredump

这个对应的是发起什么样的请求呢？

3. coredump 的位置 ngx_http_lua_ssl_sess_store_handler() 函数

最好能提供完整的 gdb backtrace，这样这容易定位问题

以下配置可以稳定复现问题 nginx.conf

server {
listen 80;
listen 443 ssl;
server_name abc.haha.com;
ssl_certificate /etc/nginx/HTTPS/haha.com.crt;
ssl_certificate_key /etc/nginx/HTTPS/haha.com.key;
include /etc/nginx/conf.d/ssl.conf;

location / {
return 200 "abc\n";
}
}

server {
listen 80;
listen 443 ssl;
ssl_certificate /etc/nginx/HTTPS/haha.com.crt;
ssl_certificate_key /etc/nginx/HTTPS/haha.com.key;
server_name 123.haha.com;
#include /etc/nginx/conf.d/ssl.conf;

location / {
return 200 "123\n";
}
}

/etc/nginx/conf.d/ssl.conf 就是简单的使用 ssl_session

ssl_session_fetch_by_lua_block {
local ssl_sess = require "ngx.ssl.session"

local sess_id, err = ssl_sess.get_session_id()
ngx.log(ngx.ERR, "session ID: ", sess_id)
if not sess_id then
ngx.log(ngx.ERR, "failed to get session ID: ", err)
-- considered a cache miss, and just return...
return
end
}

ssl_session_store_by_lua_block {
local ssl_sess = require "ngx.ssl.session"

local sess_id, err = ssl_sess.get_session_id()
ngx.log(ngx.ERR, "session ID: ", sess_id)
if not sess_id then
ngx.log(ngx.ERR, "failed to get session ID: ", err)
-- just give up
return
end
}

--

On Sunday, September 11, 2016 at 12:11:42 PM UTC+8, doujiang wrote:

Hello

我使用你提供的配置，并不能复现出来问题

curl "https://127.0.0.1/foo" -k
abc

curl "https://127.0.0.1/foo" -k -H "abc.haha.com"
abc

还请你提供详细的复现步骤，多谢配合 :)

2016-09-09 15:16 GMT+08:00 nomyself <nomyse...@gmail.com>:
问题描述：

1. 使用 https://github.com/openresty/openresty-packaging/tree/master/rpm/SPECS 打包的版本

比如这个，最好提供版本号，因为 master 总是最新的，只能根据时间来推断版本号了

2. 在多个 ssl server 片段中，后面的 server 不启用 ssl_session_store 则会出现 ssl 握手失败，nginx worker coredump

这个对应的是发起什么样的请求呢？

3. coredump 的位置 ngx_http_lua_ssl_sess_store_handler() 函数

最好能提供完整的 gdb backtrace，这样这容易定位问题

以下配置可以稳定复现问题 nginx.conf

server {
listen 80;
listen 443 ssl;
server_name abc.haha.com;
ssl_certificate /etc/nginx/HTTPS/haha.com.crt;
ssl_certificate_key /etc/nginx/HTTPS/haha.com.key;
include /etc/nginx/conf.d/ssl.conf;

location / {
return 200 "abc\n";
}
}

server {
listen 80;
listen 443 ssl;
ssl_certificate /etc/nginx/HTTPS/haha.com.crt;
ssl_certificate_key /etc/nginx/HTTPS/haha.com.key;
server_name 123.haha.com;
#include /etc/nginx/conf.d/ssl.conf;

location / {
return 200 "123\n";
}
}

/etc/nginx/conf.d/ssl.conf 就是简单的使用 ssl_session

ssl_session_fetch_by_lua_block {
local ssl_sess = require "ngx.ssl.session"

local sess_id, err = ssl_sess.get_session_id()
ngx.log(ngx.ERR, "session ID: ", sess_id)
if not sess_id then
ngx.log(ngx.ERR, "failed to get session ID: ", err)
-- considered a cache miss, and just return...
return
end
}

ssl_session_store_by_lua_block {
local ssl_sess = require "ngx.ssl.session"

local sess_id, err = ssl_sess.get_session_id()
ngx.log(ngx.ERR, "session ID: ", sess_id)
if not sess_id then
ngx.log(ngx.ERR, "failed to get session ID: ", err)
-- just give up
return
end
}

--

agentzh

Hello!

On Sun, Sep 11, 2016 at 7:35 PM, nomyself wrote:
> Program terminated with signal 11, Segmentation fault.
> #0  0x0000000000000000 in ?? ()
> Missing separate debuginfos, use: debuginfo-install
> libgcc-4.4.7-17.el6.x86_64 nss-softokn-freebl-3.14.3-17.el6.x86_64
> sssd-client-1.13.3-22.el6_8.4.x86_64
> (gdb) bt
> #0  0x0000000000000000 in ?? ()
> #1  0x000000000050fb5d in ngx_http_lua_ssl_sess_store_handler ()
> #2  0x00007f160825d1e0 in ssl_update_cache () from
> /usr/local/openresty/openssl/lib/libssl.so.1.0.0
> #3  0x00007f16082373ad in ssl3_accept () from
> /usr/local/openresty/openssl/lib/libssl.so.1.0.0
> #4  0x000000000044ba36 in ngx_ssl_handshake ()
> #5  0x000000000044bdd3 in ngx_ssl_handshake_handler ()
> #6  0x0000000000446dd7 in ngx_epoll_process_events ()
> #7  0x000000000043ccc7 in ngx_process_events_and_timers ()
> #8  0x0000000000444785 in ngx_worker_process_cycle ()
> #9  0x000000000044320c in ngx_spawn_process ()
> #10 0x00000000004449e4 in ngx_start_worker_processes ()
> #11 0x0000000000445751 in ngx_master_process_cycle ()
> #12 0x000000000041dae7 in main () at src/core/nginx.c:367
>

你这里没有安装对应的 debuginfo 包，缺少调试符号，所以 gdb backtrace 没什么用处。建议重新安装 openresty,
openresty-openssl 和 libgcc 等相关包的 debuginfo 包，再用 gdb 获取。

Regards,
-agentzh

nomyself.boy

@agentzh sorry，前面环境各种测试，可能搞乱了

Core was generated by `nginx: worker process '.

Program terminated with signal 11, Segmentation fault.

#0 0x0000000000000000 in ?? ()

(gdb) bt

#0 0x0000000000000000 in ?? ()

#1 0x00000000004fd04f in ngx_http_lua_ssl_sess_store_handler (ssl_conn=<value optimized out>, sess=<value optimized out>)

at ../ngx_lua-0.10.6/src/ngx_http_lua_ssl_session_storeby.c:272

#2 0x00007f3273ce7232 in ssl_update_cache (s=0x7f3274e3d040, mode=2) at ssl_lib.c:2643

#3 0x00007f3273cc3dd9 in ssl3_accept (s=0x7f3274e3d040) at s3_srvr.c:850

#4 0x0000000000447f7f in ngx_ssl_handshake (c=0x7f3271e332e0) at src/event/ngx_event_openssl.c:1191

#5 0x0000000000448162 in ngx_ssl_handshake_handler (ev=<value optimized out>) at src/event/ngx_event_openssl.c:1367

#6 0x0000000000443945 in ngx_epoll_process_events (cycle=<value optimized out>, timer=<value optimized out>, flags=<value optimized out>)

at src/event/modules/ngx_epoll_module.c:900

#7 0x000000000043ac45 in ngx_process_events_and_timers (cycle=0x7f3274e03bc0) at src/event/ngx_event.c:242

#8 0x00000000004417b8 in ngx_worker_process_cycle (cycle=0x7f3274e03bc0, data="" optimized out>) at src/os/unix/ngx_process_cycle.c:753

#9 0x000000000043fdf7 in ngx_spawn_process (cycle=0x7f3274e03bc0, proc=0x441780 <ngx_worker_process_cycle>, data="" name=0x5119f5 "worker process", respawn=-3)

at src/os/unix/ngx_process.c:198

#10 0x0000000000440cbc in ngx_start_worker_processes (cycle=0x7f3274e03bc0, n=1, type=-3) at src/os/unix/ngx_process_cycle.c:358

#11 0x0000000000441e44 in ngx_master_process_cycle (cycle=0x7f3274e03bc0) at src/os/unix/ngx_process_cycle.c:130

#12 0x000000000041d5bf in main (argc=<value optimized out>, argv=<value optimized out>) at src/core/nginx.c:367

On Monday, September 12, 2016 at 11:59:39 AM UTC+8, agentzh wrote:

Hello!

On Sun, Sep 11, 2016 at 7:35 PM, nomyself wrote:
> Program terminated with signal 11, Segmentation fault.
> #0 0x0000000000000000 in ?? ()
> Missing separate debuginfos, use: debuginfo-install
> libgcc-4.4.7-17.el6.x86_64 nss-softokn-freebl-3.14.3-17.el6.x86_64
> sssd-client-1.13.3-22.el6_8.4.x86_64
> (gdb) bt
> #0 0x0000000000000000 in ?? ()
> #1 0x000000000050fb5d in ngx_http_lua_ssl_sess_store_handler ()
> #2 0x00007f160825d1e0 in ssl_update_cache () from
> /usr/local/openresty/openssl/lib/libssl.so.1.0.0
> #3 0x00007f16082373ad in ssl3_accept () from
> /usr/local/openresty/openssl/lib/libssl.so.1.0.0
> #4 0x000000000044ba36 in ngx_ssl_handshake ()
> #5 0x000000000044bdd3 in ngx_ssl_handshake_handler ()
> #6 0x0000000000446dd7 in ngx_epoll_process_events ()
> #7 0x000000000043ccc7 in ngx_process_events_and_timers ()
> #8 0x0000000000444785 in ngx_worker_process_cycle ()
> #9 0x000000000044320c in ngx_spawn_process ()
> #10 0x00000000004449e4 in ngx_start_worker_processes ()
> #11 0x0000000000445751 in ngx_master_process_cycle ()
> #12 0x000000000041dae7 in main () at src/core/nginx.c:367
>

你这里没有安装对应的 debuginfo 包，缺少调试符号，所以 gdb backtrace 没什么用处。建议重新安装 openresty,
openresty-openssl 和 libgcc 等相关包的 debuginfo 包，再用 gdb 获取。

Regards,
-agentzh

nomyself.boy

@ 这里你请求 curl -v https://123.haha.com 应该就可以复现问题了，请求前面的 abc.haha.com 是正常的。

On Sunday, September 11, 2016 at 12:11:42 PM UTC+8, doujiang wrote:

Hello

我使用你提供的配置，并不能复现出来问题

curl "https://127.0.0.1/foo" -k
abc

curl "https://127.0.0.1/foo" -k -H "abc.haha.com"
abc

还请你提供详细的复现步骤，多谢配合 :)

2016-09-09 15:16 GMT+08:00 nomyself <nomyse...@gmail.com>:
问题描述：

1. 使用 https://github.com/openresty/openresty-packaging/tree/master/rpm/SPECS 打包的版本

比如这个，最好提供版本号，因为 master 总是最新的，只能根据时间来推断版本号了

2. 在多个 ssl server 片段中，后面的 server 不启用 ssl_session_store 则会出现 ssl 握手失败，nginx worker coredump

这个对应的是发起什么样的请求呢？

3. coredump 的位置 ngx_http_lua_ssl_sess_store_handler() 函数

最好能提供完整的 gdb backtrace，这样这容易定位问题

以下配置可以稳定复现问题 nginx.conf

server {
listen 80;
listen 443 ssl;
server_name abc.haha.com;
ssl_certificate /etc/nginx/HTTPS/haha.com.crt;
ssl_certificate_key /etc/nginx/HTTPS/haha.com.key;
include /etc/nginx/conf.d/ssl.conf;

location / {
return 200 "abc\n";
}
}

server {
listen 80;
listen 443 ssl;
ssl_certificate /etc/nginx/HTTPS/haha.com.crt;
ssl_certificate_key /etc/nginx/HTTPS/haha.com.key;
server_name 123.haha.com;
#include /etc/nginx/conf.d/ssl.conf;

location / {
return 200 "123\n";
}
}

/etc/nginx/conf.d/ssl.conf 就是简单的使用 ssl_session

ssl_session_fetch_by_lua_block {
local ssl_sess = require "ngx.ssl.session"

local sess_id, err = ssl_sess.get_session_id()
ngx.log(ngx.ERR, "session ID: ", sess_id)
if not sess_id then
ngx.log(ngx.ERR, "failed to get session ID: ", err)
-- considered a cache miss, and just return...
return
end
}

ssl_session_store_by_lua_block {
local ssl_sess = require "ngx.ssl.session"

local sess_id, err = ssl_sess.get_session_id()
ngx.log(ngx.ERR, "session ID: ", sess_id)
if not sess_id then
ngx.log(ngx.ERR, "failed to get session ID: ", err)
-- just give up
return
end
}

--

doujiang24

Hello

在 2016年9月13日上午10:20，nomyself <nomys...@gmail.com>写道：

@doujiang 这里你请求 curl -v https://123.haha.com 应该就可以复现问题了，请求前面的 abc.haha.com 是正常的。

你这个 curl -v https://123.haha.com 应该跟我下面这个是一个意思吧

奇怪的是，我这里并不能够复现呢

curl "https://127.0.0.1" -H "Host: 123.haha.com" -k

123

On Sunday, September 11, 2016 at 12:11:42 PM UTC+8, doujiang wrote:
Hello

我使用你提供的配置，并不能复现出来问题

curl "https://127.0.0.1/foo" -k
abc

curl "https://127.0.0.1/foo" -k -H "abc.haha.com"
abc

还请你提供详细的复现步骤，多谢配合 :)

2016-09-09 15:16 GMT+08:00 nomyself <nomyse...@gmail.com>:
问题描述：

1. 使用 https://github.com/openresty/openresty-packaging/tree/master/rpm/SPECS 打包的版本

比如这个，最好提供版本号，因为 master 总是最新的，只能根据时间来推断版本号了

2. 在多个 ssl server 片段中，后面的 server 不启用 ssl_session_store 则会出现 ssl 握手失败，nginx worker coredump

这个对应的是发起什么样的请求呢？

3. coredump 的位置 ngx_http_lua_ssl_sess_store_handler() 函数

最好能提供完整的 gdb backtrace，这样这容易定位问题

以下配置可以稳定复现问题 nginx.conf

server {
listen 80;
listen 443 ssl;
server_name abc.haha.com;
ssl_certificate /etc/nginx/HTTPS/haha.com.crt;
ssl_certificate_key /etc/nginx/HTTPS/haha.com.key;
include /etc/nginx/conf.d/ssl.conf;

location / {
return 200 "abc\n";
}
}

server {
listen 80;
listen 443 ssl;
ssl_certificate /etc/nginx/HTTPS/haha.com.crt;
ssl_certificate_key /etc/nginx/HTTPS/haha.com.key;
server_name 123.haha.com;
#include /etc/nginx/conf.d/ssl.conf;

location / {
return 200 "123\n";
}
}

/etc/nginx/conf.d/ssl.conf 就是简单的使用 ssl_session

ssl_session_fetch_by_lua_block {
local ssl_sess = require "ngx.ssl.session"

local sess_id, err = ssl_sess.get_session_id()
ngx.log(ngx.ERR, "session ID: ", sess_id)
if not sess_id then
ngx.log(ngx.ERR, "failed to get session ID: ", err)
-- considered a cache miss, and just return...
return
end
}

ssl_session_store_by_lua_block {
local ssl_sess = require "ngx.ssl.session"

local sess_id, err = ssl_sess.get_session_id()
ngx.log(ngx.ERR, "session ID: ", sess_id)
if not sess_id then
ngx.log(ngx.ERR, "failed to get session ID: ", err)
-- just give up
return
end
}

--

--

doujiang24

Hello

2016-09-12 14:59 GMT+08:00 nomyself <nomys...@gmail.com>:

@agentzh sorry，前面环境各种测试，可能搞乱了

Core was generated by `nginx: worker process '.
Program terminated with signal 11, Segmentation fault.
#0 0x0000000000000000 in ?? ()
(gdb) bt
#0 0x0000000000000000 in ?? ()
#1 0x00000000004fd04f in ngx_http_lua_ssl_sess_store_handler (ssl_conn=<value optimized out>, sess=<value optimized out>)

at ../ngx_lua-0.10.6/src/ngx_http_lua_ssl_session_storeby.c:272

这里有些 <value optimized out>，最好是编译一个没有优化的版本：

./configure --with-cc-opt="-O0"

如果你方便的话，可以用 gdb 简单调试一下

比如在这里:

frame 1

p *lscf

这样拿到 lscf 的整个值

#2 0x00007f3273ce7232 in ssl_update_cache (s=0x7f3274e3d040, mode=2) at ssl_lib.c:2643
#3 0x00007f3273cc3dd9 in ssl3_accept (s=0x7f3274e3d040) at s3_srvr.c:850
#4 0x0000000000447f7f in ngx_ssl_handshake (c=0x7f3271e332e0) at src/event/ngx_event_openssl.c:1191
#5 0x0000000000448162 in ngx_ssl_handshake_handler (ev=<value optimized out>) at src/event/ngx_event_openssl.c:1367
#6 0x0000000000443945 in ngx_epoll_process_events (cycle=<value optimized out>, timer=<value optimized out>, flags=<value optimized out>)
at src/event/modules/ngx_epoll_module.c:900
#7 0x000000000043ac45 in ngx_process_events_and_timers (cycle=0x7f3274e03bc0) at src/event/ngx_event.c:242
#8 0x00000000004417b8 in ngx_worker_process_cycle (cycle=0x7f3274e03bc0, data="" optimized out>) at src/os/unix/ngx_process_cycle.c:753
#9 0x000000000043fdf7 in ngx_spawn_process (cycle=0x7f3274e03bc0, proc=0x441780 <ngx_worker_process_cycle>, data="" name=0x5119f5 "worker process", respawn=-3)
at src/os/unix/ngx_process.c:198
#10 0x0000000000440cbc in ngx_start_worker_processes (cycle=0x7f3274e03bc0, n=1, type=-3) at src/os/unix/ngx_process_cycle.c:358
#11 0x0000000000441e44 in ngx_master_process_cycle (cycle=0x7f3274e03bc0) at src/os/unix/ngx_process_cycle.c:130
#12 0x000000000041d5bf in main (argc=<value optimized out>, argv=<value optimized out>) at src/core/nginx.c:367

On Monday, September 12, 2016 at 11:59:39 AM UTC+8, agentzh wrote:
Hello!

On Sun, Sep 11, 2016 at 7:35 PM, nomyself wrote:
> Program terminated with signal 11, Segmentation fault.
> #0 0x0000000000000000 in ?? ()
> Missing separate debuginfos, use: debuginfo-install
> libgcc-4.4.7-17.el6.x86_64 nss-softokn-freebl-3.14.3-17.el6.x86_64
> sssd-client-1.13.3-22.el6_8.4.x86_64
> (gdb) bt
> #0 0x0000000000000000 in ?? ()
> #1 0x000000000050fb5d in ngx_http_lua_ssl_sess_store_handler ()
> #2 0x00007f160825d1e0 in ssl_update_cache () from
> /usr/local/openresty/openssl/lib/libssl.so.1.0.0
> #3 0x00007f16082373ad in ssl3_accept () from
> /usr/local/openresty/openssl/lib/libssl.so.1.0.0
> #4 0x000000000044ba36 in ngx_ssl_handshake ()
> #5 0x000000000044bdd3 in ngx_ssl_handshake_handler ()
> #6 0x0000000000446dd7 in ngx_epoll_process_events ()
> #7 0x000000000043ccc7 in ngx_process_events_and_timers ()
> #8 0x0000000000444785 in ngx_worker_process_cycle ()
> #9 0x000000000044320c in ngx_spawn_process ()
> #10 0x00000000004449e4 in ngx_start_worker_processes ()
> #11 0x0000000000445751 in ngx_master_process_cycle ()
> #12 0x000000000041dae7 in main () at src/core/nginx.c:367
>

你这里没有安装对应的 debuginfo 包，缺少调试符号，所以 gdb backtrace 没什么用处。建议重新安装 openresty,
openresty-openssl 和 libgcc 等相关包的 debuginfo 包，再用 gdb 获取。

Regards,
-agentzh

--

nomyself.boy

@doujiang

https 不能这么请求，可以绑定下 hosts。 curl -v https://127.0.0.1 -H"Host: 123.haha.com" 跟 curl -v https://123.haha.com 到 nginx 处理的逻辑是有差异的。

curl man 文档也写了 (HTTP) Extra header to include in the request when sending HTTP to a server.

gdb 调试信息

(gdb) frame 1
#1 0x00000000005976d4 in ngx_http_lua_ssl_sess_store_handler (ssl_conn=0x7f2f5f65e6e0, sess=0x7f2f5f369e00) at ../ngx_lua-0.10.6/src/ngx_http_lua_ssl_session_storeby.c:272
272 rc = lscf->srv.ssl_sess_store_handler(r, lscf, L);
(gdb) info locals
L = 0x41214378
rc = 14
c = 0x7f2f5e7c22e0
fc = 0x7f2f5e7c23d0
r = 0x7f2f5f63df30
hc = 0x7f2f5f65e590
cctx = 0x7f2f5f65e658
lscf = 0x7f2f65b4a730
clcf = 0x7f2f65b48818
__PRETTY_FUNCTION__ = "ngx_http_lua_ssl_sess_store_handler"
(gdb) p *lscf
$1 = {srv = {ssl_cert_handler = 0, ssl_cert_src = {len = 0, data = "" ssl_cert_src_key = 0x0, ssl_sess_store_handler = 0, ssl_sess_store_src = {len = 0, data = "">
ssl_sess_store_src_key = 0x0, ssl_sess_fetch_handler = 0, ssl_sess_fetch_src = {len = 0, data = "" ssl_sess_fetch_src_key = 0x0}, balancer = {src = "" = 0,
data = "" src_key = 0x0, handler = 0}}
(gdb) p *clcf
$2 = {name = {len = 0, data = "" regex = 0x0, noname = 0, lmt_excpt = 0, named = 0, exact_match = 0, noregex = 0, auto_redirect = 0, gzip_disable_msie6 = 0,
static_locations = 0x7f2f65b60aa0, regex_locations = 0x0, loc_conf = 0x0, limit_except = 0, limit_except_loc_conf = 0x0, handler = 0, alias = 0, root = {len = 31,
data = "" "/usr/local/openresty/nginx/html"}, post_action = {len = 0, data = "" root_lengths = 0x0, root_values = 0x0, types = 0x7f2f65b4d000,
types_hash = {buckets = 0x7f2f65b4d088, size = 1}, default_type = {len = 10, data = "" "text/plain"}, client_max_body_size = 1048576, directio = 9223372036854775807,
directio_alignment = 512, client_body_buffer_size = 8192, send_lowat = 0, postpone_output = 1460, limit_rate = 0, limit_rate_after = 0, sendfile_max_chunk = 0,
read_ahead = 0, client_body_timeout = 60000, send_timeout = 60000, keepalive_timeout = 75000, lingering_time = 30000, lingering_timeout = 5000, resolver_timeout = 30000,
resolver = 0x7f2f65b4d960, keepalive_header = 0, keepalive_requests = 100, keepalive_disable = 5, satisfy = 0, lingering_close = 1, if_modified_since = 1,
max_ranges = 2147483647, client_body_in_file_only = 0, client_body_in_single_buffer = 0, internal = 0, sendfile = 0, aio = 0, aio_write = 0, tcp_nopush = 0,
tcp_nodelay = 1, reset_timedout_connection = 0, server_name_in_redirect = 0, port_in_redirect = 1, msie_padding = 1, msie_refresh = 0, log_not_found = 1,
log_subrequest = 0, recursive_error_pages = 0, server_tokens = 1, chunked_transfer_encoding = 1, etag = 1, gzip_vary = 0, gzip_http_version = 1001, gzip_proxied = 3,
gzip_disable = 0x0, thread_pool = 0x0, thread_pool_value = 0x0, disable_symlinks = 0, disable_symlinks_from = 0x0, error_pages = 0x0, try_files = 0x0,
client_body_temp_path = 0x7f2f65b4cf48, open_file_cache = 0x0, open_file_cache_valid = 60, open_file_cache_min_uses = 1, open_file_cache_errors = 0,
open_file_cache_events = 0, error_log = 0x7f2f65b2bbd8, types_hash_max_size = 1024, types_hash_bucket_size = 64, locations = 0x7f2f65b31408}

On Friday, September 16, 2016 at 5:51:26 PM UTC+8, doujiang wrote:
Hello

在 2016年9月13日上午10:20，nomyself <nomyse...@gmail.com>写道：
@doujiang 这里你请求 curl -v https://123.haha.com 应该就可以复现问题了，请求前面的 abc.haha.com 是正常的。

你这个 curl -v https://123.haha.com 应该跟我下面这个是一个意思吧

奇怪的是，我这里并不能够复现呢
curl "https://127.0.0.1" -H "Host: 123.haha.com" -k
123

On Sunday, September 11, 2016 at 12:11:42 PM UTC+8, doujiang wrote:
Hello

我使用你提供的配置，并不能复现出来问题

curl "https://127.0.0.1/foo" -k
abc

curl "https://127.0.0.1/foo" -k -H "abc.haha.com"
abc

还请你提供详细的复现步骤，多谢配合 :)

2016-09-09 15:16 GMT+08:00 nomyself <nomyse...@gmail.com>:
问题描述：

1. 使用 https://github.com/openresty/openresty-packaging/tree/master/rpm/SPECS 打包的版本

比如这个，最好提供版本号，因为 master 总是最新的，只能根据时间来推断版本号了

2. 在多个 ssl server 片段中，后面的 server 不启用 ssl_session_store 则会出现 ssl 握手失败，nginx worker coredump

这个对应的是发起什么样的请求呢？

3. coredump 的位置 ngx_http_lua_ssl_sess_store_handler() 函数

最好能提供完整的 gdb backtrace，这样这容易定位问题

以下配置可以稳定复现问题 nginx.conf

server {
listen 80;
listen 443 ssl;
server_name abc.haha.com;
ssl_certificate /etc/nginx/HTTPS/haha.com.crt;
ssl_certificate_key /etc/nginx/HTTPS/haha.com.key;
include /etc/nginx/conf.d/ssl.conf;

location / {
return 200 "abc\n";
}
}

server {
listen 80;
listen 443 ssl;
ssl_certificate /etc/nginx/HTTPS/haha.com.crt;
ssl_certificate_key /etc/nginx/HTTPS/haha.com.key;
server_name 123.haha.com;
#include /etc/nginx/conf.d/ssl.conf;

location / {
return 200 "123\n";
}
}

/etc/nginx/conf.d/ssl.conf 就是简单的使用 ssl_session

ssl_session_fetch_by_lua_block {
local ssl_sess = require "ngx.ssl.session"

local sess_id, err = ssl_sess.get_session_id()
ngx.log(ngx.ERR, "session ID: ", sess_id)
if not sess_id then
ngx.log(ngx.ERR, "failed to get session ID: ", err)
-- considered a cache miss, and just return...
return
end
}

ssl_session_store_by_lua_block {
local ssl_sess = require "ngx.ssl.session"

local sess_id, err = ssl_sess.get_session_id()
ngx.log(ngx.ERR, "session ID: ", sess_id)
if not sess_id then
ngx.log(ngx.ERR, "failed to get session ID: ", err)
-- just give up
return
end
}

--

--

doujiang24

Hello

2016-09-18 10:38 GMT+08:00 nomyself <nomys...@gmail.com>:

@doujiang

https 不能这么请求，可以绑定下 hosts。 curl -v https://127.0.0.1 -H"Host: 123.haha.com" 跟 curl -v https://123.haha.com 到 nginx 处理的逻辑是有差异的。

是了，-H 只是在 http header 里发，绑 host 可以在 ssl 握手阶段把 host 传过去

多谢指正 :)

我绑 host 也确实可以重现

我搞了一个简单的修复 [1] 但是觉得这样只是治标不治本

没有 ssl_session_store_by_lua_* 的 server 就不应该进入这个回调才对

但是倒腾了好久也没搞明白

@春哥，求指点 :)

1. https://github.com/doujiang24/lua-nginx-module/commit/f6b7505c47efa0581ae1b21625ab8d614d04de2d

curl man 文档也写了 (HTTP) Extra header to include in the request when sending HTTP to a server.

gdb 调试信息
(gdb) frame 1
#1 0x00000000005976d4 in ngx_http_lua_ssl_sess_store_handler (ssl_conn=0x7f2f5f65e6e0, sess=0x7f2f5f369e00) at ../ngx_lua-0.10.6/src/ngx_http_lua_ssl_session_storeby.c:272
272 rc = lscf->srv.ssl_sess_store_handler(r, lscf, L);
(gdb) info locals
L = 0x41214378
rc = 14
c = 0x7f2f5e7c22e0
fc = 0x7f2f5e7c23d0
r = 0x7f2f5f63df30
hc = 0x7f2f5f65e590
cctx = 0x7f2f5f65e658
lscf = 0x7f2f65b4a730
clcf = 0x7f2f65b48818
__PRETTY_FUNCTION__ = "ngx_http_lua_ssl_sess_store_handler"
(gdb) p *lscf
$1 = {srv = {ssl_cert_handler = 0, ssl_cert_src = {len = 0, data = "" ssl_cert_src_key = 0x0, ssl_sess_store_handler = 0, ssl_sess_store_src = {len = 0, data = "">
ssl_sess_store_src_key = 0x0, ssl_sess_fetch_handler = 0, ssl_sess_fetch_src = {len = 0, data = "" ssl_sess_fetch_src_key = 0x0}, balancer = {src = "" = 0,
data = "" src_key = 0x0, handler = 0}}
(gdb) p *clcf
$2 = {name = {len = 0, data = "" regex = 0x0, noname = 0, lmt_excpt = 0, named = 0, exact_match = 0, noregex = 0, auto_redirect = 0, gzip_disable_msie6 = 0,
static_locations = 0x7f2f65b60aa0, regex_locations = 0x0, loc_conf = 0x0, limit_except = 0, limit_except_loc_conf = 0x0, handler = 0, alias = 0, root = {len = 31,
data = "" "/usr/local/openresty/nginx/html"}, post_action = {len = 0, data = "" root_lengths = 0x0, root_values = 0x0, types = 0x7f2f65b4d000,
types_hash = {buckets = 0x7f2f65b4d088, size = 1}, default_type = {len = 10, data = "" "text/plain"}, client_max_body_size = 1048576, directio = 9223372036854775807,
directio_alignment = 512, client_body_buffer_size = 8192, send_lowat = 0, postpone_output = 1460, limit_rate = 0, limit_rate_after = 0, sendfile_max_chunk = 0,
read_ahead = 0, client_body_timeout = 60000, send_timeout = 60000, keepalive_timeout = 75000, lingering_time = 30000, lingering_timeout = 5000, resolver_timeout = 30000,
resolver = 0x7f2f65b4d960, keepalive_header = 0, keepalive_requests = 100, keepalive_disable = 5, satisfy = 0, lingering_close = 1, if_modified_since = 1,
max_ranges = 2147483647, client_body_in_file_only = 0, client_body_in_single_buffer = 0, internal = 0, sendfile = 0, aio = 0, aio_write = 0, tcp_nopush = 0,
tcp_nodelay = 1, reset_timedout_connection = 0, server_name_in_redirect = 0, port_in_redirect = 1, msie_padding = 1, msie_refresh = 0, log_not_found = 1,
log_subrequest = 0, recursive_error_pages = 0, server_tokens = 1, chunked_transfer_encoding = 1, etag = 1, gzip_vary = 0, gzip_http_version = 1001, gzip_proxied = 3,
gzip_disable = 0x0, thread_pool = 0x0, thread_pool_value = 0x0, disable_symlinks = 0, disable_symlinks_from = 0x0, error_pages = 0x0, try_files = 0x0,
client_body_temp_path = 0x7f2f65b4cf48, open_file_cache = 0x0, open_file_cache_valid = 60, open_file_cache_min_uses = 1, open_file_cache_errors = 0,
open_file_cache_events = 0, error_log = 0x7f2f65b2bbd8, types_hash_max_size = 1024, types_hash_bucket_size = 64, locations = 0x7f2f65b31408}

On Friday, September 16, 2016 at 5:51:26 PM UTC+8, doujiang wrote:
Hello

在 2016年9月13日上午10:20，nomyself <nomyse...@gmail.com>写道：
@doujiang 这里你请求 curl -v https://123.haha.com 应该就可以复现问题了，请求前面的 abc.haha.com 是正常的。

你这个 curl -v https://123.haha.com 应该跟我下面这个是一个意思吧

奇怪的是，我这里并不能够复现呢
curl "https://127.0.0.1" -H "Host: 123.haha.com" -k
123

On Sunday, September 11, 2016 at 12:11:42 PM UTC+8, doujiang wrote:
Hello

我使用你提供的配置，并不能复现出来问题

curl "https://127.0.0.1/foo" -k
abc

curl "https://127.0.0.1/foo" -k -H "abc.haha.com"
abc

还请你提供详细的复现步骤，多谢配合 :)

2016-09-09 15:16 GMT+08:00 nomyself <nomyse...@gmail.com>:
问题描述：

1. 使用 https://github.com/openresty/openresty-packaging/tree/master/rpm/SPECS 打包的版本

比如这个，最好提供版本号，因为 master 总是最新的，只能根据时间来推断版本号了

2. 在多个 ssl server 片段中，后面的 server 不启用 ssl_session_store 则会出现 ssl 握手失败，nginx worker coredump

这个对应的是发起什么样的请求呢？

3. coredump 的位置 ngx_http_lua_ssl_sess_store_handler() 函数

最好能提供完整的 gdb backtrace，这样这容易定位问题

以下配置可以稳定复现问题 nginx.conf

server {
listen 80;
listen 443 ssl;
server_name abc.haha.com;
ssl_certificate /etc/nginx/HTTPS/haha.com.crt;
ssl_certificate_key /etc/nginx/HTTPS/haha.com.key;
include /etc/nginx/conf.d/ssl.conf;

location / {
return 200 "abc\n";
}
}

server {
listen 80;
listen 443 ssl;
ssl_certificate /etc/nginx/HTTPS/haha.com.crt;
ssl_certificate_key /etc/nginx/HTTPS/haha.com.key;
server_name 123.haha.com;
#include /etc/nginx/conf.d/ssl.conf;

location / {
return 200 "123\n";
}
}

/etc/nginx/conf.d/ssl.conf 就是简单的使用 ssl_session

ssl_session_fetch_by_lua_block {
local ssl_sess = require "ngx.ssl.session"

local sess_id, err = ssl_sess.get_session_id()
ngx.log(ngx.ERR, "session ID: ", sess_id)
if not sess_id then
ngx.log(ngx.ERR, "failed to get session ID: ", err)
-- considered a cache miss, and just return...
return
end
}

ssl_session_store_by_lua_block {
local ssl_sess = require "ngx.ssl.session"

local sess_id, err = ssl_sess.get_session_id()
ngx.log(ngx.ERR, "session ID: ", sess_id)
if not sess_id then
ngx.log(ngx.ERR, "failed to get session ID: ", err)
-- just give up
return
end
}

--

--

--

agentzh

Hello!

2016-09-27 4:14 GMT-07:00 DeJiang Zhu:
> 是了，-H 只是在 http header 里发，绑 host 可以在 ssl 握手阶段把 host 传过去
> 多谢指正 :)
>
> 我绑 host 也确实可以重现
>
> 我搞了一个简单的修复 [1] 但是觉得这样只是治标不治本
> 没有 ssl_session_store_by_lua_* 的 server 就不应该进入这个回调才对
> 但是倒腾了好久也没搞明白
> @春哥，求指点 :)

这个问题应该已经在最新的 OpenResty 1.11.2.2 RC1 中修复了。多谢德江贡献的补丁，还有林孜的指导！

Regards,
-agentzh