LuaJit/Nginx crash after websocket connected 2 minutes

laurencerowe · 2014-08-25T15:15:59+00:00

Ganglia, collectd can solve the low cost data aggregating part. In the past I've tailed Nginx logs and wrote cache hit rates etc to ganglia or stated and gra...

LuaJit/Nginx crash after websocket connected 2 minutes

laurencerowe

I've installed OpenResty on my Mac 10.6 using Homebrew and have got the WebSocket example from https://medium.com/technology-and-programming/websockets-with-openresty-1778601c9e05 working. However, after the WebSocket has been connected for a couple of minutes the Nginx worker process crashes with the crash report below. I wasn't able to find similar reports, any idea what might be wrong?

Laurence

Process: openresty [19956]

Path: /usr/local/Cellar/openresty/1.7.2.1/bin/openresty

Identifier: openresty

Version: ??? (???)

Code Type: X86-64 (Native)

Parent Process: openresty [19939]

Date/Time: 2014-08-25 00:08:30.692 -0700

OS Version: Mac OS X 10.6.8 (10K549)

Report Version: 6

Interval Since Last Report: 20583212 sec

Crashes Since Last Report: 121

Per-App Crashes Since Last Report: 6

Anonymous UUID: FBE67D28-FE1C-4C51-A954-48EDA3A9F006

Exception Type: EXC_CRASH (SIGABRT)

Exception Codes: 0x0000000000000000, 0x0000000000000000

Crashed Thread: 0 Dispatch queue: com.apple.main-thread

Application Specific Information:

abort() called

Thread 0 Crashed: Dispatch queue: com.apple.main-thread

0 libSystem.B.dylib 0x00007fff839bd0b6 __kill + 10

1 libSystem.B.dylib 0x00007fff83a5d9f6 abort + 83

2 libgcc_s.1.dylib 0x000000000007cd51 uw_init_context_1 + 385

3 libgcc_s.1.dylib 0x000000000007d142 _Unwind_RaiseException + 66

4 libluajit-5.1.2.dylib 0x00007fff04c506d5 lj_err_throw + 86

5 libluajit-5.1.2.dylib 0x00007fff04c82c0f lj_trace_free + 0

6 libluajit-5.1.2.dylib 0x00007fff04c76367 recff_nyi + 66

7 libluajit-5.1.2.dylib 0x00007fff04c75bce lj_ffrecord_func + 103

8 libluajit-5.1.2.dylib 0x00007fff04c70dd3 lj_record_ins + 7797

9 libluajit-5.1.2.dylib 0x00007fff04c82ea9 trace_state + 392

10 libluajit-5.1.2.dylib 0x00007fff04c4d4f0 lj_vm_cpcall + 74

11 libluajit-5.1.2.dylib 0x00007fff04c82798 lj_trace_ins + 75

12 libluajit-5.1.2.dylib 0x00007fff04c56059 lj_dispatch_call + 245

13 libluajit-5.1.2.dylib 0x00007fff04c4eb24 lj_vm_hotcall + 30

14 openresty 0x000000010008a4cb ngx_http_lua_run_thread + 213

15 openresty 0x0000000100092209 ngx_http_lua_socket_tcp_resume_helper + 173

16 openresty 0x0000000100030076 ngx_http_request_handler + 39

17 openresty 0x0000000100018aad ngx_event_expire_timers + 105

18 openresty 0x00000001000189ab ngx_process_events_and_timers + 252

19 openresty 0x000000010001e0ae ngx_worker_process_cycle + 197

20 openresty 0x000000010001cefb ngx_spawn_process + 915

21 openresty 0x000000010001ebe7 ngx_master_process_cycle + 1062

22 openresty 0x00000001000049b9 main + 2671

23 openresty 0x0000000100003418 start + 52

Thread 0 crashed with X86 Thread State (64-bit):

rax: 0x0000000000000000 rbx: 0x00007fff5fbfe680 rcx: 0x00007fff5fbfe658 rdx: 0x0000000000000000

rdi: 0x0000000000004df4 rsi: 0x0000000000000006 rbp: 0x00007fff5fbfe670 rsp: 0x00007fff5fbfe658

r8: 0x00000000001fc0a0 r9: 0x00000000001fc0a4 r10: 0x00007fff839b90fa r11: 0x0000000000000206

r12: 0x00007fff5fbfead0 r13: 0x00007fff5fbfec10 r14: 0x00007fff04c506d5 r15: 0x00007fff5fbfec90

rip: 0x00007fff839bd0b6 rfl: 0x0000000000000206 cr2: 0x00007fff70622808

Binary Images:

0x10000 - 0x70fef +libpcre.1.dylib 4.3.0 (compatibility 4.0.0) <7F989206-DBEA-1880-E0CD-B87AFF7F7246> /usr/local/lib/libpcre.1.dylib

0x75000 - 0x81fb1 +libgcc_s.1.dylib ??? (???) <BBB4EAFF-D7EC-4B7A-9AED-C5BB3003BDE0> /usr/local/lib/libgcc_s.1.dylib

0x100000000 - 0x1000deff7 +openresty ??? (???) <BE6EEFAC-1502-E557-DBC3-DAA8B523627D> /usr/local/bin/openresty

0x7fff04c4a000 - 0x7fff04caefff +libluajit-5.1.2.dylib 2.1.0 (compatibility 2.1.0) <9A4E53B2-C7DB-E243-4675-341FE81B046D> /usr/local/Cellar/openresty/1.7.2.1/luajit/lib/libluajit-5.1.2.dylib

0x7fff5fc00000 - 0x7fff5fc3bdef dyld 132.1 (???) <B536F2F1-9DF1-3B6C-1C2C-9075EA219A06> /usr/lib/dyld

0x7fff8003c000 - 0x7fff8004dff7 libz.1.dylib 1.2.3 (compatibility 1.0.0) <FB5EE53A-0534-0FFA-B2ED-486609433717> /usr/lib/libz.1.dylib

0x7fff80812000 - 0x7fff80851ff7 libssl.0.9.8.dylib 0.9.8 (compatibility 0.9.8) <B2E93DEC-2F33-366A-995C-ACA0523D508F> /usr/lib/libssl.0.9.8.dylib

0x7fff8396e000 - 0x7fff83b2ffef libSystem.B.dylib 125.2.11 (compatibility 1.0.0) <9AB4F1D1-89DC-0E8A-DC8E-A4FE4D69DB69> /usr/lib/libSystem.B.dylib

0x7fff84455000 - 0x7fff84456ff7 com.apple.TrustEvaluationAgent 1.1 (1) <306FD9EE-A301-41D5-EBDE-2FC52F28229C> /System/Library/PrivateFrameworks/TrustEvaluationAgent.framework/Versions/A/TrustEvaluationAgent

0x7fff876eb000 - 0x7fff876efff7 libmathCommon.A.dylib 315.0.0 (compatibility 1.0.0) <95718673-FEEE-B6ED-B127-BCDBDB60D4E5> /usr/lib/system/libmathCommon.A.dylib

0x7fff876f0000 - 0x7fff8780fff7 libcrypto.0.9.8.dylib 0.9.8 (compatibility 0.9.8) <0CE8D59B-D0C7-1DCE-3654-37F27F61BEFA> /usr/lib/libcrypto.0.9.8.dylib

0x7fffffe00000 - 0x7fffffe01fff libSystem.B.dylib ??? (???) <9AB4F1D1-89DC-0E8A-DC8E-A4FE4D69DB69> /usr/lib/libSystem.B.dylib

Model: MacBookPro6,2, BootROM MBP61.0057.B0C, 2 processors, Intel Core i7, 2.66 GHz, 4 GB, SMC 1.58f15

Graphics: NVIDIA GeForce GT 330M, NVIDIA GeForce GT 330M, PCIe, 512 MB

Graphics: Intel HD Graphics, Intel HD Graphics, Built-In, 288 MB

Memory Module: global_name

AirPort: spairport_wireless_card_type_airport_extreme (0x14E4, 0x93), Broadcom BCM43xx 1.0 (5.10.131.42.4)

Bluetooth: Version 2.4.5f3, 2 service, 12 devices, 2 incoming serial ports

Network Service: AirPort, AirPort, en1

Network Service: Parallels Shared Networking Adapter, Ethernet, en2

Network Service: Parallels Host-Only Networking Adapter, Ethernet, en3

Serial ATA Device: APPLE SSD TS128B, 113 GB

Serial ATA Device: MATSHITADVD-R UJ-898

USB Device: Hub, 0x0424 (SMSC), 0x2514, 0xfa100000 / 2

USB Device: Internal Memory Card Reader, 0x05ac (Apple Inc.), 0x8403, 0xfa130000 / 5

USB Device: Apple Internal Keyboard / Trackpad, 0x05ac (Apple Inc.), 0x0237, 0xfa120000 / 4

USB Device: BRCM2070 Hub, 0x0a5c (Broadcom Corp.), 0x4500, 0xfa110000 / 3

USB Device: Bluetooth USB Host Controller, 0x05ac (Apple Inc.), 0x8218, 0xfa113000 / 7

USB Device: Hub, 0x0424 (SMSC), 0x2514, 0xfd100000 / 2

USB Device: Built-in iSight, 0x05ac (Apple Inc.), 0x8507, 0xfd110000 / 4

USB Device: IR Receiver, 0x05ac (Apple Inc.), 0x8242, 0xfd120000 / 3

agentzh

Hello!

On Mon, Aug 25, 2014 at 12:15 AM, Laurence Rowe wrote:
> I've installed OpenResty on my Mac 10.6 using Homebrew and have got the
> WebSocket example from
> https://medium.com/technology-and-programming/websockets-with-openresty-1778601c9e05
> working. However, after the WebSocket has been connected for a couple of
> minutes the Nginx worker process crashes with the crash report below. I
> wasn't able to find similar reports, any idea what might be wrong?
>

Thank you for your report! But the information you've provided is not
enough for debugging the issue. Several suggestions:

1. Please build openresty for the following latest source code tarball
for OpenResty:

    http://openresty.org/download/ngx_openresty-1.7.4.1rc1.2.tar.gz

The build instructions can be found below:

    http://openresty.org/#Installation

There has been quite some bug fixes in everything (including nginx,
luajit, and ngx_lua) since the 1.7.2.1 release you're using. Maybe it
was a bug that has already been fixed.

Also, I'm not familiar with the homebrew openresty stuff. It's better
to use the official source tarball for trouble shooting.

2. When building your openresty from source, please pass the
--with-debug option to the ./configure command line. This way we can
enable internal assertions in both LuaJIT and ngx_lua.

3. Please use gdb to get proper backtrace at the crashing site from a
core dump file. See

    http://wiki.nginx.org/Debugging#Core_dump

Also, when you're at it, please also use the "lgc", "lbt", and
"lgcstat" commands from the nginx-gdb-utils project to examine the
LuaJIT VM state in the core dump file in gdb:

    https://github.com/openresty/nginx-gdb-utils

4. When the crash happens, please check if there's anything
interesting in your nginx's error.log file around that time.

5. Please try providing a minimal and self-contained example by which
we can reproduce the issue you're seeing relatively easily on our
side.

Thanks!
-agentzh