This is an old revision of the document!


How to submit a bug report

There are two cases you can log a bug report. One case when the server crashed and core files were generated at /tmp/lshttpd/bak_core/, you can submit a server crash report. The other case is server doesn't crash at all, but LSWS doesn't work as expected, especially it works with apache but doesn't work with LSWS in the control panel environment. In this situation, you can turn on debug logging when reproducing the issue, then send the detailed debug log for us to investigate.

We will discuss the proper bug report processes for both cases in the following.

Always upgrade to the latest version of Latest build before submitting a report

New version/new build of web server may hit the bug, crash and generate core files at /tmp/lshttpd/bak_core/core.*. Normally you will receive some email alert on this situation. You are welcome to sumbit such server crash report to us, either forwarding the alert email to us or forwarding core files to bug@litespeedtech.com.

Before you log such server crash bug report to us, it is always a good practice to check which release versio and which BUILD caused such crash.

cat /usr/local/lsws/VERSION
cat /usr/local/lsws/BUILD

If it is still an old version or old build, please alway run the force upgrade command so that you are using the latest build of the latest version. This action alone may fix your problem.

/usr/local/lsws/admin/misc/lsup.sh -f -v <latest_version_number>

then monitor your server to see if it happens again or not. If yes, please take the following steps to log the server crash report.

Email alert received with debug info

You may receive an email alert similar to the following if crash happened again after force upgrade:

  Web server example.com on example.com automatically restarted
  
  ##---------- Forwarded message ----------##
  From: root <root@example.com>
  To: root@localhost
  Cc:
  Bcc:
  Date: Wed, 01 Nov 2017 01:00:27 -0500
  Subject: Web server example.com on example.com
  automatically restarted
  At [01/Nov/2017:01:00:25 -0500], web server with pid=12964 received
  unexpected signal=11, a core file has been created. A new instance of web
  server will be started automatically!
  Please forward the following debug information to [[bug@litespeedtech.com]].
  Environment:
  Server: LiteSpeed/5.2.2 Enterprise Build 2
  OS: Linux
  Release: 3.10.0-693.5.2.el7.x86_64
  Version: #1 SMP Fri Oct 20 20:32:50 UTC 2017
  Machine: x86_64
  If the call stack information does not show up here, please compress and
  forward the core file located in /tmp/lshttpd/.
  [New LWP 12964]
  [New LWP 12971]
  [New LWP 12972]
  [Thread debugging using libthread_db enabled]
  Using host libthread_db library "/lib64/libthread_db.so.1".
  Core was generated by `litespeed'.
  Program terminated with signal 11, Segmentation fault.
  #0 0x0000000000607440 in lsShm_hElem_s::getLruLinkPtr (this=0x0) at
  /home/gwang/release/litespeed/src/shm/lsshmhash.h:102
  #0 0x0000000000607440 in lsShm_hElem_s::getLruLinkPtr (this=0x0) at
  /home/gwang/release/litespeed/src/shm/lsshmhash.h:102
  No locals.
  #1 0x00000000006a6553 in lsShm_hElem_s::setLruLinkNext (this=0x0, off=...)
  at /home/gwang/release/litespeed/src/shm/lsshmhash.h:110
  No locals.
  #2 0x00000000006a681c in LsShmHash::set_linkNext (this=0x144a880,
  offThis=..., offNext=...) at
  /home/gwang/release/litespeed/src/shm/lsshmhash.h:686
  No locals.
  #3 0x00000000006a5953 in LsShmHash::linkSetTop (this=0x144a880,
  pElem=0x7f7d53dc4b80, offElem=...) at
  /home/gwang/release/litespeed/src/shm/lsshmhash.cpp:1581
  pLru = 0x7f7d53d00a1c
  prev = {m_iOffset = 610112}
  __PRETTY_FUNCTION__ = "void LsShmHash::linkSetTop(LsShmHElem*,
  LsShmHash::iteroffset)"
  pLink = 0x7f7d53dc4b9c
  next = {m_iOffset = 12479344}
  #4 0x00000000006a34c1 in LsShmHash::doGet (this=0x144a880, iterOff=...,
  key=4039776365, pParms=0x7ffd84259560, pFlag=0x7ffd84259614) at
  /home/gwang/release/litespeed/src/shm/lsshmhash.cpp:809
  iter = 0x7f7d53dc4b80
  #5 0x00000000006a4a9a in LsShmHash::getPtr (pThis=0x144a880,
  pParms=0x7ffd84259560, pFlag=0x7ffd84259614) at
  /home/gwang/release/litespeed/src/shm/lsshmhash.cpp:1206
  key = 4039776365
  iterOff = {m_iOffset = 805760}
  #6 0x000000000069f9fb in LsShmHash::getIterator (this=0x144a880,
  pParms=0x7ffd84259560, pFlag=0x7ffd84259614) at
  /home/gwang/release/litespeed/src/shm/lsshmhash.h:401
  iterOff = {m_iOffset = 44971056}
  #7 0x000000000069f8fd in LsShmHash::get (this=0x144a880, pKey=0x2ae3430,
  keyLen=8, valLen=0x7ffd84259618, pFlag=0x7ffd84259614) at
  /home/gwang/release/litespeed/src/shm/lsshmhash.h:323
  iterOff = {m_iOffset = 2217055712}
  parms = {key = {ptr = 0x2ae3430 "\357\t\304\330\326\373e8", len =
  8}, val = {ptr = 0x0, len = 12}}
  iter = 0x2ae3430
  #8 0x000000000069f2bb in ShmCacheManager::addTracking2 (this=0x15024a0,
  pEntry=0x2ae3410, pTracker=0x144a880) at
  /home/gwang/release/litespeed/src/cache/shmcachemanager.cpp:1141
  pData = 0x943b51
  valLen = 12
  flag = 0
  offVal = 0
  #9 0x000000000069f248 in ShmCacheManager::addTracking (this=0x15024a0,
  pEntry=0x2ae3410) at
  /home/gwang/release/litespeed/src/cache/shmcachemanager.cpp:1128
  pTracker = 0x144a880
  #10 0x000000000055baef in HttpSession::preparePageCache (this=0x30f9ea0) at
  /home/gwang/release/litespeed/src/http/httpsession.cpp:6463
  pEntry = 0xfffffffffffffffe
  pIP = 0x0
  ipLen = 0
  age = 604800
  priv_cache = 0
  pResp = 0x30fa358
  pCtx = 0x149afd0
  #11 0x0000000000555d79 in HttpSession::setupDynRespBody (this=0x30f9ea0,
  prepareCache=1) at
  /home/gwang/release/litespeed/src/http/httpsession.cpp:4439
  No locals.
  #12 0x0000000000556641 in HttpSession::appendDynBody (this=0x30f9ea0,
  pBuf=0xd7b420 <HttpGlobals::g_achBuf> "<!DOCTYPE html>\n<html
  lang=\"en-US\" prefix=\"og: http://ogp.me/ns# fb:
  http://ogp.me/ns/fb#\">\n<head
  >\n<meta charset=\"UTF-8\" />\n<title>My Bookmark -
  MyReadingManga</title><meta name=\"viewport\" content=\"w"..., len=16384)
  at /home/gwang/release/litespeed/src/http/httpsession.cpp:4631
  buffered = 32765
  ret = 0
  #13 0x000000000056eec8 in HttpExtConnector::processRespBodyData
  (this=0x14a4800, pBuf=0xd7b420 <HttpGlobals::g_achBuf> "<!DOCTYPE
  html>\n<html lang=\"en-US\" prefix=\"og: http://ogp.me/ns# fb:
  http://ogp.me/ns/fb#\">\n<head >\n<meta charset=\"UTF-8\" />\n<title>My
  Bookmark - MyReadingManga</title><meta name=\"viewport\" content=\"w"...,
  len=16384) at
  /home/gwang/release/litespeed/src/http/httpextconnector.cpp:371
  ret = 16384
  #14 0x000000000066d7f7 in LsapiConn::readRespBody (this=0x314a9e0) at
  /home/gwang/release/litespeed/src/extensions/lsapi/lsapiconn.cpp:943
  len = 0
  packetLen = 16384
  pBuf = 0xd7b420 <HttpGlobals::g_achBuf> "<!DOCTYPE html>\n<html
  lang=\"en-US\" prefix=\"og: http://ogp.me/ns# fb:
  http://ogp.me/ns/fb#\">\n<head
  >\n<meta charset=\"UTF-8\" />\n<title>My Bookmark -
  MyReadingManga</title><meta name=\"viewport\" content=\"w"...
  toRead = 16384
  pHEC = 0x14a4800
  ret = 16384
  bufLen = 16384
  respState = @0x14a4874: 526339
  total = 16384
  #15 0x000000000066cb13 in LsapiConn::processResp (this=0x314a9e0) at
  /home/gwang/release/litespeed/src/extensions/lsapi/lsapiconn.cpp:683
  ret = 8
  count = 2
  #16 0x000000000066c2c3 in LsapiConn::doRead (this=0x314a9e0) at
  /home/gwang/release/litespeed/src/extensions/lsapi/lsapiconn.cpp:361
  ret = 0
  #17 0x000000000067e727 in ExtConn::onRead (this=0x314a9e0) at
  /home/gwang/release/litespeed/src/extensions/extconn.cpp:344
  ret = 0
  #18 0x000000000068720f in EdStream::handleEvents (this=0x314a9e0, event=1)
  at /home/gwang/release/litespeed/src/edio/ediostream.cpp:70
  ret = 0
  #19 0x000000000068b6e2 in epoll::processEvents (this=0x135fa80, events=1)
  at /home/gwang/release/litespeed/src/edio/epoll.cpp:371
  fd = 76
  pReactor = 0x314a9e0
  p = 0x1370130
  __PRETTY_FUNCTION__ = "virtual int epoll::processEvents(int)"
  #20 0x000000000068b5c5 in epoll::waitAndProcessEvents (this=0x135fa80,
  iTimeoutMilliSec=100) at
  /home/gwang/release/litespeed/src/edio/epoll.cpp:344
  ret = 1
  #21 0x000000000054192d in EventDispatcher::run (this=0x1345c78) at
  /home/gwang/release/litespeed/src/http/eventdispatcher.cpp:244
  ret = 1
  evt_waiting = 0
  sigEvent = 0
  errors = 0
  pQuicEngine = 0x13a3a50
  #22 0x00000000004ef55b in HttpServerImpl::start (this=0x1345c50) at
  /home/gwang/release/litespeed/src/main/httpserver.cpp:529
  No locals.
  #23 0x00000000004f2bfa in HttpServer::start (this=0xd752d0
  <TSingleton<HttpServer>::getInstance()::s_instance>) at
  /home/gwang/release/litespeed/src/main/httpserver.cpp:2031
  No locals.
  #24 0x00000000004e83d1 in LshttpdMain::main (this=0x1345870, argc=1,
  argv=0x7ffd84259ca8) at
  /home/gwang/release/litespeed/src/main/lshttpdmain.cpp:1950
  ret = 0
  #25 0x00000000004e36e6 in main (argc=1, argv=0x7ffd84259ca8) at
  /home/gwang/release/litespeed/src/main.cpp:140
  ret = 0
  rax 0x0 0
  rbx 0x1345870 20207728
  rcx 0x7f7d53d00a1c 140176253782556
  rdx 0x10 16
  rsi 0xc4b80 805760
  rdi 0x0 0
  rbp 0x7ffd84259bc0 0x7ffd84259bc0
  rsp 0x7ffd84259b90 0x7ffd84259b90
  r8 0x7ffd84259614 140726820509204
  r9 0x7f7d59fca340 140176357368640
  r10 0x4 4
  r11 0x7f7d5a0032d0 140176357602000
  r12 0x4e3578 5125496
  r13 0x7ffd84259ca0 140726820510880
  r14 0x0 0
  r15 0x0 0
  rip 0x4e36e6 0x4e36e6 <main(int, char**)+102>
  eflags 0x10206 [ PF IF RF ]
  cs 0x33 51
  ss 0x2b 43
  ds 0x0 0
  es 0x0 0
  fs 0x0 0
  gs 0x0 0

This means that LiteSpeed Web Server crashed, and core files have been created at ''/tmp/lshttpd/bak_core''. You can foward the above email to bug@litespeedtech.com. 

  ll /tmp/lshttpd/bak_core
  drwxr-xr-x 2 root   nobody     4096 May  3  2018 ./
  drwxr-x--x 4 nobody nobody    20480 Dec 11 11:10 ../
  -rw------- 1 nobody nobody 53587968 May  3  2018 core.2578440

==== Email alert received without debug info ====
If you receive an email similar to the following without backtrace information:

<code>
  At [10/Dec/2018:07:15:19 +0000], web server with pid=4059 received unexpected signal=6, a core file has been created. A new instance of web server will be started automatically!

  Please forward the following debug information to bug@litespeedtech.com.
Environment:

Server: LiteSpeed/5.3.4 Enterprise Build 6
OS: Linux
Release: 3.10.0-514.21.1.el7.x86_64
Version: #1 SMP Thu May 25 17:04:51 UTC 2017
Machine: x86_64

If the call stack information does not show up here, please compress and forward the core file located in /tmp/lshttpd/. 

please install gdb first, also install LSWS debug build:

  /usr/local/lsws/admin/misc/lsup.sh -d -f -v <latest_version_number>

then wait for next crash then foward the email with debug info to bug@litespeedtech.com.

The other case is server doesn't crash at all, but LSWS doesn't work as expected, especially it works with apache but doesn't work with LSWS in the control panel environment. In this case, you may want to enable high level debug log on LSWS to reproduce the issue and send such detailed debug log for our developer to investigate along with detailed steps to reproduce the problem.

How to toggle the debug logging and capture detailed debug log?

Let's give an example on a cpanel EA4 envirment. One PHP URl/request work with Apache but doesn't work with LSWS. Go to the logs folder:

cd /etc/apache2/logs
## backup/or delete old error_log 
mv error_log error_log.old
##toggle litespeed server debug log, run once to turn on, run again to turn off. This is to turn on##
killall -USR2 litespeed
## then from your brower to reproduce the issue
## reproduce the issue
##toggle litespeed server debug log, This time to turn it off##
killall -USR2 litespeed
mv error_log error_log.debug

The error_log.debug could be very big if it is a very busy shared production server since it could be many domains servicing requests at the same time. It might be good to just focus on the IP of your location.

You can easliy find your own IP by typing “IP” from google.com.

then

grep your_IP error_log.debug > error_log.debug.ip

You can then forward error_log.debug.ip along with detailed steps to reproduce issue to bug@litespeedtech.com for investigation.

  • Admin
  • Last modified: 2019/04/02 15:13
  • by Jackson Zhang