solaris litespeed enterprise always dumps core on upgrade

[jrmarino]

I'm running litespeed on two servers, identical hardware, Sunfire X4100M2 I think. One version of litespeed is the enterprise version, the other is the standard version. Solaris 10 is the operating system for both.

When I use the upgrade feature, the standard version always works as expected. It upgrades and restarts itself, no issue.

The enterprise version never works. It upgrades itself. It works for a few seconds and then dumps core. This has happened at least 3 times, and every time I've used the upgrade.

My solution is to upgrade and to immediately disable litespeed via the SMF. With 3.3.6 it dumped core before I could disable it. When I enable it again, runs nominally.

These environments are as about as identical as you can get -- the same version of solaris, the same sun hardware, other software configurations are similar. It's something specific to the enterprise version causing the core dump.

[mistwang]

We will check this issue.

[jrmarino]

Hi mistwang,

this is possibly related. Today I restarted the enterprise webserver twice intentionally to update the awstats alias field. Both times I received this email:

At [11/Mar/2008:05:20:11 -0500], web server with pid=21294 received unexpected signal=9, no core file is created. A new instance of web server will be started automatically!

I also updated the standard webserver to update the awstats alias field and I did not receive that email.

The enterprise webserver is at version 3.3.6.
The standard webserver is at version 3.3.7

[mistwang]

Signal 9 is SIGKILL, strange.
Will you get that whenever you restart the enterprise edition?

[jrmarino]

I don't recall seeing that before. I think it just started happening with version 3.3.6.

I looked at the SMF logs -- the watchdog is taking care of the restarts and the watchdog is not terminating. Here is the tail end of the SMF logs. You can see the core dumps that I mentioned before. The last activity is when I restarted the server on March 2nd to upgrade to 3.3.6.

[ Oct 23 01:53:16 Method "start" exited with status 0 ]
[ Jan 11 16:59:24 Stopping because process dumped core. ]
[ Jan 11 16:59:25 Executing stop method ("/opt/lsws/bin/lswsctrl stop") ]
[OK] lshttpd: stopped.
[ Jan 11 16:59:25 Method "stop" exited with status 0 ]
[ Jan 11 17:00:25 Method or service exit timed out.  Killing contract 1167 ]
[ Jan 11 17:01:22 Leaving maintenance because disable requested. ]
[ Jan 11 17:01:22 Disabled. ]
[ Jan 11 17:01:30 Enabled. ]
[ Jan 11 17:01:30 Executing start method ("/opt/lsws/bin/lswsctrl start") ]
[OK] lshttpd: pid=2499.
[ Jan 11 17:01:30 Method "start" exited with status 0 ]
[ Jan 28 12:33:03 Stopping because process dumped core. ]
[ Jan 28 12:33:03 Executing stop method ("/opt/lsws/bin/lswsctrl stop") ]
[OK] lshttpd: stopped.
[ Jan 28 12:33:03 Method "stop" exited with status 0 ]
[ Jan 28 12:34:03 Method or service exit timed out.  Killing contract 1475 ]
[ Jan 28 14:20:50 Leaving maintenance because disable requested. ]
[ Jan 28 14:20:50 Disabled. ]
[ Jan 28 14:20:56 Enabled. ]
[ Jan 28 14:20:56 Executing start method ("/opt/lsws/bin/lswsctrl start") ]
[OK] lshttpd: pid=8965.
[ Jan 28 14:20:56 Method "start" exited with status 0 ]
[ Mar  2 05:41:03 Stopping because process dumped core. ]
[ Mar  2 05:41:03 Executing stop method ("/opt/lsws/bin/lswsctrl stop") ]
[OK] lshttpd: stopped.
[ Mar  2 05:41:03 Method "stop" exited with status 0 ]
[ Mar  2 05:42:04 Method or service exit timed out.  Killing contract 1549 ]
[ Mar  2 05:42:04 Leaving maintenance because disable requested. ]
[ Mar  2 05:42:04 Disabled. ]
[ Mar  2 05:42:04 Enabled. ]
[ Mar  2 05:42:04 Executing start method ("/opt/lsws/bin/lswsctrl start") ]
[OK] lshttpd: pid=28159.
[ Mar  2 05:42:04 Method "start" exited with status 0 ]

[mistwang]

[ Mar 2 05:42:04 Method or service exit timed out. Killing contract 1549 ]

Does it mean SMF kill the process with "-9"? That would explain the unexpected signal 9.
LSWS does graceful restart/stop, which will try to finish all the pending requests before exiting.

Will investigate the "Process dumped core" issue. it does not show which process dumped core.
If lshttpd does, you should receive a similar report in email.

Can you locate the core file and check it with GDB? lshttpd core files should be under /tmp/lshttpd .