Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
litespeed_wiki:troubleshooting:mysql-killed-when-pid-max-too-small [2019/03/20 14:30]
Jackson Zhang created
litespeed_wiki:troubleshooting:mysql-killed-when-pid-max-too-small [2020/01/13 18:45] (current)
Joshua Reynolds [Solution]
Line 1: Line 1:
-====== ​Small pid_max may cause MySQL using recybled pid accidentally killed ​======+~~NOTOC~~ 
 +====== ​LiteSpeed Kills Apps Using Recycled PIDs ====== 
 +A small ''​pid_max''​ value may cause applications to be accidentally killed when they are using a recycled PID.
  
-MySQL was accidentally killed somehow. When tracing the root cause, seem LiteSpeed killed ​mysql:+===== Example ===== 
 +This log shows that LiteSpeed killed ​MySQL:
   [Wed Mar 20 04:11:42 2019] SIGTERM to mysqld (12289) by litespeed (6133),0,99 <- systemd (1),0,0   [Wed Mar 20 04:11:42 2019] SIGTERM to mysqld (12289) by litespeed (6133),0,99 <- systemd (1),0,0
   [Wed Mar 20 04:11:49 2019] SIGTERM to mysqld (12289) by mysqld (12289),​994,​991 <- mysqld_safe (12121),0,0 <- systemd (1),0,0   [Wed Mar 20 04:11:49 2019] SIGTERM to mysqld (12289) by mysqld (12289),​994,​991 <- mysqld_safe (12121),0,0 <- systemd (1),0,0
   ​   ​
-Check ''/​proc/​sys/​kernel/​pid_max'' ​and it is ''​kernel.pid_max = 32768'' ​at the moment. +If you check ''/​proc/​sys/​kernel/​pid_max'' ​you may find ''​kernel.pid_max = 32768''​.
-''​32768''​ is too small for some situation when pids are used up very fast and recycled quickly. ​ LiteSpeed may try to kill the unused lsphp process but that pid may have been recycled and reuse as a mysql pid, exim pid or other applications,​ which might cause mysql or other application was killed accidentally. Sometimes CRIU feature may cause such trouble as well since as it reset the PID to restore a process, causing PID being reused too fast even with large pid_max+
  
-The fix is to increase ​the ''​pid_max''​ from '​32768'​ to a larger number such as ''​1048576'':​ +''​32768'' ​is too small for some situations, particularly where PIDs are used up very fast and recycled quickly. For example, LiteSpeed may try to kill an unused lsphp process, but that PID may already have been recycled and reused as a MySQL (or other application) PID. When LiteSpeed kills the process, MySQL is killed unintentionally. 
- ​sysctl kernel.pid_max + 
- ​kernel.pid_max = 32768 +===== Solution ===== 
- ​sysctl -w kernel.pid_max=1048576 +Increase ​''​pid_max''​ from ''32768'' to a larger numbersuch as ''​1048576''​, like so
- ​kernel.pid_max = 1048576+  sysctl kernel.pid_max 
 +  kernel.pid_max = 32768 
 +  sysctl -w kernel.pid_max=1048576 
 +  kernel.pid_max = 1048576 
 +  
 +**NOTE**:​Sometimes the CRIU feature may cause similar trouble when it resets the PID to restore a process. This causes the PID to be reused too quickly, and can be a problem even with a large ''​pid_max''​ setting. ​
  • Admin
  • Last modified: 2019/03/20 14:30
  • by Jackson Zhang