[solved] APC & litespeed restarts because of unknown reason

Discussion in 'PHP' started by semprot, Oct 6, 2013.

  1. semprot

    semprot Member

    After using lsphp 6.3, and use "suEXEC Daemon" on "Run On Start Up", APC is still restarted (my litespeed php5 config is attached below).
    When i checked the litespeed log, i see some messages that could be the reason of the litespeed to be restarted?

    Code:
    No request delivery notification has been received from LSAPI process group [-1], possible run away process.
    ...
    Retry with new process group.
    ...
    connection to [uds://tmp/lshttpd/lsphp5.sock.29027] on request #0, confirmed, 0, associated process: -1, running: 0, error: Connection timed out!
    ...
    Suspicious request timeout, associated process: -1, running: 0.
    Max retries has been reached, 503!
    oops! 503 Service Unavailable
    
    Does my litespeed config is not optimized?

    Code:
    Runtime Settings
    apc.cache_by_default	1
    apc.canonicalize	1
    apc.coredump_unmap	0
    apc.enable_cli	1
    apc.enabled	1
    apc.file_md5	0
    apc.file_update_protection	2
    apc.filters	
    apc.gc_ttl	3600
    apc.include_once_override	0
    apc.lazy_classes	0
    apc.lazy_functions	0
    apc.max_file_size	20M
    apc.mmap_file_mask	/home/apc/apc.GKhZuh
    apc.num_files_hint	1000
    apc.preload_path	
    apc.report_autofilter	0
    apc.rfc1867	0
    apc.rfc1867_freq	0
    apc.rfc1867_name	APC_UPLOAD_PROGRESS
    apc.rfc1867_prefix	upload_
    apc.rfc1867_ttl	3600
    apc.serializer	default
    apc.shm_segments	1
    apc.shm_size	7144M
    apc.slam_defense	1
    apc.stat	0
    apc.stat_ctime	0
    apc.ttl	0
    apc.use_request_time	1
    apc.user_entries_hint	400000
    apc.user_ttl	7200
    apc.write_lock	1
    

    Attached Files:

    Last edited by a moderator: Oct 8, 2013
  2. NiteWave

    NiteWave Administrator

    the lsphp5 config is normal and ok.

    use cloudlinux or not ?

    please paste the "pstree -p" output under litespeed root process
  3. semprot

    semprot Member

    1. i think i use "CENTOS 5.9" (from WHM).
    2. here is the result. (i used 4 cpu license just 15 mins ago, it was 2 cpu before, i was upgrading the license)
    Code:
    |-litespeed(9359)-+-httpd(9360)
    |                 |-litespeed(9363)-+-admin_php(9429)
    |                 |                 |-splitlogs(9430)
    |                 |                 |-splitlogs(9431)
    |                 |                 |-{litespeed}(9365)
    |                 |                 `-{litespeed}(9366)
    |                 |-litespeed(9364)-+-splitlogs(9375)
    |                 |                 |-splitlogs(9376)
    |                 |                 |-{litespeed}(9368)
    |                 |                 `-{litespeed}(9369)
    |                 |-litespeed(9367)-+-splitlogs(9508)
    |                 |                 |-splitlogs(9509)
    |                 |                 |-{litespeed}(9372)
    |                 |                 `-{litespeed}(9374)
    |                 |-litespeed(9370)-+-splitlogs(9435)
    |                 |                 |-splitlogs(9436)
    |                 |                 |-{litespeed}(9371)
    |                 |                 `-{litespeed}(9373)
    |                 `-lsphp5(9362)-+-lsphp5(9377)
    |                                |-lsphp5(9378)
    |                                |-lsphp5(9379)
    |                                |-lsphp5(9380)
    |                                |-lsphp5(9381)
    |                                |-lsphp5(9382)
    |                                |-lsphp5(9383)
    |                                |-lsphp5(9397)
    |                                |-lsphp5(9398)
    |                                |-lsphp5(9399)
    |                                |-lsphp5(9416)
    |                                |-lsphp5(9425)
    |                                |-lsphp5(9426)
    |                                |-lsphp5(9427)
    |                                |-lsphp5(9428)
    |                                |-lsphp5(9433)
    |                                |-lsphp5(9493)
    |                                |-lsphp5(9506)
    |                                `-lsphp5(9507)
    
    Last edited: Oct 6, 2013
  4. NiteWave

    NiteWave Administrator

    from the pstree output, php run in SuEXEC daemon mode as expected.

    >APC is still restarted
    because of webserver restart which because of

    >Max retries has been reached, 503!
    >oops! 503 Service Unavailable

    now should trouble-shoot the 503 error issue. when it's resolved, APC issue will be fixed too.

    regarding php 503 error, it's because of php process crash. there are many reasons a php process could crash. a simple php script won't cause 503 error.

    1st, check the php extensions you've installed. see if able to disable some or all of them to reduce 503 error. please paste here the list of your current extensions(.so)
  5. semprot

    semprot Member

    Yes i received emails too about lsws being restarted because of 503 error.

    Here is my loaded extensions :
    Code:
    apache2handler
    bcmath
    bz2
    calendar
    Core
    ctype
    curl
    date
    dom
    ereg
    exif
    filter
    ftp
    gd
    gettext
    hash
    iconv
    json
    libxml
    mbstring
    mcrypt
    mhash
    mongo
    mysql
    mysqli
    mysqlnd
    odbc
    pcre
    PDO
    pdo_mysql
    pdo_sqlite
    Phar
    Reflection
    session
    SimpleXML
    soap
    sockets
    SPL
    sqlite3
    standard
    tokenizer
    wddx
    xml
    xmlreader
    xmlrpc
    xmlwriter
    xsl
    zip
    zlib
    
    Additional information : i have doubled my RAM few hours ago hoping it will solve the issue, but i still have the issue.
  6. NiteWave

    NiteWave Administrator

    the php extensions, I meant those not build in to lsphp5

    please locate the current php.ini, then

    #grep extension /path/to/php.ini

    to see how many .so are loaded.
  7. semprot

    semprot Member

    Okay here it is. Thanks.

    Code:
    ; Directory in which the loadable extensions (modules) reside.
    extension_dir = "/usr/local/lib/php/extensions/no-debug-non-zts-20060613"
    zend_extension="/usr/local/IonCube/ioncube_loader_lin_5.2.so"
    extension = "pdo.so"
    extension = "pdo_sqlite.so"
    extension = "sqlite.so"
    extension = "pdo_mysql.so"
    extension = "ffmpeg.so"
    extension="apc.so"
    extension="timezonedb.so"
    extension = "memcache.so"
    ;eaccelerator.cache_dir="/tmp/eaccelerator"
    ;eaccelerator.check_mtime="1"
    ;eaccelerator.compress="1"
    ;eaccelerator.compress_level="9"
    ;eaccelerator.debug="0"
    ;eaccelerator.enable="1"
    ;eaccelerator.filter=""
    ;eaccelerator.optimizer="1"
    ;eaccelerator.shm_max="0"
    ;eaccelerator.shm_only="0"
    ;eaccelerator.shm_prune_period="0"
    ;eaccelerator.shm_size="16"
    ;eaccelerator.shm_ttl="0"
  8. NiteWave

    NiteWave Administrator

    >zend_extension="/usr/local/IonCube/ioncube_loader_lin_5.2.so"
    what's your php version ? php 5.2.x ?
  9. semprot

    semprot Member

    Yes i run 5.2 now.
  10. semprot

    semprot Member

    More information,

    /var/log/messages:

    Code:
    Oct 6 04:37:43 myusenamehere kernel: lsphp5[8870]: segfault at ffffffffffffffff rip 0000003d7ae0a2ca rsp 00007fff8e9d7188 error 6
    Oct 6 04:37:43 myusenamehere kernel: lsphp5[8871]: segfault at ffffffffffffffff rip 0000003d7ae0a2ca rsp 00007fffdf459398 error 6
    Oct 6 04:37:43 myusenamehere kernel: lsphp5[8872]: segfault at ffffffffffffffff rip 0000003d7ae0a2ca rsp 00007fff03c42a18 error 6
    Oct 6 04:37:43 myusenamehere kernel: lsphp5[8873]: segfault at ffffffffffffffff rip 0000003d7ae0a2ca rsp 00007fff9d566338 error 6
    Oct 6 04:37:43 myusenamehere kernel: lsphp5[8874]: segfault at ffffffffffffffff rip 0000003d7ae0a2ca rsp 00007fff569734a8 error 6
    Oct 6 04:37:44 myusenamehere kernel: lsphp5[8875]: segfault at ffffffffffffffff rip 0000003d7ae0a2ca rsp 00007fff0b35ad18 error 6
    Oct 6 04:37:44 myusenamehere kernel: lsphp5[8876]: segfault at ffffffffffffffff rip 0000003d7ae0a2ca rsp 00007fff46312478 error 6
    Oct 6 04:37:44 myusenamehere kernel: lsphp5[8877]: segfault at ffffffffffffffff rip 0000003d7ae0a2ca rsp 00007fff31bda378 error 6
    Oct 6 04:37:44 myusenamehere kernel: lsphp5[8878]: segfault at ffffffffffffffff rip 0000003d7ae0a2ca rsp 00007fff8df93ff8 error 6
    Oct 6 04:37:44 myusenamehere kernel: lsphp5[8879]: segfault at ffffffffffffffff rip 0000003d7ae0a2ca rsp 00007fffb2e0a488 error 6
    Oct 6 04:37:44 myusenamehere kernel: lsphp5[8881]: segfault at ffffffffffffffff rip 0000003d7ae0a2ca rsp 00007fff96509178 error 6
    Oct 6 04:37:44 myusenamehere kernel: lsphp5[8882]: segfault at ffffffffffffffff rip 0000003d7ae0a2ca rsp 00007fff0cb0c158 error 6
    Oct 6 04:37:45 myusenamehere kernel: lsphp5[8883]: segfault at ffffffffffffffff rip 0000003d7ae0a2ca rsp 00007fff0a422478 error 6
    Oct 6 04:37:45 myusenamehere kernel: lsphp5[8889]: segfault at ffffffffffffffff rip 0000003d7ae0a2ca rsp 00007fff893779b8 error 6
    Oct 6 04:37:45 myusenamehere kernel: lsphp5[8891]: segfault at ffffffffffffffff rip 0000003d7ae0a2ca rsp 00007fff1c32c978 error 6
    Oct 6 04:37:45 myusenamehere kernel: lsphp5[8892]: segfault at ffffffffffffffff rip 0000003d7ae0a2ca rsp 00007fff05b02028 error 6
    Oct 6 04:37:45 myusenamehere kernel: lsphp5[8893]: segfault at ffffffffffffffff rip 0000003d7ae0a2ca rsp 00007fffaa76e0f8 error 6
    Oct 6 04:37:45 myusenamehere kernel: lsphp5[8894]: segfault at ffffffffffffffff rip 0000003d7ae0a2ca rsp 00007fff298f16e8 error 6
    Oct 6 04:37:45 myusenamehere kernel: lsphp5[8895]: segfault at ffffffffffffffff rip 0000003d7ae0a2ca rsp 00007fff32f38b98 error 6
    Oct 6 04:37:45 myusenamehere kernel: lsphp5[8896]: segfault at ffffffffffffffff rip 0000003d7ae0a2ca rsp 00007fffce968918 error 6
    Oct 6 04:37:45 myusenamehere kernel: lsphp5[8897]: segfault at ffffffffffffffff rip 0000003d7ae0a2ca rsp 00007fff6f7ddba8 error 6
    
    and tons of similar message.
  11. NiteWave

    NiteWave Administrator

    so php crashed many times.

    the main problem should be at php code side. with so many crash, why you still want to enable APC ? 1st priority is to reduce the crashes. will disable APC to stop or reduce crash ?

    "Oct 6 04:37:43 myusenamehere kernel: lsphp5[8873]: segfault at ffffffffffffffff rip 0000003d7ae0a2ca rsp 00007fff9d566338 error 6"

    if above crashes happens on one user "myusenamehere" only, you can disable it temporarily. and watch some time if php crash/503 error/lsws restart has improved.

    another suggestion is to install php 5.3.x or 5.4, move some accounts to run 5.3/5.4. you can run multiple version php at the same time. here's the how-to wiki:
    http://www.litespeedtech.com/suppor...peed_wiki:multiple_phps_in_control_panel_lsws
  12. semprot

    semprot Member

    About the APC, it helps to cache queries in my website and reduce load significantly.

    And the problem appeared only about 3 days ago. I've been running APC for months, traffic is stable (no sudden increase), and 3 days ago errors popped up.

    Is it possible that crashed is caused by something else? Such as PHP coding error / lack of memory (that i doubt, because my RAM has been doubled now) / misconfigured server?

    About PHP 5.3.x or 5.4 multiple installation, does it needed for analyzing the cause of the crash / solve the crash itself?

    Thanks.
  13. NiteWave

    NiteWave Administrator

    most likely "PHP coding error" / code change

    about mis-config server, check if you added
    extension="apc.so"
    extension = "ffmpeg.so"
    extension = "memcache.so"
    ..
    etc recently
  14. semprot

    semprot Member

    Problem has been solved. The php crash was caused by mismatched binaries, and the php was rebuilt several times, and finally it is fixed :)
  15. NiteWave

    NiteWave Administrator

    great! thanks for update. particularly which binaries ?
  16. semprot

    semprot Member

    Here is what my hosting support said

    Update : after i watched the server, it seems APC is still restarted every 10-25 mins. But no 503 error e-mail on my inbox anymore. And because i've increased my RAM, the effect is not as significant as when my RAM is still small. Will check the litespeed log later.
    Last edited: Oct 8, 2013

Share This Page