503 Errors - APC with cPanel matched PHP binary

Discussion in 'PHP' started by ziggy95, Feb 18, 2009.

  1. ziggy95

    ziggy95 New Member

    Hi there. I have successfully installed litespeed and matched the PHP 5.2.8 binary from cPanel. I have installed APC and included it in php.ini correctly.

    When I restart litespeed, I confirm that APC is added to phpinfo().

    PHP pages load fine for a few seconds, and then quickly go to 503 errors. There is nothing recorded to stderr.log.

    In error_log it says:

    2009-02-18 12:36:59.838 [NOTICE] [97.77.97.51:1386-0#APVH_sitename.com:lsapi] Redirect: #2, URL: /vbseo.php
    2009-02-18 12:36:59.838 [ERROR] [97.77.97.51:1386-0#APVH_sitename.com:lsapi] detect loop redirection.
    2009-02-18 12:37:00.829 [INFO] [97.77.97.51:1389-0#APVH_sitename.com] connection to [uds://tmp/lshttpd/lsphp5.sock] on request #0, Connection reset by peer!
    2009-02-18 12:37:00.831 [INFO] [97.77.97.51:1389-0#APVH_sitename.com] connection to [uds://tmp/lshttpd/lsphp5.sock] on request #0, Connection reset by peer!
    2009-02-18 12:37:00.833 [INFO] [97.77.97.51:1389-0#APVH_sitename.com] connection to [uds://tmp/lshttpd/lsphp5.sock] on request #0, Connection reset by peer!
    2009-02-18 12:37:00.833 [NOTICE] [97.77.97.51:1389-0#APVH_sitename.com] oops! 503 Service Unavailable
    2009-02-18 12:37:00.833 [NOTICE] [97.77.97.51:1389-0#APVH_sitename.com] Content len: 0, Request line:
    GET / HTTP/1.1
    2009-02-18 12:37:00.834 [INFO] [97.77.97.51:1389-0#APVH_sitename.com:lsapi] connection to [uds://tmp/lshttpd/lsphp5.sock] on request #0, Connection reset by peer!
    2009-02-18 12:37:00.835 [INFO] [97.77.97.51:1389-0#APVH_sitename.com:lsapi] connection to [uds://tmp/lshttpd/lsphp5.sock] on request #0, Connection reset by peer!
    2009-02-18 12:37:00.836 [INFO] [97.77.97.51:1389-0#APVH_sitename.com:lsapi] connection to [uds://tmp/lshttpd/lsphp5.sock] on request #0, Connection reset by peer!
    2009-02-18 12:37:00.836 [NOTICE] [97.77.97.51:1389-0#APVH_sitename.com:lsapi] oops! 503 Service Unavailable
    2009-02-18 12:37:00.836 [NOTICE] [97.77.97.51:1389-0#APVH_sitename.com:lsapi] Content len: 0, Request line:
    GET / HTTP/1.1
    2009-02-18 12:37:00.836 [NOTICE] [97.77.97.51:1389-0#APVH_sitename.com:lsapi] Redirect: #2, URL: /vbseo.php
    2009-02-18 12:37:00.836 [ERROR] [97.77.97.51:1389-0#APVH_sitename.com:lsapi] detect loop redirection.
    2009-02-18 12:37:32.870 [INFO] [66.249.70.226:45454-0#APVH_sitename.com] connection to [uds://tmp/lshttpd/lsphp5.sock] on request #0, Connection reset by peer!
    2009-02-18 12:37:32.871 [INFO] [66.249.70.226:45454-0#APVH_sitename.com] connection to [uds://tmp/lshttpd/lsphp5.sock] on request #0, Connection reset by peer!
    2009-02-18 12:37:32.873 [INFO] [66.249.70.226:45454-0#APVH_sitename.com] connection to [uds://tmp/lshttpd/lsphp5.sock] on request #0, Connection reset by peer!
    2009-02-18 12:37:32.874 [NOTICE] [66.249.70.226:45454-0#APVH_sitename.com] oops! 503 Service Unavailable
    2009-02-18 12:37:32.874 [NOTICE] [66.249.70.226:45454-0#APVH_sitename.com] Content len: 0, Request line:
    GET /members/sexculer.html HTTP/1.1
    2009-02-18 12:37:32.874 [NOTICE] [66.249.70.226:45454-0#APVH_sitename.com] Redirect: #1, URL: /vbseo.php
    2009-02-18 12:37:32.875 [INFO] [66.249.70.226:45454-0#APVH_sitename.com:lsapi] connection to [uds://tmp/lshttpd/lsphp5.sock] on request #0, Connection reset by peer!
    2009-02-18 12:37:32.878 [INFO] [66.249.70.226:45454-0#APVH_sitename.com:lsapi] connection to [uds://tmp/lshttpd/lsphp5.sock] on request #0, Connection reset by peer!
    2009-02-18 12:37:32.880 [INFO] [66.249.70.226:45454-0#APVH_sitename.com:lsapi] connection to [uds://tmp/lshttpd/lsphp5.sock] on request #0, Connection reset by peer!
    2009-02-18 12:37:32.880 [NOTICE] [66.249.70.226:45454-0#APVH_sitename.com:lsapi] oops! 503 Service Unavailable
    2009-02-18 12:37:32.880 [NOTICE] [66.249.70.226:45454-0#APVH_sitename.com:lsapi] Content len: 0, Request line:
    GET /members/sexculer.html HTTP/1.1
    2009-02-18 12:37:32.880 [NOTICE] [66.249.70.226:45454-0#APVH_sitename.com:lsapi] Redirect: #3, URL: /vbseo.php
    2009-02-18 12:37:32.880 [ERROR] [66.249.70.226:45454-0#APVH_sitename.com:lsapi] detect loop redirection.

    I would like to use APC, but must disable it until I can resolve this.
  2. ziggy95

    ziggy95 New Member

    Static pages continue to load fine. At first I thought maybe the problem was a conflict with vbseo.php, but then after a while even phpinfo() goes to 503.
  3. mistwang

    mistwang LiteSpeed Staff

  4. ziggy95

    ziggy95 New Member

    I tried increasing soft/hard for lsphp5 to from 250M/300M to 2500M/2500M and restarted and still get same result. Do I need to change it somewhere else also?
  5. ziggy95

    ziggy95 New Member

    I also tried increasing it in the Server > Security tab. No avail.
  6. ziggy95

    ziggy95 New Member

    Any help you can provide would be appreciated.
  7. mistwang

    mistwang LiteSpeed Staff

    Do you get anything in stderr.log?
  8. ziggy95

    ziggy95 New Member

    Only if there is an error in php.ini. When APC is active there is nothing logged to stderr.log.
  9. mistwang

    mistwang LiteSpeed Staff

    Are you using a VPS?
    What is the cache memory size for APC in php.ini? maybe you need to reduce the cache memory size, especially when you use PHP suEXEC. Maybe you should disable PHP suEXEC if you use VPS to host a few web sites, not for shared hosting.
  10. ziggy95

    ziggy95 New Member

    It is a dedicated server cluster, one web server and one database server. I am not using suEXEC. I haven't specified cache size in php.ini for APC, I'll look into that, though I am doubtful that the default would cause overload - especially since I tried raising limit to 3,000MB.
  11. ziggy95

    ziggy95 New Member

    After trying again we are not getting the 503 errors anymore. There was a problem where the database connection was routing through the network LAN vs the local LAN between the two boxes. I'm not sure how this would have played into things, but now that the connection is direct the errors aren't happenning.... yet.

    Will keep you posted.
  12. mistwang

    mistwang LiteSpeed Staff

    You should use LAN for DB connection. always use the fastest available connection.
  13. ziggy95

    ziggy95 New Member

    Yes, of course we understand this. :) Someone had made a mistake and it was corrected.

    However, we are back to the 503 errors. So far we have only moved a single low traffic site to the server. We are testing and near moving a very high traffic site (vbulletin forum) over.

    I have tried setting memory in Security and External App to 3,000M. No good.

    2009-02-19 13:17:31.799 [NOTICE] [68.83.66.8:49373-0#APVH_sitename.com] Redirect: #1, URL: /vbseo.php
    2009-02-19 13:17:31.801 [INFO] [68.83.66.8:49373-0#APVH_sitename.com:lsapi] connection to [uds://tmp/lshttpd/lsphp5.sock] on request #0, Connection reset by peer!
    2009-02-19 13:17:31.804 [INFO] [68.83.66.8:49373-0#APVH_sitename.com:lsapi] connection to [uds://tmp/lshttpd/lsphp5.sock] on request #0, Connection reset by peer!
    2009-02-19 13:17:31.807 [INFO] [68.83.66.8:49373-0#APVH_sitename.com:lsapi] connection to [uds://tmp/lshttpd/lsphp5.sock] on request #0, Connection reset by peer!
    2009-02-19 13:17:31.807 [NOTICE] [68.83.66.8:49373-0#APVH_sitename.com:lsapi] oops! 503 Service Unavailable

    is what shows in error_log.

    I guess we don't have to have APC, though it would likely help.

    Any advice?
  14. mistwang

    mistwang LiteSpeed Staff

    Maybe you have set process limits of lsphp5 too low?
    check how many lsphp5 processes do you get?

    I think for some reason, APC crashes PHP, maybe look into other cache solutions like eAccelerator or xcache.
  15. ziggy95

    ziggy95 New Member

    The processes are set at default 200/200. I will try xcache instead. I've used them both before with comparable results.
  16. mistwang

    mistwang LiteSpeed Staff

    Since you do not use PHP suEXEC, you can try strace the parent lsphp5 process with command

    strace -f -p <pid_of_lsphp5>

    when you get 503 errors. It can give you some hints on why PHP crashed for some reason.
  17. ziggy95

    ziggy95 New Member

    I installed and configured xcache. Within seconds of restarting the server, the 503 errors come back. I setting max processes to 0/0, no luck. Here is the result of strace:

    # strace -f -p 16225
    Process 16225 attached - interrupt to quit
    select(6, [5], NULL, NULL, {0, 853000}) = 0 (Timeout)
    getppid() = 16224
    select(6, [5], NULL, NULL, {1, 0}) = 1 (in [5], left {0, 226000})
    read(5, "LS\1\0\355\6\0\0\315\3\0\0\0\0\0\0\201\1\0\0\372\1\0\0\265\1\0\0n\2\0\0"..., 8192) = 1773
    write(5, "LS\7\0\10\0\0\0", 8) = 8
    setitimer(ITIMER_PROF, {it_interval={0, 0}, it_value={60, 0}}, NULL) = 0
    rt_sigaction(SIGPROF, {0x685d80, [PROF], SA_RESTORER|SA_RESTART, 0x3ed4e301b0}, {0x685d80, [PROF], SA_RESTORER|SA_RESTART, 0x3ed4e301b0}, 8) = 0
    rt_sigprocmask(SIG_UNBLOCK, [PROF], NULL, 8) = 0
    getcwd("/usr/local/lsws/fcgi-bin", 4095) = 25
    chdir("/home/sitename/public_html") = 0
    setitimer(ITIMER_PROF, {it_interval={0, 0}, it_value={30, 0}}, NULL) = 0
    rt_sigaction(SIGPROF, {0x685d80, [PROF], SA_RESTORER|SA_RESTART, 0x3ed4e301b0}, {0x685d80, [PROF], SA_RESTORER|SA_RESTART, 0x3ed4e301b0}, 8) = 0
    rt_sigprocmask(SIG_UNBLOCK, [PROF], NULL, 8) = 0
    open("/home/sitename/public_html/vbseo.php", O_RDONLY) = 6
    fstat(6, {st_mode=S_IFREG|0644, st_size=48445, ...}) = 0
    mmap(NULL, 48445, PROT_READ, MAP_SHARED, 6, 0) = 0x2aaaafef6000
    munmap(0x2aaaafef6000, 48445) = 0
    close(6) = 0
    open("/home/sitename/public_html/vbseo.php", O_RDONLY) = 6
    fstat(6, {st_mode=S_IFREG|0644, st_size=48445, ...}) = 0
    fcntl(6, F_GETFL) = 0x8000 (flags O_RDONLY|O_LARGEFILE)
    fstat(6, {st_mode=S_IFREG|0644, st_size=48445, ...}) = 0
    mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaaafef6000
    lseek(6, 0, SEEK_CUR) = 0
    lseek(6, 0, SEEK_SET) = 0
    read(6, "<?php\n\n/************************"..., 4096) = 4096
    lseek(6, 4096, SEEK_SET) = 4096
    stat("/home/sitename/public_html/vbseo.php", {st_mode=S_IFREG|0644, st_size=48445, ...}) = 0
    fcntl(1, F_SETLKW, {type=F_WRLCK, whence=SEEK_SET, start=0, len=1}) = 0
    fcntl(1, F_SETLKW, {type=F_UNLCK, whence=SEEK_SET, start=0, len=1}) = 0
    lseek(6, -4096, SEEK_CUR) = 0
    close(6) = 0
    munmap(0x2aaaafef6000, 4096) = 0
    --- SIGSEGV (Segmentation fault) @ 0 (0) ---
    Process 16225 detached
  18. mistwang

    mistwang LiteSpeed Staff

    Which version of PHP is used?
    Maybe you can try downgrade PHP version.

    How about Apache mod_php? will APC or Xcache crash it?
    The problem is between PHP and opcode cache, you can try to let lsphp5 dump core file and exam the core file with GDB.

    To let lsphp5 dump core file, you need to add environment variable to lsphp5 external application

    LSAPI_ALLOW_CORE_DUMP=1
  19. ziggy95

    ziggy95 New Member

    PHP is 5.2.8 as stated. I'm not familiar with examining the core dump. I would need more help on that.
  20. mistwang

    mistwang LiteSpeed Staff

    Once you got the core file, just run

    gdb <path_to_lsphp5> <core_file>

    then type command

    bt

    to list the call stack.

Share This Page