LSPHP5 cpu issue

Discussion in 'PHP' started by MentaL, Nov 6, 2010.

  1. MentaL

    MentaL New Member

    Hello,

    I'm running a rather large vbulletin forum with VBSEO but sadly it appears I am unable to get any stability going with litespeed. My CPU will often jump up from around 16 to 100+ or more and I'm unsure as of why, assistance would be appreciated.

    Eaccelerator version; 0.9.6.1
    Memcached version; 2.2.5


    Kind regards.


    ####

    [quote="stable" top stats]
    top - 04:46:17 up 1 day, 18:49, 1 user, load average: 16.80, 28.29, 30.21
    Tasks: 87 total, 10 running, 70 sleeping, 6 stopped, 1 zombie
    Cpu(s): 53.6%us, 9.7%sy, 2.5%ni, 31.5%id, 2.7%wa, 0.0%hi, 0.0%si, 0.0%st
    Mem: 8388608k total, 1163840k used, 7224768k free, 0k buffers
    Swap: 0k total, 0k used, 0k free, 0k cached

    PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
    4037 nobody 15 0 334m 46m 15m R 73.2 0.6 0:15.01 lsphp5
    4034 nobody 16 0 340m 64m 30m R 48.7 0.8 0:10.35 lsphp5
    4058 nobody 16 0 332m 53m 27m R 48.7 0.7 0:08.40 lsphp5
    4001 nobody 16 0 333m 42m 12m R 48.0 0.5 0:09.41 lsphp5
    5523 nobody 16 0 329m 43m 18m R 47.0 0.5 0:10.26 lsphp5
    1327 nobody 16 0 333m 54m 25m R 43.7 0.7 0:21.34 lsphp5
    3998 nobody 15 0 337m 44m 13m R 30.8 0.5 0:12.03 lsphp5
    4032 nobody 15 0 317m 29m 15m S 26.2 0.4 0:14.50 lsphp5
    1326 nobody 15 0 334m 48m 19m S 23.2 0.6 0:18.46 lsphp5
    4040 nobody 25 10 331m 41m 13m R 19.9 0.5 0:08.42 lsphp5
    4028 nobody 15 0 329m 42m 16m S 15.9 0.5 0:09.78 lsphp5
    32764 nobody 16 0 338m 47m 15m R 15.2 0.6 0:19.43 lsphp5
    3572 nobody 15 0 335m 45m 13m S 8.3 0.6 0:09.93 lsphp5
    21659 nobody 15 0 337m 49m 18m S 8.3 0.6 0:26.75 lsphp5
    31980 root 15 0 190m 70m 508 S 7.0 0.9 59:22.05 memcached
    25820 nobody 15 0 27244 11m 3664 S 3.3 0.1 0:10.36 litespeed
    3568 nobody 15 0 332m 43m 16m S 1.7 0.5 0:13.18 lsphp5
    4039 nobody 15 0 334m 50m 19m S 1.3 0.6 0:10.38 lsphp5
    29786 nobody 15 0 334m 46m 17m S 1.3 0.6 0:35.67 lsphp5

    [/quote]
  2. NiteWave

    NiteWave Administrator

    run "top -c", you may see vbseo is consuming most of the CPU

    google search "vbseo slow", can find many discussion there.

    if php in non-suExec mode, APC/Xcache may be better than eAccelerator. eAccelerator 0.9.6.1 don't support user variable cache, while APC/Xcache does.

    some application(vbseo maybe,not sure) may utilize user variable cache for better performance.
  3. MentaL

    MentaL New Member

    Hello


    I'll look into your recommendations and thanks :)
  4. MentaL

    MentaL New Member

    Hello

    I tried APC but sadly always got 503 errors with VBSEO, I tried Xcache and this seems to work good so far, load is manageable (but I wont hold my breath).

    I'm using my vbulletin database cache as Memcache and vbseo's caching options as Xcache so hopefully this will assist allot.


    Thanks.

    /EDIT

    Still having load issues;

    Last edited: Nov 7, 2010
  5. NiteWave

    NiteWave Administrator

  6. MentaL

    MentaL New Member

    That didn't alter my load at all really, my current stats and have been around this for about 12 hours now except every half hour or so it will spike up to around 50.

  7. MentaL

    MentaL New Member

    Here is a random spike;


    /EDIT

    Upon checking the connections it appears that our SQL server (thats located on a different machine) is appearing to cause the spike,

    Last edited: Nov 8, 2010
  8. NiteWave

    NiteWave Administrator

  9. MentaL

    MentaL New Member

    Well it seems that the load is correcting itself...

  10. MentaL

    MentaL New Member

    Meh, It's just randomly decided to play up again.

    Code:
    top - 13:53:52 up  3:58,  1 user,  load average: 39.48, 17.76, 11.16
    Tasks: 158 total,  59 running,  84 sleeping,  15 stopped,   0 zombie
    Cpu(s): 86.7%us, 13.1%sy,  0.0%ni,  0.1%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
    Mem:   8388608k total,  3193808k used,  5194800k free,        0k buffers
    Swap:        0k total,        0k used,        0k free,        0k cached
    
      PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
    27767 nobody    15   0  270m  44m  14m S 16.2  0.5   0:23.71 lsphp5
    26029 nobody    16   0  270m  58m  31m R 10.7  0.7   0:16.62 lsphp5:/home/xxx/public_html/forum/vbseo.php
    27777 nobody    16   0  276m  51m  16m R 10.7  0.6   0:18.04 lsphp5:/home/xxx/public_html/forum/vbseo.php
    22220 nobody    16   0  280m  79m  43m R 10.1  1.0   0:49.18 lsphp5:/home/xxx/public_html/forum/vbseo.php
    27761 nobody    15   0  273m  57m  27m S  9.8  0.7   0:18.33 lsphp5:/home/xxx/public_html/forum/vbseo.php
    28355 nobody    15   0  267m  36m  11m S  9.2  0.5   0:03.88 lsphp5:/home/xxx/public_html/forum/vbseo.php
    28012 nobody    15   0  267m  51m  25m S  8.5  0.6   0:11.19 lsphp5:/home/xxx/public_html/forum/vbseo.php
    28593 nobody    15   0  269m  40m  11m S  8.5  0.5   0:01.73 lsphp5:/home/xxx/public_html/forum/vbseo.php
    28598 nobody    15   0  268m  38m  11m S  8.2  0.5   0:01.53 lsphp5:/home/xxx/public_html/forum/vbseo.php
    28023 nobody    15   0  270m  41m  14m S  7.9  0.5   0:16.75 lsphp5:/home/xxx/public_html/forum/vbseo.php
    28547 nobody    16   0  270m  41m  11m R  7.9  0.5   0:01.93 lsphp5:/home/xxx/public_html/forum/vbseo.php
    28549 nobody    15   0  271m  41m  11m S  7.9  0.5   0:02.12 lsphp5:/home/xxx/public_html/forum/vbseo.php
    28631 nobody    15   0  267m  38m  11m S  7.9  0.5   0:01.34 lsphp5:/home/xxx/public_html/forum/vbseo.php
    28528 nobody    16   0  268m  39m  11m S  7.6  0.5   0:02.52 lsphp5
    28566 nobody    15   0  266m  34m 9480 S  7.6  0.4   0:01.25 lsphp5:/home/xxx/public_html/forum/vbseo.php
    28588 nobody    15   0  274m  44m  11m S  7.6  0.5   0:02.09 lsphp5:/home/xxx/public_html/forum/vbseo.php
    28633 nobody    16   0  263m  31m 9476 R  7.6  0.4   0:01.00 lsphp5:/home/xxx/public_html/forum/vbseo.php
    28286 nobody    16   0  271m  42m  12m R  7.3  0.5   0:06.81 lsphp5:/home/xxx/public_html/forum/vbseo.php
    28530 nobody    16   0  271m  41m  11m R  7.3  0.5   0:02.48 lsphp5:/home/xxx/public_html/forum/vbseo.php
    26030 nobody    16   0  268m  60m  33m R  7.0  0.7   0:35.11 lsphp5:/home/xxx/public_html/forum/vbseo.php
    28009 nobody    16   0  267m  40m  14m R  7.0  0.5   0:09.09 lsphp5:/home/xxx/public_html/forum/vbseo.php
    28553 nobody    16   0  274m  45m  12m R  7.0  0.6   0:01.70 lsphp5:/home/xxx/public_html/forum/vbseo.php
    28564 nobody    16   0  274m  44m  11m R  7.0  0.5   0:01.53 lsphp5:/home/xxx/public_html/forum/vbseo.php
    28664 nobody    15   0  263m  31m 9456 R  7.0  0.4   0:00.66 lsphp5:/home/xxx/public_html/forum/vbseo.php
    22172 nobody    16   0  272m  69m  40m R  6.7  0.9   1:24.27 lsphp5:/home/xxx/public_html/forum/vbseo.php
    28253 nobody    16   0  271m  42m  12m R  6.7  0.5   0:06.98 lsphp5:/home/xxx/public_html/forum/vbseo.php
    28531 nobody    15   0  271m  41m  11m S  6.7  0.5   0:02.10 lsphp5:/home/xxx/public_html/forum/vbseo.php
    28567 nobody    15   0  267m  37m  10m S  6.7  0.5   0:01.26 lsphp5:/home/xxx/public_html/forum/vbseo.php
    28666 nobody    15   0  266m  34m 9468 R  6.7  0.4   0:00.41 lsphp5:/home/xxx/public_html/forum/vbseo.php
    28667 nobody    16   0  263m  31m 9604 R  6.7  0.4   0:00.65 lsphp5:/home/xxx/public_html/forum/vbseo.php
    

    It will average between 30-100, nothing has been altered, I'm using memcached and xcache.
  11. NiteWave

    NiteWave Administrator

    can you strace one of lsphp5 process which has run longest time, in your post, this one:
    22172 nobody 16 0 272m 69m 40m R 6.7 0.9 1:24.27 lsphp5:/home/xxx/public_html/forum/vbseo.php

    and see what it's doing.
  12. MentaL

    MentaL New Member

    Its just outputting allot of info from vb.

    Code:
    fcntl(5, F_SETLKW, {type=F_WRLCK, whence=SEEK_SET, start=0, len=1}) = 0
    fcntl(5, F_SETLKW, {type=F_UNLCK, whence=SEEK_SET, start=0, len=1}) = 0
    lseek(22, -4096, SEEK_CUR)              = 0
    close(22)                               = 0
    munmap(0x2abf6736d000, 4096)            = 0
    poll([{fd=21, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
    write(21, "X\0\0\0\3\n\t\t\tSELECT blockid, require"..., 92) = 92
    read(21, "\1\0\0\1\2[\0\0\2\3def\21xxx_database\23"..., 16384) = 221
    poll([{fd=21, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
    write(21, "\26\1\0\0\3\n\t\t\tSELECT pf.profilefieldc"..., 282) = 282
    read(21, "\1\0\0\1\25a\0\0\2\3def\21xxx_database\2"..., 16384) = 1908
    lstat("/home", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
    lstat("/home/xxx", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
    lstat("/home/xxx/public_html", {st_mode=S_IFDIR|0750, st_size=4096, ...}) = 0
    lstat("/home/xxx/public_html/forum", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
    lstat("/home/xxx/public_html/forum/includes", {st_mode=S_IFDIR|0755, st_size=12288, ...}) = 0
    lstat("/home/xxx/public_html/forum/includes/class_profileblock_blog.php", {st_mode=S_IFREG|0644, st_size=20646, ...}) = 0
    lstat("/home", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
    lstat("/home/xxx", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
    lstat("/home/xxx/public_html", {st_mode=S_IFDIR|0750, st_size=4096, ...}) = 0
    lstat("/home/xxx/public_html/forum", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
    lstat("/home/xxx/public_html/forum/includes", {st_mode=S_IFDIR|0755, st_size=12288, ...}) = 0
    lstat("/home/xxx/public_html/forum/includes/class_profileblock_blog.php", {st_mode=S_IFREG|0644, st_size=20646, ...}) = 0
    open("/home/xxx/public_html/forum/includes/class_profileblock_blog.php", O_RDONLY) = 22
    fstat(22, {st_mode=S_IFREG|0644, st_size=20646, ...}) = 0
    fcntl(22, F_GETFL)                      = 0x8000 (flags O_RDONLY|O_LARGEFILE)
    fstat(22, {st_mode=S_IFREG|0644, st_size=20646, ...}) = 0
    mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abf6736d000
    lseek(22, 0, SEEK_CUR)                  = 0
    lseek(22, 0, SEEK_SET)                  = 0
    read(22, "<?php\n/*========================"..., 4096) = 4096
    lseek(22, 4096, SEEK_SET)               = 4096
    stat("/home/xxx/public_html/forum/includes/class_profileblock_blog.php", {st_mode=S_IFREG|0644, st_size=20646, ...}) = 0
    fcntl(5, F_SETLKW, {type=F_WRLCK, whence=SEEK_SET, start=0, len=1}) = 0
    fcntl(5, F_SETLKW, {type=F_UNLCK, whence=SEEK_SET, start=0, len=1}) = 0
    lseek(22, -4096, SEEK_CUR)              = 0
    close(22)                               = 0
    munmap(0x2abf6736d000, 4096)            = 0
    lstat("/home", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
    lstat("/home/xxx", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
    lstat("/home/xxx/public_html", {st_mode=S_IFDIR|0750, st_size=4096, ...}) = 0
    lstat("/home/xxx/public_html/forum", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
    lstat("/home/xxx/public_html/forum/includes", {st_mode=S_IFDIR|0755, st_size=12288, ...}) = 0
    lstat("/home/xxx/public_html/forum/includes/blog_functions.php", {st_mode=S_IFREG|0644, st_size=105536, ...}) = 0
    lstat("/home", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
    lstat("/home/xxx", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
    lstat("/home/xxx/public_html", {st_mode=S_IFDIR|0750, st_size=4096, ...}) = 0
    lstat("/home/xxx/public_html/forum", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
    lstat("/home/xxx/public_html/forum/includes", {st_mode=S_IFDIR|0755, st_size=12288, ...}) = 0
    lstat("/home/xxx/public_html/forum/includes/blog_functions.php", {st_mode=S_IFREG|0644, st_size=105536, ...}) = 0
    open("/home/xxx/public_html/forum/includes/blog_functions.php", O_RDONLY) = 22
    fstat(22, {st_mode=S_IFREG|0644, st_size=105536, ...}) = 0
    fcntl(22, F_GETFL)                      = 0x8000 (flags O_RDONLY|O_LARGEFILE)
    fstat(22, {st_mode=S_IFREG|0644, st_size=105536, ...}) = 0
    mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abf6736d000
    lseek(22, 0, SEEK_CUR)                  = 0
    lseek(22, 0, SEEK_SET)                  = 0
    read(22, "<?php\n/*========================"..., 4096) = 4096
    lseek(22, 4096, SEEK_SET)               = 4096
    stat("/home/xxx/public_html/forum/includes/blog_functions.php", {st_mode=S_IFREG|0644, st_size=105536, ...}) = 0
    fcntl(3, F_SETLKW, {type=F_WRLCK, whence=SEEK_SET, start=0, len=1}) = 0
    fcntl(3, F_SETLKW, {type=F_UNLCK, whence=SEEK_SET, start=0, len=1}) = 0
    lseek(22, -4096, SEEK_CUR)              = 0
    close(22)                               = 0
    munmap(0x2abf6736d000, 4096)            = 0
    lstat("/home", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
    lstat("/home/xxx", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
    lstat("/home/xxx/public_html", {st_mode=S_IFDIR|0750, st_size=4096, ...}) = 0
    lstat("/home/xxx/public_html/forum", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
    lstat("/home/xxx/public_html/forum/includes", {st_mode=S_IFDIR|0755, st_size=12288, ...}) = 0
    lstat("/home/xxx/public_html/forum/includes/functions_xperience.php", {st_mode=S_IFREG|0644, st_size=43278, ...}) = 0
    open("/home/xxx/public_html/forum/includes/functions_xperience.php", O_RDONLY) = 22
    fstat(22, {st_mode=S_IFREG|0644, st_size=43278, ...}) = 0
    fcntl(22, F_GETFL)                      = 0x8000 (flags O_RDONLY|O_LARGEFILE)
    fstat(22, {st_mode=S_IFREG|0644, st_size=43278, ...}) = 0
    mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2abf6736d000
    lseek(22, 0, SEEK_CUR)                  = 0
    lseek(22, 0, SEEK_SET)                  = 0
    read(22, "<?php\n/*========================"..., 4096) = 4096
    lseek(22, 4096, SEEK_SET)               = 4096
    stat("/home/xxx/public_html/forum/includes/functions_xperience.php", {st_mode=S_IFREG|0644, st_size=43278, ...}) = 0
    fcntl(3, F_SETLKW, {type=F_WRLCK, whence=SEEK_SET, start=0, len=1}) = 0
    fcntl(3, F_SETLKW, {type=F_UNLCK, whence=SEEK_SET, start=0, len=1}) = 0
    lseek(22, -4096, SEEK_CUR)              = 0
    close(22)                               = 0
    munmap(0x2abf6736d000, 4096)            = 0
    poll([{fd=21, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
    write(21, "\221\0\0\0\3\n\t\tSELECT COUNT(DISTINCT(us"..., 149) = 149
    read(21, "\1\0\0\1\3\37\0\0\2\3def\0\0\0\tusercount\0\f?\0\25\0"..., 16384) = 165
    poll([{fd=21, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
    write(21, "\205\0\0\0\3\n\t\tSELECT COUNT(*) AS userc"..., 137) = 137
    read(21, "\1\0\0\1\2\37\0\0\2\3def\0\0\0\tusercount\0\f?\0\25\0"..., 16384) = 127
    poll([{fd=21, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
    write(21, "\205\0\0\0\3\n\t\tSELECT COUNT(*) AS userc"..., 137) = 137
    read(21, "\1\0\0\1\2\37\0\0\2\3def\0\0\0\tusercount\0\f?\0\25\0"..., 16384) = 127
    poll([{fd=21, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
    write(21, "\205\0\0\0\3\n\t\tSELECT COUNT(*) AS userc"..., 137) = 137
    read(21, "\1\0\0\1\2\37\0\0\2\3def\0\0\0\tusercount\0\f?\0\25\0"..., 16384) = 127
    
    

    Update;

    It seems to be correcting itself again, allthough I made two alterations, I disabled vbulletins live topic and vboptimize, given that the optimize was already built into the configuration of vbulletin (memcached) and using xcache with vbseo , it appears to be "ok" as of this moment, but I'm sure it will go haywire shortly.


    Code:
    top - 05:50:47 up  1:21,  1 user,  load average: 2.40, 3.69, 9.94
    Tasks:  71 total,   2 running,  60 sleeping,   9 stopped,   0 zombie
    Cpu(s): 17.1%us,  3.0%sy,  1.8%ni, 78.0%id,  0.1%wa,  0.0%hi,  0.0%si,  0.0%st
    Mem:   8388608k total,   991484k used,  7397124k free,        0k buffers
    Swap:        0k total,        0k used,        0k free,        0k cached
    
      PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
    29766 nobody    15   0  271m  46m  16m S 54.9  0.6   0:02.71 lsphp5
    17722 nobody    15   0  272m  76m  47m S 35.0  0.9   2:35.97 lsphp5
    19916 nobody    16   0  277m  80m  46m R 30.0  1.0   3:08.23 lsphp5
    14326 nobody    15   0  271m  70m  42m S 20.0  0.9   3:15.78 lsphp5
    15846 nobody    25  10  283m  87m  47m S 17.6  1.1   4:23.42 lsphp5
    20004 nobody    15   0  269m  75m  49m S 14.3  0.9   2:27.27 lsphp5
     1514 nobody    15   0 26976  10m 1580 S  1.0  0.1   1:15.07 litespeed
     9960 root      15   0 88060 3348 2584 S  0.3  0.0   0:01.46 sshd
    21977 root      15   0 81580  28m  500 S  0.3  0.3   2:58.26 memcached
    26252 root      15   0 12724 1260  924 R  0.3  0.0   0:00.81 top
    
    Last edited: Nov 22, 2010
  13. NiteWave

    NiteWave Administrator

    please note those lines with "Timeout", I think this may be an important clue. Many lsphp5 process are waiting something and get timeout, this will cause high load. I guess maybe waiting for responses from MySQL. most likely code bug somewhere, cause high load under some conditions.

Share This Page