Optimization Request

Discussion in 'General' started by bhanuprasad1981, Jan 9, 2012.

  1. bhanuprasad1981

    bhanuprasad1981 New Member

    i am running server with 2 cpu license past some days load is too high tries a lot of configs but fail, please help me to find issue , let me know what details you need .
  2. webizen

    webizen New Member

    run 'top' first and see what process consumes the most resources/cpu cycles.
  3. bhanuprasad1981

    bhanuprasad1981 New Member

    here is top result :-

    Code:
    Tasks: 191 total,   1 running, 190 sleeping,   0 stopped,   0 zombie
    Cpu(s):  1.0%us,  1.0%sy,  0.0%ni, 86.5%id, 10.7%wa,  0.5%hi,  0.2%si,  0.0%st
    Mem:  12289824k total, 12075560k used,   214264k free,   409056k buffers
    Swap: 12594940k total,        0k used, 12594940k free,  9524700k cached
    
      PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
     6641 mysql     10  -5 10.5g 1.5g 4728 S  5.7 12.8  26:02.59 /usr/sbin/mysqld --
    11590 nobody    15   0  551m  31m  16m S  4.7  0.3   0:00.14 lsphp5
    11588 nobody    15   0  544m  22m  14m S  2.0  0.2   0:00.06 lsphp5
    11491 nobody    15   0  548m  87m  78m S  1.0  0.7   0:01.27 lsphp5
    11589 nobody    15   0  542m  14m 8796 S  0.7  0.1   0:00.02 lsphp5
     2490 root      10  -5     0    0    0 D  0.3  0.0   0:23.56 [kjournald]
        1 root      15   0 10368  688  572 S  0.0  0.0   0:01.37 init [3]
        2 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 [migration/0]
        3 root      34  19     0    0    0 S  0.0  0.0   0:00.01 [ksoftirqd/0]
        4 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 [watchdog/0]
        5 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 [migration/1]
        6 root      34  19     0    0    0 S  0.0  0.0   0:00.00 [ksoftirqd/1]
        7 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 [watchdog/1]
        8 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 [migration/2]
        9 root      34  19     0    0    0 S  0.0  0.0   0:00.00 [ksoftirqd/2]
       10 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 [watchdog/2]
       11 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 [migration/3]
    
  4. webizen

    webizen New Member

    you missing the first line (contains 'load average'). pls add back.
  5. bhanuprasad1981

    bhanuprasad1981 New Member

    top -c result during normal time not peak* during peak load touches 20+ and sites start to crawl.

    Code:
    root@ecat [~]# top -c
    top - 05:34:02 up 17:19,  1 user,  load average: 5.77, 6.10, 6.44
    Tasks: 194 total,   1 running, 193 sleeping,   0 stopped,   0 zombie
    Cpu(s):  2.7%us,  1.2%sy,  0.0%ni, 82.5%id, 13.3%wa,  0.2%hi,  0.2%si,  0.0%st
    Mem:  12289824k total, 11967808k used,   322016k free,   462640k buffers
    Swap: 12594940k total,   102876k used, 12492064k free,  8291120k cached
    
      PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
    20436 nobody    15   0  550m  98m  87m S  4.0  0.8   0:13.11 lsphp5
    16045 nobody    18   0  553m 131m 117m D  3.3  1.1   0:52.27 lsphp5:/home/cluefo
    20488 nobody    15   0  553m  97m  79m S  3.3  0.8   0:01.41 lsphp5
    20589 nobody    15   0  543m  39m  33m S  3.3  0.3   0:00.26 lsphp5
     6641 mysql     10  -5 10.5g 2.2g 4872 S  3.0 19.1 483:48.96 /usr/sbin/mysqld --
    20588 nobody    15   0  545m  51m  43m S  2.0  0.4   0:00.34 lsphp5
    20591 nobody    15   0  544m  31m  24m S  1.3  0.3   0:00.19 lsphp5
    20594 nobody    15   0  541m  19m  15m S  1.3  0.2   0:00.07 lsphp5
    20521 nobody    15   0  550m  91m  81m S  1.0  0.8   0:00.87 lsphp5
    20596 nobody    15   0  544m  18m  10m S  1.0  0.2   0:00.06 lsphp5
     2490 root      10  -5     0    0    0 D  0.7  0.0  11:55.34 [kjournald]
    20404 root      15   0 12760 1192  824 R  0.7  0.0   0:00.30 top -c
    20437 nobody    15   0  562m 121m 100m S  0.7  1.0   0:01.84 lsphp5
    20516 nobody    15   0  556m  82m  65m S  0.7  0.7   0:00.86 lsphp5
    13377 nobody     4 -19 63908  22m 9.9m D  0.3  0.2   0:23.16 litespeed (lshttpd)
    20593 nobody    15   0  543m  19m  12m S  0.3  0.2   0:00.07 lsphp5
        1 root      15   0 10368  688  572 S  0.0  0.0   0:01.40 init [3]
    
  6. bhanuprasad1981

    bhanuprasad1981 New Member

    cpanel daily process logs show :-

    mysql 73.0 /usr/sbin/mysqld --basedir/ --datadir/var/lib/mysql --usermysql --log-error/var/lib/mysql/ecat.10gbin.com.err --open-files-limit8192 --pid-file/var/lib/mysql/ecat.10gbin.com.pid
    mysql 64.1 /usr/sbin/mysqld --basedir/ --datadir/var/lib/mysql --usermysql --log-error/var/lib/mysql/ecat.10gbin.com.err --open-files-limit8192 --pid-file/var/lib/mysql/ecat.10gbin.com.pid


    another top :-

    Code:
    root@ecat [~]# top -c
    top - 06:27:28 up 18:12,  1 user,  load average: 10.74, 11.85, 10.50
    Tasks: 198 total,   1 running, 197 sleeping,   0 stopped,   0 zombie
    Cpu(s):  5.4%us,  1.7%sy,  0.0%ni, 79.6%id, 12.9%wa,  0.1%hi,  0.3%si,  0.0%st
    Mem:  12289824k total, 11967816k used,   322008k free,   517812k buffers
    Swap: 12594940k total,      188k used, 12594752k free,  8252116k cached
    
      PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
     6641 mysql     10  -5 10.5g 2.3g 4924 S 50.9 19.5 499:53.38 /usr/sbin/mysqld --
    25090 nobody    15   0  549m  24m  13m S  1.7  0.2   0:00.16 lsphp5
      795 root      10  -5     0    0    0 D  0.7  0.0   1:03.64 [kjournald]
    25083 nobody    15   0  547m  34m  23m S  0.7  0.3   0:00.20 lsphp5
        1 root      15   0 10368  688  572 S  0.0  0.0   0:01.40 init [3]
        2 root      RT  -5     0    0    0 S  0.0  0.0   0:00.01 [migration/0]
        3 root      34  19     0    0    0 S  0.0  0.0   0:00.29 [ksoftirqd/0]
        4 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 [watchdog/0]
        5 root      RT  -5     0    0    0 S  0.0  0.0   0:00.06 [migration/1]
        6 root      34  19     0    0    0 S  0.0  0.0   0:00.01 [ksoftirqd/1]
        7 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 [watchdog/1]
        8 root      RT  -5     0    0    0 S  0.0  0.0   0:00.04 [migration/2]
        9 root      34  19     0    0    0 S  0.0  0.0   0:00.00 [ksoftirqd/2]
       10 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 [watchdog/2]
       11 root      RT  -5     0    0    0 S  0.0  0.0   0:00.01 [migration/3]
       12 root      34  19     0    0    0 S  0.0  0.0   0:00.00 [ksoftirqd/3]
       13 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 [watchdog/3]
    
    
    Last edited: Jan 11, 2012
  7. NiteWave

    NiteWave Administrator

    looks like the bottle-neck at io(12.9%wa) and mysql.
    how many CPU/cores of this server ?

    "top -c" is most helpful when the load is high.

    also lsws console->Real-time Stats when high-load is helpful.
  8. bhanuprasad1981

    bhanuprasad1981 New Member

    this is 8 physical core server with raid 0 running php 5.3.x and xcache issue raised past 5 days back only i cant understand whats going on
  9. bhanuprasad1981

    bhanuprasad1981 New Member

    running apache with latest 5.3.8 php mpm prefork,without any opcode cache server load is around 7-10 now, sites are running ok not fast as litespeed but better than 1 min page load time of litespeed here is top :-



    Code:
    top - 13:16:54 up 2 days,  1:02,  1 user,  load average: 6.91, 8.34, 10.24
    Tasks: 213 total,   8 running, 205 sleeping,   0 stopped,   0 zombie
    Cpu(s): 27.3%us,  6.8%sy,  0.0%ni, 50.3%id, 14.6%wa,  0.2%hi,  0.8%si,  0.0%st
    Mem:  12289824k total, 11957784k used,   332040k free,   340720k buffers
    Swap: 12594940k total,      188k used, 12594752k free,  8660456k cached
    
      PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
     7309 mysql     10  -5 10.5g 2.0g 4824 S 46.9 17.1 273:28.81 mysqld
    27702 nobody    17   0  116m  26m 5984 R  3.7  0.2   0:00.11 php
    27703 nobody    18   0  108m  19m 6136 R  3.7  0.2   0:00.11 php
    27704 nobody    18   0  106m  15m 5848 R  2.0  0.1   0:00.06 php
    22071 nobody    16   0  328m  11m 1872 S  1.7  0.1   0:01.85 httpd
     8004 nobody    16   0 71768 3348  900 S  1.3  0.0   0:24.95 httpd
    22147 nobody    15   0  326m  10m 1832 S  1.3  0.1   0:01.49 httpd
    22270 nobody    15   0  370m  10m 1828 S  1.0  0.1   0:01.54 httpd
    22296 nobody    15   0  296m  10m 1832 S  1.0  0.1   0:01.49 httpd
    24805 nobody    15   0  275m 9380 1828 S  1.0  0.1   0:00.83 httpd
    24032 nobody    16   0  326m  10m 1832 S  0.7  0.1   0:01.13 httpd
    27333 nobody    15   0  262m 6656 1800 S  0.7  0.1   0:00.11 httpd
    27705 nobody    18   0  101m  10m 5472 R  0.7  0.1   0:00.02 php
    27706 nobody    18   0  100m  10m 5464 R  0.7  0.1   0:00.02 php
    32203 nobody    18   0  121m  31m 6136 D  0.7  0.3   0:11.16 php
     2490 root      10  -5     0    0    0 S  0.3  0.0  15:08.80 kjournald
     9317 root      18   0  126m  25m 1568 S  0.3  0.2   0:28.40 lfd
    
    
  10. NiteWave

    NiteWave Administrator

    yes, this is not acceptable.

    can you post the real-time stats?

    maybe you can increase
    Server->General->PHP suEXEC Max Conn

    load time should be less than apache, otherwise abnormal.
  11. bhanuprasad1981

    bhanuprasad1981 New Member

    yes it is strange , i am not phpsuexec realtime stats are normal except one account which was showing some 20+ process everytime, even suspending that account doesnt increase page load time on litespeed , i suspect settings responsible for it, what you gusy think ? i am using latest version of litespeed.
  12. bhanuprasad1981

    bhanuprasad1981 New Member

    thanks for no reply i think i may need stay with apache on one server
  13. webizen

    webizen New Member

    sorry for missing this. pls pm your server temporary root access for us to look at further if you would like.
  14. bhanuprasad1981

    bhanuprasad1981 New Member

    not possible atm for privacy issues , i can attach screens what ever details you need
  15. NiteWave

    NiteWave Administrator

    please give screenshot of real-time stats when high-load

Share This Page