Optimization Request

Discussion in 'General' started by bhanuprasad1981, Jan 9, 2012.

  1. bhanuprasad1981

    bhanuprasad1981 Well-Known Member

    i am running server with 2 cpu license past some days load is too high tries a lot of configs but fail, please help me to find issue , let me know what details you need .
     
  2. webizen

    webizen Well-Known Member

    run 'top' first and see what process consumes the most resources/cpu cycles.
     
  3. bhanuprasad1981

    bhanuprasad1981 Well-Known Member

    here is top result :-

    Code:
    Tasks: 191 total,   1 running, 190 sleeping,   0 stopped,   0 zombie
    Cpu(s):  1.0%us,  1.0%sy,  0.0%ni, 86.5%id, 10.7%wa,  0.5%hi,  0.2%si,  0.0%st
    Mem:  12289824k total, 12075560k used,   214264k free,   409056k buffers
    Swap: 12594940k total,        0k used, 12594940k free,  9524700k cached
    
      PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
     6641 mysql     10  -5 10.5g 1.5g 4728 S  5.7 12.8  26:02.59 /usr/sbin/mysqld --
    11590 nobody    15   0  551m  31m  16m S  4.7  0.3   0:00.14 lsphp5
    11588 nobody    15   0  544m  22m  14m S  2.0  0.2   0:00.06 lsphp5
    11491 nobody    15   0  548m  87m  78m S  1.0  0.7   0:01.27 lsphp5
    11589 nobody    15   0  542m  14m 8796 S  0.7  0.1   0:00.02 lsphp5
     2490 root      10  -5     0    0    0 D  0.3  0.0   0:23.56 [kjournald]
        1 root      15   0 10368  688  572 S  0.0  0.0   0:01.37 init [3]
        2 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 [migration/0]
        3 root      34  19     0    0    0 S  0.0  0.0   0:00.01 [ksoftirqd/0]
        4 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 [watchdog/0]
        5 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 [migration/1]
        6 root      34  19     0    0    0 S  0.0  0.0   0:00.00 [ksoftirqd/1]
        7 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 [watchdog/1]
        8 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 [migration/2]
        9 root      34  19     0    0    0 S  0.0  0.0   0:00.00 [ksoftirqd/2]
       10 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 [watchdog/2]
       11 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 [migration/3]
    
     
  4. webizen

    webizen Well-Known Member

    you missing the first line (contains 'load average'). pls add back.
     
  5. bhanuprasad1981

    bhanuprasad1981 Well-Known Member

    top -c result during normal time not peak* during peak load touches 20+ and sites start to crawl.

    Code:
    root@ecat [~]# top -c
    top - 05:34:02 up 17:19,  1 user,  load average: 5.77, 6.10, 6.44
    Tasks: 194 total,   1 running, 193 sleeping,   0 stopped,   0 zombie
    Cpu(s):  2.7%us,  1.2%sy,  0.0%ni, 82.5%id, 13.3%wa,  0.2%hi,  0.2%si,  0.0%st
    Mem:  12289824k total, 11967808k used,   322016k free,   462640k buffers
    Swap: 12594940k total,   102876k used, 12492064k free,  8291120k cached
    
      PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
    20436 nobody    15   0  550m  98m  87m S  4.0  0.8   0:13.11 lsphp5
    16045 nobody    18   0  553m 131m 117m D  3.3  1.1   0:52.27 lsphp5:/home/cluefo
    20488 nobody    15   0  553m  97m  79m S  3.3  0.8   0:01.41 lsphp5
    20589 nobody    15   0  543m  39m  33m S  3.3  0.3   0:00.26 lsphp5
     6641 mysql     10  -5 10.5g 2.2g 4872 S  3.0 19.1 483:48.96 /usr/sbin/mysqld --
    20588 nobody    15   0  545m  51m  43m S  2.0  0.4   0:00.34 lsphp5
    20591 nobody    15   0  544m  31m  24m S  1.3  0.3   0:00.19 lsphp5
    20594 nobody    15   0  541m  19m  15m S  1.3  0.2   0:00.07 lsphp5
    20521 nobody    15   0  550m  91m  81m S  1.0  0.8   0:00.87 lsphp5
    20596 nobody    15   0  544m  18m  10m S  1.0  0.2   0:00.06 lsphp5
     2490 root      10  -5     0    0    0 D  0.7  0.0  11:55.34 [kjournald]
    20404 root      15   0 12760 1192  824 R  0.7  0.0   0:00.30 top -c
    20437 nobody    15   0  562m 121m 100m S  0.7  1.0   0:01.84 lsphp5
    20516 nobody    15   0  556m  82m  65m S  0.7  0.7   0:00.86 lsphp5
    13377 nobody     4 -19 63908  22m 9.9m D  0.3  0.2   0:23.16 litespeed (lshttpd)
    20593 nobody    15   0  543m  19m  12m S  0.3  0.2   0:00.07 lsphp5
        1 root      15   0 10368  688  572 S  0.0  0.0   0:01.40 init [3]
    
     
  6. bhanuprasad1981

    bhanuprasad1981 Well-Known Member

    cpanel daily process logs show :-

    mysql 73.0 /usr/sbin/mysqld --basedir/ --datadir/var/lib/mysql --usermysql --log-error/var/lib/mysql/ecat.10gbin.com.err --open-files-limit8192 --pid-file/var/lib/mysql/ecat.10gbin.com.pid
    mysql 64.1 /usr/sbin/mysqld --basedir/ --datadir/var/lib/mysql --usermysql --log-error/var/lib/mysql/ecat.10gbin.com.err --open-files-limit8192 --pid-file/var/lib/mysql/ecat.10gbin.com.pid


    another top :-

    Code:
    root@ecat [~]# top -c
    top - 06:27:28 up 18:12,  1 user,  load average: 10.74, 11.85, 10.50
    Tasks: 198 total,   1 running, 197 sleeping,   0 stopped,   0 zombie
    Cpu(s):  5.4%us,  1.7%sy,  0.0%ni, 79.6%id, 12.9%wa,  0.1%hi,  0.3%si,  0.0%st
    Mem:  12289824k total, 11967816k used,   322008k free,   517812k buffers
    Swap: 12594940k total,      188k used, 12594752k free,  8252116k cached
    
      PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
     6641 mysql     10  -5 10.5g 2.3g 4924 S 50.9 19.5 499:53.38 /usr/sbin/mysqld --
    25090 nobody    15   0  549m  24m  13m S  1.7  0.2   0:00.16 lsphp5
      795 root      10  -5     0    0    0 D  0.7  0.0   1:03.64 [kjournald]
    25083 nobody    15   0  547m  34m  23m S  0.7  0.3   0:00.20 lsphp5
        1 root      15   0 10368  688  572 S  0.0  0.0   0:01.40 init [3]
        2 root      RT  -5     0    0    0 S  0.0  0.0   0:00.01 [migration/0]
        3 root      34  19     0    0    0 S  0.0  0.0   0:00.29 [ksoftirqd/0]
        4 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 [watchdog/0]
        5 root      RT  -5     0    0    0 S  0.0  0.0   0:00.06 [migration/1]
        6 root      34  19     0    0    0 S  0.0  0.0   0:00.01 [ksoftirqd/1]
        7 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 [watchdog/1]
        8 root      RT  -5     0    0    0 S  0.0  0.0   0:00.04 [migration/2]
        9 root      34  19     0    0    0 S  0.0  0.0   0:00.00 [ksoftirqd/2]
       10 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 [watchdog/2]
       11 root      RT  -5     0    0    0 S  0.0  0.0   0:00.01 [migration/3]
       12 root      34  19     0    0    0 S  0.0  0.0   0:00.00 [ksoftirqd/3]
       13 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 [watchdog/3]
    
    
     
    Last edited: Jan 11, 2012
  7. NiteWave

    NiteWave Administrator

    looks like the bottle-neck at io(12.9%wa) and mysql.
    how many CPU/cores of this server ?

    "top -c" is most helpful when the load is high.

    also lsws console->Real-time Stats when high-load is helpful.
     
  8. bhanuprasad1981

    bhanuprasad1981 Well-Known Member

    this is 8 physical core server with raid 0 running php 5.3.x and xcache issue raised past 5 days back only i cant understand whats going on
     
  9. bhanuprasad1981

    bhanuprasad1981 Well-Known Member

    running apache with latest 5.3.8 php mpm prefork,without any opcode cache server load is around 7-10 now, sites are running ok not fast as litespeed but better than 1 min page load time of litespeed here is top :-



    Code:
    top - 13:16:54 up 2 days,  1:02,  1 user,  load average: 6.91, 8.34, 10.24
    Tasks: 213 total,   8 running, 205 sleeping,   0 stopped,   0 zombie
    Cpu(s): 27.3%us,  6.8%sy,  0.0%ni, 50.3%id, 14.6%wa,  0.2%hi,  0.8%si,  0.0%st
    Mem:  12289824k total, 11957784k used,   332040k free,   340720k buffers
    Swap: 12594940k total,      188k used, 12594752k free,  8660456k cached
    
      PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
     7309 mysql     10  -5 10.5g 2.0g 4824 S 46.9 17.1 273:28.81 mysqld
    27702 nobody    17   0  116m  26m 5984 R  3.7  0.2   0:00.11 php
    27703 nobody    18   0  108m  19m 6136 R  3.7  0.2   0:00.11 php
    27704 nobody    18   0  106m  15m 5848 R  2.0  0.1   0:00.06 php
    22071 nobody    16   0  328m  11m 1872 S  1.7  0.1   0:01.85 httpd
     8004 nobody    16   0 71768 3348  900 S  1.3  0.0   0:24.95 httpd
    22147 nobody    15   0  326m  10m 1832 S  1.3  0.1   0:01.49 httpd
    22270 nobody    15   0  370m  10m 1828 S  1.0  0.1   0:01.54 httpd
    22296 nobody    15   0  296m  10m 1832 S  1.0  0.1   0:01.49 httpd
    24805 nobody    15   0  275m 9380 1828 S  1.0  0.1   0:00.83 httpd
    24032 nobody    16   0  326m  10m 1832 S  0.7  0.1   0:01.13 httpd
    27333 nobody    15   0  262m 6656 1800 S  0.7  0.1   0:00.11 httpd
    27705 nobody    18   0  101m  10m 5472 R  0.7  0.1   0:00.02 php
    27706 nobody    18   0  100m  10m 5464 R  0.7  0.1   0:00.02 php
    32203 nobody    18   0  121m  31m 6136 D  0.7  0.3   0:11.16 php
     2490 root      10  -5     0    0    0 S  0.3  0.0  15:08.80 kjournald
     9317 root      18   0  126m  25m 1568 S  0.3  0.2   0:28.40 lfd
    
    
     
  10. NiteWave

    NiteWave Administrator

    yes, this is not acceptable.

    can you post the real-time stats?

    maybe you can increase
    Server->General->PHP suEXEC Max Conn

    load time should be less than apache, otherwise abnormal.
     
  11. bhanuprasad1981

    bhanuprasad1981 Well-Known Member

    yes it is strange , i am not phpsuexec realtime stats are normal except one account which was showing some 20+ process everytime, even suspending that account doesnt increase page load time on litespeed , i suspect settings responsible for it, what you gusy think ? i am using latest version of litespeed.
     
  12. bhanuprasad1981

    bhanuprasad1981 Well-Known Member

    thanks for no reply i think i may need stay with apache on one server
     
  13. webizen

    webizen Well-Known Member

    sorry for missing this. pls pm your server temporary root access for us to look at further if you would like.
     
  14. bhanuprasad1981

    bhanuprasad1981 Well-Known Member

    not possible atm for privacy issues , i can attach screens what ever details you need
     
  15. NiteWave

    NiteWave Administrator

    please give screenshot of real-time stats when high-load
     

Share This Page