Run "vmstat" and check for swap in/out columns to make sure that your OS is not hitting the swapping partition like mad which would kill your performance. This can happen if you overallocate too much memory than available to mysql.
If vmstat command is not available, install the "yum install sysstat" package.
Make sure that mysql's my.conf threadcache value is not too high and not to low and just enough to that the "new threads created" vaule within mysql "show variables" collection is not increasing much if at all. Proper mysql threadcache level can make a huge different by minimizing constant thread creation, memory allocation, which can lead to swapping on overburdened servers.