Standard 2.2.5 + LSAPI + Rails uses up all processes

Discussion in 'Bug Reports' started by dirkbj, Nov 26, 2006.

  1. dirkbj

    dirkbj New Member

    I saw this bug on earlier versions - 2.2.3 with an LSAPI Ruby module earlier to 1.11. I am still seeing this bug running 2.2.5 with the latest LSAPI Ruby module (1.11)

    Here are the details:

    1) OS type and Version:

    Mac OS X Server 10.4.8

    2) LiteSpeed Version:

    2.2.5 Standard + LSAPI Ruby Module 1.11

    3) If PHP related:

    Not PHP related

    4) All possible related info that you observed before and after, the bug occurred:

    The site handles a Rails application. It runs great for days at a time, but then, for no apparant reason, it spawns a bunch of ruby threads and takes up all system resources. Our fix is to restart the server. We see no general pattern - it seems to occur when the site is busy or when it is sitting idle.

    5) If you can reproduce the bug, please detail what the steps are:

    Just let it run.

    6) Note any errors logged in error.log and stderr.log of LiteSpeed logs directory:

    No errors were logged.

    7) Check /tmp/lshttpd for "core dumps"/core files:

    No core dumps
  2. mistwang

    mistwang LiteSpeed Staff

    Maybe the ruby processes gets into a state that cannot be killed by normal SIGTERM signal.

    Can you "truss" or "ktrace" those ruby processes to check the state of those hanging processes.

    It would be best if we can logon to your server to check though ruby processes when it happenes.
  3. dirkbj

    dirkbj New Member

    If this were true, I believe we would see ruby processes building over time. This is not the case. The ruby process count stays generally constant over a period of days and then all-of-a-sudden grows out of control.

    I will watch and try to get a trace the next time it happens.

    Login access to our server, unfortunately, is not an option.

    Dirk
  4. mistwang

    mistwang LiteSpeed Staff

    Also, please make sure the latest ruby-lsapi has been used. If you have installed it manually, then switch to gem installation, you need to remove the file installed manually.
  5. dirkbj

    dirkbj New Member

    Yes, I am and have been running the latest - installed with ruby gems.

    Dirk
  6. admin

    admin Administrator Staff Member

    What is your rails settings within litespeed? If you set instances/children value too high and a search crawler/bot comes to hammer your site on a periodic basis, it can send your server resource skyrocketing if the backend cannot support the number of simultaneous script connections.

    Since we do not know the exact rail script that's causing this or detailed parameters of the incident, it would be extremely difficult to isolate/replicate on our end right now. Hopefully we can get more answers once we have some trace data if and when this happens again.
  7. fantasydreaming

    fantasydreaming New Member

    My similar problem to this was caused by a search using the ruby-google module... apparently http-access and http-access2 can both get stuck waiting 'forever' for a google (or anywhere, likely) response.

    The solution for me was to wrap it in a Timeout::timeout(5) do() block.
  8. mistwang

    mistwang LiteSpeed Staff

    fantasydreaming,

    Thank you for sharing your finding with us. It is really helpful.

    We may improve the ruby-lsapi implementation to somehow kill those hanging processes when we got a chance. :)

Share This Page