A different Error 503 problem

Discussion in 'Ruby/Rails' started by lanfusion, Mar 5, 2008.

  1. lanfusion

    lanfusion New Member

    I am new to LSWS. I installed it last night after experiencing performance issues issues with Lighttpd and Ruby on Rails. All of my sites work except http://community.lanfusion.com, a social networking site built in RoR. http://community.lanfusion.com was working fine with Lighttpd and works with Webbrick.

    After following the instructions found in this thread: http://www.litespeedtech.com/support/forum/showthread.php?t=1845, I'm still having a 503 issue. Here is some vital info:

    Excerpt from /usr/local/lsws/logs/error.log

    2008-03-05 16:05:13.254 [INFO] [66.249.66.12:44591-0#community.lanfusion.com] connection to [uds://tmp/lshttpd/community.lanfusion.com:_.sock] on request #0, error: Connection reset by peer!
    2008-03-05 16:05:17.503 [INFO] [66.249.66.12:44591-0#community.lanfusion.com] connection to [uds://tmp/lshttpd/community.lanfusion.com:_.sock] on request #0, error: Connection reset by peer!
    2008-03-05 16:05:23.587 [INFO] [66.249.66.12:44591-0#community.lanfusion.com] connection to [uds://tmp/lshttpd/community.lanfusion.com:_.sock] on request #0, error: Connection reset by peer!
    2008-03-05 16:05:26.819 [NOTICE] [66.249.66.12:44591-0#community.lanfusion.com] oops! 503 Service Unavailable
    2008-03-05 16:05:26.819 [NOTICE] [66.249.66.12:44591-0#community.lanfusion.com] Content len: 0, Request line:
    GET /topic/daily/Lookin%20for%20some%20Software?date=Feb282008 HTTP/1.1
    2008-03-05 16:05:26.819 [NOTICE] [66.249.66.12:44591-0#community.lanfusion.com] Redirect: #2, URL: /dispatch.lsapi

    Excerpt from stderr.log (these three lines are added each time I restart the server)

    [FATAL] failed to allocate memory
    [FATAL] failed to allocate memory
    [FATAL] failed to allocate memory

    [​IMG]

    Any help would be greatly appreciated.

    Dan Matthews - LANFusion LLC - www.lanfusion.com
  2. mistwang

    mistwang LiteSpeed Staff

    set "Memory Soft/hard Limit" to "500M" or above.
  3. lanfusion

    lanfusion New Member

    Updated... still having the problem.

    [​IMG]

    2008-03-05 16:44:37.897 [INFO] [Rails:community.lanfusion.com:/] add child process pid: 9862
    2008-03-05 16:44:37.897 [INFO] [Rails:community.lanfusion.com:/] pid list size: 1
    2008-03-05 16:46:14.735 [INFO] [66.249.66.12:58218-0#community.lanfusion.com] connection to [uds://tmp/lshttpd/community.lanfusion.com:_.sock] on request #0, error: Connection reset by peer!
    2008-03-05 16:46:26.419 [INFO] [66.249.66.12:58218-0#community.lanfusion.com] connection to [uds://tmp/lshttpd/community.lanfusion.com:_.sock] on request #0, error: Connection reset by peer!
    2008-03-05 16:46:39.416 [INFO] [66.249.66.12:58218-0#community.lanfusion.com] connection to [uds://tmp/lshttpd/community.lanfusion.com:_.sock] on request #0, error: Connection reset by peer!
    2008-03-05 16:46:39.416 [NOTICE] [66.249.66.12:58218-0#community.lanfusion.com] oops! 503 Service Unavailable
    2008-03-05 16:46:39.416 [NOTICE] [66.249.66.12:58218-0#community.lanfusion.com] Content len: 0, Request line:
    GET /topic/classic/Posting?page=1 HTTP/1.1
    2008-03-05 16:46:39.416 [NOTICE] [66.249.66.12:58218-0#community.lanfusion.com] Redirect: #2, URL: /dispatch.lsapi
  4. mistwang

    mistwang LiteSpeed Staff

    Please set the process soft/hard limit as well, maybe to ">200".
  5. lanfusion

    lanfusion New Member

    [​IMG]

    Now the application just hangs. After around 5 minutes I get a "500 Internal Server Error" page.
  6. mistwang

    mistwang LiteSpeed Staff

    Please check the production.log and lsws/logs/stderr.log.
    If the ruby process is running, do

    "strace -f -p <pid_of_ruby>"
  7. lanfusion

    lanfusion New Member

    nothing in the stderr.log file

    this is the production log:

    # Logfile created on Wed Mar 05 17:30:44 -0500 2008

    DRb::DRbConnError (druby://localhost:9010 - #<Errno::ECONNREFUSED: Connection refused - connect(2)>):
    /usr/lib/ruby/1.8/drb/drb.rb:736:in `open'
    /usr/lib/ruby/1.8/drb/drb.rb:729:in `each'
    /usr/lib/ruby/1.8/drb/drb.rb:729:in `open'
    /usr/lib/ruby/1.8/drb/drb.rb:1189:in `initialize'
    /usr/lib/ruby/1.8/drb/drb.rb:1169:in `new'
    /usr/lib/ruby/1.8/drb/drb.rb:1169:in `open'
    /usr/lib/ruby/1.8/drb/drb.rb:1085:in `method_missing'
    /usr/lib/ruby/1.8/drb/drb.rb:1103:in `with_friend'
    /usr/lib/ruby/1.8/drb/drb.rb:1084:in `method_missing'
    /usr/lib64/ruby/gems/1.8/gems/acts_as_ferret-0.4.3/lib/remote_index.rb:16:in `send'
    /usr/lib64/ruby/gems/1.8/gems/acts_as_ferret-0.4.3/lib/remote_index.rb:16:in `method_missing'
    /usr/lib64/ruby/gems/1.8/gems/acts_as_ferret-0.4.3/lib/act_methods.rb:189:in `acts_as_ferret'
    /app/models/user.rb:31
    /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in `gem_original_require'
    /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in `require'
    /usr/lib64/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:496:in `require'
    /usr/lib64/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:342:in `new_constants_in'
    /usr/lib64/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:496:in `require'
    /usr/lib64/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:104:in `require_or_load'
    /usr/lib64/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:248:in `load_missing_constant'
    /usr/lib64/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:453:in `const_missing'
    /usr/lib64/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:465:in `const_missing'
    /usr/lib64/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:260:in `load_missing_constant'
    /usr/lib64/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:469:in `const_missing'
    /app/controllers/application.rb:37:in `authorize'
    /usr/lib64/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/filters.rb:469:in `send!'
    /usr/lib64/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/filters.rb:469:in `call'
    /usr/lib64/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/filters.rb:441:in `run'
    /usr/lib64/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/filters.rb:716:in `run_before_filters'
    /usr/lib64/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/filters.rb:695:in `call_filters'
    /usr/lib64/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/filters.rb:689:in `perform_action_without_benchmark'
    /usr/lib64/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/benchmarking.rb:68:in `perform_action_without_rescue'
    /usr/lib/ruby/1.8/benchmark.rb:293:in `measure'
    /usr/lib64/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/benchmarking.rb:68:in `perform_action_without_rescue'
    /usr/lib64/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/rescue.rb:199:in `perform_action_without_caching'
    /usr/lib64/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/caching.rb:678:in `perform_action'
    /usr/lib64/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/abstract/query_cache.rb:33:in `cache'
    /usr/lib64/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/query_cache.rb:8:in `cache'
    /usr/lib64/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/caching.rb:677:in `perform_action'
    /usr/lib64/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/base.rb:524:in `send'
    /usr/lib64/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/base.rb:524:in `process_without_filters'
    /usr/lib64/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/filters.rb:685:in `process_without_session_management_support'
    /usr/lib64/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/session_management.rb:123:in `process'
    /usr/lib64/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/base.rb:388:in `process'
    /usr/lib64/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/dispatcher.rb:171:in `handle_request'
    /usr/lib64/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/dispatcher.rb:115:in `dispatch'
    /usr/lib64/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/dispatcher.rb:126:in `dispatch_cgi'
    /usr/lib64/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/dispatcher.rb:9:in `dispatch'
    /usr/local/lsws/fcgi-bin/RailsRunner.rb:24

    LoadError (Expected /var/www/domains/community.lanfusion.com/app/models/user.rb to define User):
    /usr/lib64/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:249:in `load_missing_constant'
    /usr/lib64/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:453:in `const_missing'
    /usr/lib64/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:465:in `const_missing'
    /usr/lib64/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:260:in `load_missing_constant'
    /usr/lib64/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:469:in `const_missing'
    /app/controllers/application.rb:37:in `authorize'


    that's all unique... then this block of log repeats infinitely:

    LoadError (Expected /var/www/domains/community.lanfusion.com/app/models/user.rb to define User):
    /usr/lib64/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:249:in `load_missing_constant'
    /usr/lib64/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:453:in `const_missing'
    /usr/lib64/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:465:in `const_missing'
    /usr/lib64/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:260:in `load_missing_constant'
    /usr/lib64/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:469:in `const_missing'
    /app/controllers/application.rb:37:in `authorize'
  8. mistwang

    mistwang LiteSpeed Staff

    That's ruby on Rails errors. Your guys are RoR developer, should be able to fix them one by one. ;)
  9. lanfusion

    lanfusion New Member

    Except the app works fine in webbrick and lighttpd. Only with LiteSpeed does it give me these errors.

    Here it is running on the Webbrick server: http://secure1.lanfusion.net:3000/
  10. mistwang

    mistwang LiteSpeed Staff

    The difference between litespeed and others are the stripped environment variables and one ruby process fork children processes to handle all the requests.
    The children process will inherit all file handles from parent. I don't know exactly what is DRb, but looks like it need to connect to a local TCP port for some services, it could be a problem when children inherit the file handle for that connection. So, you need to find a way to close the DRb connection established during initializing Rails framework before parent fork the child process.

    Like what we did for the ActiveRecord connections in RailsRunner.rb
    Code:
    #Close all DB connections established during initialization
    ActiveRecord::Base.clear_active_connections! if defined?(ActiveRecord::Base)
    You need to figure out how to do that to the DRb connection.
  11. lanfusion

    lanfusion New Member

    Thanks for the help. I'll have my lead developer look into it.
  12. mistwang

    mistwang LiteSpeed Staff

    Please keep us posted if you find a solution. :)

Share This Page