whatever popen does isnt working in a rails app

I

inspirix

Active Member
#1
#1
Not sure if this is a bug or something that just needs configuration tweaks to work correctly, I'll do my best to explain:

Im running into a Broken Pipe issue with the following code running on 2.2.5. The same thing under Mongrel works fine, so I assume there is something I can do to get this to work under LS, but just dont know what that would be. The broken pipe error happens extremely quick, so i believe the case of the external process taking too long is ruled out.

Summary: The run method executes an external process (htmldoc), feeds it some text, then should close the write process and send the PDF back.

My guess: I'll take a stab and say I think it works in Mongrel because mongrel is one ongoing process that obtains the ability to fork/lock/etc once and has that ability thereafter, where as LS works differently?

It wont work for any external process as far as I can tell (whatever popen does, doesnt work in a LSAPI instance maybe?), but since I have this code already, Im using htmldoc as the example.

Here is the code and the error:

Code: 
class StuffController < ApplicationController

	def run
	  generator = IO.popen('htmldoc -t pdf14 --webpage -', 'w+')
	  generator.puts render_to_string("stuff/_preview")          #LINE 53 in my actual controller
	  generator.close_write
	  send_data generator.read, :filename => "preview.pdf", :type => 'application/pdf', :disposition => 'inline'
	end

end

Errno::EPIPE (Broken pipe):
    /app/controllers/stuff_controller.rb:53:in `write'
    /app/controllers/stuff_controller.rb:53:in `run'
    /usr/local/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/base.rb:941:in `perform_action_without_filters'
    /usr/local/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/filters.rb:368:in `perform_action_without_benchmark'
    /usr/local/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/benchmarking.rb:69:in `perform_action_without_rescue'
    /usr/local/lib/ruby/1.8/benchmark.rb:293:in `measure'
    /usr/local/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/benchmarking.rb:69:in `perform_action_without_rescue'
    /usr/local/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/rescue.rb:82:in `perform_action'
    /usr/local/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/base.rb:408:in `process_without_filters'
    /usr/local/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/filters.rb:377:in `process_without_session_management_support'
    /usr/local/lib/ruby/gems/1.8/gems/actionpack-1.12.5/lib/action_controller/session_management.rb:117:in `process'
    /usr/local/lib/ruby/gems/1.8/gems/rails-1.1.6/lib/dispatcher.rb:38:in `dispatch'
    /usr/local/lsws/fcgi-bin/RailsRunner.rb:12
 
I

inspirix

Active Member
#2
#2
Ok, so I just found a solution (in the form of a workaround, but it seems like a GOOD workaround, this 'session' gem seems great so far):

This seems to be working great (so far):

require the session gem and:

Code: 
	def run
		cmd = "htmldoc -t pdf14 --webpage -"
		text = render_to_string("stuff/_preview")
		se = Session.new
		result, err = se.execute cmd, :stdin => text
		send_data result, :filename => "preview.pdf", :type => 'application/pdf', :disposition => 'inline'
	end
I still dont get whats wrong with popen itself, but it seems like this may be a preferred anyway as it looks like the session gem is handling all of the possible issues that could arise.

So I currently have no issue, but am still a bit curious on why popen doesnt work if anyone has a guess.

Joe
 
M

mistwang

LiteSpeed Staff
#3
#3
I did a simple IO.popen() test, and it works. Maybe it has something to do with your application. If you really want to find out why, you can try "strace" the ruby process that executes IO.popen().

Aother possibility is that the child process created by IO.popen() dead for some reason, you can try strace that process as well. :)
 
M

mistwang

LiteSpeed Staff
#4
#4
More finding.

If "generator.close" is not called at the end, there will be a new zombie process for each request, since LiteSpeed set the "Number of Process Limit", soon the limit will be reached, and 'popen' will fail. Maybe that is the reason. :)
 
J

jmuollo

New Member
#5
#5
Check HTMLDOC_NOCGI environment variable

I had this problem in a PRODUCTION environment because I hadn't set the HTMLDOC_NOCGI environment variable.

Htmldoc was outputting an info message and exiting, causing the broken pipe.

The info message was:

HTMLDOC Version 1.8.27 Copyright 1997-2006 Easy Software Products, All Rights Reserved.
This software is based in part on the work of the Independent JPEG Group.

HTMLDOC is running in CGI mode. To disable CGI mode when running
from a server-side script/page, set the HTMLDOC_NOCGI environment
variable prior to running HTMLDOC.

If you are trying to use CGI mode, make sure that the ServerName
for the web server is accessible from the local system. If you
are using Apache 2.0.30 or later, make sure you set 'AcceptPathInfo'
to 'On' for the HTMLDOC/cgi-bin directory.

To fix this I did the following:

Code: 
      htmldoc_env = "HTMLDOC_NOCGI=TRUE;export HTMLDOC_NOCGI"
         
      generator = IO.popen("#{htmldoc_env};htmldoc -t pdf14 --webpage -", "w+")
 
Last edited:
O

ostraaten

Member
#7
#7
Also broke my pipe

Hi,
I'm experiencing a similar problem.

I'm also using the session gem. My code is working fine on my laptop Ubuntu intrepid, I'm using the built WebRick webserver.

I deploy to a web server running SLES and LSWS standard edition 3.3.22.

Reproducing the error is simple enough with a piece of code like this

def tags
bash = Session::Bash.new
bash_out, bash_err = bash.execute 'cd /home/user/epf_libraries'
end

I call this method from my controller and then I get a 500 internal server error in the browser. In the log file is the broken pipe. Request immediately after the the request that calls my method tags also get an error that "MySQL has gone away". This goes away after a few request and everything is fine again.

Any ideas or suggestions on this would be very much appreciated as I really don't have a clue why the session gem would fail on such a simple command.

Thanks and Regards,
Onno



ActionView::TemplateError (Broken pipe) on line #198 of app/views/sites/description.rhtml:
195: page.visual_effect :toggle_appear, "#{c.name}_tags"
196: end %>
197: <ul id="<%= c.name %>_tags" style="display:none;" >
198: <% c.tags.each do |tag| %>
199: <li><%= link_to tag, {:action => 'update_cvs', :tag => tag, :configuration_id => c.id, :id => @site.id},
200: :confirm => "Update Wiki '#{@site.title}' with configuration '#{c.name}' (#{c.relative_path}) using tag '#{tag}'?", :method => :post %></li>
201: <% end %>

/usr/lib/ruby/gems/1.8/gems/session-2.4.0/lib/session.rb:503:in `write'
/usr/lib/ruby/gems/1.8/gems/session-2.4.0/lib/session.rb:503:in `puts'
/usr/lib/ruby/gems/1.8/gems/session-2.4.0/lib/session.rb:503:in `clear'
/usr/lib/ruby/gems/1.8/gems/session-2.4.0/lib/session.rb:358:in `execute'
/usr/lib/ruby/gems/1.8/gems/session-2.4.0/lib/session.rb:612:in `execute'
app/models/cvs_configuration.rb:125:in `tags'
app/views/sites/description.rhtml:198
app/views/sites/description.rhtml:193:in `each'
app/views/sites/description.rhtml:193
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_view/renderable.rb:39:in `send'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_view/renderable.rb:39:in `render'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_view/template.rb:73:in `render_template'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_view/base.rb:256:in `render'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_view/base.rb:367:in `_render_with_layout'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_view/base.rb:254:in `render'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/base.rb:1174:in `render_for_file'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/base.rb:896:in `render_without_benchmark'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/benchmarking.rb:51:in `render'
/usr/lib/ruby/gems/1.8/gems/activesupport-2.2.2/lib/active_support/core_ext/benchmark.rb:8:in `realtime'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/benchmarking.rb:51:in `render'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/base.rb:868:in `render_without_benchmark'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/benchmarking.rb:51:in `render'
/usr/lib/ruby/gems/1.8/gems/activesupport-2.2.2/lib/active_support/core_ext/benchmark.rb:8:in `realtime'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/benchmarking.rb:51:in `render'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/base.rb:1248:in `default_render'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/base.rb:1254:in `perform_action_without_filters'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/filters.rb:617:in `call_filters'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/filters.rb:638:in `run_before_filters'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/filters.rb:189:in `call'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/filters.rb:189:in `call'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/filters.rb:635:in `run_before_filters'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/filters.rb:615:in `call_filters'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/filters.rb:610:in `perform_action_without_benchmark'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/benchmarking.rb:68:in `perform_action_without_rescue'
/usr/lib/ruby/1.8/benchmark.rb:293:in `measure'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/benchmarking.rb:68:in `perform_action_without_rescue'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/rescue.rb:136:in `perform_action_without_caching'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/caching/sql_cache.rb:13:in `perform_action'
/usr/lib/ruby/gems/1.8/gems/activerecord-2.2.2/lib/active_record/connection_adapters/abstract/query_cache.rb:34:in `cache'
/usr/lib/ruby/gems/1.8/gems/activerecord-2.2.2/lib/active_record/query_cache.rb:8:in `cache'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/caching/sql_cache.rb:12:in `perform_action'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/base.rb:524:in `send'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/base.rb:524:in `process_without_filters'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/filters.rb:606:in `process_without_session_management_support'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/session_management.rb:134:in `process'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/base.rb:392:in `process'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/dispatcher.rb:183:in `handle_request'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/dispatcher.rb:110:in `dispatch_unlocked'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/dispatcher.rb:123:in `dispatch'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/dispatcher.rb:122:in `synchronize'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/dispatcher.rb:122:in `dispatch'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/dispatcher.rb:132:in `dispatch_cgi'
/usr/lib/ruby/gems/1.8/gems/actionpack-2.2.2/lib/action_controller/dispatcher.rb:39:in `dispatch'
/opt/lsws/fcgi-bin/RailsRunner.rb:28

User Load (0.0ms) Mysql::Error: MySQL server has gone away: SELECT * FROM `users` WHERE (admin='C') LIMIT 1
Mysql::Error: MySQL server has gone away: SELECT * FROM `users` WHERE (admin='C') LIMIT 1
Redirected to actionerrorcontrollerother


Processing SitesController#description (for 83.87.70.69 at 2009-03-26 16:26:40) [GET]
Session ID: 43844b9ef477c425ddc6041078df05b4
Parameters: {"id"=>"2"}


ActiveRecord::StatementInvalid (Mysql::Error: MySQL server has gone away: UPDATE `sessions` SET `data` = 'BAh7CCIJdXNlcm86CVVzZXIUOhZAYXR0cmlidXRlc19jYWNoZXsAOg5AY29t\nbWVudHMwOgpAc2l0ZTA6C0BzaXRlczA6DUB1cGxvYWRzMDoQQGF0dHJpYnV0\nZXN7GiIdbG9nb25fdXNpbmdfY29va2llX2NvdW50aQgiD2NyZWF0ZWRfb24i\b250cm9sbGVyOjpGbGFzaDo6Rmxhc2hIYXNoewciCmVycm9yIhBCcm9rZW4g\ncGlwZSILbm90aWNlInVXZSdyZSBzb3JyeSwgYnV0IHNvbWV0aGluZyB3ZW50\nIHdyb25nLiBXZSd2ZSBiZWVuIG5vdGlmaWVkIGFib3V0IHRoaXMgaXNzdWUg\nYW5kIHdlJ2xsIHRha2UgYSBsb29rIGF0IGl0IHNob3J0bHkuBjoKQHVzZWR7\nB0A0RkA2Rg==\n', `updated_at` = '2009-03-26 16:26:40' WHERE `id` = 11):
/usr/lib/ruby/gems/1.8/gems/activerecord-2.2.2/lib/active_record/connection_adapt
bash = Session::Bash.new
bash_out, bash_err = bash.execute cmdline
 
You must log in or register to reply here.
Top