How to fix hanging popen3 in Ruby?

别来无恙 提交于 2019-12-29 05:19:45

问题


I am getting unexpected behaviour using popen3, which I want to use to run a command like tool ala cmd < file1 > file2. The below example hangs, so that stdout done is never reached. Using other tools than cat may cause hanging, so that stdin done is never reached. I suspect, I am suffering from buffering, but how do I fix this?

#!/usr/bin/env ruby

require 'open3'

Open3.popen3("cat") do |stdin, stdout, stderr, wait_thr|
  stdin.puts "foobar"

  puts "stdin done"

  stdout.each_line { |line| puts line }

  puts "stdout done"

  puts wait_thr.value
end

puts "all done"

回答1:


stdout.each_line is waiting for further output from cat because cat's output stream is still open. It's still open because cat is still waiting for input from the user because its input stream hasn't been closed yet (you'll notice that when you open cat in a terminal and type in foobar, it will still be running and waiting for input until you press ^d to close the stream).

So to fix this, simply call stdin.close before you print the output.




回答2:


Your code is hanging, because stdin is still open!

You need to close it with IO#close or with IO#close_write if you use popen3.

If you use popen then you need to use IO#close_write because it only uses one file descriptor.

 #!/usr/bin/env ruby
 require 'open3'

 Open3.popen3("cat") do |stdin, stdout, stderr, wait_thr|
   stdin.puts "foobar"

   stdin.close   # close stdin like this!  or with stdin.close_write

   stdout.each_line { |line| puts line }

   puts wait_thr.value
 end

See also:

Ruby 1.8.7 IO#close_write

Ruby 1.9.2 IO#close_write

Ruby 2.3.1 IO#close_write




回答3:


The answers by Tilo and by sepp2k are right: If you close stdin, your simple test will end. Problem solved.

Though in your comment to the answer of sepp2k, you indicate that you still experience hangs. Well, there are some traps that you might have overlooked.

Stuck on full buffer for stderr

If you call a program that prints more to stderr than the buffer of an anonymous pipe can hold (64KiB for current Linuxes), the program gets suspended. A suspended program neither exits nor closes stdout. Consequently, reading from its stdout will hang. So if you want to do it right, you have to use threads or IO.select, non-blocking, unbuffered reads in order to read from both stdout and stderr in parallel or by turns without getting stuck.

Stuck on full buffer for stdin

If you try to feed more (much more) than "foobar" to your program (cat), the buffer of the anonymous pipe for stdout will get full. The OS will suspend cat. If you write even more to stdin, the buffer of the anonymous pipe for stdin will get full. Then your call to stdin.write will get stuck. This means: You need to write to stdin, read from stdout and read from stderr in parallel or by turns.

Conclusion

Read a good book (Richards Stevens, "UNIX Network Programming: Interprocess communications") and use good library functions. IPC (interprocess communications) is just too complicated and prone to indeterministic run-time behavior. It is for too much hassle to try to get it right by try-and-error.

Use Open3.capture3.



来源:https://stackoverflow.com/questions/8952043/how-to-fix-hanging-popen3-in-ruby

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!