Edit Question - KnowledgeHub

Title Body

I have some Python code that executes an external app which works fine when the app has a small amount of output, but hangs when there is a lot. My code looks like: p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) errcode = p.wait() retval = p.stdout.read() errmess = p.stderr.read() if errcode: log.error('cmd failed <%s>: %s' % (errcode,errmess)) There are comments in the docs that seem to indicate the potential issue. Under wait, there is: Warning: This will deadlock if the child process generates enough output to a stdout or stderr pipe such that it blocks waiting for the OS pipe buffer to accept more data. Use communicate() to avoid that. though under communicate, I see: Note The data read is buffered in memory, so do not use this method if the data size is large or unlimited. So it is unclear to me that I should use either of these if I have a large amount of data. They don't indicate what method I should use in that case. I do need the return value from the exec and do parse and use both the stdout and stderr . So what is an equivalent method in Python to exec an external app that is going to have large output?

Tags (comma-separated) Cancel