I have a memory intensive Python application (between hundreds of MB to several GB).
I have a couple of VERY SMALL Linux executables the main application needs to run, e.g.
child = Popen("make html", cwd = r'../../docs', stdout = PIPE, shell = True)
child.wait()
When I run these external utilities (once, at the end of the long main process run) using subprocess.Popen I sometimes get OSError: [Errno 12] Cannot allocate memory.
I don't understand why... The requested process is tiny!
The system has enough memory for many more shells.
I'm using Linux (Ubuntu 12.10, 64 bits), so I guess subprocess calls Fork.
And Fork forks my existing process, thus doubling the amount of memory consumed, and fails??
What happened to "copy on write"?
Can I spawn a new process without fork (or at least without copying memory - starting fresh)?
Related:
The difference between fork(), vfork(), exec() and clone()