KnowledgeHub
Questions
Tags
Users
Search
Alex Rivera
|
Logout
Edit Question
Title
Body
In py2.6+, the multiprocessing module offers a Pool class, so one can do: class Volatile(object): def do_stuff(self, ...): pool = multiprocessing.Pool() return pool.imap(...) However, with the standard Python implementation at 2.7.2, this approach soon leads to "IOError: [Errno 24] Too many open files". Apparently the pool object never gets garbage collected, so its processes never terminate, accumulating whatever descriptors are opened internally. I think this because the following works: class Volatile(object): def do_stuff(self, ...): pool = multiprocessing.Pool() result = pool.map(...) pool.terminate() return result I would like to keep the "lazy" iterator approach of imap ; how does the garbage collector work in that case? How to fix the code?
Tags (comma-separated)
Save Edits
Cancel