Edit Question - KnowledgeHub

Title Body

I have a Rails app that processes a large (millions) number of records in a mysql database. Once it starts working, its memory use quickly grows at a speed of 50MB per second. With tools like oink I was able to narrow down the root cause to one loop that goes through all the records in a big table in the database. I understand if I use something like Person.all.each , all the records will be loaded into memory. However if I switch to find_each , I still see the same memory issue. To further isolate the problem I created the following test controller, which does nothing but looping through the records. I suppose find_each only keeps a small number of objects in memory each time, but memory use grows linearly as it executes. class TestController < ApplicationController def memory_test Person.find_each do |person| end end I suspect it has to do with ActiveRecord caching the query results. But I checked my environment settings and I do have all the caching related options set to false in development (I am using the default settings created by rails). I did some search online but couldn't find a solution. I am using rails 3.1.0 rc1 and ruby 1.9.2 Thanks!

Tags (comma-separated) Cancel