Edit Question - KnowledgeHub

Title Body

I have to insert 8000+ records into a SQLite database using Django's ORM. This operation needs to be run as a cronjob about once per minute. At the moment I'm using a for loop to iterate through all the items and then insert them one by one. Example: for item in items: entry = Entry(a1=item.a1, a2=item.a2) entry.save() What is an efficient way of doing this? Edit: A little comparison between the two insertion methods. Without commit_manually decorator (11245 records): nox@noxdevel marinetraffic]$ time python manage.py insrec real 1m50.288s user 0m6.710s sys 0m23.445s Using commit_manually decorator (11245 records): [nox@noxdevel marinetraffic]$ time python manage.py insrec real 0m18.464s user 0m5.433s sys 0m10.163s Note: The test script also does some other operations besides inserting into the database (downloads a ZIP file, extracts an XML file from the ZIP archive, parses the XML file) so the time needed for execution does not necessarily represent the time needed to insert the records.

Tags (comma-separated) Cancel