Alex Rivera | Logout

Splitting a large XML file in Python

Asked 2009-01-25T00:22:01.163
11

I'm looking to split a huge XML file into smaller bits. I'd like to scan through the file looking for a specific tag, then grab all info between and , then save that into a file, then continue on through the rest of the file.

My issue is trying to find a clean way to note the start and end of the tags, so that I can grab the text inside as I scan through the file with "for line in f"

I'd rather not use sentinel variables. Is there a pythonic way to get this done?

The file is too big to read into memory.

python xml
Edit
Report

1 Answer

9

There are two common ways to handle XML data.

One is called DOM, which stands for Document Object Model. This style of XML parsing is probably what you have seen when looking at documentation, because it reads the entire XML into memory to create the object model.

The second is called SAX, which is a streaming method. The parser starts reading the XML and sends signals to your code about certain events, e.g. when a new start tag is found.

So SAX is clearly what you need for your situation. Sax parsers can be found in the python library under xml.sax and xml.parsers.expat.

answered 2009-01-25T00:49:08.970

Your Answer