Alex Rivera | Logout

How can I split a large text file into smaller files with an equal number of lines?

Asked 2010-01-06T22:40:39.823
774

I've got a large (by number of lines) plain text file that I'd like to split into smaller files, also by number of lines. So if my file has around 2M lines, I'd like to split it up into 10 files that contain 200k lines, or 100 files that contain 20k lines (plus one file with the remainder; being evenly divisible doesn't matter).

I could do this fairly easily in Python, but I'm wondering if there's any kind of ninja way to do this using Bash and Unix utilities (as opposed to manually looping and counting / partitioning lines).

Edit
Report

2 Answers

116

Use the split command:

split -l 200000 mybigfile.txt
answered 2010-01-06T22:45:05.973
16

You can also use AWK:

awk -vc=1 'NR%200000==0{++c}{print $0 > c".txt"}' largefile
answered 2010-01-07T01:03:23.767

Your Answer