Alex Rivera | Logout

Maximum number of files/directories on Linux?

Asked 2011-11-23T08:01:38.323
56

I'm developing a LAMP online store, which will allow admins to upload multiple images for each item.

My concern is - right off the bat there will be 20000 items meaning roughly 60000 images.

Questions:

  1. What is the maximum number of files and/or directories on Linux?

  2. What is the usual way of handling this situation (best practice)?

My idea was to make a directory for each item, based on its unique ID, but then I'd still have 20000 directories in a main uploads directory, and it will grow indefinitely as old items won't be removed.

Thanks for any help.

linux directory directory-structure
Edit
Report

1 Answer

12

If your server's filesystem has the dir_index feature turned on (see tune2fs(8) for details on checking and turning on the feature) then you can reasonably store upwards of 100,000 files in a directory before the performance degrades. (dir_index has been the default for new filesystems for most of the distributions for several years now, so it would only be an old filesystem that doesn't have the feature on by default.)

That said, adding another directory level to reduce the number of files in a directory by a factor of 16 or 256 would drastically improve the chances of things like ls * working without over-running the kernel's maximum argv size.

Typically, this is done by something like:

/a/a1111
/a/a1112
...
/b/b1111
...
/c/c6565
...

i.e., prepending a letter or digit to the path, based on some feature you can compute off the name. (The first two characters of md5sum or sha1sum of the file name is one common approach, but if you have unique object ids, then 'a'+ id % 16 is easy enough mechanism to determine which directory to use.)

answered 2011-11-23T08:07:51.900

Your Answer