Alex Rivera | Logout

Alternative to BerkeleyDB?

Asked 2008-11-04T03:44:47.150
43

I'm looking for a dbm-like library that I can use in place of Berkeley DB, which I'm currently using. My main reason for switching is the licensing fees for BDB are pretty high (free for open source apps, but my employer does not want to open source this particular app for various reasons).

I've looked briefly at qdbm but it doesn't look like it will fill my needs -- lots of keys (several million) and large data items (> 1-5 megabytes). Before I continue my search I figured I'd ask because it seems there are tons of dbm-like libraries out there.

berkeley-db dbm okvs
Edit
Report

3 Answers

9

You can get much improved performance out of any dbm (even qdbm) and improved parallelism with a simple level of indirection: Simply take your keys and hash them, and use data_dir/H(key)/ as the database to store those keys. Limit the hash output to a small value (say) 255 for best results.

This approach has a number of benefits, easily summarized:

  • Conceptually simple
  • Easy to implement and test
  • Doesn't lock the entire database for updates
  • Can support much larger databases
  • Easy to replace the DBM component

The hash probably doesn't even need to be cryptographically secure; just mostly uniform. DJB's cdb hash works well in most cases.

answered 2008-11-14T16:00:07.717
3

If you are on Windows then you can use the builtin esent database engine. http://blogs.msdn.com/windowssdk/archive/2008/10/23/esent-extensible-storage-engine-api-in-the-windows-sdk.aspx

answered 2008-12-15T17:26:08.983
0

db4o is pretty cheap and fast but it can only be used with java or .net

answered 2008-11-04T03:47:06.957

Your Answer