Alex Rivera | Logout

Most efficient way to store URLs in Mysql?

Asked 2009-06-17T17:46:15.300
13

I want to store a large set of URLs in MySql and create a unique index on the column. If I make the column utf8 then I'll be limited to a varchar(333), which is not enough to hold some of my URLs. If I declare the column to be latin1 then I get the full 1000 characters (don't think I need that much). However, I'll have to encode the URL and be consistent about always working with the encoded URL. Is there a better way to manage large sets of URLs?

mysql
Edit
Report

1 Answer

9

three good ways to do this:

1) use TEXT instead of VARCHAR. to ensure uniqueness, you'll have to also create a separate VARCHAR column to store an MD5() or SHA1() hash and add a UNIQUE or PRIMARY index. this has the unfortunate consequence of an additional disk seek to retrieve the URL, but depending on your use case that might be OK.

2) use VARCHAR with a binary collation and compress the URL using COMPRESS().

3) i forgot the third one as i was typing the first two. grr...

answered 2009-06-17T21:08:14.113

Your Answer