Doctypes tell the browser in what language the page is written in, be it HTML or XHTML. For example,
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN"
"http://www.w3.org/TR/html4/strict.dtd">
tell the browser to render the page as HTML4 strict. Older browsers used to render pages incorrectly and therefore newer browsers simulate errors of the older browsers when they find an old doctype.
Today you should use at least HTML4 or better XHTML.
A blog entry about doctypes is Fix Your Site With the Right DOCTYPE! (from A List Apart).
On the web, a doctype does nothing but tell the brower if you want standards, almost standards, or quirks mode.
What changes in quirks mode depends on the browser: Firefox, Opera, Safari, and Chrome implement a limited set of quirks, like removing the space for text descenders in code like <table><tr><td><img></td></tr></table> (solution: td img { vertical-align:bottom; }). IE, on the other hand, reverts to the rendering engine in IE5.5. That means that you won't be able to use any of the new features implemented since 2000.
To trigger standards mode, I suggest using the HTML5 doctype, <doctype html>, as it is the easiest to remember.