Why use anything but div?

In HTML/XHTML, why use anything other than div elements?

They are basic blocks that one can use to build entire pages. Sure, they don't add to the semantics of the page, but HTML isn't about semantics, it's about structure. And why should I be concerned about semantics?

Any thoughts?

Edit: So the statement "...HTML isn't about semantics, it's about structure." has caused some fuss; it is argued as being all about semantics, although div and span (perhaps the most common elements) break this rule. Perhaps then, a better question would be to ask: should HTML/XHTML be about semantics?