How can I test a custom DelegatingHandler in the ASP.NET MVC 4 Web API?

I've seen this question come up in a few places, and not seen any great answers. As I've had to do this myself a few times, I thought I'd post my solution. If you have anything better, please post.

N.B. This is using ASP.NET MVC 4 Beta 2 version of Web API - future versions may change!

Update: This still works in ASP.NET MVC 4 RC