How to tell Mathematica to quit upon a debug-error

Following an idea borrowed from the Mathematica toolbag:

myMessage::debug = "Something went horribly wrong";

Unprotect[Message];
Message[mm : myMessage::debug] :=
 Block[{$inMsg = True, result},
   Message[mm];
   Quit[]] /; ! TrueQ[$inMsg]
Protect[Message];

Here, replace myMessage::debug with the actual message type you want to intercept.

Normal message:

Message[Power::infy]

(* ===> 
        StringForm::sfr: Item 1 requested in "Infinite expression `1` 
        encountered. >>" out of range; 0 items available. >> 

        Power::infy: Infinite expression `1` encountered. >> 
*)

Your 'debug' message:

Message[myMessage::debug]

(* ===>
myMessage::debug: Something went horribly wrong

 [kernel quits...]
*)

Update

Assert messages get some additional arguments, so you have to catch those too. And, by the way, you have to put the actual message name in the definition (not using the example message I used above):

Unprotect[Message];
Message[mm : Assert::asrtfl, m___] :=
 Block[{$inMsg = True, result},
   Message[mm, m];
   Quit[]] /; ! TrueQ[$inMsg]
Protect[Message];

In addition to the other answers you may want to investigate using:

$MessagePrePrint

Which allows you to execute code when any message is triggered.

For example, you can evaluate this:

$MessagePrePrint := (Print[Stack[_]]; Dialog[])

Which prints the evaluation stack (sometimes very large!) of the code that is evaluating. The dialog function then puts you in a sub-evaluation loop, allowing you to inspect the values of variables. To get out of the dialog loop, you can evaluate:

Return[]

which takes you back to the main loop (the main evaluation then either finishes or breaks at a new message).