WPF window crashes on startup, or it starts but hangs and does not render contents

I've had a couple nasty problems with WPF applications crashing on certain client machines but working everywhere else - every developer's nightmare.

I'm posting the problems / solution here to help preserve the sanity of fellow developers. These were not easy to troubleshoot.

Both of these issues are classic "it works on my machine but not theirs" issues.

1. WPF app crashes on startup. No errors, no idea why.
2. WPF app starts up, the window border shows, but the contents do not paint. It just hangs (reported as a "transparent window") Clicking to close triggers a crash report (machine was windows XP).