WCF error: "The X.509 certificate CN=localhost chain building failed ..."

I'm getting this error while attempting to make my WCF client and server talk to each other.

The X.509 certificate CN=localhost chain building failed. The certificate that was used has a trust chain that cannot be verified. Replace the certificate or change the certificateValidationMode. A certificate chain processed, but terminated in a root certificate which is not trusted by the trust.

Everything works perfectly if I turn SSL certificates off.