How to decode a Base64 string?

I have a normal string in Powershell that is from a text file containing Base64 text; it is stored in $x. I am trying to decode it as such:

$z = [System.Text.Encoding]::Unicode.GetString([System.Convert]::FromBase64String($x));

This works if $x was a Base64 string created in Powershell (but it's not). And this does not work on the $x Base64 string that came from a file, $z simply ends up as something like 䐲券.

What am I missing? For example, $x could be YmxhaGJsYWg= which is Base64 for blahblah.

In a nutshell, YmxhaGJsYWg= is in a text file then put into a string in this Powershell code and I try to decode it but end up with 䐲券 etc.