powershell 2.0 command line redirection

I'm looking for an explanation of the following discrepancy:

Given the following powershell script foo.ps1:

write-host "normal"
write-error "error"
write-host "yay"

Running it with

C:\>powershell .\foo.ps1 > out.txt 2>&1

Produces:

normal
C:\foo.ps1 : error
At line:1 char:10
+ .\foo.ps1 <<<< 
    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,foo.ps1

Write-Host : The OS handle's position is not what FileStream expected. Do not use a handle simultaneously in one FileStream and in Win32 co
de or another FileStream. This may cause data loss.
At C:\foo.ps1:3 char:11
+ write-host <<<<  "yay"
    + CategoryInfo          : NotSpecified: (:) [Write-Host], IOException
    + FullyQualifiedErrorId : System.IO.IOException,Microsoft.PowerShell.Commands.WriteHostCommand

But running with:

C:\>powershell .\foo.ps1 2>&1 > out.txt

Produces (correctly):

normal
C:\foo.ps1 : error
At line:1 char:10
+ .\foo.ps1 <<<< 
    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,foo.ps1

yay

I almost resolved myself into thinking that the order of the redirection mattered in Windows, however all of the examples in the TechNet usage page for command redirection show the file redirection preceding the stderr redirection.

Can someone please explain this to me?

For reference, this is being done on Server 2003 x64 SP2 with:

C:\>powershell get-host


Name             : ConsoleHost
Version