Length of the data to decrypt is invalid

I'm trying to encrypt and decrypt a file stream over a socket using RijndaelManaged, but I keep bumping into the exception

CryptographicException: Length of the data to decrypt is invalid.
    at System.Security.Cryptography.RijndaelManagedTransform.TransformFinalBlock(Byte[] inputBuffer, Int32 inputOffset, Int32 inputCount)
    at System.Security.Cryptography.CryptoStream.FlushFinalBlock()
    at System.Security.Cryptography.CryptoStream.Dispose(Boolean disposing)

The exception is thrown at the end of the using statement in receiveFile, when the whole file has been transferred.

I tried searching the web but only found answers to problems that arise when using Encoding when encrypting and decrypting a single string. I use a FileStream, so I don't specify any Encoding to be used, so that should not be the problem. These are my methods:

private void transferFile(FileInfo file, long position, long readBytes)
{
    // transfer on socket stream
    Stream stream = new FileStream(file.FullName, FileMode.Open);
    if (position > 0)
    {
        stream.Seek(position, SeekOrigin.Begin);
    }
    // if this should be encrypted, wrap the encryptor stream
    if (UseCipher)
    {
        stream = new CryptoStream(stream, streamEncryptor, CryptoStreamMode.Read);
    }
    using (stream)
    {
        int read;
        byte[] array = new byte[8096];
        while ((read = stream.Read(array, 0, array.Length)) > 0)
        {
            streamSocket.Send(array, 0, read, SocketFlags.None);
            position += read;
        }
    }
}

private void receiveFile(FileInfo transferFile)
{
    byte[] array = new byte[8096];
    // receive file
    Stream stream = new FileStream(transferFile.FullName, FileMode.Append);
    if (UseCipher)
    {
        stream = new CryptoStream(stream, streamDecryptor, CryptoStreamMode.Write);
    }
    using (stream)
    {
        long position = new FileInfo(transferFile.Path).Length;
        whi