A PowerShell script to find the file size and file count of a folder with millions of files?

If you use streaming and pipelining, you should be reduce problem with (3) a lot, because when you stream, each object is passed along the pipeline as and when they are available and do not take up much memory and you should be able to process millions of files (though it will take time).

Get-ChildItem $directory -recurse | Measure-Object -property length -sum

I don't believe @Stej's statement, Get-ChildItem probably reads all entries in the directory and then begins pushing them to the pipeline., is true. Pipelining is a fundamental concept of PowerShell (provide the cmdlets, scripts, etc. support it). It both ensures that processed objects are passed along the pipeline one by one as and when they are available and also, only when they are needed. Get-ChildItem is not going to behave differently.

A great example of this is given in Understanding the Windows PowerShell Pipeline.

Quoting from it:

The Out-Host -Paging command is a useful pipeline element whenever you have lengthy output that you would like to display slowly. It is especially useful if the operation is very CPU-intensive. Because processing is transferred to the Out-Host cmdlet when it has a complete page ready to display, cmdlets that precede it in the pipeline halt operation until the next page of output is available. You can see this if you use the Windows Task Manager to monitor CPU and memory use by Windows PowerShell.

Run the following command: Get-ChildItem C:\Windows -Recurse. Compare the CPU and memory usage to this command: Get-ChildItem C:\Windows -Recurse | Out-Host -Paging.

Benchmark on using Get-ChildItem on c:\ (about 179516 files, not milions, but good enough):

Memory usage after running <

Get-ChildItem probably reads all entries in the directory and then begins pushing them to the pipeline. In case that Get-ChildItem doesn't work well, try to switch to .NET 4.0 and use EnumerateFiles and EnumeratedDirectories:

function Get-HugeDirStats($directory) {
    function go($dir, $stats)
    {
        foreach ($f in [system.io.Directory]::EnumerateFiles($dir))
        {
            $stats.Count++
            $stats.Size += (New-Object io.FileInfo $f).Length
        }
        foreach ($d in [system.io.directory]::EnumerateDirectories($dir))
        {
            go $d $stats
        }
    }
    $statistics = New-Object PsObject -Property @{Count = 0; Size = [long]0 }
    go $directory $statistics

    $statistics
}

#example
$stats = Get-HugeDirStats c:\windows

Here the most expensive part is the one with New-Object io.FileInfo $f, because EnumerateFiles returns just file names. So if only count of files is enough, you can comment the line.

See Stack Overflow question How can I run PowerShell with the .NET 4 runtime? to learn how to use .NET 4.0.

You may also use plain old methods which are also fast, but read all the files in directory. So it depends on your needs, just try it. Later there is comparison of all the methods.

function Get-HugeDirStats2($directory) {
    function go($dir, $stats)
    {
        foreach ($f in $dir.GetFiles())
        {
            $stats.Count++
            $stats.Size += $f.Length
        }
        foreach ($d in $dir.GetDirectories())
        {
            go $d $stats
        }
    }
    $statistics = New-Object PsObject -Property @{Count = 0; Size = [long]0 }
    go (new-object IO.DirectoryInfo $directory) $statistics

    $statistics
}

Comparison