Compare Files with PowerShell: a faster way

Sometimes you need to test whether two files are the same. As files get larger, comparisons take longer, so we need to consider performance. In this article, I'll show how to compare two files using a buffered approach in PowerShell.

When talking about performance, it is better to measure multiple times on multiple systems. BenchmarkDotNet's good practices also warn against extrapolating results across environments. I only measured once on one system here, so treat these numbers as a relative indication rather than a general benchmark.

Strategies

There are several ways to compare files:

• Start by comparing the file sizes. Files with different sizes cannot be equal.
• Do a byte-by-byte comparison of both files. You can stop the comparison when a byte differs, so you don't have to read the entire file. However, reading many bytes at once is usually faster.
• Do a buffer comparison, reading several bytes at once and comparing the arrays.
• Generate a checksum of both files and compare the results. For a one-off comparison, this requires reading both files completely and cannot stop at the first difference. Checksums become more useful when you can reuse previously calculated values, although hash collisions mean they do not provide the same guarantee as a byte-exact comparison.

Extra performance: use LINQ

PowerShell has native comparison operators, but calling LINQ from PowerShell makes the buffer comparison much faster. We'll import the System.Linq namespace with using namespace System.Linq and call SequenceEqual like this: [Enumerable]::SequenceEqual($one, $two). In my single test, this LINQ method was substantially faster than the PowerShell comparison.

FilesAreEqual function

Let's create a function that takes two files, compares them, and returns $true when both files are equal:

using namespace System.Linq

function FilesAreEqual {
    param(
        [System.IO.FileInfo] $first,
        [System.IO.FileInfo] $second,
        [uint32] $bufferSize = 524288)

    function Read-FullBuffer {
        param(
            [System.IO.Stream] $stream,
            [byte[]] $buffer)

        $totalBytesRead = 0

        while ($totalBytesRead -lt $buffer.Length) {
            $bytesRead = $stream.Read(
                $buffer,
                $totalBytesRead,
                $buffer.Length - $totalBytesRead)

            if ($bytesRead -eq 0) { break }
            $totalBytesRead += $bytesRead
        }

        return $totalBytesRead
    }

    if ($first.Length -ne $second.Length) { return $false }
    if ($bufferSize -eq 0) { $bufferSize = 524288 }

    $fs1 = $null
    $fs2 = $null

    try {
        $fs1 = $first.OpenRead()
        $fs2 = $second.OpenRead()

        $one = New-Object byte[] $bufferSize
        $two = New-Object byte[] $bufferSize

        while ($true) {
            $bytesRead1 = Read-FullBuffer $fs1 $one
            $bytesRead2 = Read-FullBuffer $fs2 $two

            if ($bytesRead1 -ne $bytesRead2) { return $false }
            if ($bytesRead1 -eq 0) { return $true }

            if ($bytesRead1 -lt $bufferSize) {
                [Array]::Clear($one, $bytesRead1, $bufferSize - $bytesRead1)
                [Array]::Clear($two, $bytesRead2, $bufferSize - $bytesRead2)
            }

            if (-Not [Enumerable]::SequenceEqual($one, $two)) {
                return $false
            }
        }
    }
    finally {
        if ($null -ne $fs1) { $fs1.Dispose() }
        if ($null -ne $fs2) { $fs2.Dispose() }
    }
}

Buffering works

So what happens when we plug different values into $bufferSize? This is what I got on my machine by comparing the Wars of Liberty binary of 140 MB with itself:

A sufficiently large buffer can make a file comparison much faster, which is why I settled on 524,288 bytes (512 KiB) for this PowerShell function. On my machine, comparing the entire 2.2 GB Wars of Liberty setup file took 42 seconds.

So what about a byte-by-byte comparison? In the same test, that took 32 seconds for the 140 MB file: more than 13 times slower than using a 524,288-byte buffer.

Just need a script?

If you just need a script, copy this text to compare.ps1 and run it from the command line like .\compare.ps1 .\article.html .\article2.html. The result will be printed to the console.

<#
.SYNOPSIS
    Compares two files. Returns True if the files are equal.
.DESCRIPTION
    Compares two files. Returns True if the files are equal; otherwise False.
    Use the bufferSize to optimize for speed. Might depend on your system.
.PARAMETER file1
    The first file.
.PARAMETER file2
    The second file.
.PARAMETER bufferSize
    The size of the buffer will influence the speed of the script.
.OUTPUTS
    True when equal otherwise False.
.LINK
    More info: https://keestalkstech.com/comparing-files-with-powershell/
#>
using namespace System.Linq

param(
    [Parameter(Mandatory = $true)]
    [string]
    $file1,
    [Parameter(Mandatory = $true)]
    [string]
    $file2,
    [uint32]
    $bufferSize = 524288)

function Read-FullBuffer {
    param(
        [System.IO.Stream] $stream,
        [byte[]] $buffer)

    $totalBytesRead = 0

    while ($totalBytesRead -lt $buffer.Length) {
        $bytesRead = $stream.Read(
            $buffer,
            $totalBytesRead,
            $buffer.Length - $totalBytesRead)

        if ($bytesRead -eq 0) { break }
        $totalBytesRead += $bytesRead
    }

    return $totalBytesRead
}

$ErrorActionPreference = "Stop"
$PSDefaultParameterValues['*:ErrorAction'] = 'Stop'

$first = Get-Item $file1
$second = Get-Item $file2

# Files with different sizes cannot be equal.
if ($first.Length -ne $second.Length) { return $false }
if ($bufferSize -eq 0) { $bufferSize = 524288 }

$fs1 = $null
$fs2 = $null

try {
    $fs1 = $first.OpenRead()
    $fs2 = $second.OpenRead()

    $one = New-Object byte[] $bufferSize
    $two = New-Object byte[] $bufferSize

    while ($true) {
        $bytesRead1 = Read-FullBuffer $fs1 $one
        $bytesRead2 = Read-FullBuffer $fs2 $two

        if ($bytesRead1 -ne $bytesRead2) { return $false }
        if ($bytesRead1 -eq 0) { return $true }

        if ($bytesRead1 -lt $bufferSize) {
            [Array]::Clear($one, $bytesRead1, $bufferSize - $bytesRead1)
            [Array]::Clear($two, $bytesRead2, $bufferSize - $bytesRead2)
        }

        if (-Not [Enumerable]::SequenceEqual($one, $two)) {
            return $false
        }
    }
}
finally {
    if ($null -ne $fs1) { $fs1.Dispose() }
    if ($null -ne $fs2) { $fs2.Dispose() }
}

Further reading

While working on the subject, I found some interesting reads:

• High Performance PowerShell with LINQ - shows how to use LINQ instead of native PowerShell.
• Comparing two byte arrays in .NET - discusses fast byte-array comparison in .NET. Most upvotes go to Enumerable.SequenceEqual, but there are faster alternatives.

Changelog

• 2026-07-18 corrected and tested partial-read handling and stream cleanup, clarified the benchmark limitations and file-comparison strategies, and simplified the LINQ calls with using namespace System.Linq.
• 2020-10-10 added the Just need a script? section and fixed the first two if statements on systems where they did not work, based on code from caspertone2003.
• 2020-06-08 the original article used a byte-by-byte comparison, slowing things down on larger files. After writing about the impact of buffering on file streams, I rewrote this article to include buffering and .NET LINQ to improve performance.