Compare many Gigs of files

[Timelord]

I have a situation where a coworker copied several very large volumes of data from 1 location to another. The smallest copy is about 500 Gig. I need to compare the copy with the original. We are in the process of moving the files from 1 piece of hardware to another and need to validate we got everything. Does anyone know of free software that can handle this without crashing or bringing the servers to a near stop?

[JoeP]

Do you need to compare individual file content or just the tables of content?

[PaulB]

You could try using the File Checksum Integrity Verifier (FCIV) utility to compute the MD5 or SHA-1 cryptographic hash values of your source file and of your destination file. Hashes are used to ensure the data integrity of files. Because the hash algorithm always produces the same output for the same given input, you can compare a hash of the source file with a newly created hash of the destination file to check that it is intact and unmodified.

Information about the File Checksum Integrity Verifier (FCIV) utility can be found in the Microsoft Knowledge Base:

841290 Availability and description of the File Checksum Integrity Verifier utility.

I have no idea if this utility will handle files the size you are dealing with. You may want to look into other hash generators.

[Timelord]

JoeP, I need to compare the files but not their content at this time. Most files will be no larger than 1 or 2 Meg and no folder over 4Gig in size so we are talking a very large number of files and folders. I am working a simple space compare (does this top level folder hold as much as this other top level folder) but I am not confident that will actually tell me if there is an issue.

[PJ_in_FL]

Can you get the "Properties" pop-up for the directory structure of each backup and compare the the "Properties" of the original directory structure?

In other words, do you get the same numbers for "Size:" and number of files and folders in "Contains:" in the Properties dialog pop-up windows when you right click on the top level folder of each group of files and click "Properties"?

The "Size on disk" may be different if cluster sizes differ between storage media so don't compare those numbers.

[JoeP]

See if Easily compare two folders by using PowerShell helps.

[Timelord]

PJ_in_FL. That is what I was trying but some of this is so large it takes a very long time to get the information, and as always, this is due yesterday.
JoeP. Thanks, I will look into that.