defragmenting encrypted drives

[ChrisGreaves]

Following on from a recent thread Defragmenting By Stale Date and the discussion surrounding SSD drives, I was struck by the thought that (to me) any sort of drive that isn't circular platters with retractable arms (as in the past 40 years of my life) are a Black Box.

SSDs in particular; If I'd not been told/warned I'd not have known that defragmenting an SSD is futile/dangerous etc.
OK. I get that.

So then I got to thinking about encrypted drives, specifically TrueCrypt and now VeraCrypt.

If I run a defragmenting program on an opened encrypted drive, the defragmenting program shows that the disk is fragmented and (if I weren't me) I am tempted to defragment it.

But TrueCrypt is a black box to me. How do I know that what the defragmenting program sees as fragmentation really is having an impact on speed?

To make an extreme example, suppose TrueCrypt sucked the entire drive into RAM memory and then accessed it from there. Wouldn't speed of access be irrelevant?
Yes, directory structures might be an issue, but TrueCrypt might be smart enough to defragment directories as they were loaded into RAM (A b it like RoboCopy's suggestion that copying a drive be done first with the "/CREATE" switch, and then repeated without the switch).


In short, the more I think about it the less useful or advisable defragmenting seems to be.

[StuartR]

You may need to defrag BOTH the encrypted Truecrypt container AND the disk that holds that container.

In other words you should dismount your Truecrypt file, then defrag the real disk, then mount the Truecrypt container and then defrag the virtual disk it creates.

[John Gray]

If you are defragmenting the partition, a Truecrypt container therein located gets defragmented just in the same way as does any ordinary non-system, movable, file. But, as Stuart says, its contents don't get defragmented.

Windows neither knows or cares that the Truecrypt container file is encrypted.
And neither do I!

[ChrisGreaves]

You may need to defrag BOTH the encrypted Truecrypt container AND the disk that holds that container.

Hi Stuart.
I might need to defragment the container if it were a file on a (Windows-like) partition, but in my case I partitioned the physical hard drive into a boot partition (C:) and a data partition (D:).
I then told TruCrypt to use the entire D: partition, so AFAIK that container can't be defragmented.

In other words you should dismount your Truecrypt file, then defrag the real disk, then mount the Truecrypt container and then defrag the virtual disk it creates.

The point I was trying and so gloriously failing to make (grin) is that unless and until I know how TrueCrypt organizes its Black Box of Tricks, I am unlikely to develop an effective defragmentation algorithm.

As an extreme example, suppose TrueCrypt's encryption techniques involved writing a byte at a time separated by at least another hundred bytes (I said it was an extreme example) so that a file's bytes were guaranteed to be non-contiguous. This extreme example would involve massive overhead in data that organizes the user's data, but nonetheless it would make nonsense of any attempt to defragment the contents of the container as we currently think of defragmenting, because TrueCrypt's design would be fragmenting user file data on purpose!

A less extreme example would have TrueCrypt writing a user's file in (?)sectors(?) each sector of a file having other sector of data interspersed. For example if my user fil;e data is represented by sectors labeled "A", we would see, by design, "ABAGALKATAEAJHAGADAKJANALAJAHG" and any attempt by us to coagulate the user file to appears as "AAAAAAAA etc." would be thwarted by TrueCrypt.

Bottom Line: Unless I know the workings of the black box, be it TrueCrypt, SSD,or TTE etc. I am unlikely to develop a useful file defragmenting algorithm.

[ChrisGreaves]

If you are defragmenting the partition, a Truecrypt container therein located gets defragmented just in the same way as does any ordinary non-system, movable, file. But, as Stuart says, its contents don't get defragmented.

Agreed!

Windows neither knows or cares that the Truecrypt container file is encrypted.
And neither do I!

Agreed and agreed.

[StuartR]

You do need to defrag your Truecrypt volume occasionally, depending on how it is used. The internal organization of Truecrypt causes each write to have to write a somewhat larger block than on a physical drive, but fragmentation of files will still slow down read operations, and fragmentation of free space will slow down file creation and extension.

[ChrisGreaves]

You do need to defrag your Truecrypt volume occasionally, depending on how it is used. The internal organization of Truecrypt causes each write to have to write a somewhat larger block than on a physical drive, but fragmentation of files will still slow down read operations, and fragmentation of free space will slow down file creation and extension.

Alright. So now we are into some sort of knowledge of what goes on inside the Black Box known as TrueCrypt.

Encryption, I can accept, requires a bit more overhead, so I can see that a 100KB file of data could require more than 100KB of bytes (just as a Word document is 20K to hold about 20 characters of text).
Knowing that TrueCrypt data takes more space than unencrypted data accentuates any perceived slowing of access to data. But until it can be measured, it is only perceived.

I also accept that fragmentation of the data of a file into more than one contiguous area slows down access (I'm thinking still of rotational speed and arm access times of traditional drives).

I'm just not sure that defragmenting the data blocks for the sake of defragmenting the data blocks is effective when compared to a targeted defragmentation, for examples, active files (vs. stale files) or directories (vs. data blocks).

I am sure we are on the same wavelength and I continue to be academically curious about the real/true effectiveness of defragmentation.

[JoeP]

If you have a modern (i.e. large) spinning disk let Windows take care of the defragmentation and quit worrying about it. You should read Disk Defragmentation – Background and Engineering the Windows 7 Improvements.

Joe

[ChrisGreaves]

You should read Disk Defragmentation – Background and Engineering the Windows 7 Improvements.

Thanks Joe, and I will read it, first downloading it to peruse at home.

I had not realised that for the past four or five years Windows defragmentation tool also acts as a virus-tracker:-

Code: Select all

Cynthia Hubbard
6 Oct 2011 3:12 PM
I am using Windows XP. I have a Disk defragmentation and run it once a week. There is white for space, green for unmovable files, blue for contagious files and the red is for fragment files. Now every time I ran the disk defragmentation to analyze it would say you do not have to defragment this file up until last night. So, I did the defragment but I have about 80% blue contagious file and about 30% unmovable.