defragmenting encrypted USB hard drive

User avatar
ChrisGreaves
PlutoniumLounger
Posts: 15498
Joined: 24 Jan 2010, 23:23
Location: brings.slot.perky

defragmenting encrypted USB hard drive

Post by ChrisGreaves »

I am not a big fan of defragmenting hard drives, but it's been about two months since I reloaded my drive so ...
Win10-2004 File Explorer, Right-click, Properties, Tools, Optimize(?!!), and nada!
Untitled.png
Drive C: is visible and is (now) optimized, but there is no sign of my drive T:

Under Win7, I could de-fragment my drive T, which was assigned that letter via TrueCrypt.
On my new Win10 laptop I am using VeraCrypt (quite happily), but Win10 doesn't seem to see my mounted data partition.
Untitled2.png
PiriForm's Defraggler sees my mounted encrypted drive but I now notice that the partition is exFat, and I was warned against FAT, so possibly Win10 does not see drive T because it is not NTFS?
I am not sure what exFat is or how I stumbled into it, but the referenced page suggests it is good for SST drives, which my USB hard drive is NOT, so maybe Win10 sees "exFat" and thinks "SSD" and steers clear of it? (in which case Win10 is more stupid than I thought; Win10 ought to know that this is a platter hard drive?)

I can change the partition to NTFS tonight if suggested.

In the meantime, does Win10 have an aversion to USB hard drives or encrypted partitions?

Thanks for any steering
Chris
You do not have the required permissions to view the files attached to this post.
An expensive day out: Wallet and Grimace

User avatar
stuck
Panoramic Lounger
Posts: 8127
Joined: 25 Jan 2010, 09:09
Location: retirement

Re: defragmenting encrypted USB hard drive

Post by stuck »

I've always been told you should NOT defrag a SSD, you should just let Windows manage it. defragging just wastes the SSD's (limited number of) write cycles

Ken

User avatar
ChrisGreaves
PlutoniumLounger
Posts: 15498
Joined: 24 Jan 2010, 23:23
Location: brings.slot.perky

Re: defragmenting encrypted USB hard drive

Post by ChrisGreaves »

stuck wrote:
02 Sep 2020, 13:31
I've always been told you should NOT defrag a SSD, you should just let Windows manage it. defragging just wastes the SSD's (limited number of) write cycles
Hi Ken, and thanks for the confirmation. That is my understanding too, that an SSD is rather like regular core/ram memory.
But my USB hard drives are platters with moving heads (I can hear them chuttering away when they are mounted), so (a) de-fragmentation shouldn't harm them and (b) de-fragmentation overnight once every six months might make a small improvement.

Right now the defragmenmtation is not critical - after all, my nightly backup is finished long before I have finished washing my dishes in the kitchen (!), but I am more concerned about why Win10 doesn't seem to see the mounted VeraCrypted drive. when Piriform Defraggler does. In terms of SSD de-fragmentation I would expect both defragmenter programs to recommend against SSD drives, or both programs to ignore the drive, but one recognizes the drive and the other does not.

I recall too being cautioned against FAT file types when I was setting up this laptop two months ago, so I am worried that I might have set it up with the wrong type - exFat instead of NFTS.

Thanks again
Chris
An expensive day out: Wallet and Grimace

User avatar
stuck
Panoramic Lounger
Posts: 8127
Joined: 25 Jan 2010, 09:09
Location: retirement

Re: defragmenting encrypted USB hard drive

Post by stuck »

https://www.howtogeek.com/235596/whats- ... -and-ntfs/

Why did you choose something (exFAT) that is NOT Window's default (NTFS) file system? On second thoughts, I don't want to know.

Ken

User avatar
ChrisGreaves
PlutoniumLounger
Posts: 15498
Joined: 24 Jan 2010, 23:23
Location: brings.slot.perky

Re: defragmenting encrypted USB hard drive

Post by ChrisGreaves »

stuck wrote:
02 Sep 2020, 14:47
Why did you choose something (exFAT) that is NOT Window's default (NTFS) file system?
80% ignorance and 20% confusion, although it might have been 80% confusion and 20% ignorance. Or indecision even ...

I was overwhelmed with the new laptop, the new (to me) O/S, plus everything else that is going on here at The Landfall Garden House.

Ken, I think that you are saying to me "after tonight's data backup, reformat that partition as NTFS", is that correct? I can do that now.

Two months ago I was juggling the technical stuff while trying to do productive work on the old Toshiba laptop (sans Ctrl and Alt keys) and staying on top of the gardening, hauling rocks out of the ground, and looking for electrical contractors and ...

Cheers, and Thanks
Chris
An expensive day out: Wallet and Grimace

User avatar
stuck
Panoramic Lounger
Posts: 8127
Joined: 25 Jan 2010, 09:09
Location: retirement

Re: defragmenting encrypted USB hard drive

Post by stuck »

ChrisGreaves wrote:
02 Sep 2020, 15:18
Ken, I think that you are saying to me "after tonight's data backup, reformat that partition as NTFS", is that correct? I can do that now.
No, I wasn't advising you to do anything, only you know what your requirements are. For example, if you ever want to attach this external HDD to a Mac, exFAT is a sensible choice. A Mac can both read and write to that type of file system but it can only read an NTFS drive (see the How to Geek page I linked to previously).

Ken

PJ_in_FL
5StarLounger
Posts: 1090
Joined: 21 Jan 2011, 16:51
Location: Florida

Re: defragmenting encrypted USB hard drive

Post by PJ_in_FL »

Chris,

Unless you see a noticeable degradation in accessing files on the external drive then -- IF IT AIN'T BROKE DON'T FIX IT.

Perhaps a large disk with an active database might be improved (by milliseconds) by defragmentation, but more modern drives are much faster and have internal caching that decreases latency delays significantly. No metrics to cite, but just experience with a dozen plus external drives with about 20T of data across the home network. YMMV.

Suggestion, since you did ask for input: save the wear and tear on the drives.
PJ in (usually sunny) FL

User avatar
ChrisGreaves
PlutoniumLounger
Posts: 15498
Joined: 24 Jan 2010, 23:23
Location: brings.slot.perky

Re: defragmenting encrypted USB hard drive

Post by ChrisGreaves »

PJ_in_FL wrote:
02 Sep 2020, 19:12
Unless you see a noticeable degradation in accessing files on the external drive then -- IF IT AIN'T BROKE DON'T FIX IT. ... Suggestion, since you did ask for input: save the wear and tear on the drives.
Hi PJ. As stated above, I am not a big fan of defragmenmtation. It made sense back in the days of 5.25" floppies, which we de-fragmented by copying them from one drive to another.
Today I do not de-fragment after an initial post-installation de-fragmentation.
Indeed, I am intrigued when an inspection of files shows that EXEs and DLLs that have not been upgraded or updated (Office 2003 components as an example) show up as fragments in a list. What has caused unchanged files to become fragmented?

That said, I was using "Properties, Tools" to inspect the drives because I have been receiving BSOD's after my evening backups. I reasoned that there might be some form of file table corruption on a drive, but whether the boot drive (C) or the de-crypted data drive (T) or the de-crypted backup drive (Y), how to tell? And it made sense to me to check the drives before posting a question here.

"Properties, Tools, Check disk" might have given a clue. That is how I ended up looking at fragmentation (which pops up with "scan disk for errors") , because I have TWO such tools - the native Windows 7/10 tool and a 3rd-party tool from PiriForm. The fact that my data drive was NOT listed as a candidate for de-fragmentation is what gave me a pause for thought. Not that I couldn't de-fragment it - I don't want to de fragment it anyway - but that it did not even appear in a list of drives!. De-fragmentation can be seen as yet-another-test of file integrity.

Cheers
Chris
An expensive day out: Wallet and Grimace

User avatar
ChrisGreaves
PlutoniumLounger
Posts: 15498
Joined: 24 Jan 2010, 23:23
Location: brings.slot.perky

Re: defragmenting encrypted USB hard drive

Post by ChrisGreaves »

stuck wrote:
02 Sep 2020, 17:54
No, I wasn't advising you to do anything, only you know what your requirements are. For example, if you ever want to attach this external HDD to a Mac, exFAT is a sensible choice. A Mac can both read and write to that type of file system but it can only read an NTFS drive (see the How to Geek page I linked to previously).
Hi Ken.
I re-built the data partition as NTFS; I do remember receiving advice (from someone) two months ago, after I had formatted the data drive to get started.
I am now using NTFS on the data drive, with no plans to send the data anywhere except to the daily backup drive.

For all I know the exFat format could have resulted from aberrant behavior of a too-sensitive track-pad, which I appear to have fixed just two days ago. I remember making an set of bad choices by clumsy use of this track-pad in the early days!
Cheers, and thanks again.
Chris
An expensive day out: Wallet and Grimace

JoeP
SilverLounger
Posts: 2051
Joined: 25 Jan 2010, 02:12

Re: defragmenting encrypted USB hard drive

Post by JoeP »

Most modern hard drives are of a size that defragmenting is a waste of time. Just let Windows take care of it. While it is true that SSD's do not need defragmenting as HDD do they still need defragmenting occasionally. Windows 10 will handle it. Windows 10 will also optimize SSDs. Again let Windows 10 handle it automatically.

Here's an old blog post from Scott Hanselman that still applies - The real and complete story - Does Windows defragment your SSD?.
Joe

User avatar
ChrisGreaves
PlutoniumLounger
Posts: 15498
Joined: 24 Jan 2010, 23:23
Location: brings.slot.perky

Re: defragmenting encrypted USB hard drive

Post by ChrisGreaves »

JoeP wrote:
03 Sep 2020, 19:31
Most modern hard drives are of a size that defragmenting is a waste of time. Just let Windows take care of it. While it is true that SSD's do not need defragmenting as HDD do they still need defragmenting occasionally. Windows 10 will handle it. Windows 10 will also optimize SSDs. Again let Windows 10 handle it automatically. Here's an old blog post from Scott Hanselman that still applies - The real and complete story - Does Windows defragment your SSD?.
Right.
Right, Right.
Right!
Untitled.png
But look at the screen shot at the head of this thread - Win10 Optimizer does not recognize/see my data drive T (whereas File Explorer does, and Piriform's Dfraggler does ...)
Cheers
Chris
You do not have the required permissions to view the files attached to this post.
An expensive day out: Wallet and Grimace

JoeP
SilverLounger
Posts: 2051
Joined: 25 Jan 2010, 02:12

Re: defragmenting encrypted USB hard drive

Post by JoeP »

That's because it is exFat
Joe

JoeP
SilverLounger
Posts: 2051
Joined: 25 Jan 2010, 02:12

Re: defragmenting encrypted USB hard drive

Post by JoeP »

You need to use a third party tool such as Defraggler or UltraDefrag.
Joe

User avatar
ChrisGreaves
PlutoniumLounger
Posts: 15498
Joined: 24 Jan 2010, 23:23
Location: brings.slot.perky

Re: defragmenting encrypted USB hard drive

Post by ChrisGreaves »

That's because it is exFat
Ho Joe. I'm sorry but I don't see why Windows defragmenter should NOT recognize "exFat" when another defragmenter (piriform) does recognize exFat.

I can see that, as an example, a MAC o/s may not want to/be able to write to an exFat drive, but in this thread I am in Windows-only mode.

Why should Windows defrag NOT recognise an exFat format when
(a) Windows File Explorer does recognize exFat drives and
(b) Piriform defraggler does recognize exFat drives?

There may well be a rational reason, but I haven't heard it yet.

From my basic knowledge of memory fragmentation (RAM memory, hard drive memory, interpreter managed memory etc), I know that fragmentation occurs.
That said, we write defragmentation routines to re-locate chunks of memory in whatever media they are housed. (RAM, drives, interpreter-managed storage)

Or to pose the question differently, how is it that Piriform defraggler says it can, and will, (and DOES!), defragment the data drive T when Windows says it can/will not?
JoeP wrote:
03 Sep 2020, 22:36
You need to use a third party tool such as Defraggler or UltraDefrag.
Which I would, if I were keen on defragmentation. Remember that I am using "defrag" solely because it highlights some sort of problem with the data drive T.

FWIW I re-built the VeraCrypt drive a couple of nights ago. It is now NTFS format.
Untitled.png
And while Windows defrag now sees/recognizes the data drive T, it refuses to have anything to do with it.
Data drive T is now an NTFS drive, and (to my mind) ought to behave like any other NTFS partition.
Untitle2.png
Here is a current screenshot of Piriform Defraggler dealing with my (now) NTFS data drive T:, yet Win10 Optimizer still refuses to deal with it.
Again, it is not "defragmenmtation" of the drive that interests me; it is the discrepancy between Win10 optimizer and (just about) any other application.

Right now I am of the mind that either:-
(a) Win10 Optimizer is afraid of Veracrypt drives or
(b) Veracrypt has some sort of flag or data on the partition that makes Win10 Optimizer (but not File Explorer or any other aspect of Win10) believe that my Veracrypt drive T is "not a real drive".

Thanks
Chris
You do not have the required permissions to view the files attached to this post.
An expensive day out: Wallet and Grimace

User avatar
Leif
Administrator
Posts: 7193
Joined: 15 Jan 2010, 22:52
Location: Middle of England

Re: defragmenting encrypted USB hard drive

Post by Leif »

ChrisGreaves wrote:
04 Sep 2020, 13:27
Why should Windows defrag NOT recognise an exFat format when
(a) Windows File Explorer does recognize exFat drives and
(b) Piriform defraggler does recognize exFat drives?
Perhaps it's because "Windows defrag"' is not part of Windows?

Read on, young man and let the rest of us get some rest :grin:
Leif

User avatar
StuartR
Administrator
Posts: 12577
Joined: 16 Jan 2010, 15:49
Location: London, Europe

Re: defragmenting encrypted USB hard drive

Post by StuartR »

I don't think the problem here is the exFAT format, I have veracrypt partitions that are formatted with NTFS and they also don't show up as disk drives under Windows, and can't be defragged with Windows defrag.
StuartR


JoeP
SilverLounger
Posts: 2051
Joined: 25 Jan 2010, 02:12

Re: defragmenting encrypted USB hard drive

Post by JoeP »

Windows defrag was originally a stripped down version of Diskeeper. It probably has not been updated to handle exFat. If you look at some of the third party products that support exFat, the support for exFat was added after the initial release.

@Stuart,
See https://github.com/veracrypt/VeraCrypt/issues/374
Joe

User avatar
StuartR
Administrator
Posts: 12577
Joined: 16 Jan 2010, 15:49
Location: London, Europe

Re: defragmenting encrypted USB hard drive

Post by StuartR »

That was interesting Joe, thank you.

When I try to enable that option in VeraCrypt it gives me this warning message. I think I will leave the default alone!
You do not have the required permissions to view the files attached to this post.
StuartR


User avatar
ChrisGreaves
PlutoniumLounger
Posts: 15498
Joined: 24 Jan 2010, 23:23
Location: brings.slot.perky

Re: defragmenting encrypted USB hard drive

Post by ChrisGreaves »

Leif wrote:
04 Sep 2020, 14:07
Perhaps it's because "Windows defrag"' is not part of Windows?
Well!
I didn't know that!!
No-one ever tells me anything!!!
Why am I always the last one to know?

I know that Microsoft is not a software company; that it is a S&M company, but ever since I was this high I have thought of "Windows defragmenter" as being part of Windows, rather than a 3rd-party grafted-on assemblage, as has been done over the years with MSWord 2.0/6.0/97/etc.

FWIW the bond (of defragmenter) is seamless. I see no trace of 3rd-party in displays given off by Task manager, Services.msc, Task Scheduler ("Author: Microsoft Corporation") etc
FWISW PiriForm and, I note, AusLogic are able to deal with exFat/VeraCrypt data drives.

That Diskeeper Corporation has not kept its code up to date is one thing, but that Win10 has not insisted that the code be kept up to date is another.

Whatever, I am eternally grateful for one of my super-peeriors for enlightening me as to why the VeraCrypted data drive T is persona non data to the grafted-on defragmentation routine of Win10.

Now please get some well-deserved rest. I have been kept awake most of the night by two bleating cats and a dog with a gimp leg. :scream:

Cheers
Chris
An expensive day out: Wallet and Grimace

User avatar
ChrisGreaves
PlutoniumLounger
Posts: 15498
Joined: 24 Jan 2010, 23:23
Location: brings.slot.perky

Re: defragmenting encrypted USB hard drive

Post by ChrisGreaves »

StuartR wrote:
04 Sep 2020, 16:21
I don't think the problem here is the exFAT format, I have veracrypt partitions that are formatted with NTFS and they also don't show up as disk drives under Windows, and can't be defragged with Windows defrag.
Thanks Stuart. Now that I have reloaded my VeraCrypt encrypted partition as NTFS I have an option to re-reload it as a Truecrypt encrypted partition in NTFS format.
How I have loved this world of computing for so many years I will never know. :evilgrin:

The biggest part of the mystery to me was that Win10 was happy with the data drive T in every respect I could try except defragmentation.
Cheers
Chris
P.S. I acknowledge that reverting to TrueCrypt only makes sense for me if I want to defrag, but I could achieve defragmentation faster by a two pass RoboCopy [/Create] after my weekly backup (/MIR). That is, by simply reloading the data partition from the cleaned backup.
C
An expensive day out: Wallet and Grimace