(Vera/Truecrypt) Store files larger than 4GB?

User avatar
ChrisGreaves
PlutoniumLounger
Posts: 15498
Joined: 24 Jan 2010, 23:23
Location: brings.slot.perky

(Vera/Truecrypt) Store files larger than 4GB?

Post by ChrisGreaves »

Here i am, yet again encrypting a data partition, and the perennial question pops up.
The largest (visible) file on my old laptop's C-Drive is in "C:\Windows\winsxs\ManifestCache\" and is about 111_MB
The largest (visible) file on my old laptop's data partition, apart from a Windows.ISO which I downloaded this morning at 4+GB, is in "T:\Appl\MicrosoftUpdates\Office2013\" and is about 806_MB.
I think of a DVD image as 4.7GB or thereabouts, and while I play with aggregates of movies on DVDs, I don't store the aggregates on my data partition.

So I chose "No, never more than 4GB" and got on with life.

Have I now prevented myself from ever having a 5GB file stored?
Or will Windows get usefully smart for once and split a 5GB file into a somewhat inefficient two portions?


I suspect that the 4GB boundary is something to do with how Windows file system maps chunks of files to the hard drive, and that larger files have to be allocated in larger chunks.
If this supposition is true, then choosing "yes, larger than 4GB" suggests that all the other files, including the 1KB notepad files that contain 32 bytes, will each occupy significantly more space, just for the sake of the odd large file.

Right now, if someone offered me a 5GB file containing The Meaning Of Life, I'd buy another 64GB memory key and use it as storage, and not worry about the large file being on my day-to-day use encrypted data partition.

Thanks for any confirmation or correction.
Chris
An expensive day out: Wallet and Grimace

User avatar
John Gray
PlatinumLounger
Posts: 5401
Joined: 24 Jan 2010, 08:33
Location: A cathedral city in England

Re: (Vera/Truecrypt) Store files larger than 4GB?

Post by John Gray »

Are you just revisiting the fact that the FAT32 file system can hold files no larger than 4 GB? If you want to save files greater than 4 GB you need to use the NTFS or exFAT file format (no, I've never used that either) on your disk partition.
This is all 1993 stuff, though... :thumbup:
John Gray

Venison is quiet deer, and quite dear.

User avatar
StuartR
Administrator
Posts: 12577
Joined: 16 Jan 2010, 15:49
Location: London, Europe

Re: (Vera/Truecrypt) Store files larger than 4GB?

Post by StuartR »

Why not just format the encrypted partition with NTFS and forget about all this
StuartR


User avatar
ChrisGreaves
PlutoniumLounger
Posts: 15498
Joined: 24 Jan 2010, 23:23
Location: brings.slot.perky

Re: (Vera/Truecrypt) Store files larger than 4GB?

Post by ChrisGreaves »

John Gray wrote:
27 Jul 2020, 19:37
Are you just revisiting the fact that the FAT32 file system can hold files no larger than 4 GB? If you want to save files greater than 4 GB you need to use the NTFS or exFAT file format (no, I've never used that either) on your disk partition.
This is all 1993 stuff, though... :thumbup:
Hi John. I think that you are saying "Yes" and "No" respectively in answer to the two questions.

Yes, it may well be 1993 stuff, but I had no files that size back then, and it is only in the past half-dozen years that I have pondered DVDs with 4.7GB capacity. I am prompted into this by having to use UEFI which leads into GPT which is for "hard drives greater than 2TB", which is not my concern.

Cheers
Chris
An expensive day out: Wallet and Grimace

User avatar
ChrisGreaves
PlutoniumLounger
Posts: 15498
Joined: 24 Jan 2010, 23:23
Location: brings.slot.perky

Re: (Vera/Truecrypt) Store files larger than 4GB?

Post by ChrisGreaves »

StuartR wrote:
27 Jul 2020, 20:46
Why not just format the encrypted partition with NTFS and forget about all this
Hi Stuart.
In answer to your question, I suppose that it is because the Vercrypt/Truecrypt question isn't proposed as NTFS-or-not, but seems more like a question about my personal data habits.
I already use NTFS, otherwise I would not have been able to d/l a Windows ISO file yesterday, right? Or to make use of Burn-ing software.

I consider the Veracrypt/Truecrypt question to be misleading. It does not explicitly state that I will not be able to store a file greater than 4GB; that is what led me to wonder if there was a way, albeit inefficient, for Vera/Truecrypt to split a >4GB file into smaller chunks.

Cheers
Chris
An expensive day out: Wallet and Grimace

User avatar
StuartR
Administrator
Posts: 12577
Joined: 16 Jan 2010, 15:49
Location: London, Europe

Re: (Vera/Truecrypt) Store files larger than 4GB?

Post by StuartR »

Any file system will split files into smaller chunks to fit into the available free space.

The 4GB limit is not to do with the number of chunks. It is due to the limited size of a data structure that maps each block of the file to a physical location.
StuartR


User avatar
ChrisGreaves
PlutoniumLounger
Posts: 15498
Joined: 24 Jan 2010, 23:23
Location: brings.slot.perky

Re: (Vera/Truecrypt) Store files larger than 4GB?

Post by ChrisGreaves »

StuartR wrote:
28 Jul 2020, 12:01
Any file system will split files into smaller chunks to fit into the available free space.
Hi Stuart.
I guess this is what EaseUs did when it made a system image backup for me:-
Untitled.png
The 4GB limit is not to do with the number of chunks. It is due to the limited size of a data structure that maps each block of the file to a physical location.
So I believe that if I specify "No, never more than 4GB" I will be using smaller physical writes, so that storage is a little more efficient, excepting when I go over the 4GB file size, and even then the introduced inefficiency is related to dealing with several file-portions.
In the screen snapshot above will have to issue five calls to write to a DVD, one for each version.
Cheers
Chris
You do not have the required permissions to view the files attached to this post.
An expensive day out: Wallet and Grimace

User avatar
StuartR
Administrator
Posts: 12577
Joined: 16 Jan 2010, 15:49
Location: London, Europe

Re: (Vera/Truecrypt) Store files larger than 4GB?

Post by StuartR »

No Chris. You CAN'T create a file bigger than 4GB on file systems like FAT. What EaseUs did was create 5 different files, none of which is bigger than 4GB.
StuartR


User avatar
ChrisGreaves
PlutoniumLounger
Posts: 15498
Joined: 24 Jan 2010, 23:23
Location: brings.slot.perky

Re: (Vera/Truecrypt) Store files larger than 4GB?

Post by ChrisGreaves »

StuartR wrote:
01 Aug 2020, 22:01
No Chris. You CAN'T create a file bigger than 4GB on file systems like FAT. What EaseUs did was create 5 different files, none of which is bigger than 4GB.
Yes Yes Stuart.
I figured that that was the work of EaseUs, not Windows, but I classed it under the general heading of "Any file system will split files ..."; including EaseUs and not restricted to Windows.
Untitled.png
I note with interest that the largest file on my encrypted T: data drive is the paging file.
My original question about 4GB files was related to an encrypted backup data drive.
Cheers
Chris
You do not have the required permissions to view the files attached to this post.
An expensive day out: Wallet and Grimace

User avatar
StuartR
Administrator
Posts: 12577
Joined: 16 Jan 2010, 15:49
Location: London, Europe

Re: (Vera/Truecrypt) Store files larger than 4GB?

Post by StuartR »

EaseUs is not a file system, it is a program. A file system is something like NTFS or FAT which actually organizes the bits and bytes of physical disk space.
StuartR


User avatar
John Gray
PlatinumLounger
Posts: 5401
Joined: 24 Jan 2010, 08:33
Location: A cathedral city in England

Re: (Vera/Truecrypt) Store files larger than 4GB?

Post by John Gray »

StuartR wrote:
27 Jul 2020, 20:46
Why not just format the encrypted partition with NTFS and forget about all this
I'm beginning to think that this would solve All Known Problems... :thumbup:
John Gray

Venison is quiet deer, and quite dear.

PJ_in_FL
5StarLounger
Posts: 1090
Joined: 21 Jan 2011, 16:51
Location: Florida

Re: (Vera/Truecrypt) Store files larger than 4GB?

Post by PJ_in_FL »

IIRC, another benefit to NTFS is files smaller than a certain size (4K?) are actually stored in the "directory" which I believe is the Master File Table, and don't actually get allocated their own separate cluster.
PJ in (usually sunny) FL