Eggs and Baskets

[BobH]

Because I've been looking at hardware upgrades recently, I've run across 2TB hard drives from Western Digital (Caviar Black) for $140. I've seen Seagate drives of he same size for $80 (and in my experience not worth half that). I also note that the new standard for writing to HDs will write 4KB blocks of data instead of 512 bytes.

The latter doesn't concern me overmuch but it does seem that this is going to leave larger chunks of unusable space when partial blocks are written. Given the price, size, and speed of HD's today the lost space is not likely to be a significant factor, but it does seem, somehow, wasteful. I wonder if any studies have been done - and results published - that would shed light on how many partial blocks are written as a percentage of total blocks written on media. I suppose that faster channel and bus speeds make moving larger blocks of data more efficient and thereby warrant the trade off in lost addressable disk space making the overall function for efficient and more economic.

As to the former, I'm a bit leary of such large HDs. It seems to me that failures occur in these devices that render the devices entirely unusable (when they occur). How does it make sense to have such high capacity drives (unless MTBF is greatly reduced in the large drives - and I've seen no data to suggest that)? Wouldn't it be more advisable from a data management perspective to have multiple smaller drives? Maybe in a RAID5 implementation where data is redundantly stored and failed or failing drives can be replaced 'on the fly' these large capacity drives would be ok; but, then again, wouldn't it make sense to have a larger storage cabinet and more, smaller drives?

I'm certainly not an engineer, but these two developments seem to be inconsistent with what I thought I knew about the technology. Does anyone else share these reservations about putting so many eggs in one basket?

[StuartR]

A few things to think about here.

• The bigger block size means that the metadata describing where everything lives on the disk can be much smaller, so you won't be wasting quite as much space as you suspect
• The bigger block size means less fragmentation and less time needed to allocate space on the disk when creating and extending files
• RAID5 really isn't suitable for home users. Unless you are in a position to document the exact recovery steps you will take when a disk fails and to test these steps every few months you are too likely to lose all your data when something goes wrong.
• Many people store lots of audio and video files, which can be quite large, not too much space will be wasted by having large block sizes for these files. Also this may be sufficient reason for using a very large hard drive
• I use a very large disk drive for storing all my backups, this enables me to keep multiple generations of backups, some of these go back a very long time. Then if I discover that something became corrupted a long time ago and I didn't notice at the time I have a chance of recovering.

[Timelord]

You do have the ability to change the block size that is written. I used to know where and will try to remember / find that setting. When I do I will post it.

[ChrisGreaves]

Does anyone else share these reservations about putting so many eggs in one basket?

What Stuart says, PLUS:-

It's not that I'm paranoid or anything, but I don't sleep so good nights unless I've backed up to twin external drives and they are parked in my little fire-proof box in the closet in the next room.

Regardless of the quality of the manufacturer's process, I figure know that drives are pre-programmed to fail at the worst possible time.
So to have TWO drives is my primary tactic.

I guess you could call that "Having all my eggs in TWO baskets".

P.S. I recognize that a drive is more likely to fail through my handling it twice each evening than from any manufacturer's defect.
I have no figures to support this; just a hunch that I am clumsier than a finely-engineered piece of metal.

[Timelord]

Got it, you can select to format the drive. In the format window is an option for the allocation size. Here you can change that size.

[Skitterbug]

It's not that I'm paranoid or anything, but I don't sleep so good nights unless I've backed up to twin external drives and they are parked in my little fire-proof box in the closet in the next room.

What's the point in placing an EX. HD in a fire-proof box? If you had the tragic event of a fire, wouldn't the little fire-proof box still be heated to the point to cause ruin to the EX. HD and the data it contains? :just curious:

[ChrisGreaves]

What's the point in placing an EX. HD in a fire-proof box? If you had the tragic event of a fire, wouldn't the little fire-proof box still be heated to the point to cause ruin to the EX. HD and the data it contains?

Hello "Just Curious" from Ohio.

I think it most unlikely that a fire would cause any damage.

Here's a thought experiment:

Picture my two ED/HD in a thin sheet steel box (two sheets, about 1mm thick, with about 1/2 cm of insulation between them, the box sitting in a closet with the closet door closed in a bedroom.

Now picture a lighted match in the kitchen.
No chance of damage, right?
Now picture a fire in the kitchen.
No chance of damage, right?
Now picture a raging inferno in the kitchen
No chance of damage, right?

Only if a fire spreads to the bedroom do we have a problem.
And first it has to penetrate the closet door.
Now the fire has spread and has burned away the closet door.
But still the steel box with its tiddy-little insulation resists the heat.

But by now the fire-brigade have arrived, and the apartments are all steel-walled, concrete etc, so unless the fire starts in my bedroom while I'm away, I've very little risk of loss of the data.

To my mind I'm in most danger when I have the ED near my computer; when I'm being stupid and deleting files I didn't oughta, I'm also stupid enough to mess up the ED.
Making myself walk next door gives me time to think and eliminates 99.99% of my problems.

If a fire did break out in the apartment, I'd probably toss the safe over the balcony, and reclaim it from the lawn the next day.

I am under two flight paths here, so I have contemplated leaving an ED at the accountant's office a mile away, but that seems to me to be ultra-paranoid.

After all, when was the last time a plane crashed into this bui

[Skitterbug]

What's the point in placing an EX. HD in a fire-proof box? If you had the tragic event of a fire, wouldn't the little fire-proof box still be heated to the point to cause ruin to the EX. HD and the data it contains?

Hello "Just Curious" from Ohio.

I think it most unlikely that a fire would cause any damage.

Here's a thought experiment:

Picture my two ED/HD in a thin sheet steel box (two sheets, about 1mm thick, with about 1/2 cm of insulation between them, the box sitting in a closet with the closet door closed in a bedroom.

Now picture a lighted match in the kitchen.
No chance of damage, right?
Now picture a fire in the kitchen.
No chance of damage, right?
Now picture a raging inferno in the kitchen
No chance of damage, right?

Only if a fire spreads to the bedroom do we have a problem.
And first it has to penetrate the closet door.
Now the fire has spread and has burned away the closet door.
But still the steel box with its tiddy-little insulation resists the heat.

But by now the fire-brigade have arrived, and the apartments are all steel-walled, concrete etc, so unless the fire starts in my bedroom while I'm away, I've very little risk of loss of the data.

To my mind I'm in most danger when I have the ED near my computer; when I'm being stupid and deleting files I didn't oughta, I'm also stupid enough to mess up the ED.
Making myself walk next door gives me time to think and eliminates 99.99% of my problems.

If a fire did break out in the apartment, I'd probably toss the safe over the balcony, and reclaim it from the lawn the next day.

I am under two flight paths here, so I have contemplated leaving an ED at the accountant's office a mile away, but that seems to me to be ultra-paranoid.

After all, when was the last time a plane crashed into this bui

Well.......now I know your reasoning for your choice of security measures! You've certainly have put a lot of thought into it! Hopefully, you never have to test any of it!

[jonwallace]

What's the point in placing an EX. HD in a fire-proof box? If you had the tragic event of a fire, wouldn't the little fire-proof box still be heated to the point to cause ruin to the EX. HD and the data it contains?

...

I am under two flight paths here, so I have contemplated leaving an ED at the accountant's office a mile away, but that seems to me to be ultra-paranoid.

After all, when was the last time a plane crashed into this bui

My escape plan involves forgetting all my data, retrieving that fake passport from the storage locker and making a new life for myself somewhere temperate. but don't tell the wife...

[ChrisGreaves]

Hopefully, you never have to test any of it!

Au contraire ma petite!
It gets tested every week, at least, as should all backup systems.
At least once a week I shoot myself in the foot and get up, walk away, calm down, retrieve a backup drive and restore a file. That restoration of a recently-modified file re-assures me that my backups are working. Without that incident I wouldn't know if the files were getting written.

I did some work for a medical diagnosis clinic ten years ago (Think: "Patient medical-history records"), and on the second day casually asked about backups. "Oh Yes! Every day I pop a tape in here, one for Monday, one for Tuesday, etc and go home. In the morning I take it out ...".
Sure enough I shot myself in the foot. On purpose or by accident, can't remember which.
Waddyaknow! Nothing on the previous night's tape. Nor the other 4 tapes.
For several years they'd been putting in a tape and walking away without ever checking to see that backups were being made.

That made the feathers fly for a few days!

Reminds me of the old cartoon that has a SysOps guy evilly lurching towards the power line coming out of the wall socket: "You've got backup systems? let's see if they work ..."

[ChrisGreaves]

My escape plan involves forgetting all my data,

I confess when things get rough, to a reverie of being able to abandon 40+ years of punched cards etc. and re-starting as curator of a small farm, or a librarian in a small town, or ...
A bit like John Grisham's hero's THIRD start to life in "The Partner"

[ChrisGreaves]

Hopefully, you never have to test any of it!

And now this:

Airline tycoon Richard Branson: "Running a business, we have a meticulous computer backup system so I'd assumed all that was completely safe. But it turns out the backup was also in the house."

[Skitterbug]

Hopefully, you never have to test any of it!

And now this:

Airline tycoon Richard Branson: "Running a business, we have a meticulous computer backup system so I'd assumed all that was completely safe. But it turns out the backup was also in the house."

He notes that he plans on buying a fire proof vault but I'm still wondering if delicate back up medium would even survive in such a device if it had to go through a fire such as he had.
Our friends had a home fire and everything that had any weight to it fell through the main floor and into their basement. The flames were dark blue and very hot. In spite of efforts to put the flames out, the coals smoldered for a few days after. So do you think back up material would survive even if in a special vault?

[DaveA]

So do you think back up material would survive even if in a special vault?

Yes, if it was built like a "Flight Data Recorder" box! http://en.wikipedia.org/wiki/Flight_data_recorder" onclick="window.open(this.href);return false;

[Skitterbug]

So do you think back up material would survive even if in a special vault?

Yes, if it was built like a "Flight Data Recorder" box! http://en.wikipedia.org/wiki/Flight_data_recorder" onclick="window.open(this.href);return false;

I wonder how many home vaults are built to these standards?

[ChrisGreaves]

So do you think back up material would survive even if in a special vault?

No, I doubt they would survive, any more than my backups would survive a plane crash while taking off/landing at the airport. I'm literally under the approach for 33L.
The aim is not to survive the worst event ever possible (I suppose being struck by an asteroid would be about it), but to make an economic choice as to the expense vs. surety of survival).

I think I'm less likely to have a fire in my apartment than most other people in this building (I don't smoke, I don't use electric heaters or fans, and cook only twice a week), and my limited experience with apartment building fires is that one apartment might "melt", but the adjacent apartments are merely smoke-damaged.
So for me a small steel box is probably all the cover I need.

[Sundog]

One March day I got curious about our IT Dept. backups on a big project with a lot of time put in. Upon investigation, the last backup that could be found was run on December 31. Nothing was run from January to March. A line in the auto-executing code specified the year, and no one from IT had changed it on January 1.

Fortunately my curiosity was not based on a need for a backup.

[ChrisGreaves]

... and no one from IT had changed it on January 1. Fortunately my curiosity was not based on a need for a backup.

Classic!
Typical!

Remind IT that they should make a backup before they go changing the batch file.

[BobH]

Heads should roll in the IT department!

I mean that seriously. I worked in IT from 1964 into 2001 at varying levels of responsibility. Checking and verifying backups was just a routine part of the scheduled work performed by operators. We occasionally found bad media that caused lost data be we always had earlier generations that could be updated to bring us current.

Not having run backups and tested them on a daily basis is a criminal lapse for anyone who claims to earn their pay from IT. Writing a routine that requires an annual date change to run correctly is criminally stupid for anyone who claims a paycheck from IT. I would not have hesitated to fire as many as were involved . . . especially the 'manager' responsible.

There is no acceptable excuse for such lapses.

[ChrisGreaves]

Heads should roll in the IT department!

Hi Bob. I agree with your sentiments.

The devil in me wants to ask "Would they be dual head drives?"

The case I had 10 years back, I found myself thinking that the medical imaging lab should have sued/clawed back the installation consultant's fee.