Robocopy correct behaviour

[OneSecondStreet]

Win 10. Let's say I have my source directory E, which contains folders folder1, folder2, folder3 and various files which may or may not change between runs.

What is the correct robocopy command to copy all the folders and files in directory E, but NOT delete any folders on the backup drive which are not in directory E?

Currently I'm running a mir, but I accidentally just deleted a bunch of files in the directory on the backup drive which is not on the source drive, before I realised it.

Thankyou.

[ChrisGreaves]

What is the correct robocopy command to copy all the folders and files in directory E, but NOT delete any folders on the backup drive which are not in directory E?

I think that this is what I call "my nightly backup command"

Code: Select all

ROBOCOPY E: F: /r:0 /w:0 /s 

This supposes that your backup drive is F:

Currently I'm running a mir, but I accidentally just deleted a bunch of files in the directory on the backup drive which is not on the source drive, before I realised it.

Have you considered first backing up your hair. I neglected to do that back when I was 40 years old.

I use the /MIR to make a Mirror image of my data drive (T:) to my nightly backup drive (Y:) AFTER I have backed up Y: to my weekly cumulative backup Z:

This means that every Sunday night, my nightly backup drive is an exact copy of my data drive. If I have to restore my data drive T: from my backup drive Y: first thing Monday morning, everything is sweet. The worst case is if I restore data T: from nightly Y: Sunday afternoon, because then I will get some data files that I had house-cleaned away during the week. But at least I won't be getting cluttered up with years of archived or outdated files.

Might I ask: What brought about your decision to use the /MIR parameter?

In (drive) location is not available accessible I suggested using environment variables in batch files. If you post your current batch file I'll have a shot at modifying it for you.

Cheers, Chris

[kdock]

Have you considered first backing up your hair. I neglected to do that back when I was 40 years old.

Kim

[OneSecondStreet]

Have you considered first backing up your hair. I neglected to do that back when I was 40 years old.

Well, while one of my parents is, let's say, follically challenged, the other is most definitely not. In fact, I could probably go into business providing hair backups for others.

Might I ask: What brought about your decision to use the /MIR parameter?

You might well ask that. I made that decision after seeing it on some forum somewhere years ago. Clearly, that's the extent of my research skills these days.

If you post your current batch file I'll have a shot at modifying it for you.

OK, here are the details and the current command.

I have three backup drives: U (Ultimate), O (Offsite) and P (Pneumonoultramicroscopicsilicovolcanoconiosis).

I have the U plugged in "permanently" to my computer (I know, I know, if I get a virus that wipes out my computer it'll copy over to all my files...haven't sorted out this yet). It gets backed up each night using the command below.

The O is stored at work, and brought home once a week. It is then, along with the P, backed up using the same commands.

So the O (sitting at work in case the house burns down) and the P (sitting at home in case the house doesn't burn down) would be missing at most a week of stuff. The U would be missing at most a day.

My PC (desktop) has a partition on its drive so C holds the windows files and E is the one that holds my files. So it's essentially

robocopy "E:\Documents" "U:\Documents" /MIR /FFT /R:3 /W:10 /Z /NP /NDL
robocopy "E:\Music" "U:\Music" /MIR /FFT /R:3 /W:10 /Z /NP /NDL
robocopy "E:\Pictures" "U:\Pictures" /MIR /FFT /R:3 /W:10 /Z /NP /NDL
robocopy "E:\Videos" "U:\Videos" /MIR /FFT /R:3 /W:10 /Z /NP /NDL

This is what happens whenever I backup. The offsite drive also has a backup of my work computer files, which I do once a week. These are the ones that got deleted (but restored today at work).

I also have individual files on the E drive not sitting in any of those 4 folders. I don't currently have them backed up but would like to.

Please, feel free to simplify my life

[ChrisGreaves]

This response is excessively long, so I have broken it up into what I hope are digestible portions. I anticipate kind criticism from other Robocopy users.
1SS>>> I have the U plugged in "permanently" to my computer (I know, I know, if I get a virus that wipes out my computer it'll copy over to all my files...haven't sorted out this yet). It gets backed up each night using the command below.

If you were asking for my advice, I’d allow a backup drive to be plugged in, but ONLY while you are backing up or copying files. My backups, like my data partitions, are encrypted. As soon as I have recovered the file(s) I want, I dismount and unplug the drives. The next time I rise from my desk, the drive goes back into the fire-resistant safe.
You are using a fire-resistant safe, aren’t you?

1SS>>> The O is stored at work, and brought home once a week. It is then, along with the P, backed up using the same commands.

Excellent! I lived for years under the flight path to two of Toronto’s runways. I should have stored my backups at my accountant’s office, a mile away to the East of me.

1SS>>> So the O (sitting at work in case the house burns down) and the P (sitting at home in case the house doesn't burn down) would be missing at most a week of stuff. The U would be missing at most a day.

Alright. So the U is the daily backup; the O and P are the weekly backups. Correct? NO backup should EVER be stored in the same room as the computer. I learned this in Newcastle NSW back in 1968.

1SS>>> My PC (desktop) has a partition on its drive so C holds the windows files and E is the one that holds my files. So it's essentially

Backups are, therefore, from E to U and then from U on to O and P. Have I got that right? What follows is based on my understanding.

Code: Select all

robocopy "E:\Documents" "U:\Documents" /MIR /FFT /R:3 /W:10 /Z /NP /NDL
robocopy "E:\Music" "U:\Music" /MIR /FFT /R:3 /W:10 /Z /NP /NDL
robocopy "E:\Pictures" "U:\Pictures" /MIR /FFT /R:3 /W:10 /Z /NP /NDL
robocopy "E:\Videos" "U:\Videos" /MIR /FFT /R:3 /W:10 /Z /NP /NDL

Your daily backup can/should be condensed to a single command as follows:-

Code: Select all

robocopy E:\ U:\ /MIR /FFT /R:3 /W:10 /Z /NP /NDL

I removed the quotes; quotes are useful if/when there are spaces within a path, but since there are no paths, you won’t need quotes, right?
I removed the paths because you cannot honestly tell me that in your future you will not/never have other folders to be backed up. So just backup the entire data partition E: to the backup drives(1)
/MIR A file-for-file mirror image is a good idea, but I make that ONLY after I have accumulated the daily backup drive to the weekly backup drive. Once that is done, my daily backup drive is reset to be a mirror image of the data drive so that IF I need to restore the data partition it has never more than a week of clutter. This is what you are doing (somewhere I think), so the concept of /MIR is not foreign to you. Yes. I make TWO RoboCopy each Sunday night; one is the regular daily backup of new or changed files; the second is the /MIR once the daily backup is backed up to the weekly.
/FFT “Assume FAT File Times (2-second granularity).” Never heard of it (grin), but why are you concerned with time? I set my backups running and then wander off to pour another mug of tea. When I return the backup is completed.
/R:3 Retries. Why? Robocopy is being used for backups, right? If you anticipate retries perhaps you should be replacing that backup drive (or the data drive )
/W:0 Wait-between-retries: If you aren’t using retries, then you won’t need wait-times, right?
/Z Restartable copies? Why? I think that an IT guy backing up a LAN might be interested in restartable backups, but an individual home-operator? Apart from which “with /Z, failed copies will usually restart from” how are you detecting that your RoboCopy run failed, and if you aren’t detecting it, why bother with restartable?
/NP “use the /NP switch to suppress the display of progress information.” Why suppress the data? If you are off pouring a mug of tea, you won’t see it anyway, and if your logging data goes to CON (your screen) it is too fast to read, and if you are sending logging data to a file (which I do), then searching the file is much more efficient than staring, now that we have computers, yes?
/NDL “/NDL (No Directory List) suppresses output of the directory list. … so that problems can be pinpointed” I am not clear on this. Why suppress data at all? We have computers to process (sort, filter etc) data. But on top of that, how are you at pinpointing problems with RoboCopying?
Now I’m not being just my normal curmudgeonly self here, and I thank you for introducing me to several parameters of RoboCopy that, in 20? Years I’d not bothered to read. I have seen your “after seeing it on some forum somewhere years ago. Clearly, that's the extent of my research skills these days”, and I really do understand how we grab at responses on forums and run with them; done that all my life; and you are now doing it again!”.

(continued in next post)

[ChrisGreaves]

(continued)
So now I think that your nightly backup at home can be expressed as:-

Code: Select all

robocopy E:\ U:\ /R:0 /W:0 /S

I recommend that you obtain a copy of RoboCopy.doc, about ??? bytes from the RKTools.exe. The PDF is 311KB on my system so I can’t attach that. Zipped is not much better at 304 KB. If you can’t find a copy let me know and I will locate it or u/l to my web site for your d/l.

1SS>>> This is what happens whenever I backup. The offsite drive also has a backup of my work computer files, which I do once a week. These are the ones that got deleted (but restored today at work).

I make the assumption that you backup each night. There are people who say “Oh, I only backup up when I’ve done something really important on my computer”, to which I respond “Then why are you wasting your time doing things that are NOT important (to you) on your computer?”

The rest of the quoted paragraph suggests that you might have TWO backup systems in use – one operates at home and backs up your home computer (good!) and the other operates at work and backs up files at work (also good).
But then how do I interpret “These are the ones that got deleted (but restored today at work).”? This sounds as if you did something that failed, and your IT manager fixed your mistake by restoring from the IT backup.
I used to do local backups on a client’s network computer. When development was rapid I would create folders on the network and drag a copy of my development folder across to make named/timed snapshots 0912, 1037, 1158 and so on, so that I could roll back to an earlier time without having to wait for IT to get around to setting development back to 6pm the previous day.

1SS>>> I also have individual files on the E drive not sitting in any of those 4 folders. I don't currently have them backed up but would like to.

Surely. But see “can/should be condensed” above; just backup the entire drive already!

1SS>>> Please, feel free to simplify my life

You will live to regret that.

(1) This statement will change if you use one backup drive for several computers, in which case you will be tempted to backup to different folder trees for each machine:-

Code: Select all

Robocopy E:\ U:\HomeMachine\

Then

Code: Select all

Robocopy F:\ U:\WorkMachine\

Then

Code: Select all

Robocopy W:\ U:\WifesMachine\

Etc.
But even Penny-Pincher Greaves has got into the habit of buying 1+TB drives in pairs as SSD storage is so cheap, and since you have at least one off-site site, you have plenty of room to start a few 4”x8”x½“ drives, yes?

I hope this helps. There are more comments, but with all this, your home computer should be being backed up well. The weekly backups, including /MIR are a second step, once you are happy with your daily backups at home.

My copy of rktools.exe

Untitled.png

Cheers, Chris

[PJ_in_FL]

Chris,

I'm glad there is a bookmark feature in the lounge. Appreciate the long discussion of backups and using ROBOCOPY.

[ChrisGreaves]

I'm glad there is a bookmark feature in the lounge. Appreciate the long discussion of backups and using ROBOCOPY.

PJ, do I detect a certain :snideness: in yur response? If so, then you are welcome. It was an essay because (a) it was related to what remains a puzzling scenario (three backup streams spread scross two hard drives in two sites and (b) the original poster appeared to comprehend "/MIR /FFT /R:3 /W:10 /Z /NP /NDL", most of which I knew nothing - or at best little - about.
And I just can't bear the feeling that someone here knows more than do i!
That you feel the need to bookmark it suggests that you, like me, didn't know some of this stuff, (which raises you in my esteem) and that makes me fell strong enough to go back outside and tackle some more grass clippings.
Cheers, Chris

[OneSecondStreet]

PJ, do I detect a certain :snideness: in yur response?

For what it's worth, i didn't detect any snideness, i assumed it was genuine thanks...as I now offer

If you were asking for my advice, I’d allow a backup drive to be plugged in, but ONLY while you are backing up or copying files.

I know this is good practice in theory. I haven't done it so far because my desktop sits under my desk (with a TV screen monitor on top) and the backup drive plugs into the usb port under the desk. It's purely that I'm getting too old to crawl under my desk twice a day for this. I should probably invest in a longer cord so it can be plugged in sitting on top of my desk or something similar. There's also the issue of me forgetting to plug it in by the time the scheduled backup runs.

Alright. So the U is the daily backup; the O and P are the weekly backups. Correct?

Correct.

Backups are, therefore, from E to U and then from U on to O and P. Have I got that right?

Nope. from E to U daily. From E to O and from E to P weekly. Same backup batch file script just with different destination drive letters. Three different batch files.

Your daily backup can/should be condensed to a single command as follows

Good. Thankyou. I'll change my U code and try that tonight.

/FFT ............. how are you at pinpointing problems with RoboCopying?

Yep. I read what they did and didn't feel they would harm anything so I left them in.

robocopy E:\ U:\ /R:0 /W:0 /S

Since I want a mirror, what's wrong with

robocopy E:\ U:\ /MIR

Why do I need the /S? I want a mirror, so I don't mind if empty folders are copied over.

I make the assumption that you backup each night.

I backup to U each night, but to O and P only weekly.

The rest of the quoted paragraph suggests that you might have TWO backup systems in use – one operates at home and backs up your home computer (good!) and the other operates at work and backs up files at work (also good).

Sort of. I bring O home from work on Tuesday, and do my weekly backups onto O and P that night. I take O back to work on Wednesday, and then backup myself, using robocopy, my work files. This is a separate folder on O (which got partially deleted and which precipitated this thread).

your IT manager fixed your mistake by restoring from the IT backup

I appreciate the laugh. My IT manager wouldn't lift a finger to help in any way... Besides which, none of this has anything to do with him. I'm backing this up myself. Their response would be "why bother? It's being backed up to the cloud..."

So, if I use

robocopy E:\ U:\

in my batch file to my O drive, will that
1) backup everything from my E drive to my O drive (as a backup), and
2) NOT delete any extra folders on my work drive (eg. "Work Files Backup")

[ChrisGreaves]

Nope. from E to U daily. From E to O and from E to P weekly. Same backup batch file script just with different destination drive letters. Three different batch files.

That’s Fine. But if the procedures are identical except for the destination drive, why not make the drive letter a parameter of the batch file?
The following batch file is untested.

Code: Select all

REM 20240620.bat - backup batch file; the only parameter MUST be a valid drive letter, but not E or ...
if "%1"=="O" goto 20_MakeBackup
if "%1"=="P" goto 20_MakeBackup
if "%1"=="U" goto 20_MakeBackup
echo %0 - Drive letter must be one of O, P, or U
pause
goto 90Exit

:20_MakeBackup
pause
RoboCopy e:\ %1\

:90exit
REM end of 20240620.bat

You could then type at the command prompt:-

Code: Select all

20240620.bat U

Or

Code: Select all

20240620.bat P

Or

Code: Select all

20240620.bat O

Or have a batch file called Weekly.bat:-

Code: Select all

REM Weekly bat
Call 20240620.bat U
Call 20240620.bat P
Call 20240620.bat O
REM End of Weekly bat

Note: Batch files are now old-fashioned; Scripts are the new thing, and possibly quite good, too, but you seem to be familiar with batch files, so I have used batch files as examples.

Yep. I read what they did and didn't feel they would harm anything so I left them in.

Fair enough. I wasn’t sure/didn’t know that.

Since I want a mirror, what's wrong with

Nothing. I am used to doing a daily backup to 465GB drive, then a cumulative backup to 931GB drive. I run the /MIR only after I have accumulated ALL files on the weekly backup 931GB.

Why do I need the /S? I want a mirror, so I don't mind if empty folders are copied over.

Understood. I am in the habit of populating \MUSIC\ and \CAMERA\ a month in advance with empty folders.
I have a scheduled backup to an online backup that runs at regular intervals

I backup to U each night, but to O and P only weekly.

So “A mirror image backup runs automatically once a day to an online backup drive; I make a mirror image once a week to two off-line drives, one of which is stored off-site. " Am I getting close?

Sort of. I bring O home from work on Tuesday, and do my weekly backups onto O and P that night. I take O back to work on Wednesday, and then backup myself, using robocopy, my work files. This is a separate folder on O (which got partially deleted and which precipitated this thread).

So: is P a second drive kept at home? O travels from and to work each week, correct? So that O is a MIRor image stored off-site. If P is kept at home, then it is identical (in content) to U, except that P, like O, is unplugged 99% of the time. U is plugged in 100%.

So, if I use
robocopy E:\ U:\
in my batch file to my O drive, will that
1) backup everything from my E drive to my O drive (as a backup), and
2) NOT delete any extra folders on my work drive (eg. "Work Files Backup")

I am not sure of my answer; my general understanding of RobCopy is that, wherever possible, it copies whatever it can, so I would expect the default (neither /S nor /E switches) to copy everything, including empty folders, from the source drive to the target drive.
There is, of course, an easy way to find out.
(1) delete those empty folders (if they exist) from the backup drive U:
(2) run the command “robocopy E:\ U:\”
(3) inspect the backup drive to see if the empty folders have been copied across.
I think that we are getting close.
Cheers, Chris

[ChrisGreaves]

The following batch file is untested.

I am not sure how you feel about batch/script files. I feel that they are great time-savers on two counts:-
(1) They save time typing commands manually every day, week, ...
(2) Once tested, they avoid typing mistakes, and in my humble opinion, fixing the results of typing errors is perhaps the biggest waste of time sitting at a computer.

In these examples, a backup command with /MIR to the wrong path could be disastrous.
Cheers, Chris

[PJ_in_FL]

I'm glad there is a bookmark feature in the lounge. Appreciate the long discussion of backups and using ROBOCOPY.

PJ, do I detect a certain :snideness: in yur response? If so, then you are welcome. It was an essay because (a) it was related to what remains a puzzling scenario (three backup streams spread scross two hard drives in two sites and (b) the original poster appeared to comprehend "/MIR /FFT /R:3 /W:10 /Z /NP /NDL", most of which I knew nothing - or at best little - about.
And I just can't bear the feeling that someone here knows more than do i!
That you feel the need to bookmark it suggests that you, like me, didn't know some of this stuff, (which raises you in my esteem) and that makes me fell strong enough to go back outside and tackle some more grass clippings.
Cheers, Chris

No snideness intended.

I'm unfortunately a poor backer-upper, and seeing how you set up your backups give me a start to getting my own under control.

[ChrisGreaves]

I'm unfortunately a poor backer-upper, and seeing how you set up your backups give me a start to getting my own under control.

I recommend that you start by :-
(1) obtaining a copy of RKTools.exe
(2) telling us where you found it
(3) setting aside a 10 GB USB memory key
(4) noting its drive letter when inserted
(5) trying the suggested

Code: Select all

robocopy E:\ U:\ /MIR

where E:\ can be replaced by D:\PJ\Documents\MSWord\ or similar
and
U: can be replaced with the drive noted in (4) above.

See also Advanced Search.
Cheers, Chris
P.S. My 'phone says "32◦", but I think that that is the outside temperature right now. It's a bit like being on holiday in Florida! C

[OneSecondStreet]

I think I've got the right syntax:

robocopy E:\ P:\ /E /DCOPY:DAT

The issue is it seems to be stalling and it's to do with recycle bins...

image3.png

I'm not sure if I'm supposed to exclude the directory E:\$RECYCLE.BIN or not. I've not encountered this previously. Any ideas?

[ChrisGreaves]

robocopy E:\ P:\ /E /DCOPY:DAT
The issue is it seems to be stalling and it's to do with recycle bins...
I'm not sure if I'm supposed to exclude the directory E:\$RECYCLE.BIN or not. I've not encountered this previously. Any ideas?

Yes.
(1) My RoboCopy.PDF file does not tell me about the /DCOPY switch, so perhaps you could tell me?
(2) I can understand that Windows may not want to copy the recycle bin, but I can't come up with a logical reason why it ought not to be able to make a copy. I usually have a $Recycle bin or two floating around on a backup drive, in the root folder AND sometimes in a sub-folder such as "DELL075" or similar (representing a separate machine that has been backed up to a folder), or one of those "S345784r9827503750" types of recycle files/folders.
(3) While I am not using the DCOPY parameter, I do use the R:0 and W:0 parameter stringss.
(4) Here are some lines pasted directly from my one batch file that does both daily and weekly backups:

Code: Select all

%robocopy% c:\users\%username%\appdata\roaming T:\users\%username% /r:0 /w:0 /s /log:%LoggingFile%
%robocopy% t:\      y:\%computername% /s /r:0 /w:0  /A-:SH /log:%LoggingFile%
%robocopy% y: z: /s /r:0 /w:0 /M  /log:%LoggingFile%
%robocopy% t:\      y:\%computername% /s /r:0 /w:0 /XJ /MIR /M   /log:%LoggingFile%

We note that they all include "/r:0 /w:0 ".
Try it; it might work.
Then of course there is the philosophical question of "Do we care whether or not our recycle bin is copied (completely or partially)?" For me, if ever I need to recover a recycle bin to get an older copy of a file, it would mean that the file itself was somehow missing from the backup file.
Cheers, Chris

[OneSecondStreet]

Well, I finally got it sorted after some strange behaviour. I tried it with the source being E:\ only. It copied over system files and hidden files and for some reason also turned some folders hidden and "system". In the end I've made a new folder "MyStuff" in the E drive, and done a copy from there to corresponding on P. It all works fine now, at least I'm happy with it. This obviously also means it won't affect anything else on the P drive, which was the original concern.

Thanks for all your help with it!