Stop rsync from deleting large amounts of data when disk accidentally umounted
I use rsync to mirror my hard drive onto a couple of devices with cron.
The command is essentially this:
rsync -avr --delete /home/username username@example.com:/home/username
The problem I have is that some of the directories within the structure are on separate drives. For example /home/username/portable is a USB stick. As a result, if the cron job runs when that file system is umounted, a large amount of data is deleted from the mirror and then needs to be re-copied when it is next run with the drive mounted. This is time consuming and, of course, I assume that when I need to restore from the mirror it will be immediately after this has happened!
Could anyone suggest a good way to militate against this problem? I appreciate that two possible ways would be to see if
- The device is mounted by grep’ing df or similar; or
- The target directory has no contents by running du /home/username/portable or similar.
Whilst these are probably the Correct Way of dealing with this issue, both of these require manual adjustment every time a new mount point is added to the file system which does not appeal to my lazy outlook.
What I was particularly wondering is whether there is a way of getting rsync to test how much data is going to be deleted from the remote drive by running --delete? That way, the cron script could be tweaked to require manual involvement if, say, more than 1GB is going to be deleted. It would therefore provide a sanity-check against large-scale accidental data loss.
I’ve had a look through the manual but can’t see a straight-forward way of doing this. Can anyone help?
Thanks!
linux unix backup rsync mirroring
asked Dec 31 '18 at 15:19
SarrusophoneSarrusophone
61
add a comment |
I use rsync to mirror my hard drive onto a couple of devices with cron.
The command is essentially this:
rsync -avr --delete /home/username username@example.com:/home/username
The problem I have is that some of the directories within the structure are on separate drives. For example /home/username/portable is a USB stick. As a result, if the cron job runs when that file system is umounted, a large amount of data is deleted from the mirror and then needs to be re-copied when it is next run with the drive mounted. This is time consuming and, of course, I assume that when I need to restore from the mirror it will be immediately after this has happened!
Could anyone suggest a good way to militate against this problem? I appreciate that two possible ways would be to see if
- The device is mounted by grep’ing df or similar; or
- The target directory has no contents by running du /home/username/portable or similar.
Whilst these are probably the Correct Way of dealing with this issue, both of these require manual adjustment every time a new mount point is added to the file system which does not appeal to my lazy outlook.
What I was particularly wondering is whether there is a way of getting rsync to test how much data is going to be deleted from the remote drive by running --delete? That way, the cron script could be tweaked to require manual involvement if, say, more than 1GB is going to be deleted. It would therefore provide a sanity-check against large-scale accidental data loss.
I’ve had a look through the manual but can’t see a straight-forward way of doing this. Can anyone help?
Thanks!
linux unix backup rsync mirroring
asked Dec 31 '18 at 15:19
SarrusophoneSarrusophone
61
add a comment |
I use rsync to mirror my hard drive onto a couple of devices with cron.
The command is essentially this:
rsync -avr --delete /home/username username@example.com:/home/username
The problem I have is that some of the directories within the structure are on separate drives. For example /home/username/portable is a USB stick. As a result, if the cron job runs when that file system is umounted, a large amount of data is deleted from the mirror and then needs to be re-copied when it is next run with the drive mounted. This is time consuming and, of course, I assume that when I need to restore from the mirror it will be immediately after this has happened!
Could anyone suggest a good way to militate against this problem? I appreciate that two possible ways would be to see if
- The device is mounted by grep’ing df or similar; or
- The target directory has no contents by running du /home/username/portable or similar.
Whilst these are probably the Correct Way of dealing with this issue, both of these require manual adjustment every time a new mount point is added to the file system which does not appeal to my lazy outlook.
What I was particularly wondering is whether there is a way of getting rsync to test how much data is going to be deleted from the remote drive by running --delete? That way, the cron script could be tweaked to require manual involvement if, say, more than 1GB is going to be deleted. It would therefore provide a sanity-check against large-scale accidental data loss.
I’ve had a look through the manual but can’t see a straight-forward way of doing this. Can anyone help?
Thanks!
linux unix backup rsync mirroring
asked Dec 31 '18 at 15:19
SarrusophoneSarrusophone
61
I use rsync to mirror my hard drive onto a couple of devices with cron.
The command is essentially this:
rsync -avr --delete /home/username username@example.com:/home/username
The problem I have is that some of the directories within the structure are on separate drives. For example /home/username/portable is a USB stick. As a result, if the cron job runs when that file system is umounted, a large amount of data is deleted from the mirror and then needs to be re-copied when it is next run with the drive mounted. This is time consuming and, of course, I assume that when I need to restore from the mirror it will be immediately after this has happened!
Could anyone suggest a good way to militate against this problem? I appreciate that two possible ways would be to see if
- The device is mounted by grep’ing df or similar; or
- The target directory has no contents by running du /home/username/portable or similar.
Whilst these are probably the Correct Way of dealing with this issue, both of these require manual adjustment every time a new mount point is added to the file system which does not appeal to my lazy outlook.
What I was particularly wondering is whether there is a way of getting rsync to test how much data is going to be deleted from the remote drive by running --delete? That way, the cron script could be tweaked to require manual involvement if, say, more than 1GB is going to be deleted. It would therefore provide a sanity-check against large-scale accidental data loss.
I’ve had a look through the manual but can’t see a straight-forward way of doing this. Can anyone help?
Thanks!
linux unix backup rsync mirroring
linux unix backup rsync mirroring
asked Dec 31 '18 at 15:19
SarrusophoneSarrusophone
61
asked Dec 31 '18 at 15:19
SarrusophoneSarrusophone
61
asked Dec 31 '18 at 15:19
SarrusophoneSarrusophone
61
asked Dec 31 '18 at 15:19
SarrusophoneSarrusophone
61
asked Dec 31 '18 at 15:19
SarrusophoneSarrusophone
61
61
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "3"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1389286%2fstop-rsync-from-deleting-large-amounts-of-data-when-disk-accidentally-umounted%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Super User!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1389286%2fstop-rsync-from-deleting-large-amounts-of-data-when-disk-accidentally-umounted%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
