RMIRR
- Introduction
- Why not just use rsync directly?
- Usage
- Configuration
- Examples
- Mirroring Selected File Objects
Introduction
rmirr provides a safe and easy to use way to set up and execute file storage mirroring instructions. And because it uses rsync to actually perform the synchronization, you get all the performance and availability benefits of rsync.
Features:
- specify only the source file object (file, directory) to mirror
- mirror settings selected by best match path
- support one or more mirror destinations
- ensure mirroring from specific sources only
- ensure single instance of mirroring operation per mirror path
- log history of mirroring invocations
- save to file report of mirrored objects
- email report file
Why not just use rsync directly?
Simple cases of mirroring files/directories can easily be done by using rsync directly. But, it does not provide any safeguards. For example, the following are not the same:
and:
Assuming a directory is being synced, call #2 is correct, as it uses a trailing / on the source path. rmirr ensures that will be done properly.
Usage
rmirr [<options>] [-n <name>] -s <suitename>
rmirr [<options>] -n <name>
rmirr -l
Mirror file objects which can be specified by <path> (for which a
mirror is selected), <suite> (representing one or more paths), or
<name> (a mirror configuration).
Where:
-l List mirror configurations.
-n <name>
Mirror name.
-p <path>
Mirror path.
-s <suitename>
Alias for one or more paths.
Options:
-c <path>
Location of paths.json configuration file.
-d <hostname>[,...]
Destinations to mirror to from list in configuration.
--debug
Enable debugging.
--delete
Allow file object deletion.
--dry Dry run. Do not execute.
--dry-rsync
Dry run for rsync.
--mailto <emailaddr>[,...]
Set/override recipients when mailing report.
--mailreport
Mail report.
--nolock
Do not use/require lock to run.
--safeoff
Disable safemode.
--showreport
Show report on console.
--verbose
Enable verbosity.
-y Do not ask for confirmation before executing.
Configuration
Configuration file is located at ~/.rmirr/rmirr.json.
Section | Name | Description |
|---|---|---|
defaults | email_recipients | List of email recipients when --mailreport is used. |
smtp_server | Set SMTP server. Default is to use "localhost". | |
suites | <suitename> | List of paths to be mirrored. |
mirrors | comment | Description of what is being mirrored. |
destinations | List of destinations to which the source is to be mirrored each of the format [<user>@]]<host>:<path>. | |
email_recipients | List of email recipients when --mailreport is used. Overrides the value from the defaults section. | |
excludes | List of file/dir names to exclude from the mirroring. | |
name | Mirror configuration name. | |
source | Source of the mirror: [<user>@]<host>:<path>. |
Examples
Given the configuration at ~/.rmirr/rmirr.json:
"defaults": {
"email_recipients": []
},
"suites": {
"all": [
"/data/main",
"/data/finance",
"/data/hr"
]
},
"mirrors": [
{
"name": "data-main--home--finance-hr",
"comment": "data main: home -> finance, hr",
"source": "data@home.abc.xyz:/data/main",
"excludes": [".snapshot", ".snapshots"],
"destinations": ["finance.abc.xyz", "hr.abc.xyz"]
},
{
"name": "data-finance--home--finance",
"comment": "data finance: home -> finance",
"email_recipients": ["bob@finance.abc.xyz"],
"source": "data@home.abc.xyz:/data/finance",
"excludes": [".snapshot", ".snapshots"],
"destinations": ["finance.abc.xyz"]
},
{
"name": "data-hr--home--hr",
"comment": "data hr: home -> hr",
"source": "data@home.abc.xyz:/data/hr",
"excludes": [".snapshot", ".snapshots"],
"destinations": ["hr.abc.xyz"]
}
]
}
List suite and mirror settings:
Mirror /data/main from home.abc.xyz:
Notes:
- mirrors to both finance.abc.xyz and hr.abc.xyz
- .snapshot and .snapshots are excluded
- by default, mirroring does not delete at the destination
- by default, mirroring requires user interaction ("y" to proceed)
Show what rmirr would do without executing it (dry-run):
Mirror /data/main/opt:
Notes:
- because /data/main/opt is under /data/main, the settings for /data/main are used
- only /data/main/opt is mirrored
Mirror /data/finance and allow deletion at the destination:
Do a dry run (at rsync level) to mirror /data/finance with deletion and no user interaction:
Mirror /data/main to specific destination (hr.abc.xyz):
Request report be emailed:
Request report be output to console:
Mirror the "all" suite:
Mirroring Selected File Objects
Sometimes only a subset of file objects (files, directories) need to be mirrored rather than a full tree. rmirr supports this with the names setting in the configuration file.
Configuration for mirroring bash files:
"suites": {
"bashfiles": [
"~/.bash_completion.d",
"~/.bash_profile",
"~/.bash_logout",
"~/.bashrc",
"~/.profile",
"~/.profile.d"
]
},
"mirrors": [
{
"name": "my-bash-files",
"comment": "my .bash files",
"source": "jdm@home.abc.xyz:~",
"excludes": [
".snapshot",
".snapshots"
],
"destinations": [
"finance.abc.xyz",
"hr.abc.xyz"
]
}
]
}
Notes:
- the suites section defines a suite to reference those bash-specific file objects by a single name
- of course, any changes/tweaks done at the destinations will be overwritten when rmirr is called
To mirror all the file objects of the "bashfiles" suite:
To mirror specific file objects: