Show last authors
1 {{info}}
2 2018-12-20: Version 0.5 released.
3 {{/info}}
4
5 {{warning title="Disclaimer"}}
6 As with any copy/sync tool, make sure to test thoroughly with data that can be lost.
7
8 Test with the ##~-~-dry## and ##~-~-dry-rsync## options to verify that the operations match what you want to do.
9 {{/warning}}
10
11 {{toc-local/}}
12
13 = Introduction =
14
15 rmirr provides a **safe** and easy to use way to set up and execute file storage mirroring instructions. And because it uses rsync to actually perform the synchronization, you get all the performance and availability benefits of rsync.
16
17 Features:
18
19 * specify only the source file object (file, directory) to mirror
20 * mirror settings selected by best match path
21 * support one or more mirror destinations
22 * ensure mirroring from specific sources only
23 * ensure single instance of mirroring operation per mirror path
24 * log history of mirroring invocations
25 * save to file report of mirrored objects
26 * email report file
27
28 = Why not just use rsync directly? =
29
30 Simple cases of mirroring files/directories can easily be done by using rsync directly. But, it does not provide any safeguards. For example, the following are not the same:
31
32 {{noformat}}
33 rsync -avz --delete /a/b/c mach2:/a/b/c
34 {{/noformat}}
35
36 and:
37
38 {{noformat}}
39 rsync -avz --delete /a/b/c/ mach2:/a/b/c
40 {{/noformat}}
41
42 Assuming a directory is being synced, call #2 is correct, as it uses a trailing / on the source path. rmirr ensures that will be done properly.
43
44 = Usage =
45
46 {{noformat}}
47 usage: rmirr [<options>] [-n <name>] -p <path>
48 rmirr [<options>] [-n <name>] -s <suitename>
49 rmirr [<options>] -n <name>
50 rmirr -l
51
52 Mirror file objects which can be specified by <path> (for which a
53 mirror is selected), <suite> (representing one or more paths), or
54 <name> (a mirror configuration).
55
56 Where:
57 -l List mirror configurations.
58 -n <name>
59 Mirror name.
60 -p <path>
61 Mirror path.
62 -s <suitename>
63 Alias for one or more paths.
64
65 Options:
66 -c <path>
67 Location of paths.json configuration file.
68 -d <hostname>[,...]
69 Destinations to mirror to from list in configuration.
70 --debug
71 Enable debugging.
72 --delete
73 Allow file object deletion.
74 --dry Dry run. Do not execute.
75 --dry-rsync
76 Dry run for rsync.
77 --mailto <emailaddr>[,...]
78 Set/override recipients when mailing report.
79 --mailreport
80 Mail report.
81 --nolock
82 Do not use/require lock to run.
83 --safeoff
84 Disable safemode.
85 --showreport
86 Show report on console.
87 --verbose
88 Enable verbosity.
89 -y Do not ask for confirmation before executing.
90 {{/noformat}}
91
92 = Configuration =
93
94 Configuration file is located at ##~~/.rmirr/rmirr.json##.
95
96 (% class="wrapped" %)
97 |=(((
98 Section
99 )))|=(((
100 Name
101 )))|=(((
102 Description
103 )))
104 |(% rowspan="2" %)(% rowspan="2" %)
105 (((
106 defaults
107 )))|(((
108 email_recipients
109 )))|(((
110 List of email recipients when ~-~-mailreport is used.
111 )))
112 |(% colspan="1" %)(% colspan="1" %)
113 (((
114 smtp_server
115 )))|(% colspan="1" %)(% colspan="1" %)
116 (((
117 Set SMTP server. Default is to use "localhost".
118 )))
119 |(% colspan="1" %)(% colspan="1" %)
120 (((
121 suites
122 )))|(% colspan="1" %)(% colspan="1" %)
123 (((
124 <suitename>
125 )))|(% colspan="1" %)(% colspan="1" %)
126 (((
127 List of paths to be mirrored.
128 )))
129 |(% rowspan="6" %)(% rowspan="6" %)
130 (((
131 mirrors
132 )))|(((
133 comment
134 )))|(((
135 Description of what is being mirrored.
136 )))
137 |(% colspan="1" %)(% colspan="1" %)
138 (((
139 destinations
140 )))|(% colspan="1" %)(% colspan="1" %)
141 (((
142 List of destinations to which the source is to be mirrored each of the format [<user>@]]<host>:<path>.
143 )))
144 |(((
145 email_recipients
146 )))|(((
147 List of email recipients when ~-~-mailreport is used. Overrides the value from the defaults section.
148 )))
149 |(((
150 excludes
151 )))|(((
152 List of file/dir names to exclude from the mirroring.
153 )))
154 |(((
155 name
156 )))|(((
157 Mirror configuration name.
158 )))
159 |(((
160 source
161 )))|(((
162 Source of the mirror: [<user>@]<host>:<path>.
163 )))
164
165 = Examples =
166
167 Given the configuration at ##~~/.rmirr/rmirr.json##:
168
169 {{code language="none"}}
170 {
171 "defaults": {
172 "email_recipients": []
173 },
174 "suites": {
175 "all": [
176 "/data/main",
177 "/data/finance",
178 "/data/hr"
179 ]
180 },
181 "mirrors": [
182 {
183 "name": "data-main--home--finance-hr",
184 "comment": "data main: home -> finance, hr",
185 "source": "data@home.abc.xyz:/data/main",
186 "excludes": [".snapshot", ".snapshots"],
187 "destinations": ["finance.abc.xyz", "hr.abc.xyz"]
188 },
189 {
190 "name": "data-finance--home--finance",
191 "comment": "data finance: home -> finance",
192 "email_recipients": ["bob@finance.abc.xyz"],
193 "source": "data@home.abc.xyz:/data/finance",
194 "excludes": [".snapshot", ".snapshots"],
195 "destinations": ["finance.abc.xyz"]
196 },
197 {
198 "name": "data-hr--home--hr",
199 "comment": "data hr: home -> hr",
200 "source": "data@home.abc.xyz:/data/hr",
201 "excludes": [".snapshot", ".snapshots"],
202 "destinations": ["hr.abc.xyz"]
203 }
204 ]
205 }
206 {{/code}}
207
208 List suite and mirror settings:
209
210 {{noformat}}
211 rmirr -l
212 {{/noformat}}
213
214 Mirror ##/data/main## from home.abc.xyz:
215
216 {{noformat}}
217 rmirr -p /data/main
218 {{/noformat}}
219
220 Notes:
221
222 * mirrors to both finance.abc.xyz and hr.abc.xyz
223 * .snapshot and .snapshots are excluded
224 * by default, mirroring **does not** delete at the destination
225 * by default, mirroring **requires** user interaction ("y" to proceed)
226
227 Show what rmirr would do **without** executing it (dry-run):
228
229 {{noformat}}
230 rmirr --dry -p /data/main
231 {{/noformat}}
232
233 Mirror /data/main/opt:
234
235 {{noformat}}
236 rmirr -p /data/main/opt
237 {{/noformat}}
238
239 Notes:
240
241 * because ##/data/main/opt## is under ##/data/main##, the settings for ##/data/main## are used
242 * only ##/data/main/opt## is mirrored
243
244 Mirror /data/finance and allow deletion at the destination:
245
246 {{noformat}}
247 rmirr --delete -p /data/finance
248 {{/noformat}}
249
250 Do a dry run (at rsync level) to mirror /data/finance with deletion and no user interaction:
251
252 {{noformat}}
253 rmirr --dry-rsync --delete -y -p /data/finance
254 {{/noformat}}
255
256 Mirror /data/main to specific destination (hr.abc.xyz):
257
258 {{noformat}}
259 rmirr --delete -y -d hr.abc.xyz -p /data/main
260 {{/noformat}}
261
262 Request report be emailed:
263
264 {{noformat}}
265 rmirr -p /data/main --mailreport
266 {{/noformat}}
267
268 Request report be output to console:
269
270 {{noformat}}
271 rmirr -p /data/main --showreport
272 {{/noformat}}
273
274 Mirror the "all" suite:
275
276 {{noformat}}
277 rmirr -s all
278 {{/noformat}}
279
280 = Mirroring Selected File Objects =
281
282 Sometimes only a subset of file objects (files, directories) need to be mirrored rather than a full tree. rmirr supports this with the names setting in the configuration file.
283
284 Configuration for mirroring bash files:
285
286 {{code language="none"}}
287 {
288 "suites": {
289 "bashfiles": [
290 "~/.bash_completion.d",
291 "~/.bash_profile",
292 "~/.bash_logout",
293 "~/.bashrc",
294 "~/.profile",
295 "~/.profile.d"
296 ]
297 },
298 "mirrors": [
299 {
300 "name": "my-bash-files",
301 "comment": "my .bash files",
302 "source": "jdm@home.abc.xyz:~",
303 "excludes": [
304 ".snapshot",
305 ".snapshots"
306 ],
307 "destinations": [
308 "finance.abc.xyz",
309 "hr.abc.xyz"
310 ]
311 }
312 ]
313 }
314 {{/code}}
315
316 Notes:
317
318 * the suites section defines a suite to reference those bash-specific file objects by a single name
319 * of course, any changes/tweaks done at the destinations will be overwritten when rmirr is called
320
321 To mirror all the file objects of the "bashfiles" suite:
322
323 {{noformat}}
324 rmirr -s bashfiles
325 {{/noformat}}
326
327 To mirror specific file objects:
328
329 {{noformat}}
330 rmirr -p /home/jdm/.bash_completion.d
331 {{/noformat}}

Contact

About

Name

rmirr - Mirroring Tool

Requirements

Python 2, rsync

License

GPL-2

Links

Repository