561

You might not even like rsync. Yeah it's old. Yeah it's slow. But if you're working with Linux you're going to need to know it.

In this video I walk through my favorite everyday flags for rsync.

Support the channel:
https://patreon.com/VeronicaExplains
https://ko-fi.com/VeronicaExplains
https://thestopbits.bandcamp.com/

Here's a companion blog post, where I cover a bit more detail: https://vkc.sh/everyday-rsync

Also, @BreadOnPenguins made an awesome rsync video and you should check it out: https://www.youtube.com/watch?v=eifQI5uD6VQ

Lastly, I left out all of the ssh setup stuff because I made a video about that and the blog post goes into a smidge more detail. If you want to see a video covering the basics of using SSH, I made one a few years ago and it's still pretty good: https://www.youtube.com/watch?v=3FKsdbjzBcc

Chapters:
1:18 Invoking rsync
4:05 The --delete flag for rsync
5:30 Compression flag: -z
6:02 Using tmux and rsync together
6:30 but Veronica... why not use (insert shiny object here)

top 50 comments
sorted by: hot top controversial new old
[-] NuXCOM_90Percent@lemmy.zip 72 points 1 week ago

I would generally argue that rsync is not a backup solution. But it is one of the best transfer/archiving solutions.

Yes, it is INCREDIBLY powerful and is often 90% of what people actually want/need. But to be an actual backup solution you still need infrastructure around that. Bare minimum is a crontab. But if you are actually backing something up (not just copying it to a local directory) then you need some logging/retry logic on top of that.

At which point you are building your own borg, as it were. Which, to be clear, is a great thing to do. But... backups are incredibly important and it is very much important to understand what a backup actually needs to be.

[-] tal@olio.cafe 25 points 1 week ago* (last edited 1 week ago)

I would generally argue that rsync is not a backup solution.

Yeah, if you want to use rsync specifically for backups, you're probably better-off using something like rdiff-backup, which makes use of rsync to generate backups and store them efficiently, and drive it from something like backupninja, which will run the task periodically and notify you if it fails.

rsync: one-way synchronization

unison: bidirectional synchronization

git: synchronization of text files with good interactive merging.

rdiff-backup: rsync-based backups. I used to use this and moved to restic, as the backupninja target for rdiff-backup has kind of fallen into disrepair.

That doesn't mean "don't use rsync". I mean, rsync's a fine tool. It's just...not really a backup program on its own.

[-] melfie@lemy.lol 9 points 1 week ago

Having a synced copy elsewhere is not an adequate backup and snapshots are pretty important. I recently had RAM go bad and my most recent backups had corrupt data, but having previous snapshots saved the day.

load more comments (3 replies)
load more comments (4 replies)
[-] Colloidal@programming.dev 4 points 1 week ago

Borg gang represent!

load more comments (2 replies)
[-] mesamunefire@piefed.social 49 points 1 week ago

Ive personally used rsync for backups for about....15 years or so? Its worked out great. An awesome video going over all the basics and what you can do with it.

[-] Eldritch@piefed.world 22 points 1 week ago

And I generally enjoy Veronica's presentation. Knowledgable and simple.

[-] mesamunefire@piefed.social 19 points 1 week ago

Her https://tinkerbetter.tube/w/ffhBwuXDg7ZuPPFcqR93Bd made me learn a new way of looking at data. There was some tricks I havent done before. She has such good videos.

[-] Eldritch@piefed.world 5 points 1 week ago

Yep, I found her through YouTube. Her and action retro's content is always great.with some Adrian black on the side.

load more comments (1 replies)
[-] eager_eagle@lemmy.world 7 points 1 week ago* (last edited 1 week ago)

It works fine if all you need is transfer, my issue with it it's just not efficient. If you want a "time travel" feature, your only option is to duplicate data. Differential backups, compression, and encryption for off-site ones is where other tools shine.

load more comments (5 replies)
load more comments (1 replies)

Why videos? I feel like an old man yelling at clouds every time something that sounds interesting is presented in a fucking video. Videos are so damn awful. They take time, I need audio and I can't copy&paste. Why have they become the default for things that should've been a blog post?

[-] Wawe@lemmy.world 14 points 1 week ago* (last edited 1 week ago)

They linked blog post with the video: https://vkc.sh/everyday-rsync/

[-] czardestructo@lemmy.world 12 points 1 week ago

Thank you for putting into words what ive subconsciously been thinking for years. Every search result prioritizes videos at the top and I'm still annoyed every time. Or even worst I have to hunt through a 10 minute video for the 30 seconds of info I needed. Stoohhhhpppp internet of new! Make it good again!

[-] kchr@lemmy.sdf.org 6 points 1 week ago

Hear hear. Knowledge should be communicated in an easily shareable way that can also be archived as easily, in contrast to a video requiring hundreds of MB:s.

load more comments (2 replies)
[-] atk007@lemmy.world 23 points 1 week ago

Rsnapshot. It uses rsync, but provides snapshot management and multiple backup versioning.

[-] Tja@programming.dev 16 points 1 week ago

Yes, but a few hours writing my own scripts will save me from several minutes of reading its documentation...

load more comments (3 replies)
[-] BonkTheAnnoyed 4 points 1 week ago

Yah, I really like this approach. Same reason I set up Timeshift and Mint Backup on all the user machines in my house. For others rsync + cron is aces.

[-] sugar_in_your_tea@sh.itjust.works 19 points 1 week ago

Yeah it’s slow

What's slow about async? If you have a reasonably fast CPU and are merely syncing differences, it's pretty quick.

[-] pathief@lemmy.world 5 points 1 week ago

It's single thread, one file at a time.

That would only matter if it's lots of small files, right? And after the initial sync, you'd have very few files, no?

Rsync is designed for incremental syncs, which is exactly what you want in a backup solution. If your multithreaded alternative doesn't do a diff, rsync will win on larger data sets that don't have rapid changes.

load more comments (1 replies)
[-] quick_snail@feddit.nl 19 points 1 week ago
[-] HereIAm@lemmy.world 9 points 1 week ago

Compared to something multi threaded, yes. But there are obviously a number of bottlenecks that might diminish the gains of a multi threaded program.

[-] Tja@programming.dev 7 points 1 week ago

With xargs everything is multithreaded.

[-] okamiueru@lemmy.world 9 points 1 week ago

That part threw me off. Last time i used it, I did incremental backups of a 500 gig disk once a week or so, and it took 20 seconds max.

load more comments (1 replies)
[-] clif@lemmy.world 15 points 1 week ago

I'll never not upvote Veronica Explains. Excellent creator and excellent info on everything I've seen.

[-] probable_possum@leminal.space 13 points 1 week ago* (last edited 1 week ago)

rsnapshot is a script for the purpose of repeatedly creating deduplicated copies (hardlinks) for one or more directories. You can chose how many hourly, daily, weekly,... copies you'd like to keep and it removes outdated copies automatically. It wraps rsync and ssh (public key auth) which need to be configured before.

load more comments (3 replies)
[-] ryper@lemmy.ca 9 points 1 week ago* (last edited 1 week ago)

I was planning to use rsync to ship several TB of stuff from my old NAS to my new one soon. Since we're already talking about rsync, I guess I may as well ask if this is right way to go?

[-] Suburbanl3g3nd@lemmings.world 6 points 1 week ago* (last edited 1 week ago)

I couldn't tell you if it's the right way but I used it on my Rpi4 to sync 4tb of stuff from my Plex drive to a backup and set a script up to have it check/mirror daily. Took a day and a half to copy and now it syncs in minutes tops when there's new data

[-] Cyber@feddit.uk 4 points 1 week ago

It depends

rsync is fine, but to clarify a little further...

If you think you'll stop the transfer and want it to resume (and some data might have changed), then yep, rsync is best.

But, if you're just doing a 1-off bulk transfer in a single run, then you could use other tools like xcopy / scp or - if you've mounted the remote NAS at a local mount point - just plain old cp

The reason for that is that rsync has to work out what's at the other end for each file, so it's doing some back & forwards communications each time which as someone else pointed out can load the CPU and reduce throughput.

(From memory, I think Raspberry Pi don't handle large transfers over scp well... I seem to recall a buffer gets saturated and the throughput drops off after a minute or so)

Also, on a local network, there's probably no point in using encryption or compression options - esp. for photos / videos / music... you're just loading the CPU again to work out that it can't compress any further.

load more comments (2 replies)
[-] solrize@lemmy.ml 9 points 1 week ago* (last edited 1 week ago)

I've been using borg because of the backend encryption and because the deduplication and snapshot features are really nice. It could be interesting to have cross-archive deduplication but maybe I can get something like that by reorganizing my backups. I do use rsync for mirroring and organizing downloads, but not really for backups. It's a synchronization program as the name implies, not really intended for backups.

load more comments (1 replies)
[-] i_stole_ur_taco@lemmy.ca 9 points 1 week ago

The thing I hate most about rsync is that I always fumble to get the right syntax and flags.

This is a problem because once it’s working I never have to touch it ever again because it just works and keeping working. There’s not enough time to memorize the usage.

[-] mesamunefire@piefed.social 5 points 1 week ago

I feel this too. I have a couple of "spells" that work wonders in a literal small notebook with other one liners over the years. Its my spell book lol.

load more comments (7 replies)
[-] calliope@retrolemmy.com 8 points 1 week ago

Tangentially, I don’t see people talk about rclone a lot, which is like rsync for cloud storage.

It’s awesome for moving things from one provider to another, for example.

load more comments (11 replies)
[-] surph_ninja@lemmy.world 8 points 1 week ago

Use borg/borgmatic for your backups. Use rsync to send your differentials to your secondary & offsite backup storage.

[-] Landless2029@lemmy.world 6 points 1 week ago

I need a breakdown like this for Rclone. I've got 1TB of OneDrive free and nothing to do with it.

I'd love to setup a home server and backup some stuff to it.

[-] RestrictedAccount@lemmy.world 6 points 1 week ago

I use syncthing.

Is rsync better?

Syncthing works pretty well for me and my stable of Ubuntu, pi, Mac, and Windows

[-] MangoPenguin 9 points 1 week ago

Different tools for different use cases IMO.

But neither do backups.

load more comments (3 replies)
[-] EncryptKeeper@lemmy.world 8 points 1 week ago

I’m not super familiar with Syncthing, but judging by the name I’d say Syncthing is not at all meant for backups.

[-] conartistpanda@lemmy.world 6 points 1 week ago

Syncthing is technically to synchronize data across different devices in real time (which I do with my phone), but I also use it to transfer data weekly via wi-fi to my old 2013 laptop with a 500GB HDD and Linux Mint (I only boot it to transfer data, and even then I pause the transfers to this device when its done transferring stuff) so I can have larger data backups that wouldn't fit in my phone, since LocalSend is unreliable for large amounts of data while Synchting can resume the transfer if anything goes wrong. On top of that Syncthing also works in Windows and Android out of the box.

[-] WhyJiffie@sh.itjust.works 5 points 1 week ago

its for a different purpose. I wouldn't use syncthing the way I use rsync

[-] Appoxo@lemmy.dbzer0.com 5 points 1 week ago

Veeam for image/block based backups of Windows, Linux and VMs.
syncthing for syncing smaller files across devices.

Thank you very much.

[-] TheWilliamist@lemmy.world 5 points 1 week ago* (last edited 1 week ago)

I still prefer tar for quick and dirty same box copies.

tar cf - * | (cd /target; tar xfp -)
load more comments (1 replies)
[-] Mio@feddit.nu 5 points 1 week ago* (last edited 1 week ago)

I think the there are better alternatives for backup like kopia and restic. Even seafile. Want protection against ransomware, storage compression, encryption, versioning, sync upon write and block deduplication.

load more comments (2 replies)
[-] vext01@lemmy.sdf.org 4 points 1 week ago

I used to use rsnapshot, which is a thin wrapper around rsync to make it incremental, but moved to restic and never looked back. Much easier and encrypted by default.

[-] tal@olio.cafe 4 points 1 week ago* (last edited 1 week ago)

slow

rsync is pretty fast, frankly. Once it's run once, if you have -a or -t passed, it'll synchronize mtimes. If the modification time and filesize matches, by default, rsync won't look at a file further, so subsequent runs will be pretty fast. You can't really beat that for speed unless you have some sort of monitoring system in place (like, filesystem-level support for identifying modifications).

load more comments (2 replies)
[-] Xylight@lemdro.id 4 points 1 week ago

rsync for backups? I guess it depends on what kind of backup

for redundant backups of my data and configs that I still have a live copy of, I use restic, it compresses extremely well

I have used rsync to permanently move something to another drive though

load more comments
view more: next ›
this post was submitted on 03 Oct 2025
561 points (100.0% liked)

Selfhosted

52275 readers
268 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 2 years ago
MODERATORS