66

I just need to preserve some old data that I have on my computers, so I was wondering what would be the best way to archive stuff long term.

Blu-ray disks ? Multiple HDDs ? What do you guys suggest ?

top 50 comments
sorted by: hot top controversial new old
[-] nottelling@lemmy.world 38 points 1 year ago* (last edited 1 year ago)

Self hosting principals aside, is this data actually important? If so, then don't fuck around with self hosting it. Are you looking for lowest cost? Then don't waste a bunch of money spinning your own disks.

Amazon glacier to guarantee availability and your own encryption to guarantee privacy.

It's currently running me about $4/month for around 10tb that I don't want to lose but just don't want to deal with. An equivalent HDD solution would be around $500, that's 10 years to break even assuming zero disk failures and zero personal maintenance time.

Plus it's guaranteed. Inherent multiple copies, has SLA, and there's no worry about the service just disappearing. It's they decide to shut down or raise prices or whatever, you can reevaluate and move.

Edit: Glacier and similar services are meant for archival which is the term OP used. You never expect to need it again, but can't get rid of it. Retrieval cost is mostly irrelevant, but yes much more expensive. (I'd wager still less expensive than a home RAID array.)

[-] Dran_Arcana@lemmy.world 8 points 1 year ago* (last edited 1 year ago)

What would it cost to retrieve though? You probably still have the appropriate cost-effective solution but it's an important consideration for newcomers to have complete math.

[-] howrar@lemmy.ca 3 points 1 year ago* (last edited 1 year ago)

Retrieving from S3 glacier is approximately ~~10 times the monthly cost of storing the data~~ 100 times actually. Didn't realize retrieval from Glacier isn't actually downloading it onto your local, but rather just moving it into a frequent access tier S3 bucket from which you can then download, and this download is the expensive part.

[-] nottelling@lemmy.world 2 points 1 year ago

OP said "archive", not "backup". Glacier is for days you need to keep but rarely touch.

[-] tryptaminev@feddit.de 4 points 1 year ago

So instead of "fucking around" with putting it on a long lasting storage device to keep in a wardrobe, he should give up control of the data, hand it to a company and risk forgetting to inform them about an adress change, so everything is lost, when the bills arent paid?

How is that more secure?

load more comments (3 replies)
[-] Appoxo@lemmy.dbzer0.com 4 points 1 year ago

How is it to get the data back?
Can I do it in real time so I could mount it as a media storage or would I need to rent one of the faster S3 tiers?

load more comments (1 replies)
[-] howrar@lemmy.ca 2 points 1 year ago

Are you sure it's $4/month and not $40/month? If so, which region is this in?

[-] nottelling@lemmy.world 3 points 1 year ago

Us-East. Look specifically at glacier, which is long term, near free to store, expensive to remove.

[-] howrar@lemmy.ca 1 points 1 year ago

Is it Glacier Deep Archive? I just realized I was looking at the Glacier flexible retrieval prices earlier. US-East lists it as $0.00099/GB (about $1/TB), which is still higher than what you're getting.

[-] nottelling@lemmy.world 2 points 1 year ago

Last months bill for my entire Amazon account was $4.72. most of that was the glacier storage.

load more comments (2 replies)
[-] Donebrach@lemmy.world 12 points 1 year ago

encode your data into the dna of an alien world so that it lasts for all time.

[-] zzzzzz@lemmy.ml 5 points 1 year ago

Even better, so it mutates into superior data!

[-] 9point6@lemmy.world 8 points 1 year ago* (last edited 1 year ago)

Depends how important the data is, how long is long term and how budget is budget, but assuming you don't want to risk losing anything, backup best practice is the 321 rule

3 copies 2 different media 1 off-site

I'd almost always say a cloud provider for your off-site backup, but if you don't want to do that, it depends how much you want to spend.

There's no guaranteed do-it-once-and-you're-done approach here, as all data can degrade. For instance if one of your backup media is hard disks, you're probably going to want it setup in at least RAID 5 and you want to be on top of swapping out disks when they fail. If you're thinking of the Blu-ray or tape approach, you're going to want to periodically check that the media hasn't degraded. You'll probably also want to plan to replace the backup media every half decade or so to be extra safe (e.g. BD-Rs have a lifespan of 5-10 years).

[-] xmunk@sh.itjust.works 7 points 1 year ago

Honestly? If it's not that much data just throw an additional copy up on Google Drive.

[-] gunpachi@lemmings.world 10 points 1 year ago

I appreciate your honest answer. I want to completely own my data, so I would not go the Cloud route. After all the Cloud is basically someone else's computer.

[-] nottelling@lemmy.world 5 points 1 year ago

The data remains yours if you encrypt it. Someone else's computer saves you all the time and effort of maintaining and monitoring hardware.

You want to use the actual services meant for this. S3 or glacier or something, not just consumer cloud storage like Google drive or Dropbox.

[-] xmunk@sh.itjust.works 2 points 1 year ago

That's quite fair, personal preference is an important factor.

[-] I_Miss_Daniel@lemmy.world 1 points 1 year ago

But if it's encrypted does it matter?

[-] ShadowRam@kbin.social 4 points 1 year ago

Ask all those that had shit on Megaupload in 2012.

Encrypted or not. Still lost.

[-] I_Miss_Daniel@lemmy.world 4 points 1 year ago

Yeah. It should not be your only backup, but it can be one of them.

[-] nottelling@lemmy.world 1 points 1 year ago

Lol imagine ever having considered megaupload as your backup solution.

[-] ares35@kbin.social 1 points 1 year ago

there are many ways to encrypt locally and store the encrypted data remotely; either a container (like veracrypt), or individual files with a file-based encryption schemes (such as cryptomator) or one of numerous backup or sync utilities with built-in encryption.

[-] KillerTofu@lemmy.world 7 points 1 year ago

Blasphemy in the hallowed halls of FOSS.

[-] xmunk@sh.itjust.works 2 points 1 year ago

Cool, but an actually useful answer.

[-] WhyJiffie@sh.itjust.works 6 points 1 year ago* (last edited 1 year ago)

Yes, that may be an option.. except that google can irreversibly lock you out of your account, or they can delete your files if their content scanning think it goes against some of their terms, but also simply there are people who don't want to lose their privacy to google.

[-] xmunk@sh.itjust.works 3 points 1 year ago

It's more likely that a Google data center exists in 100 years than your house. If you have a personal aversion to it then I can understand - but, realistically, it's more likely that an offsite copy on Google Drive exists in 2123 than a random piece of furniture you own - and furniture is pretty damage resistant.

[-] fratermus@lemmy.sdf.org 1 points 1 year ago

It’s more likely that a Google data center exists in 100 years than your house.

Yes, but it's more likely that Google will have killed a particular service like Drive. Cf. Google Reader, Hangouts, Data Saver extension, Buzz, etc.

google graveyard

load more comments (1 replies)
[-] Extrasvhx9he@lemmy.today 3 points 1 year ago

Good for a 3,2,1 backup method but bad for archival. We don't know if google will even exist in whatever number of years OP wants to archieve for or if the data will be deleted/modified by google themeselves due to some crap policy like their 2 year inactive account one for example. Just too many factors that will be out of OP's control

[-] nomecks@lemmy.world 5 points 1 year ago

If you have enough data: Tapes. Tapes are so hilariously cheap to keep. Write them and keep them in a fire proof box. No power needed to keep platters spinning. 45TB/tape!

load more comments (3 replies)
[-] darvocet@infosec.pub 5 points 1 year ago

CDs degrade over time and so aren’t the best way to archive data if you know you will need it again. If it’s just an ‘in case’ then it may be ok. Best bet is to buy a USB disk and then keep a second copy of it offsite. Also best practice to not use two of the same manufacturer drive.

[-] Extrasvhx9he@lemmy.today 5 points 1 year ago* (last edited 1 year ago)

Would probably help to know for how long, how much capacity do you need and what budget. Should also be stated external factors play a massive factor on how long a storage device can survive like enviroment, humidity and heat being the biggies

Edit in case I fall asleep: for the budget I usually would go with an external ssd just refresh the data every year or 2 it should be ok for 8ish years maybe even 10. For a write it and forget it method you'll want m-disc instead which are more expensive but if properly stored will last lifetimes so the failure point will be a usuable drive that can read it. If you decide to go the spinning mechanical drive route make sure to buy 2 (a backup for the backup) since they are a lot more fragile. Gold plated dvds/cds are also another write and forget option but have less capacity than m-discs

[-] ShellMonkey@lemmy.socdojo.com 4 points 1 year ago

A couple different threat models to consider, hardware failure vs human failure. Things like RAID can effectively cover the hardware failure side and be fully transparent. Human failure is a bit more tricky. There are a number of old expressions about backups but one that's good to keep in mind is snapshots are not backups. They're convenient and easy to automate but if the system making them goes kerplooie they're pretty useless.

A tiered version is good for off device backup, using diff backups routinely to only copy the new or changed data with a periodic full backup.

Cold disks are great but make sure to test them periodically, nothing worse than looking to restore a chunk of data only to find the backup can't be read.

load more comments (5 replies)
[-] Kit 4 points 1 year ago

How much data? The answer will differ if it's a few megs of text vs a terabyte of videos.

load more comments (1 replies)
[-] calypsopub@lemmy.world 3 points 1 year ago

Multiple methods, not really important which ones. I use an external hard drive plus I email zip files to myself.

[-] afunkysongaday@lemmy.world 2 points 1 year ago
[-] taladar@sh.itjust.works 4 points 1 year ago

Generally speaking obscure formats are not great for long term storage for your chances to read it again years later.

[-] afunkysongaday@lemmy.world 2 points 1 year ago

You can read those with a regular blu ray drive. No special hard- or software needed.

load more comments (3 replies)
[-] riquisimo@lemmy.world 3 points 1 year ago

Are those... cheap? They don't look cheap.

[-] foofiepie@lemmy.world 1 points 1 year ago

About £90 for 4 discs, and £30 for the writer, gets you 400Gb storage.

load more comments (1 replies)
[-] Atemu@lemmy.ml 2 points 1 year ago* (last edited 1 year ago)

I use multiple offline HDDs with a policy to keep n copies between them because it's by far the cheapest way to still own the data. It requires regular checks because HDDs are likely to fail after a decade or so and a bunch of HDDs are a pain to manage, so you will need tooling for this. I use git-annex for this purpose but it's not particularly user-friendly.

[-] xilliah@beehaw.org 2 points 1 year ago

I use m discs. It's like a kind of high quality Blu-ray and should last thousands of years. I think your burner must support it though.

[-] Dyskolos@lemmy.zip 1 points 1 year ago

I don't know about your budget, but I'd do it onto HDDs. They're cheap and large. But use two and regularly check them like every other month or so. If one breaks, get a replacement. That's the most simple (if you have an external dock) and cheap solution that you OWN.

Also copy it at least twice onto each in case of corruption. Also use a copier that verifies (fastcopy, teracopy etc)

load more comments
view more: next ›
this post was submitted on 15 Nov 2023
66 points (100.0% liked)

Asklemmy

43831 readers
775 users here now

A loosely moderated place to ask open-ended questions

Search asklemmy 🔍

If your post meets the following criteria, it's welcome here!

  1. Open-ended question
  2. Not offensive: at this point, we do not have the bandwidth to moderate overtly political discussions. Assume best intent and be excellent to each other.
  3. Not regarding using or support for Lemmy: context, see the list of support communities and tools for finding communities below
  4. Not ad nauseam inducing: please make sure it is a question that would be new to most members
  5. An actual topic of discussion

Looking for support?

Looking for a community?

~Icon~ ~by~ ~@Double_A@discuss.tchncs.de~

founded 5 years ago
MODERATORS