For bioinformatics crowd 🧬🖥️ looking at different backup solutions. I know backblaze had been mentioned by some before, but they seem to have issues "Backblaze: A Loss-Making Data Storage Business Mired in Lawsuits, Sham Accounting, and Brazen Insider Dumping" www.morpheus-research.com/backblaze/ - ThreadSky

About ThreadSky

binomicalabs.org • 7 days ago

For bioinformatics crowd 🧬🖥️ looking at different backup solutions. I know backblaze had been mentioned by some before, but they seem to have issues

"Backblaze: A Loss-Making Data Storage Business Mired in Lawsuits, Sham Accounting, and Brazen Insider Dumping" https://www.morpheus-research.com/backblaze/

Comments

bsky.tdunning.com•7 days ago

How much data are you talking about?

Is your budget structured to favor capex or opex?

What is your failure scenario(s)?

binomicalabs.org•7 days ago

Oh I'm nowhere near the scale where all this begins to be meaningful! Tens of terabytes at most. Institutional bioinformaticians might need to take all of that into account though.

bsky.tdunning.com•7 days ago

Well it does matter if you need off site backup or not. Also whether you need selective recovery versus restart from zero after a complete disaster.

binomicalabs.org•7 days ago

So for low-low budget operation like mine (I still pay for most of my experiments working warehouse jobs!) I settled into pattern of backing up key data in public repo, and then just archiving analysis processes as containers which cost nothing

bsky.tdunning.com•6 days ago

IF size allows this, then you have a fine plan.

One nuance might be to use a filesystem that allows replication or RAID. Ideally, you could also have snapshots.

The threats are:

- fat fingers (snapshots)
- disk failure (volume rebuild)
- lab burns down (auto-replicate to repo)

bsky.tdunning.com•6 days ago

Potential solutions include ZFS (single machine, good snapshots and RAID, Linux centric). BTRFS (single machine, snapshots, RAID, Linux centric). Seaweed (snapshots, multi-machine, replication, Linux centric).

Anything with snapshots can allow replication elsewhere.

bsky.tdunning.com•7 days ago

Why not just generic cloud storage, then?

binomicalabs.org•7 days ago

In my case it's cost - granted I do have some Google cloud integration, but size is small enough to just keep some drives/ssds around for key data. Not well thought out, of course

Posting Rules

Be respectful to others
No spam or self-promotion
Stay on topic
Follow Bluesky's terms of service

Comments

Posting Rules

Reply