Our latest paper, in which @brinda.eu (along with @zaminiqbal.bsky.social and others) introduces phylogenetic compression for storage and search of enormous microbial genome libraries, was published today in @naturemethods.bsky.social: rdcu.be/eg4OA 1/ - ThreadSky

baym.lol • 9 days ago

Our latest paper, in which @brinda.eu (along with @zaminiqbal.bsky.social and others) introduces phylogenetic compression for storage and search of enormous microbial genome libraries, was published today in @naturemethods.bsky.social:

https://rdcu.be/eg4OA

1/

Comments

triggerloop.bsky.social•9 days ago

Nice job buddy!

robp.bsky.social•9 days ago

w00t; very happy to see this out --- great work! Congrats Michael, Karel, Zamin & all those not on bsky :).

baym.lol•9 days ago

Now that it's identified, thank you for your thoughtful reviews! They truly helped

zaminiqbal.bsky.social•8 days ago

Yes, thanks!!!

tgrohens.bsky.social•9 days ago

Wow, awesome work!!

vscooper.micropopbio.org•9 days ago

Bout time! Congrats

baym.lol•9 days ago

Thank you! It was a long slog through review, but I'm grateful to the reviewers for insisting that it needed more space than the journal was originally willing to allot!

baym.lol•9 days ago

To copy the preprint explainer from the other place: Ever noticed how when you throw a sequence into BLAST, it takes a while to search? And it’s not really any faster than fifteen years ago, when the smartphones barely even existed yet? 2/

baym.lol•9 days ago

It's because we've been sequencing so many things, we just can't keep up. In fact for bacteria, just to keep things running at a reasonable speed, the fraction of the total bacteria in NCBI BLAST searches has been decreasing exponentially. 3/

baym.lol•9 days ago

So we asked: what sets the fundamental limit on computation on large genomic databases? Evolution! The irreducible entropy in genome collections is bounded by the most parsimonious path to introduce that variability.

In other words, optimal compression should echo phylogeny.
4/

baym.lol•9 days ago

Here we introduce a technique to compress using phylogeny, by clustering phylogenetically (too disparate things don’t compress), then resorting by phylogeny (local compressors care about order), and compressing. We show this on sequences, de Bruijn graphs, and Bloom filters

5/

baym.lol•9 days ago

We find that this gives enormous gains in both storage and computational efficiency, to the point where we named it MOF, or Microbes On a Flash drive, since, purely speaking theoretically here, you could compress every sequenced bacterium onto a flash drive.
6/

barrygoldman1.bsky.social•9 days ago

clever

baym.lol•9 days ago

*in the review process we updated MOF to MiniPhy

baym.lol•9 days ago

Side note: you could, speaking purely theoretically, also fit every microbe onto an SD card, which is within the weight limit for a carrier pigeon. For some distances, it would be faster than the internet for transmitting sequence libraries
7/