tell me you are proud of me. I resized my my swap on zfs. No more will the OOM killer mean that updating my Pixi dependencies causes the murder of my Spotify - ThreadSky

oxinabox.bsky.social • 164 days ago

tell me you are proud of me.
I resized my my swap on zfs.
No more will the OOM killer mean that updating my Pixi dependencies causes the murder of my Spotify

Comments

thejono.bsky.social•163 days ago

Great work!

spijet.bsky.social•164 days ago

Please do not use swap on ZFS in any form, Linux is unable to properly live in this kind of setup, which is a known and long-standing issue in ZFSonLinux. If you have some spare unpartitioned space — use it, otherwise use ZRAM.

oxinabox.bsky.social•164 days ago

What's ZRAM?

mathieu.fenniak.net•164 days ago

zram creates a swap space that is actually stored in ram, but compressed. Trades cpu cycles for more memory, in a way. It's cool and effective.

More ram is even better.

oxinabox.bsky.social•164 days ago

Oh that's very clever
I don't think it's a good idea when i don't have much to spare though.

mathieu.fenniak.net•164 days ago

It is typically used for low-memory deployments -- I think Ubuntu and Fedora deploy with a default zram size of 50% of your available RAM, these days. But your mileage may vary! $50 for more RAM is a better option. 🤣

oxinabox.bsky.social•164 days ago

Welp back down to 4GB swap.
Should have checked the swap size before partitioning.
Will know for next time.
Doubling my RAM should render it moot

spijet.bsky.social•163 days ago

You can save yourself some latency and either use ZRAM with your old swap partition as a writeback device (if your kernel is recent enough), or use swap on a partition + Zswap instead. Zswap is similar to ZRAM in that it stores inactive pages in a compressed pool in RAM before flushing it to disk.

spijet.bsky.social•163 days ago

As always, Arch Linux Wiki will tell you everything about it and then some more, even if you don't use Arch. :)

ripperoni.com•164 days ago

Hell yeah, good work

cofactorstrudel.bsky.social•164 days ago

I don't know what any of these things mean but I am proud of you 🫶

oxinabox.bsky.social•164 days ago

I wish i didn't

oxinabox.bsky.social•164 days ago

It did not.
Now my whole computer has frozen up trying to update dependencies

The fuck is wrong with python package management that it uses this much CPU and memory.

extremely.gay•164 days ago

canonically dependency resolution is SAT

never mind attempting to run parallel package builds

oxinabox.bsky.social•164 days ago

Yeah. I am pretty aware of that.
But i haven't ever had this problem with Julia's package manager.
And i have built things with 400 dependencies.

extremely.gay•164 days ago

python made a deal with the devil back in 2002 to become the most readable, obvious programming language and in exchange the devil took the concept of reasonable package management

oxinabox.bsky.social•164 days ago

I think the nightmare is in part due to pypi not storing dependency information anywhere you can access it without downloading and decompressing wheels.

extremely.gay•164 days ago

there's some dependency information in the API but pypi also makes no claims that it matches the uploaded files 🙃

oxinabox.bsky.social•164 days ago

I'm gonna ask my boss for more RAM.
It's like $50.
I have probably wasted that in pay dicking around with this

extremely.gay•164 days ago

is the swap in the pool itself (e.g. a zvol)? because that is a cursed configuration

oxinabox.bsky.social•164 days ago

Yes?
I just followed
https://github.com/zfsonlinux/pkg-zfs/wiki/HOWTO-use-a-zvol-as-a-swap-device#step-1-create-a-volume-dataset-zvol-for-use-as-a-swap-device

extremely.gay•164 days ago

so zfs really, really likes to defer flushing writes to disk, which historically means that memory pressure -> kernel asks zfs to write some pages to the swap vol -> zfs sticks another (maybe compressed) copy in its pending write cache, eating more RAM until the write happens (maybe 5s later)

extremely.gay•164 days ago

of course, that makes the memory pressure worse, so the kernel wants to swap more pages out, so zfs sticks more pages into the write cache...

oxinabox.bsky.social•164 days ago

Screaming

extremely.gay•164 days ago

it's been a few years, maybe it's better now? but I've heard the zfs project is low on maintainers who actually understand its runtime behavior

Comments

Posting Rules

Reply