actually trying this against this account has removed any performance concerns I had, since it's running in debug mode on my Mac with a load of other things *running through a tunnel* and it's not too much slower (in getPostThread, my worst code imo) than bluesky (noticable, but still useable)
me: posts that I'm quite happy with the performance, actually.
the code: what if the firehose consumer performance became bad?
for some reason, driving the consumer in a loop{} doing *nothing else* only pulls 200 commit events/second and I'm *pretty sure* there's more than that most of the time
at fully speed, it'll do 600 commits per second whilst storing them in the DB and doing handle resolution for identity events inbetween.
it's multithreaded to clear what I actually need to peak at, but the profiler says most time is spent waiting and increasing the thread count doesn't help
Comments
the code: what if the firehose consumer performance became bad?
for some reason, driving the consumer in a loop{} doing *nothing else* only pulls 200 commit events/second and I'm *pretty sure* there's more than that most of the time
it's multithreaded to clear what I actually need to peak at, but the profiler says most time is spent waiting and increasing the thread count doesn't help
going to try to reproduce single-threaded in a fresh repo tomorrow and see. might try to switch websocket libraries but that seems extreme?