yongyuanxi.bsky.social
outerport.com (yc s24)
former research scientist @ nvidia
systems, ml, graphics, llms, and data
personal web: tovacinni.github.io
33 posts
1,961 followers
770 following
Regular Contributor
Active Commenter
comment in response to
post
Thanks for sharing, Sung!!
comment in response to
post
lol def dont disagree with that- these days uv makes things a little better though (but still a nightmare once you have cuda extensions etc)
comment in response to
post
Added!
comment in response to
post
added!
comment in response to
post
Definitely agree wrt race conditions! Rust makes concurrency actually fun the work with
comment in response to
post
Note the “for C++-like systems programming”. Python is not a language meant for that.
comment in response to
post
Would love to be added :D
research.nvidia.com/labs/dir/mag...
comment in response to
post
Would love to be on there as well 😃
comment in response to
post
👋 working on high throughput LLMs for data enrichment
comment in response to
post
You can definitely replace the kernel launch code with Rust / PyO3 (although I’ve never tried this myself) but the kernels themselves will still be in CUDA C++ (just runtime or compile time built by Rust)
comment in response to
post
Yep cudarc! (didn't exist the last time I considered learning Rust which was a long time ago)
comment in response to
post
(Hopefully I'll never have to find out)
comment in response to
post
To be fair maybe CMake wouldn't be so bad now with Claude
comment in response to
post
I've definitely wasted many hours of my life fighting CMake 😆
comment in response to
post
Yes the functional features are definitely very nice too! The status quo on teams I've worked on has generally unfortunately just been CMake 😭
comment in response to
post
CUDA support in Rust is pretty good these days which was also a reason why Rust was easier to adopt now! (but of course muddles the whole 'memory safety' thing which is also why I don't think that's the main feature of Rust)
comment in response to
post
Haven't built a sim engine with Rust, but specifically for concurrency code Rust feels way less scary than C++. So application might matter a bit.
Also it definitely would've taken longer for me to learn Rust if I didn't also have Claude to explain all the concepts / compiler errors for me!
comment in response to
post
"MIT of the North" as they say 🤣
comment in response to
post
(made me think of this since this is what searching “towaki” on the gif search on bluesky gets me)
comment in response to
post
Done!
comment in response to
post
Added!
comment in response to
post
Very cool! We've been working on making batch inference faster for this use case too (extraction from 10000 PDFs vs. just 1 extraction, or extracting 10000 things from 1 PDF)
comment in response to
post
Hi I’m Towaki! I used to work on ML / computer graphics research at NVIDIA but now I work on systems for large scale data / document processing with LLMs. I’ll post about GPUs and LLMs for data stuff 😄
comment in response to
post
I haven’t found anything but I’m building something like this since I’m also drowning in arxiv PDFs with cryptic filenames lol
comment in response to
post
Thanks!!!
comment in response to
post
Just joined- would love to be on the researcher ones 🙏