Profile avatar
sraval.bsky.social
Physics, Visualization and AI PhD @ Harvard | Embedding visualization and LLM interpretability | Love pretty visuals, math, physics and pets | Currently into manifolds Wanna meet and chat? Book a meeting here: https://zcal.co/shivam-raval
41 posts 684 followers 1,238 following
Prolific Poster

This map shows the hour of sunrise globally through the year. It reveals time zones following national and, sometimes, regional boundaries, and slicing through the oceans.

Update on the VIS+AI meetup: I'm a speaker now!

Can we understand the mechanisms of a frontier AI model? πŸ“ Blog post: www.anthropic.com/research/tra... πŸ§ͺ "Biology" paper: transformer-circuits.pub/2025/attribu... βš™οΈ Methods paper: transformer-circuits.pub/2025/attribu... Featuring basic multi-step reasoning, planning, introspection and more!

bsky.app/profile/ieee...

Join us for our first Vis+AI meetup on April 3rd at Northeastern University, a meetup to gather people interested in the intersection of Data Visualization and Artificial Intelligence. Sign up as soon as possible! We have a limited number of spots. lnkd.in/e8whS6v2.

The wind map at hint.fm/wind/ has been running since 2012, relying on weather data from NOAA. We added a notice like this today. Thanks to @cambecc.bsky.social for the inspiration.

Great thread describing the new ARBOR open interpretability project, which has some fascinating projects already. Take a look!

Today we're launching a multi-lab open collaboration, the ARBOR project, to accelerate AI interpretability research for reasoning models. Please join us! github.com/ARBORproject... (ARBOR = Analysis of Reasoning Behavior through Open Research)

DeepSeek R1 shows how important it is to be studying the internals of reasoning models. Try our code: Here @canrager.bsky.social shows a method for auditing AI bias by probing the internal monologue. dsthoughts.baulab.info I'd be interested in your thoughts.

In 1897, Alfred G. Mayer created his butterfly wing projections, an attempt to gain new insights into natural patterns and laws. Vertical blocks denote individual wings, distorted and stretched mathematically to fill a tidy rectangular space. More here: publicdomainreview.org/collection/m...

DeepSeek is a side project πŸ”₯

Tailscan website now uses v4! Also updated the Tailwind CSS color palette cheat sheet πŸ‘€ added a button to see the old v3 and new v4 color Tailwind color palette. #buildinpublic

I'm teaching my first course! A seminar on "Machine Behavior." Readings are a mix of NLP, CSS-y, and ML work on how machines (focus LLMs) "behave" within sociotechnical systems and on how they can be used to study human behavior. Syllabus: manoelhortaribeiro.github.io/teaching/spr...

hahahahah there were actually two technical reports for RL reasoning models today, kimi 1.5 also has good stuff on reward shaping + RL infra kimi 1.5 report: https://buff.ly/4jqgCOa

Dimensionality reduction beyond neural subspaces with slice tensor component analysis www.nature.com/articles/s41...

What a beauty! This is comet C/2024 G3 (ATLAS) passing through the field of view of the LASCO C3 coronagraph. It wasn't for certain whether it would survive it's closest approach to the sun on January 13th, but it did and delivered us a spectacular show! #comet #C2024G3 πŸ”­

Pie and donut charts get a bad rep, but they work well if used for the right data and tasks. Read about what the science has to say about them in our new blog post: https://buff.ly/3DURbnS

*Deep Learning Through A Telescoping Lens* by @alanjeffares.bsky.social @aliciacurth.bsky.social Shows that tracking 1st-order approximations to the training dynamics provides insights into many phenomena (e.g., double descent, grokking). arxiv.org/abs/2411.00247

New paper <3 Interested in inference-time scaling? In-context Learning? Mech Interp? LMs can solve novel in-context tasks, with sufficient examples (longer contexts). Why? Bc they dynamically form *in-context representations*! 1/N

I've started a Research Integrity Feed populated by hashtags below & choice users. For the #SciPub / #AcademicPublishing sleuths πŸ“ŠπŸ”πŸ‘€ Plus it's got a cute furry mascot! 😘 bsky.app/profile/did:... #ResearchIntegrity #PredatoryPublisher #PredatoryPublishing #EditorialIndependence #SciRetraction

ByteDance has open-sourced a lip-sync model called LatentSync. LatentSync is an end-to-end lip-sync framework that does not rely on any intermediate motion representation, but instead models complex audio-visual correlations directly in the latent space.

Genuary 2025, Day 3: "Exactly 42 lines of code." A tool for drawing with the osculating (kissing) circles of one's stroke. #genuary #genuary2025 #genuary3

What was the most important machine learning paper in 2024? My Famous Deep Learning Papers list (that I use in teaching) does not include any new ideas from the last year. papers.baulab.info Which single new paper would you add?

Incredibly interesting Roundup of AI papers. open.substack.com/pub/sebastia...

Happy New Year! πŸŽ†πŸŽ‡ [Throwback to 2016 when I was in Sydney on New Year’s Eve]

Our new paper! "Analytic theory of creativity in convolutional diffusion models" lead expertly by @masonkamb.bsky.social arxiv.org/abs/2412.20292 Our closed-form theory needs no training, is mechanistically interpretable & accurately predicts diffusion model outputs with high median r^2~0.9

www.youtube.com/watch?v=UGO_... A super cool explanation of recent LLM mechanistic interpretability studies from anthropic etc. from Welch Labs (an awesome channel btw). see also "The Dark Matter of Neural Networks?" by @colah.bsky.social transformer-circuits.pub/2024/july-up...

niklaselmqvist.medium.com/what-you-sho...

An international team of scientists has launched 115TB of validated scientific data to enable AI breakthroughs across disciplines! Two huge open-source datasets across dozens of fields, from bacteria to supernova! The Well: github.com/PolymathicAI... Multimodal Universe: github.com/MultimodalUn...

Merry Christmas and so forth

A new paper dropped from DeepMind: Deliberation in Latent Space via Differentiable Cache Augmentation The trouble is, it's not very readable. I tried making a thread here, but it got far too long, so it's a blog now: timkellogg.me/blog/2024/12...

TrustMLVis Browser has been updated with eXplainable AI (XAI) work up to 2023! Check it out here: trustmlvis.lnu.se

Fractions can be weirdly beautiful, for something so mundane. This visualization just plots points of the form (a/b, c/d). Bigger dots mean smaller denominators. The biggest dot is (0, 0).

I’m releasing a series of experiment to enhance Retrieval augmented generation using attention scores. colab.research.google.com/drive/1HEUqy... Basic idea is to leverage the internal reading process, as the model goes back and forth to the sources to find information and potential quotes.

the algorithm is not some deity but a landscape, the feed is an uber ride across the manifold, only the windows are blacked out. what if you had a map of the algorithm? what if the UX of the feed let you look out of the window? musing with @infowetrust.com image from distill.pub/2017/aia/

Ilya Sutskever NeurIPS talk [video] Discussion

Literally me right now…

I'll be at @unireps.bsky.social this Saturday presenting a new experimental pipeline to visually explore structured neural network representations. The core idea is to take thousands of prompts that activate a concept, and then cluster and draw them using MultiDiffusion. πŸ§΅πŸ‘‡

Hi NeurIPS! Explore ~4,500 NeurIPS papers in this interactive visualization: jalammar.github.io/assets/neuri... (Click on a point to see the paper on the website) Uses @cohere.com models and @lelandmcinnes.bsky.social's datamapplot/umap to help make sense of the overwhelming scale of NeurIPS.

A comprehensive list of all interpretability papers from NeurIPS (complied by @neelnanda.bsky.social): docs.google.com/spreadsheets...

Sometimes our anthropocentric assumptions about how intelligence "should" work (like using language for reasoning) may be holding AI back. Letting AI reason in its own native "language" in latent space could unlock new capabilities, improving reasoning over Chain of Thought. arxiv.org/pdf/2412.06769