Hello Bluesky! 🔵 We start our account by having our third guess for Ask Me Anything session #3DV2025AMA! Noah Snavely @snavely.bsky.social from Cornell & Google DeepMind! 🌟 🕒 You have now 24 HOURS to ask him anything — drop your questions in the comments below! Keep it engaging but respectful! - ThreadSky | a Reddit-style client for Bluesky

3dvconf.bsky.social • 93 days ago

Hello Bluesky! 🔵 We start our account by having our third guess for Ask Me Anything session #3DV2025AMA!

Noah Snavely @snavely.bsky.social from Cornell & Google DeepMind! 🌟

🕒 You have now 24 HOURS to ask him anything — drop your questions in the comments below!

Keep it engaging but respectful!

Comments

akashsharma02.bsky.social•93 days ago

Which problem in 3d computer vision are you most excited about at the moment?

snavely.bsky.social•92 days ago

I'm really excited about structure from motion, which is having a real renaissance (ACE0, FlowMap, MASt3R-SfM, VGGSfM, GLOMAP, etc). It will be cool to see how SfM methods interact with VLMs -- SfM always seems like the last problem to jump on the latest bandwagon, but maybe that will change.

snavely.bsky.social•92 days ago

I also really want to see truly photorealistic scene reconstruction methods. With NeRF and Gaussian Splats, we are getting closer, but I feel like we aren't able to capture in a 3D reconstruction the amount of detail that is present in, say, a 24-megapixel photograph.

snavely.bsky.social•92 days ago

I'd really like to see multimodal 3D reconstruction of scenes, which capture motion (of things like water, people, clouds) and other modalities like audio. Dynamic scene reconstructions would likely be much more immersive and compelling that static 3D scenes.

willsmithvision.bsky.social•93 days ago

Do generative video models solve 3D vision?

snavely.bsky.social•93 days ago

That is a cool question! I had to spend a lot of time thinking about it. I think it depends on part what it means to "solve 3D vision".

snavely.bsky.social•93 days ago

On the one hand, you have problems like visual SLAM, where maybe generative video models could solve it, somehow, but it seems like they would always be way too slow for real-time performance, and also overkill -- it could probably be "solved" in much more direct, faster ways.

snavely.bsky.social•93 days ago

Then you have problems like photorealistic reconstruction from a (maybe sparse) set of images. For the sparse-view setting, it seems like generative video models could be really useful for posing views and filling in missing parts, etc.

snavely.bsky.social•93 days ago

But they also might be overkill -- maybe a more targeted model, something like a fancier version of DUSt3R, could work just as well, or better.

snavely.bsky.social•93 days ago

Personally, what I want "solving 3D vision" to mean is "recovering, from zero or more images, a perfectly realistic simulation of the underlying scene". Where I could go into the scene and treat it as an open-world game. I could knock over a lamp and the right thing would happen.

ericzzj.bsky.social•92 days ago

Hi Noah @snavely.bsky.social, you have so many great papers in 3D vision. Do you have a favorite one among them? Also, I’d love to hear your thoughts on the future development of geometry-based vision. Thanks!!

snavely.bsky.social•90 days ago

Thank you for the nice comment! I spent a lot of time thinking about it, but I don't think I have a favorite paper. Maybe the paper with the most creative use of data is one called "Learning the Depths of Moving People by Watching Frozen People".

snavely.bsky.social•90 days ago

I also think it's neat that we wrote papers (10 years apart) called "Where's Waldo" and "Who's Waldo". What's next, "When's Waldo?"

snavely.bsky.social•90 days ago

For the future, I hope that we will have reconstruction methods that can be much more high-resolution -- I think quality is still an issue with things like NeRF or 3DGS, though these are getting better.

snavely.bsky.social•90 days ago

I also hope that we can figure out how to reconstruct scenes using multiple input modalities -- images, text, audio, etc.

ericzzj.bsky.social•89 days ago

Thanks Noah @snavely.bsky.social !!

camomille5000.bsky.social•93 days ago

Which publication do you feel best represents your life's work ?

snavely.bsky.social•93 days ago

Interesting question! I honestly don't think my work is all that important in the grand scheme of things, and that is okay with me. I would be happy to tinker here and there and make weird discoveries along the way.

snavely.bsky.social•93 days ago

So it is hard for me to think about what my "life's work" is, because that sounds like something that ought to be grand. I feel like the process of research -- things like mentoring students, who might end up doing much more important things than me -- is as (or more) important than any result.

snavely.bsky.social•93 days ago

If I were to come up with an answer that maybe sounds grand, I would say our recent work on MegaScenes that reconstructs many, many scenes from the Internet. Because a long-term aspiration of mine is to reconstruct as much 3D as possible from online photos, just to see what happens if you try.

snavely.bsky.social•93 days ago

But an answer that feels more correct to me is a paper like Visual Chirality, which explores a question that seems a bit silly (determining if an image has been mirror flipped or not), but where I was kept awake thinking about it, and where we had a lot of fun along the way.

bokmangeorg.bsky.social•93 days ago

What's the most important/interesting failure case of classic SfM that you have found through the MegaScenes work?

snavely.bsky.social•92 days ago

Nice question! The one I don't know how to solve is when an object moves occasionally. This happens with some sculptures.

snavely.bsky.social•92 days ago

The Cucumber statue I mentioned in another thread is an example, where the barrel and pickle are not always in the same position. It took us ages to notice that. https://commons.wikimedia.org/wiki/Category:Statue_of_cucumber_in_Nizhyn

snavely.bsky.social•92 days ago

Other examples: the Venus de Milo in the Louvre is moved slightly (by just a few inches) every once in a while. That Charging Bull sculpture in Wall Street moved a couple times too (you can see this in the nice Internet time lapse work from Ricardo Martin Brualla et al.) https://youtu.be/oQpq4TM96Ow?si=nXAOlEtjbpbdV7oE&t=278

snavely.bsky.social•92 days ago

This kind of causes havoc with SfM, especially if the motion is small enough that it could be mistaken as noise, but then everything doesn't *quite* fit together nicely.

snavely.bsky.social•92 days ago

On the other hand, I think detecting really subtle changes in scenes, like a building sinking into the ground very slowly over time (say by 1cm per year), might be a really interesting research problem!

jaredheinly.bsky.social•92 days ago

What's something fun, interesting, or surprising that went into making some of your work possible, but that most people might not be aware of?

snavely.bsky.social•92 days ago

It's really hard to think of surprising things, but one thing that comes to mind is how much time I spent early on in getting permission to use images. For instance, SIGGRAPH historically has been really insistent that authors have rights to publish imagery in papers, for good reason.

snavely.bsky.social•92 days ago

For each of the 100s of images that appeared in the Photo Tourism paper and video (SIGGRAPH 2006), I emailed the Flickr photographer and asked for permission. At the time, Flickr had an internal mailing system where you could send a message to any user. (I don't think that still exists.)

snavely.bsky.social•92 days ago

Almost all of the users responded, and most were really gracious. They were really glad to be asked. Thinking back, it seems like a totally different world. But this all feels really timely again in the age of scraping massive datasets.

snavely.bsky.social•92 days ago

(Creative Commons would have made my life much easier back then, but it wasn't yet so prevalent. We rely on it a lot now.)

snavely.bsky.social•92 days ago

For that paper, I also got in touch with the estate of Ansel Adams (centered at the Center for Creative Photography, at the University of Arizona) for permission to use one of Ansel Adams's photographs for the publication.

csprofkgd.bsky.social•93 days ago

Which coordinate system do you prefer to work with? 🤓

Comment image

snavely.bsky.social•93 days ago

Interesting! I guess these all seem horrible to me? Is good old (+x, -y, +z) not an option?

akashsharma02.bsky.social•93 days ago

That's the opencv convention, which imo feels natural to most CV folks!
The above list seems limited to rendering/graphics software conventions

craigreynolds.bsky.social•92 days ago

In my libraries, I use the top row (“Unity”).

But I've felt it was too much “cognitive load” to ask the application programmer to remember these conventions. I always provide writeable accessors with names like right(), up(), forward(). Similarly for the basis vectors in 4x4 transform types.

nandometzger.bsky.social•90 days ago

Do you have advice on how to choose a research topic? What's your process?

mhalber.bsky.social•93 days ago

What's your favorite movie, and why?

snavely.bsky.social•93 days ago

Interesting! I don't know how to choose a favorite. Here are some that come to mind:

The Empire Strikes Back: In high school, my favorite movie. I love Star Wars, and this is the best Star Wars movie. Still my favorite film score.

snavely.bsky.social•93 days ago

Koyaanisqatsi: Made a big impression on me as a kid. I'd never seen anything like it. The visuals are stunning. The consonance between imagery and score is amazing. Vision/graphics folks especially might find the time-lapse and slow-motion footage appealing / mesmerizing.

snavely.bsky.social•93 days ago

The Apartment.

snavely.bsky.social•93 days ago

Dungeons & Dragons: Honor Among Thieves is recent-ish movie that is way more fun than it has any right to be!

snavely.bsky.social•93 days ago

The Social Network is movie that I think was prescient and that also has a great score.

parskatt.bsky.social•93 days ago

Favorite MegaDepth / MegaScenes scene?

snavely.bsky.social•93 days ago

How did you guess that I have a secret list of scenes that I really like?

All-around favorite: "Statue of cucumber in Nizhyn" (https://megascenes.github.io/web-viewer/?id=135136&rec_no=0). Just a wonderfully weird thing that I'm glad exists! There is also a secret problem for 3D reconstruction methods in this scene.

Comment image

ducha-aiki.bsky.social•92 days ago

That's the cucumber and pickle capital of Ukraine!

snavely.bsky.social•93 days ago

Most majestic: "Genoese fortress (Sudak)" (https://megascenes.github.io/web-viewer/?id=238964&rec_no=1). A fortress built on the side of mountain, leading up to a grand peak. I like learning about new places like this!

Comment image

snavely.bsky.social•93 days ago

Place I'd love to visit: "Badami Cave Temples" (https://megascenes.github.io/web-viewer/?id=207371&rec_no=0).

snavely.bsky.social•93 days ago

Furthest south: "Trinity Church, Antarctica" (https://megascenes.github.io/web-viewer/?id=87412&rec_no=0). Not the most dramatic point cloud, but it is nice to know that there are reconstructions on all continents.

snavely.bsky.social•93 days ago

There's a bunch of other really amazing places and point clouds. Makes me wish I could find the time to travel more.

I encourage folks to explore the MegaScenes webviewer -- it's really fun finding new gems in there! (And also finding interesting ways SfM fails.)

https://megascenes.github.io/web-viewer/

t-martyniuk.bsky.social•92 days ago

wow, excited to see a Ukrainian monument climbing to the top!

parskatt.bsky.social•93 days ago

Awesome choices!
I love Saint Peter's Square (and friends, aka 0015)
https://megascenes.github.io/web-viewer/?id=216019&rec_no=0

khoant.bsky.social•93 days ago

Can you share the research toolkits you use for 3D visualization and evaluation in your work?
For instance, tools like Rerun, Blender, Matplotlib, PyVista, CloudCompare, or others? Thank you

snavely.bsky.social•93 days ago

I am so horribly out of date when it comes to 3D visualization tools. I am still using octave and gnuplot to visualize things, when I need to do it myself. But people I work with who are much more savvy than me have created wonderful custom tools in javascript, using basic libraries like WebGL.

snavely.bsky.social•93 days ago

Having a lightweight 3D web viewer that can be easily modified for a custom visualization is really handy for this sort of thing, but depends on honing the skills to build such tools (or, in my case working with someone like Richard Tucker who is amazing at this).

craigreynolds.bsky.social•92 days ago

For the last year or so, I've been using Open3D, originally from Intel Labs. Still a work in progress but comprehensive and actively maintained.

https://www.open3d.org
https://github.com/isl-org/Open3D

craigreynolds.bsky.social•92 days ago

Forgot to mention: Open3D is written in c++ and also has a full set of Python bindings. While I'm using it from c++ I think most users call it from Python.

fguney.bsky.social•93 days ago

you always work on the most interesting problems, how do you decide what to work on next?

snavely.bsky.social•92 days ago

Thank you, Fatma! That is really kind of you. I'm a big fan of your work, too! A few thoughts:

- Often it is students or collaborators who come up with interesting directions (sometimes over my skepticism, which I think is really great). I try to be open-minded about ideas.

snavely.bsky.social•92 days ago

- I have a big list of project ideas that I've slowly built up over the years. (I write down all ideas, no matter how bad.) Some of the ideas I pitch for years before they get traction. (How excited I am about an idea is often inversely proportional to how easy it is to get others excited about it.)

snavely.bsky.social•92 days ago

- I like projects where the seed of the project is some interesting dataset that is underexplored. Often the novelty of data can sustain excitement (for me) even if the end goal isn't totally clear.

snavely.bsky.social•92 days ago

- I am pretty allergic to working on things that I feel are too popular. That's a harder kind of strategy, though. It's also really hard to come up with interesting ideas that are off the beaten path. I honestly don't have that many ideas that are think are really good.

snavely.bsky.social•92 days ago

- I'm not too sure about this one, but maybe it is healthy not to read too much of the latest literature when coming up with ideas, and spending time doing other things? Sometimes ideas will come to me when I'm doing something totally different, like reading a good book or watching a fun movie.

martin-r-oswald.bsky.social•93 days ago

Having worked and collected tons of experiences with all kinds of scene representations, what is your favorite one and which representations or combinations thereof you believe are important in the future?

snavely.bsky.social•93 days ago

Nice question! My favorite scene representation is code -- procedurally generated scenes. One of my most beloved videos is Inigo Quilez's "Painting a Landscape with Maths" (https://www.youtube.com/watch?v=BFld4EBO2RE). This nicely shows the power of composing simple functions in synthesizing visual complexity.

snavely.bsky.social•93 days ago

But, I can't say I've really used code representations in my own work, or that they are very popular right now in computer vision.

There are some pioneering efforts in this vein, though (and some folks like Daniel Aliaga and Daniel Ritchie have been doing inspiring work for some time).

snavely.bsky.social•93 days ago

Since LLMs are good at generating code, I could see hybrid representations that combine code in various forms (scene graphs, procedural models, shaders) becoming more popular in vision.

snavely.bsky.social•93 days ago

That said, probably the best scene representation is likely task-depending -- one thing for SLAM, another for view synthesis, yet another for scene generation.

snavely.bsky.social•93 days ago

BTW, this fascination with procedural graphics started when I was a kid and I came across demoscene-style demos (64K Commodore demos and the like). Procedurally generated scenes are also used to great effect in games like Nethack, Minecraft, etc.

martin-r-oswald.bsky.social•92 days ago

Thanks for the detailed answer! I love these representations too as they were popular in early outdoor 3D games, but I haven't thought of them much from a computer vision perspective. Probably an interesting topic to revisit ;-)