Wish we had polls here but in their absence: can ML people tell me how they would define "ablation" or "ablation study"?
As someone who frequently translates between neuroscience and ML, I fear this is another term that we both use, but slightly differently. #MLsky
As someone who frequently translates between neuroscience and ML, I fear this is another term that we both use, but slightly differently. #MLsky
Comments
PS. Here is a way to do polls. i havent tried it yet https://poll.blue/post
paper reviewers love ablation studies! it is sometimes required for approval.
paper reviewers have not heard of this opinion.
A fancy/obfuscatory word was needed because the words it stands in for would imply there is a control condition or sensible baseline
anyway, ablation is for narrowing down your findings by taking away some part of it to find out what has the most effect or isn't relevant to the findings at all
e.g. You lesion/ablate part of an algorithm or a component in an architecture and show how the perf changes. If there’s a huge drop in perf, it shows the component is crucial. See for instance: https://arxiv.org/abs/2310.00194
the specific instance I have in mind is changing the standard recurrent matrix to complex diagonal in a vanilla RNN, so it's rather adding something. But it's subtractive if you think from the other direction so it's an inconsequential semantic debate?
I've called freezing RNN gates during training 'ablation'. It removes a component from the network, with the behavior of interest being the learning trajectory.
Would you call this something different?
This wouldn’t be too far from the use of the word ablation in neuroscience.
If you are deleting a term in the loss, then yes, call it an ablation study.
But, I dislike it when people use the word to refer to a replacement of the loss.
https://www.amazon.com/Experimental-Design-Biologists-Second-David/dp/1621820416
But I'm also an old fogey at this point so don't listen to me :)
https://bsky.app/profile/bwyble.bsky.social/post/3lbzingtwv22o