excited to say that our Substance Beats Style paper was accepted to NAACL! We investigate *why* student-written programming prompts don't work well for LLMs, and find that while students think it's because of technical vocabulary gaps, it's actually information content that matters
Comments
Francesca came up with a really neat way of visualizing edits to the information content of prompts
Just as much of being a good programmer is thinking about the problem specification, much of being a good writer is thinking about the audience’s attention span and path into the topic.
Also, I think our visualization methodology could very easily be adapted to other tasks. I'm not sure what this would look like for writing specifically, but if you can come up with a way of annotating the data, the graphs make it easy to look at where people get "stuck".