I am impressed that an LLM is able to do the parroting in the first question. It's impressive that it recognized what to parrot and pulled off all the steps correctly.
(The chain of thought does suggest it called out to Wolfram Alpha for a calculation en route. But that's the right thing to do!)
(The chain of thought does suggest it called out to Wolfram Alpha for a calculation en route. But that's the right thing to do!)
Comments
It also concretely demolishes the naysaying that LLMs “parrot” their answers. It’s borderline impossible for the training data to have had that exact question. It had to perform actual reasoning to get there.