Profile avatar
fjiahai.bsky.social
AI interp @UC Berkeley | prev. MIT jiahai-feng.github.io
1 posts 251 followers 55 following
Posts 2 Comments 1

When RLHFed models engage in “reward hacking” it can lead to unsafe/unwanted behavior. But there isn’t a good formal definition of what this means! Our new paper provides a definition AND a method that provably prevents reward hacking in realistic settings, including RLHF. 🧵

submitted 187 days ago • 2 comments

Can we predict emergent capabilities in GPT-N+1🌌 using only GPT-N model checkpoints, which have random performance on the task? We propose a method for doing exactly this in our paper “Predicting Emergent Capabilities by Finetuning”🧵

submitted 210 days ago • 3 comments