Delighted to be a minor co-author on this work, led by
Pranav Nair: Combining losses for different Matyroshka-nested groups of bits in each weight within a neural network leads to an accuracy improvement for models (esp. 2-bit reps).
Paper: "Matryoshka Quantization" at https://arxiv.org/abs/2502.06786
Pranav Nair: Combining losses for different Matyroshka-nested groups of bits in each weight within a neural network leads to an accuracy improvement for models (esp. 2-bit reps).
Paper: "Matryoshka Quantization" at https://arxiv.org/abs/2502.06786
Comments
Other paper authors, including co-first authors Pranav Nair (https://pranavn1008.bsky.social), and Puranjay Datta (https://puranjay1412.bsky.social), and Aditya Kusupati (https://adityakusupati.bsky.social) and Prateek Jain.