Profile avatar
roymiles.bsky.social
Research Scientist @Noah's Ark Lab, Huawei PhD Imperial College London roymiles.github.io Working on multi-modality and efficient ML
28 posts 591 followers 252 following
Getting Started
Active Commenter
Posts 5 Comments 24

Hey! I will be at NeurIPS @neuripsconf.bsky.social this week (friday 13th) presenting our paper VeLoRA. tldr; simple method for compressing gradients with a coarse reconstruction during the backwards pass. Significant memory reductions while being complimentary to LoRA! github.com/roymiles/VeL...

submitted 80 days ago • 1 comment

I made a starter pack for people working or interested in multi-modality learning. It would be good to add lots more people so do comment and I'll add! go.bsky.app/97fAH2N

submitted 92 days ago • 15 comments

Our BMVC poster is today! Knowledge distillation can work in cross-task settings and even with a randomly initialised teacher! Our inverted projection decomposes into knowledge transfer and spectral regularisation, enabling teacher-free distillation with improvements on ImageNet-1K at no extra cost

submitted 92 days ago • 0 comments

Some related work worth checking out if you find this cool. mbaradad.github.io/shaders21k/ - learning good visual features from procedurally generated images. arxiv.org/abs/2403.14494 - distillation from randomly weighted teachers.

submitted 97 days ago • 0 comments

Hi Bluesky! I'm currently working on multi-modality learning and efficient ML (mobile devices). Hoping to regularly post about any interesting topics I find cool (ai, maths, physics). Looking forward to seeing how this platforms grows and meeting you all!

submitted 98 days ago • 1 comment