eval'd 24 retrieval method/chunking combos w/ fastbook-benchmark in an order of mag less time than manual evals. answerai-colbert-small-v1 had the best MRR@10, single-vec cos similarity (!) had the best Recall@10 Colab: drive.google.com/file/d/1joMU... Blog: vishalbakshi.github.io/blog/posts/2... - ThreadSky | a Reddit-style client for Bluesky

vishal-learner.bsky.social • 90 days ago

eval'd 24 retrieval method/chunking combos w/ fastbook-benchmark in an order of mag less time than manual evals. answerai-colbert-small-v1 had the best MRR@10, single-vec cos similarity (!) had the best Recall@10

Colab: https://drive.google.com/file/d/1joMUryzM8eNFWW4ZgmRt0lllaDfBhN4n/view?usp=sharing

Blog: https://vishalbakshi.github.io/blog/posts/2024-11-26-fastbook-benchmark-results/

1 / 2

Comments