We apply our recipe to train 2 Static Embedding models that we release today! We release:
2️⃣ an English Retrieval model and a general-purpose Multilingual similarity model (classification, clustering, etc.), both Apache 2.0. Fully integrated in Sentence Transformers, etc.
🧵
2️⃣ an English Retrieval model and a general-purpose Multilingual similarity model (classification, clustering, etc.), both Apache 2.0. Fully integrated in Sentence Transformers, etc.
🧵
Comments
📜 my training scripts, using the Sentence Transformers library
📊 my Weights & Biases reports with losses & metrics
📕 my list of 30 training and 13 evaluation datasets
🧵
I apply this simple architecture, but train it like a modern embedding model: Contrastive Learning with Matryoshka support.
🧵
🏎️ Extremely fast, e.g. 107500 sentences per second on a consumer CPU, compared to 270 for all-mpnet-base-v2 and 56 for gte-large-en-v1.5
📏 No maximum sequence length! Embed texts at any length (at your own risk)
🧵
🪆 Matryoshka support: allow you to truncate embeddings with minimal performance loss (e.g. 4x smaller with a 0.56% performance decrease for English Similarity tasks)
🧵
- @langchain.bsky.social