Wrote up my notes on ModernBERT, the brand new modern alternative to 2018-era BERT released by @benjaminwarner.dev and @howard.fm and team simonwillison.net/2024/Dec/24/... - ThreadSky

simonwillison.net • 62 days ago

Wrote up my notes on ModernBERT, the brand new modern alternative to 2018-era BERT released by @benjaminwarner.dev and @howard.fm and team https://simonwillison.net/2024/Dec/24/modernbert/

Comments

davidhuang.blog•62 days ago

📌

masoudmaani.bsky.social•62 days ago

sorry for the bother, but there's a doppelganger of you out here with the ID
https://bsky.app/profile/simon.fedi.simonwillison.net.ap.brid.gy
It would be a good idea to post it for report if it's not related to you.
best regards

simonwillison.net•62 days ago

It's me - that's my mastodon account automatically reposted to here

masoudmaani.bsky.social•62 days ago

ok, nice, you might wanna know that handle doesn't register as a valid DID, which might lower its visibility.

simonwillison.net•62 days ago

I didn't see that one up, someone created it a while ago using @ap.brid.gy

masoudmaani.bsky.social•62 days ago

ah, I see, in any case, this is how it shows up as a DID, or rather doesn't show up, good people at @ap.bird.gy might wanna look into it.

simonwillison.net•62 days ago

More on ModernBERT here https://bsky.app/profile/benjaminwarner.dev/post/3ldur45oz322b

scottonote.bsky.social•62 days ago

do we still need an NER model fine-tuned on top of ModernBERT?

zehavoc.bsky.social•62 days ago

It’s a pity you didn’t mention the French core and corresponding authors

simonwillison.net•62 days ago

What's the French core?

zehavoc.bsky.social•62 days ago

(A and B) C

zehavoc.bsky.social•62 days ago

Anyway, I meant Benjamin Clavier and Antoine Chaffin

howard.fm•62 days ago

Thanks for checking it out Simon :)

adambrosiomd.bsky.social•62 days ago

Hi! Can you suggest some resources on practical use cases of such models?

rasmus1610.bsky.social•62 days ago

Sentence classification, information retrieval, named entity recognition are among the top use cases.

Everything that isn’t concerned with generating text but doing „discriminative“ tasks on text.

callmephilip.com•62 days ago

Original announcement also stresses the fact that ModernBERT was trained on a whole lot of code. This unlocks all sorts of code search related use cases

adambrosiomd.bsky.social•62 days ago

How does it do name entity recognition!?

rasmus1610.bsky.social•61 days ago

https://huggingface.co/dslim/bert-base-NER

arnicas.bsky.social•62 days ago

Sad lol: “I'm not sure why I had to use numpy<2.0 but without that I got an error”

Posting Rules

Be respectful to others
No spam or self-promotion
Stay on topic
Follow Bluesky's terms of service

Comments

Posting Rules

Reply