Are Clinical T5 Models Better for Clinical Text? That's the question we asked in our #ML4H2024 paper. Turns out clinical models may not be worth it. 🧵 arxiv.org/abs/2412.05845 - ThreadSky

About ThreadSky

mdredze.bsky.social • 67 days ago

Are Clinical T5 Models Better for Clinical Text? That's the question we asked in our #ML4H2024 paper.

Turns out clinical models may not be worth it. 🧵

https://arxiv.org/abs/2412.05845

Comments

mdredze.bsky.social•67 days ago

T5 models are the workhorse of many clinical text applications (e.g. information extraction.) Several clinical T5 models have been trained using clinical data to improve performance on these tasks. Do these models work better than general T5 models?

mdredze.bsky.social•67 days ago

Comparing 2 clinical with 3 general models on 6 clinical datasets, we find that some clinical models improve. However, these clinical test sets come from the same domain as the clinical training data. Maybe the clinical models are better on THIS clinical data, but not in general?

mdredze.bsky.social•67 days ago

We try a new clinical task and dataset/domain. In this case, the clinical T5 benefits disappear.

mdredze.bsky.social•67 days ago

Maybe the real advantage for domain-tuned models lies in the low resource setting. With lots of supervised data, an out of domain model can do well. What about with just a few training examples?

mdredze.bsky.social•67 days ago

It turns out that when you have just a little supervised data, the models trained on more data and tasks, even when out of domain, do BETTER on the new clinical domain.

mdredze.bsky.social•67 days ago

Takeaways: If you can fine-tune a model on a specific clinical domain, that's great. If you can't, you should probably use models that are better overall, even if they aren't trained on clinical data.

Many more details in the paper!
https://arxiv.org/abs/2412.05845

Posting Rules

Be respectful to others
No spam or self-promotion
Stay on topic
Follow Bluesky's terms of service

Comments

Posting Rules

Reply