shan23chen.bsky.social
PhDing @AIM_Harvard @MassGenBrigham｜PhD Fellow @Google | Previously @Bos_CHIP @BrandeisU
More robustness and explainabilities 🧐 for Health AI.
shanchen.dev
36 posts
1,400 followers
223 following
Regular Contributor
Active Commenter
comment in response to
post
congrats！
comment in response to
post
Source: t.co/mV27ZZg5MN
comment in response to
post
Yea… he does have problems portraying female in stereotypical ways, big critics in China too
comment in response to
post
During the QA session, one stood up to her regarding this issue really respectfully and her response was: “That was not based on my judgment. That was based on the student's quote saying that the school was not teaching it, which meant that it applied to a lot of people from there."
comment in response to
post
Most of the talk discussed about bad practices. But only one slide mentioned specific group of people.
comment in response to
post
Haha which one has more nowadays?
comment in response to
post
Haha transformers really transformed both.
However, I feel like the division is even further… currently, seems like RL is taking over LM post training and many NLProc are dealing with language model enabled new applications
comment in response to
post
Thanks!
comment in response to
post
Imagine a world where these will be positively correlated
comment in response to
post
Quite possible!
Here, we found some early evidence that SAE features trained on language models are still meaningful to LLaVA.
More details will be provided in the post, and more details will be provided soon!
@JackGallifant
@oldbayes.bsky.social
@daniellebitterman.bsky.social
comment in response to
post
More on future potential reliance on LLM agent doing reviews and audits
comment in response to
post
I’m terrified by the massive openreview data. Potentially gonna bite back on us 🥲😥
comment in response to
post
END/🧵 Thanks to all our awesome co-authors:
@jannahastings.bsky.social
@daniellebitterman.bsky.social
And all our awesome collaborators who are not on the right platform yet! 🦋
Happy Thanksgiving! 🍂
comment in response to
post
5/🧵 Dive deeper into our methods, findings, and the implications of our research by checking out the full 📜 paper here: arxiv.org/abs/2405.05506
All our data can be downloaded from our website: crosscare.net
comment in response to
post
4.5/🧵 For the arxiv pretraining dataset, we also have an overall trend based on entity mentions! Guess which two terms are the big bump there back in 2019
comment in response to
post
4/🧵 We've also developed a new data visualization tool, available at [http://crosscare.net], to allow researchers and practitioners to explore these biases from different pretraining corpus and understand their implications better. Tools in progress! 🛠️📊
comment in response to
post
3.5/🧵 Moreover, alignment methods don’t resolve inconsistencies in disease prevalence across languages (EN 🇺🇸, ES 🇪🇸, FR 🇫🇷, ZH 🇨🇳). And tuning on English usually only affects English prompt output
comment in response to
post
3/🧵 By analyzing models across various architectures and sizes, we show that traditional alignment methods barely scratch the surface in fixing these discrepancies. This highlights the challenge in deploying LLMs for medical applications without reinforcing biases.
comment in response to
post
2.5/🧵How misaligned are things here? 📈Figure 2 shows the misalignment between real-world disease prevalence, pretraining data representation, and Llama3 70B.
comment in response to
post
2/🧵 Our study systematically explores how demographic biases embedded in pre-training corpora like ThePile affect LLM outputs. We reveal substantial misalignments between LLM representations of disease prevalence and actual data across demographics. 📷 👥
comment in response to
post
Oh dang, I gotta read this thanks
comment in response to
post
I recommend this!
apex-magazine.com/short-fictio...
comment in response to
post
Maybe highly interesting to you
aclanthology.org/2024.emnlp-m...
comment in response to
post
xai historian
comment in response to
post
Thank you!
comment in response to
post
Me and @daniellebitterman.bsky.social would love to be added! Thanks!
comment in response to
post
What were the questions?
comment in response to
post
Thanks!
comment in response to
post
I would love to join this! Thanks!
comment in response to
post
Thanks for organizing! Saving us from X is important!!
comment in response to
post
Thanks for putting this together!! May I be added too? Thanks!