I hope I am not late to the party (was away post-quals chilling) but here are some thoughts on why this is bad IMO: First, a disclaimer that I am writing this as an African who is a speaker of multiple African languages, NLP researcher of African languages, and HCI researcher focusing broadly on.. - ThreadSky

hellinanigatu.bsky.social • 100 days ago

I hope I am not late to the party (was away post-quals chilling) but here are some thoughts on why this is bad IMO:

First, a disclaimer that I am writing this as an African who is a speaker of multiple African languages, NLP researcher of African languages, and HCI researcher focusing broadly on..

Reposted from Dr Abeba Birhane

this is a green flag for openai & meta to formally be arbitrators of our languages & mass exploit the population (& researcher that've poured their souls into low resource languages),all to throw unreliable AI that has so far proven to result in more harm than benefit
www.reuters.com/technology/a...

Comments

hana.goldenplums.name•99 days ago

📌

chrisgoss.bsky.social•99 days ago

📌

melrosa.bsky.social•99 days ago

📌 for later

ryancallihan.bsky.social•99 days ago

Love this. Not to mention that whatever is SOTA for English and languages sharing similar properties to English, are not necessarily the best way to work with other languages and language families.

rmlockley.bsky.social•95 days ago

📌

mrmarkham.bsky.social•100 days ago

Great. And a warm and hearty welcome to you!

marielza.bsky.social•99 days ago

📌

femtetv.bsky.social•99 days ago

📌

lovesdaryl.bsky.social•96 days ago

@mglass.bsky.social Relevant to your interests?

hellinanigatu.bsky.social•100 days ago

Global Majority communities. However, I DO NOT claim to know what is best for ALL Africans.

1. The inclusion of "low-resourced" languages in whatever is STOA for "higher-resourced" languages distracts from the actual community needs for Language Technologies. In our recent EMNLP paper

hellinanigatu.bsky.social•100 days ago

(https://aclanthology.org/2024.emnlp-main.983.pdf) we frame this as a Zeno's paradox where the trend in low-resourced NLP is like chasing a constantly moving goalpost instead of building useful tech. Machine Translation is still BAD for the majority of low-resourced languages (eg https://aclanthology.org/2024.emnlp-main.1102.pdf).

hellinanigatu.bsky.social•100 days ago

ASR is bad for these languages; Whisper's OWN REPORTED results show a staggering WER of 120% for Amharic 🫠 However, when these languages are "included" only on a surface level in research (i.e are simply lines in a Table on the released report or published paper) it distracts from how bad

hellinanigatu.bsky.social•100 days ago

the performance actually is because it is 1 language from 109 languages or so.

2. The surface-level inclusion makes it difficult to publish research that is practical and community-centered. For researchers like myself who only have the resources and linguistic knowledge to work on a few languages

hellinanigatu.bsky.social•100 days ago

the fact that the languages are included in large-scale projects by Big Tech (re: ASR performance of Whisper) makes it so that reviewers believe 1. working on a handful of languages is not enough and 2. that the problem has already been solved, even though the performance is bad.

marcel.bollmann.me•95 days ago

Wait, but isn't that a problem of the reviewers, and not the Big Tech? Maybe I'm idealistic but I don't see why your points should follow from the premise. Ofc I see that some reviewers will have these attitudes but then it's the reviewing processes I would see as the primary thing that needs change

yoavgo.bsky.social•99 days ago

doesnt this argument go both ways, though? eg, reporting on one amharic system beating another amharic system, improving WER from 90% to 80% could be perceived as "niche and not high quality research", vs "an amharic system beat the best system on the Whisper leaderboard improving by 40 points",

hellinanigatu.bsky.social•100 days ago

3. The surface-level inclusion also diverts financial resources from on-the-ground researchers. The most recent call from Lacuna Funds explicitly stated they are not looking for Machine Translation datasets (https://lacunafund.org/wp-content/uploads/sites/11/2024/06/Natural-Language-Processing-RFP-2024-Lacuna-Fund.pdf) even though, again, MT is BAD for these languages.

Comments

Posting Rules

Reply