cfoster.bsky.social - Profile | ThreadSky | a Reddit-style client for Bluesky

cfoster.bsky.social

Twitter: @CFGeek Mastodon: @[email protected] Researcher @ Finetune Learning. When I choose to speak, I speak for myself. 🪄 Tensor-enjoyer 🧪

54 posts 602 followers 133 following

Posts 17 Comments 33

Why aren’t our AI evaluations better? AFAICT a key reason is that the incentives around them are kinda bad. In a new post, I explain how the standardized testing industry works and write about lessons it may have for the AI evals ecosystem. open.substack.com/pub/contextw...

submitted 2 days ago • 0 comments

When we optimize automation, we sometimes optimize hard. Like this automated loom working away at an inhuman 1200 RPM. Wild. youtu.be/WweMNDqDYhc?...

submitted 48 days ago • 1 comment

Is there a website/database out there that tracks what major AI company executives say about the future of AI?

submitted 68 days ago • 2 comments

Transformers and other parallel sequence models like Mamba are in TC⁰. That implies they can't internally map (state₁, action₁ ... actionₙ) → stateₙ₊₁ But they can map (state₁, action₁, state₂, action₂ ... stateₙ, actionₙ) → stateₙ₊₁ Just reformulate the task!

submitted 72 days ago • 0 comments

Atticus Geiger gave a take on when sparse autoencoder (SAEs) are/aren’t what you should use. I basically agree with his recommendations. youtube.com/clip/UgkxKWI...

submitted 79 days ago • 0 comments

These days, flow-based models are typically defined via (neural) differential equations, requiring numerical integration or simulation-free alternatives during training. This paper revisits autoregressive flows, using Transformer layers to define the sequence of flow transformations directly.

submitted 80 days ago • 0 comments

Re: instruction-tuning and RLHF as “lobotomy” I’m interested in experiments that look into how much finetuning can “roll back” a post-trained model to its base model perplexity on the original distribution. Has anyone seen an experiment like this run?

submitted 86 days ago • 1 comment

I’ve been wondering when it would make sense for “AI agent” services to offer money-back guarantees. Wrote a short post about this on a flight. open.substack.com/pub/contextw...

submitted 88 days ago • 1 comment

Neat thing about real-money prediction markets is that you can get paid for doing this.

submitted 90 days ago • 0 comments

A bit of clever mechanism design: prediction markets + randomized auditing. If you have 100 verifiable claims you want information on but can only afford to check 10, fund markets on each. Later, use a randomized ordering of them to check the first 10. Resolve those to yes/no, refund the rest.

submitted 92 days ago • 3 comments

Still gathering my thoughts on @TheCurveConf, but for now, a short reflection on why I like “the curve” as a way of thinking about the future of AI. (1/6)

submitted 93 days ago • 1 comment

RT-ed and endorsed

submitted 94 days ago • 0 comments

CLAIM: In areas where we can’t measure what (we claim) we want & where we won’t change our minds about that, we’ll struggle to make AI systems that give us better—rather than merely cheaper, faster, more consistent—outputs. But I think that’ll really pressure us to revise our wants.

submitted 94 days ago • 1 comment

Timothy B. Lee here gives a good short list of what human attributes might still have value (at least temporarily) in a hypothetical world where AI systems are capable of acting as “remote worker substitutes”. open.substack.com/pub/understa...

submitted 98 days ago • 1 comment

For those of us that rely on earned income, a key concern about the future is “Will automation soon put me out of work?” But at the moment, we can’t do much about it. Would you pay 1% of your earnings per year to protect a year’s worth of future earnings if most jobs are suddenly automated away?

submitted 99 days ago • 1 comment

Whenever faced with a hard problem, some AI folks say “I know, I’ll use reinforcement learning.” Now, they have two hard problems.

submitted 100 days ago • 0 comments

Using Bluesky to reboot your character arc for the LLMs

submitted 102 days ago • 0 comments