Finally got this prototype to work after 1000 years of debugging. Sweet release 🥲
The idea is I dump a bunch of unstructured notes on a topic in, feed that into gpt-4o-mini, ask it to label sections with a set of types – claim, evidence, assumption, etc. – and display them as coloured highlights
The idea is I dump a bunch of unstructured notes on a topic in, feed that into gpt-4o-mini, ask it to label sections with a set of types – claim, evidence, assumption, etc. – and display them as coloured highlights
Comments
Getting OpenAI results to play nicely with Remirror (the text editor) was most of the challenge. Also having to database when I haven't done much databasing before.
Can I design an LM interface that enables me to be a more rigorous critical thinker; meaning better at understanding my arguments, assumptions, etc.
This is a simple first pass to get it to return the right data structures, but whether the things it labelled as "claims" or "implications" are actually correct is a whole other kettle of fish
Next step is trying CoT and composition.
What’s CoT?
Essentially getting models to "think step-by-step" to improve their answers.
My Brain: Came of Thrones
I could be wrong but aren’t the o1 models supposed to be better on CoT? I’d be curious to know how the 4o vs o1 perform on your task.
We also call this "rude questioning" which is critical to find weak arguments and strengthen others.
(though the code is hot trash in flux right now... poke around but it's far from usable by others 😅)
I was trying to figure out how to humbly ask for the prompt info. This is such an amazing project, such an direct #Engelbart ian #AugmentIntellect by making visible effort.
Basic prompt (also a detailed in directory):
https://github.com/MaggieAppleton/lodestone/blob/main/src/evals/prompts/basic.ts
Thanks so much, I will have a poke!
https://huggingface.co/spaces/trohith89/Electronics-Sales-Classification
If you're having to pull text out of images then yep splitting the steps up certainly sound sensible to me!