So one of the things that I think is lost on AI proponents is what I call the card catalog effect, a thing I shouldn’t call it because a lot of people probably have no experience with a card catalog.
Comments
Log in with your Bluesky account to leave a comment
Exactly the same concurrence of events happening in software engineering. Learning how to work through problems - training your brain to do that - is the only way to get good at coding. The *only* real way is iteration and problem-solving. Relying too much >
.. relying too much on blocks of code delivered via AI could result in large coding projects that never *really* run right, because the people gluing blocks of code together lost that skill - or never had it.
Also, this code will be deeply insecure, at least first generations of it.
Loved this thread. I knew where the Library of Congress hid their card catalog after they computerized their catalog, and I would sneak back to consult it. I hope it is still there.
There’s also a structural issue in that the libraries had knowledgeable people creating the best possible sources of information available given their resources. The AI builders seem to just scrape anything they come across, without even fully knowing what those sources are.
Yeah, the form of the media (physical, digital, whatever) isn’t important, it’s the curation. Like, there’s a fair number of librarians and archivists whose job is to figure out which records are important and which can be thrown out.
But with many AI, the source of the information isn’t clear.
I remember making some primitive data base cards for a small gallery many years ago. The process of making the cards made me very familiar with the collection. Each card has the info of where to find the stored artwork in the middle, and then categories around the frame. So we had to make categories
I've run into this with my job. I have a new position that I basically have to train myself on, which has required a lot of googling. The AI answers are both often wrong and don't help me actually learn. The chains of hyperlinks from reddit, to blogs, to YouTube videos, back to more blogs, has.
Having spent considerable time on line diagnosing and treating my own obscure health issue, I know have some kind of mini phd on the science as well as insight into all sorts of human suffering, wellness grift, and the slow dispersion of medical information. Wisdom is different than data.
Yes! In libraries we call this the information search process and a really cool thing is no matter your age or education level our brains go through all the same steps to reach a conclusion.
The best thing I did when I was teaching college was bring them in and have a librarian teach them not only about searching, but about how to evaluate and read what they found
Here’s how it went when I was a kid: you want to know about something, and it took some amount of work to find out. You’d go to the library. You’d look things up in the computer or the card catalog.
Just in LOOKING for the right card in the catalog, you'd see other cards and go "Huh. That looks interesting." And you'd leave the library with an armful of books about all kinds of things....
Before I knew what protein electrophoresis was, I knew it was important because the pages of the NIH library’s volume of the journal Science that held Laemmli’s paper had fallen out and been replaced by photocopies taped in by library staff.
I think this was how they first found the first-digit law. Printed logarithm tables were grubbier and more worn for the 1.xx numbers than for the 9.xx numbers because real-world numbers are more likely to start with low digits.
When I was in grad school, I had the dewey decimal number range for my area of study memorized. I'd just stand in the stacks and flip through books. Eventually I found what I was looking for, but I absorbed so many other things along the way.
But you pretty quickly learned that the chances of the answer being in an adjacent book were higher, and in the meantime, you’d look in the book you found and end up with some different stuff that made you ask the question differently.
Also, the journey to the shelf with the book that didn't have the answer, but was near the other book that did, took you past many other books. Opening them revealed answers to questions you hadn't considered.
This process forced you to learn in pieces: first, what were you looking for broadly, where the information was contained, then pieces as you flipped through the book looking for the right thing, and finally the information, now contextualized.
The same thing happens on line, in slightly different ways. You’ll find a Reddit forum where people are arguing about (my recent obsession) the best way to treat or not treat wood for a raised bed garden.
They will not agree! Some people will go with pressure treated wood and others will say “but that leaches off into the soil, you’re gonna eat that food!”
YES. The other thing this does (if there are enough titles) is gets me a broad overview and solid starting point on a new-to-me subject by scanning (TOC, index, intro, bibliography, etc) multiple similar works. I can see what's held in common and where there's controversy or differening opinions.
One of my middle school teachers (maybe history? could have been geography) taught this as a relatively quick way to establish what you do and don't know about a subject and to clarify what you're actually looking for.
Great thread! It reminds me of Bates’ berrypicking model from 1989, which I learned at uni and never forgot. It describes how people refine their information searches and adapt it based on the results they get, even changing their retrieval strategies as they go. https://pages.gseis.ucla.edu/faculty/bates/berrypicking.html
This is why I have a library at home, and why I used the London Library which has 100% open shelving. Adjacent shelves, books, chapters and paragraphs have information you need but that you weren't aware about in advance so could not ask for.
This process is what I missed terribly in law once Westlaw, etc. became everyday tools. I used to come up with some of my most creative arguments by “wandering” the digests.
Even the process of looking up a case in bound case reports, then abstract in adjoining case drags you in…
Or even just reading the dissenting judgment…
One of my favorite parts of doing research papers in college was exactly this. I loved finding books I hadn’t even known existed, and had such great information. And the libraries at UIUC were amazing. I still do this when I’m in the nonfiction section of my local library-see what else is nearby.
Yeah that's why profs I had required we used actual physical books in our assignments. We could use some web info, but had to have at least one or two books in every assignment.
i liked card catalogs.
It was like a manual rabbit hole & I discovered a lot of new things using it.
plus the sound the cards made as you flipped them, and the drawers...
I mean, I remember the way I felt, riffling through them...it was like an adventure.
I feel sorry for younger ppl who have never had this experience. It's different than an internet rabbit hole. I think part of it is the feeling that other ppl, years before you, riffled through these same cards.
Yep. As a teen I was interested in programming. Dewey decimal 1.6ish. Sadly my local library had nearly zilch. But 1.6ish is also weird phenomena including UFOs. I stuck to programming. :-)
Even going to school after card catalogues stopped being a thing, I realized that it was always a good idea to spend an extra 20 minutes in the stacks, checking everything on the shelf surrounding what I was there to pick up. Always revealed things that hadn't shown up in the initial search.
I often found interesting patterns in the numbers. I'd look up five books related to [whatever]. Four of them would have very similar numbers and one of them would be completely different.
The similar numbers told me where to go to look for the bulk of my information (including neighboring shelves). The outlier varied. Sometimes it was not going to be useful at all, but sometimes it showed me an additional factor I hadn't thought of.
A further point: AI is only as good as its training. For historical research, it is not yet able to access archives of news organizations that are hidden behind paywalls. BIG problem. So depending on your research subject, the Dewey decimal content can be fatally limited.
It's been strange teaching this to my dad, who is a semi-academic computer programmer and def knows card cats and research but often doesn't "know how to find something" on the WEB -- bc he forgets the first 1-3 (often more) searches are abt the path not the goal. Always (even when google worked).
And the Periodical Index volumes! I used to love looking up magazine references in those because there was always a topic pretty close to what I was looking for that would end up being useful.
For many powerful AI proponents, this is a feature not a bug.
Getng people used to accepting a single, authoritative answer is the goal.
The control of 'truth' is the point 😕
Completely relate & relive these descriptions of the joys of this. But the skill to extract & recount some essential extract from this is a separate matter. Particularly in mid process (& I’m often in mid process) I irritate others with the bits & connections.
I would just like to say that I’ve never really sat down and thought about the mechanics of knowledge acquisition and this is a delightfully genius description of it. Bravo.👏
I find this with online mapping tools: with a paper map, you see where you are going in context. If you just accept turn-by-turn directions, you don't learn where you are relative to everywhere else. I like to go off book so I can understand where I am better....
I try to look at a full map (not necessarily paper) before starting out, for exactly this reason. It's gotten worse since GPS systems no longer tell you if you've made a mistake, so you don't realize you're correcting, you're just circling. I AM NOT THAT TENDER, GPS, I CAN TAKE IT.
I framed this in a parallel way comparing two related skills
1. Curiosity is a skill
2. Serendipity is a skill
Together they are part of knowledge gaining where the process and final result are valuable.
The closest equivalent is probably the Public Access Catalog (PAC or OPAC for "Online" PAC), which is a library's public-facing digital catalog that looks a bit like a specialized search engine. The same principles apply: you can search for a keyword & use that to find what you're looking for.
Yes, my Library school called it Serendipity. Also discussed Librarian as gate keeper to information. Search engines looking for profits from shopping searches (clicks or sales) have a narrow commercial view.
I'm old enough to have done all this 🤣 curiously, this is how AI is best used. My experience is AI is biased AF and learns to tell you what you want to hear. I argue with it intensely. And then use post arguments for summaries, etc. The best insights are on the margins.
I don’t really agree with this, because you can have the same type of exploration with LLMs only faster. In the worst case, you can obtain search terms from the output of the LLM which aid your internet search. I tend to use ChatGPT after a fruitless internet search.
Courtney, this is brilliant and so astute. This is spot on and mirrors exactly how I have learned things offline and now online, to this day … it always follows the sort of flow you've described 🤩
Yep, well put. I have referred to this as "the browsing effect" and it is how looking for more Dickens in the library during my early teens led to Dostoevsky even though I had little clue about "literature", let alone Russian literature, at the time.
I teach research skills. Lately students have a harder and harder time conceptualizing what you describe—that knowledge might be organized by sets of relationships. (Google is also had for that—ranking isn’t categorizing.)
That was my process too, until I started doing research in Germany and the UK in grad school. Then I had to request the books and they were brought to the reading room by librarians. It really cramped my style.
I remember...the thrill of the hunt as you discovered or thought up new ways to squeeze information out of the catalog, the joy of discovering new ways to cross reference. Using a search function and keyboard doesn't quite have the same tangible feel of the weight and heft of the card drawers...
This marries up with something I had drummed into me at various points in time when approaching learning: "You don't know what you don't know" - understanding is like slowly drawing a map from the inside out. Sometimes the entire shape of what you thought the landscape was can change in an instant.
Indeed. The path to being an expert on something has many of these high spots that occur just before you discover a whole heap of things you didn't know existed.
This is off topic but is it weird that if someone mentions a claymore or chariot or stirrup people can generally be expected to know what that is, but if it's 8 track tape or stick shift or card catalog the kids are utterly baffled, helpless to figure it out? What's the rule on obsolete technology?
I think a lot of people have seen, or heard of, for example, the Fast and the Furious series, or Buffy the Vampire Slayer. Or Ghostbusters.
I suppose there probably are more movies where a character repeats the meme that nobody remembers what an 8-track is than movies where someone uses one
More of the newer movies, sure. But it also depends on what someone watches... or notices. If they are curious enough to look up/ask about such things.
My prof during my cinema course noted that shared culture [via movies] was less all the time. And that was in 1987, iirc.
There's nothing going on here except as people age they start to tell themselves young people are inferior, ignorant.
People use phrases like "the proof is in the pudding"? Proof? Pudding? What? You have to go back hundreds of years to unpack this phrase. Unnecessary because we only need the gist
The meaning of "card catalog" is in the term. It's data storage, on cards. You can understand an old story that refers to a buckler knowing it's a shield. You can look up why it's called a buckler but you don't need to.
8-track or Betamax; not quite the same: they were unpopular and not around long
I don't think young people are ignorant just b/c they might not know something I know. Nor do I tell myself -- or others -- that someone's ignorant or inferior.
That is your interpretation.
Hell, my mom was a teacher decades ago. But now she capitalizes common nouns and it makes me crazy.
Relatedly, one of my favorite pastimes is falling into wiki-holes where an initial search about one item leads me to open five other entries, with cascading results where I emerge hours later with new knowledge that may or may not be related to the original topic.
This is how curious people learn. Others just want an answer. As black and white and easy to communicate as possible. Opinion will do if facts are thin.
I call this serendipity research. The things you find from proximity in the card catalogue, or from running your fingers along the spines of the books next to and around the one you found on your target topic. The great web of information.
The library I work in has a card catalog in the archives. I got to add new subject cards to it when I worked in that department. Not many millennials can say that!
I've heard it called branch learning. Apparently common in intelligent/autistic people. You are after an answer but your journey takes you to so many different branches, that then branch out from there. By the time you find the answer, you have learned dozens of different but related things as well.
I thought about this a lot as my academic library increasingly purchased ebooks rather than physical ones. Searching for an ebook in the catalog is not like walking up and down the shelves. As a grad student, I found weird stuff I would never have read otherwise through the card catalog effect!
I'm just hearing "trust this ai NOW, it will do everything and even miracles I swear!!" which isn't useful to an untrusting person like me. Sounds like snake oil.
My experience with AI is awfully similar. I ask proceedingly detailed and sometimes branching questions related to my subject.
I dont necessarily trust the answers, but I have an idea or trial I want to do based on the info Ive collected. Else a place to go do other research via other sources
This isnt a difference between AI and not though. It is one between treating AI as a one prompt generator versus a socratic learning machine. Ironically AI can ingrain that process within a socratic learning method instead when doing a multiprompt method
Teak resists rot, but good luck being able to find that. I've got a bunch (1/2 cord) of cut wood that came w/ my house (but no fireplace🤷🏻♀️) so I'm making low raised garden beds with them. We'll see how well it works. I think it'll be fine for squash and lettuce and such.
So good!! Gonna age myself but we didn’t even have computers for searching - strictly card catalog/dewey decimal and tons of coins for photo copying the snippets you did find!
I wholeheartedly agree with your thread but 1)why couldn't the same happen with AI? 2)sadly there are moments in life/work when you actually need a certain information, quickly, and you don't have time to mess up with the card catalog. Having this opportunity seems positive to me.
there are very few moments when you need "a certain information" quickly. we survived without this immediacy for a very long time, and we can continue to survive without it if we didn't already have the tools for it.
I can guarantee that, at least in my job (science writer) there are A LOT of such moments.
The "we survived without" argument is weak IMHO. We survived with barely any technology, this doesn't ring as a good reason *per se* to avoid technologies.
so, what you seem to be telling me is you don't know how to do research, and you are willing to accept the word of unreliable sources. neither of these is good traits for a "science writer."
Where I am coming from: I feel in a sort of grey area with AI. While I detest the techbros pushing it and even more the capitalistic system in which it prospers, I am not so skeptical on the tech itself. I believe generative AI tecnology has potential and does not have to be the slopdevil.
I get what you’re saying but I feel like this is two different things. What you’re talking about is data analysis and aggregation. Some learning might be done but you want a definite answer, general research allows you to stumble across things making connections learning relationships.
Assuming you mean what my husband (a retired bookseller who lost his biz when SF techies would quiz him for info and then order from Amazon) calls the browsing effect, or the open stacks effect: nosing around for a spectrum of related info. If not, please elaborate.
It's funny because we could build "AIs" that work like that... e.g. a top K nearest neighbor latent space search 🔍👀 that would let you explore adjacent knowledge... but noooo, we keep building LLMs that hallucinate 🫠
If AI is to enhance research, it must learn from librarianship, not erase it. The precision of a card catalog came from controlled vocabularies and authority records. Future AI systems should integrate human metadata expertise, not just scrape data indiscriminately (Svenonius, 2000).
The “card catalog effect” is real—AI proponents often overlook how structured, human-curated metadata shapes discovery. Solution? Strengthen bibliographic standards in AI training data and ensure human oversight in knowledge organization (Buckland, 1992).
Or, how about we ditch the whole genAI chatbot/LLM model and build on stuff that actually WORKS.
We already HAD search engines that worked! Many kinds! For many tasks! Sprinkling genAI chatbots on top of them has only made them worse! Why can’t we strip out the genAI junk and go back to what works?
During high school, many eons ago, I worked at the local library. One of my jobs was to type card catalog cards for new books that came in and then file them in the catalog for public access. This was before correcting typewriters, so accuracy was crucial. No strikeouts allowed.
Oh how I miss the card catalog! (And physical encyclopedias to flip through and find new things on the way to finding the thing you thought you were looking for.)
I Irish a “turloch” is a lake in limestone bedrock that disappears when it’s dry (potentially inaccurate generalisation). Might be an anglicisation of that…
Comments
Exactly the same concurrence of events happening in software engineering. Learning how to work through problems - training your brain to do that - is the only way to get good at coding. The *only* real way is iteration and problem-solving. Relying too much >
Also, this code will be deeply insecure, at least first generations of it.
But with many AI, the source of the information isn’t clear.
Me: Did you try playing around with the search terms?
Them: "I did a search."
Their teacher: "Sorry I can't bring them in for a class on doing searches, too much material to cover!"
🥲
I do feel like AI caaaan help with research - as did the internet - but one needs to be very careful not to give up autonomy.
The chances of the answer you were looking for being in that book were not actually high.
Also, the journey to the shelf with the book that didn't have the answer, but was near the other book that did, took you past many other books. Opening them revealed answers to questions you hadn't considered.
Or even just reading the dissenting judgment…
Most of them don't even look at physical books. If its not available in electronic form they just move on.
Which sounds super "OK Boomer" but that doesn't change it.
It was like a manual rabbit hole & I discovered a lot of new things using it.
plus the sound the cards made as you flipped them, and the drawers...
I feel sorry for younger ppl who have never had this experience. It's different than an internet rabbit hole. I think part of it is the feeling that other ppl, years before you, riffled through these same cards.
Getng people used to accepting a single, authoritative answer is the goal.
The control of 'truth' is the point 😕
1. Curiosity is a skill
2. Serendipity is a skill
Together they are part of knowledge gaining where the process and final result are valuable.
THIS is why AI is so limited in it's benefits. It's such a cheap, error riddled shortcut.
#LoveLibraries
Google: Do evil!
The others? Those are retro, or more rare, or no longer used. Even though you and I know what those are too.
I suppose there probably are more movies where a character repeats the meme that nobody remembers what an 8-track is than movies where someone uses one
My prof during my cinema course noted that shared culture [via movies] was less all the time. And that was in 1987, iirc.
People use phrases like "the proof is in the pudding"? Proof? Pudding? What? You have to go back hundreds of years to unpack this phrase. Unnecessary because we only need the gist
8-track or Betamax; not quite the same: they were unpopular and not around long
That is your interpretation.
Hell, my mom was a teacher decades ago. But now she capitalizes common nouns and it makes me crazy.
You can get this "card catalog effect" from them, by looking at the linked sources the machine used to build your response.
Still net bad technologies tho.
They have a narrow band of utility while leaving a wide swath of damage.
Weird Al is the GOAT
I dont necessarily trust the answers, but I have an idea or trial I want to do based on the info Ive collected. Else a place to go do other research via other sources
https://www.edutopia.org/article/teaching-students-how-use-their-notes
https://bsky.app/profile/did:plc:3yysjejil7nczusktlpuftv3/post/3ligtswv3522s
The "we survived without" argument is weak IMHO. We survived with barely any technology, this doesn't ring as a good reason *per se* to avoid technologies.
We already HAD search engines that worked! Many kinds! For many tasks! Sprinkling genAI chatbots on top of them has only made them worse! Why can’t we strip out the genAI junk and go back to what works?