I often see a lot of people with outdated understanding of modern LLMs.
This is probably the best interpretability research to date, by the leading interpretability research team.
It’s worth a read if you want a peek behind the curtain on modern models.
This is a really good science communication article, it describes their work in clear terms (finding structures that relate to abstract concepts, seeing when they are activated and how strengthening and weaking them modifies outputs) and goes into the implications for it. I’m probably going to save this link as a rebuttal for the people who claim LLMs just predict the next word and have no concepts embedded in them.
Yes, good topic, good research…
( you have a few typos : intobthe … into the, predicr … predict, im … i am. )I doubt that anyone saying that LLM are calculating next word solely based on previous sequence. It’s still statistics, regardless of complexity.
Yes, but people forget that our brains, and therefore our minds, are also “simply” statistics, albeit very complex.
Yeah I found this kind of reductionist talk pushes people to overlook the emerging properties of the system, which is where the meat of the topic is. It’s like looking at a living cell and saying “yeah well this is just chemistry”.
Saying that it’s “statistics” is, at best, unhelpful. It conveys no useful information. At worst, it’s misleading. What goes on with neural nets has very little to do with what one learns in a stats course.
It’s a well written article that raises some good points, however it’s also a bit of an ad for their particular ai, the research is practically only within that LLM and every objection about LLM’s it raises, their model seems to be well adjusted or adjustable to it.
There is no mind. It’s pretty clear that these people don’t understand their own models. Pretending that there’s a mind and the other absurd anthropomorphisms doesn’t inspire any confidence. Claude is not a person jfc.
Ah yes, it must be the scientists specializing in machine learning studying the model full time who don’t understand it.
You’re reading the title too literally. “Mind” is only mentioned once in the entire article, and that’s in the title.
