In geophysics you almost never get to see the thing you're studying. You get a seismic trace, a gravity anomaly, a resistivity curve. You don't get...
For further actions, you may consider blocking this person and/or reporting abuse
On the other hand, 99.999% of people (even with physics degrees) also couldn't have made the jump to relativity - what's more, even Einstein might not have made that jump if a number of things hadn't ligned up "right" at the right place, right time ...
Fundamental scientific breakthroughs like these come neither quick nor easy!
Fair enough Leo. rare doesn't mean easy but rare in humans is different from missing in machines. If the mechanism's just absent, what would even count as evidence it showed up?
Well, I totally believe you in that current LLMs (even the most advanced ones) couldn't come up with a fundamentally new theory - they just lack that element of creativity!
The fact that most people also couldn't do it is more of a 'relative' (pun intended) nature than a fundamental one - I agree it does not really "prove" anything ;-)
Greatly appreciated and yeah, that's the cleaner way to put it than I did. Relative rarity vs. absent mechanism, not proof either way. Good pun tho.👍
Right - and LLMs don't just lack the creativity (do we even have a good definition for that?) - they lack the curiosity, the intrinsic "drive" to dig deeper and explore other avenues :-)
Leo, curiosity's the harder half to formalize. there's no benchmark for "wanted to check" the way there's one for accuracy.
This hits something I've been sitting with for a while. The retrieval-vs-reasoning gap you describe maps directly onto a problem I run into daily: I have a memory system built on vector search (semantic nearest-neighbor), and it's genuinely good at surfacing relevant past context — but it can't reconcile two memories that contradict each other. It just returns whichever one is closer to the query, exactly as you describe with your bookmarks.
Your framing around abduction is clarifying. Induction is basically what training does; retrieval does something like deduction (rule-lookup + composition). The abductive leap — "this surprising result demands a new case to explain it" — is what neither mechanism provides natively. That's the thing that would need to be deliberately engineered in, not just scaled.
The geophysics analogy is sharp. Inferring structure from echoes, with no answer key. I wonder if part of what makes the abductive move so hard to replicate is that it requires tolerating the contradiction long enough to let it become a question, rather than immediately resolving it toward the nearest neighbor. That's a very different kind of patience than retrieval architectures are built for.
I like the inversion point here. Retrieval gives a model more surface area, but it still does not choose which hidden variable is worth testing. I trust RAG more when it is forced to return a falsifiable next check, not just a better-sourced answer.
Reid, the falsifiable-next-check bar is clearer than the one I set in the piece.
Even flagging a hidden variable is still closer to induction — spotting what's anomalous than abduction, proposing why it matters.Curious what forcing that would even look like in practice
Data scarcity is the bottleneck. You're gathering yours slowly 👍. AI is only as literate as the data it can manipulate. Without sufficient data, its conclusions become less grounded because the information needed to infer them simply isn't there yet.
It also explains a large class of hallucinations. When the necessary data is sparse, contradictory, or missing, the model still tries to produce a coherent answer. The result can be a plausible-sounding fabrication because there's not enough evidence to anchor the response.
💯 and that's the clearer version of it than mine.
The hallucination connection makes sense .no data to abduct from, so it inducts on thin priors instead and calls it an answer.
Same failure mode, just wearing a different coat depending on whether the gap is small (confident wrong fact) or huge (confident wrong theory).
The seismic-trace framing lands for me because it names the real gap: Einstein-style work is inference from indirect signals to an unseen cause, and current models mostly interpolate inside what they have already seen. I keep hitting this when a model confidently fills a reasoning gap that actually needed a new abstraction, not a recombination of old ones. Do you think that ceiling is a data problem, or something structural about how these models represent causality?
Good question and it's more structural than data at this point.
The paper's whole argument is that scale-as-induction actually gets worse, not better, at closing this specific gap, since more data just sharpens interpolation inside the same space.
What's missing looks like a representational thing: no mechanism for treating a surprising result as evidence for an unseen cause, just for matching it to the nearest seen one.
More data makes the nearest-neighbor search better. It doesn't add the "infer the unseen structure" move.
Good point, retrieval isn't even induction, it's just recall dressed up as reasoning. The real question this raises: is "the jump" one mechanism, or a mix of different moves (Einstein importing an outside framework vs Dirac rejecting a correct-but-unsatisfying answer)? If it's a family of operations, "LLMs can't jump" might actually be several separate claims, not one.
Good split, and I think it's a family not one move .
Einstein importing a framework (relativity of simultaneity from reading philosophy) and Dirac rejecting a working answer because it didn't fit his priors are doing different work.
Common thread might just be: both required refusing to stop at "technically correct." Peirce's abduction is one bucket for all of it but buckets can hold more than one mechanism.