Interesting post. I agree with your general framing: people constantly make reference to mental models that are developed in large part through physical experiences, not acquired from language.
However, I believe that many useful mental models of the physical CAN be constructed from language alone. Assertions that this is impossible typic…
Interesting post. I agree with your general framing: people constantly make reference to mental models that are developed in large part through physical experiences, not acquired from language.
However, I believe that many useful mental models of the physical CAN be constructed from language alone. Assertions that this is impossible typically seem to be based on thinking of the form, "Well, *I* don't see how it can be done, so it must be impossible..." That line of reasoning seems inadequate to me, and I hope you won't fall prey to that!
More compelling to me are experiments with toy models trained on (somewhat contrived) language from which one can extract and validate the learned mental model from the model parameters. Here are a few examples:
- Given a list statements of the form "San Francisco is west of Reno" that give the position of one US city relative to another, a simple model generates a reasonably accurate map, which can be extracted from the model parameters and shown graphically. This seems to be exactly the sort of mental model that people use to reason about the physical world.
- Extending this example, if a model is then given statements of the form "Fargo is in North Dakota", a simple model learns state boundaries. After the city positions and boundaries are learned, the model can quite accurately guess which state contains a city outside this second-stage training data. Again, this "mental model" map can be extracted and displayed graphically. A human might acquire such a model by walking, driving, or looking at a map, but language alone is actually sufficient.
- Given a list of simple arithmetic assertions, even a toy language model develops algorithms to perform basic arithmetic. In some cases, these algorithms are comprehensible enough to extract and explain. This works even though the model starts with no a priori notion of quantity or addition, which a human might acquire through physical experience, e.g. seeing a group of two sheep join a group of three sheep.
In summary, I believe that learning mental models of the physical world from language alone is MUCH HARDER than learning from language together with physical experience, but I've seen no evidence that this is impossible. To the contrary, available evidence proves this is possible in some nontrivial cases, and I expect many more examples will follow.
Thank you for the comment. I don't disagree that many useful world models can be created from language alone. (But I'd still be cautious reading too much into the current model capabilities .I have played around with the spatial/directions examples in pre-trained models. I don't think they are that robust. I think I have many test cases to show that they don't have a consistent spatial reasoning).
Also my comment was about practical impossibility, not theoretical impossibility. Lempel-Ziv algorithm is a good language model. Can asymptotically approach any other language model's performance. We can even theoretically prove the asymptotic optimality. But in practice the rate is just too slow. If we poke around in LZ, we might see evidence of world-models in it too. And since we have transformers which are more efficient learners, we won't be attempting to scale up LZ. So while I consider the implicit world models in transformers to be interesting, I don't consider them as evidence of learning all world models efficiently through language. I think we'll come up with new multi-modal architecture that are more efficient learners, and act as grounding constraints for language. If approach A is "get everything into language form" and approach B is "multi-modal grounded language model", I think B will eventually overtake A before A can completely solve the problem. And once B overtakes A, we won't follow approach A. (And of course, "plannability" of the world models is another aspect in itself, which also hopefully will get tackled in new architectures. )
Interesting post. I agree with your general framing: people constantly make reference to mental models that are developed in large part through physical experiences, not acquired from language.
However, I believe that many useful mental models of the physical CAN be constructed from language alone. Assertions that this is impossible typically seem to be based on thinking of the form, "Well, *I* don't see how it can be done, so it must be impossible..." That line of reasoning seems inadequate to me, and I hope you won't fall prey to that!
More compelling to me are experiments with toy models trained on (somewhat contrived) language from which one can extract and validate the learned mental model from the model parameters. Here are a few examples:
- Given a list statements of the form "San Francisco is west of Reno" that give the position of one US city relative to another, a simple model generates a reasonably accurate map, which can be extracted from the model parameters and shown graphically. This seems to be exactly the sort of mental model that people use to reason about the physical world.
- Extending this example, if a model is then given statements of the form "Fargo is in North Dakota", a simple model learns state boundaries. After the city positions and boundaries are learned, the model can quite accurately guess which state contains a city outside this second-stage training data. Again, this "mental model" map can be extracted and displayed graphically. A human might acquire such a model by walking, driving, or looking at a map, but language alone is actually sufficient.
- Given a list of simple arithmetic assertions, even a toy language model develops algorithms to perform basic arithmetic. In some cases, these algorithms are comprehensible enough to extract and explain. This works even though the model starts with no a priori notion of quantity or addition, which a human might acquire through physical experience, e.g. seeing a group of two sheep join a group of three sheep.
In summary, I believe that learning mental models of the physical world from language alone is MUCH HARDER than learning from language together with physical experience, but I've seen no evidence that this is impossible. To the contrary, available evidence proves this is possible in some nontrivial cases, and I expect many more examples will follow.
Thank you for the comment. I don't disagree that many useful world models can be created from language alone. (But I'd still be cautious reading too much into the current model capabilities .I have played around with the spatial/directions examples in pre-trained models. I don't think they are that robust. I think I have many test cases to show that they don't have a consistent spatial reasoning).
Also my comment was about practical impossibility, not theoretical impossibility. Lempel-Ziv algorithm is a good language model. Can asymptotically approach any other language model's performance. We can even theoretically prove the asymptotic optimality. But in practice the rate is just too slow. If we poke around in LZ, we might see evidence of world-models in it too. And since we have transformers which are more efficient learners, we won't be attempting to scale up LZ. So while I consider the implicit world models in transformers to be interesting, I don't consider them as evidence of learning all world models efficiently through language. I think we'll come up with new multi-modal architecture that are more efficient learners, and act as grounding constraints for language. If approach A is "get everything into language form" and approach B is "multi-modal grounded language model", I think B will eventually overtake A before A can completely solve the problem. And once B overtakes A, we won't follow approach A. (And of course, "plannability" of the world models is another aspect in itself, which also hopefully will get tackled in new architectures. )