"Our world model can be trained quickly in an unsupervised manner to learn a compressed spatial and temporal representation of the environment. By using features extracted from the world model as inputs to an agent, we can train a very compact and simple policy that can solve the required task."
They key difference between the "World Models" paper and "real life business" is they used a game engine to simulate car racing and playing Doom, but in "real life" we don't have (yet) "fixed and accurate business simulation engines"... contd
Game Environments:
- Clear boundaries, rules, and success metrics
- Finite state/action spaces
- Well-defined physics/transitions
Business Environments:
- Semi-structured or unstructured states
- Multiple, sometimes conflicting success metrics
- Hidden variables and incomplete information
If I understand, one of the key insights is that instead of running simulations in the actual environment (like VizDoom), they can run much faster simulations purely within the learned model M. The agent can then learn and improve its policy by interacting with these hallucinated environments.
listened to this 5 times, but, I think I get it. I'm working on a project to allow people to import their Excel files and then create views that link them together (rather than say a limited VLookup). I now see this creates a 'world view' that would normally require a SQL query or a Python notebook.
100% — any task which requires stitching together data from two or more sources assumes there is a “real world” that each dataset partially captures. So to successfully complete the task, the user has to have a mental model of the world to know if they’ve done the task right or not!
I don’t! Technically a single table that captures real-world information (even if it has noise and errors) has a latent world model.
I just like relational data because that latent model has to be manifest in order to actually query it! That manifest world model is (mostly) the JOIN statement
Only if you have to observe one instance multiple times, or if the phenomena change over time! But you are correct — almost all useful world models will require that
Comments
You've introduced a new way to articulate key ideas for my presentation, experimented with this doodle today.
I need to read the paper article and papers referenced first, but this was my immediate takeaway. Keep sharing more please :)
https://arxiv.org/abs/1803.10122
- Clear boundaries, rules, and success metrics
- Finite state/action spaces
- Well-defined physics/transitions
Business Environments:
- Semi-structured or unstructured states
- Multiple, sometimes conflicting success metrics
- Hidden variables and incomplete information
I just like relational data because that latent model has to be manifest in order to actually query it! That manifest world model is (mostly) the JOIN statement
(https://www.linkedin.com/posts/yann-lecun_lots-of-confusion-about-what-a-world-model-activity-7165738293223931904-vdgR)