Can AI agents adapt zero-shot, to complex multi-step language instructions in open-ended environments? We present MaestroMotif, a method for skill design that produces highly capable and steerable hierarchical agents. Paper: arxiv.org/abs/2412.08542 Code: github.com/mklissa/maestromotif - ThreadSky

martinklissarov.bsky.social • 136 days ago

Can AI agents adapt zero-shot, to complex multi-step language instructions in open-ended environments?

We present MaestroMotif, a method for skill design that produces highly capable and steerable hierarchical agents.

Paper: https://arxiv.org/abs/2412.08542
Code: https://github.com/mklissa/maestromotif

Comments

martinklissarov.bsky.social•136 days ago

MaestroMotif builds on our previous work, Motif, which pioneered learning RL policies from AI feedback. At the time, it set a new state-of-the-art on the open-ended domain of NetHack. With MaestroMotif, we improve on this performance by two orders of magnitude. But, how are these gains obtained?

martinklissarov.bsky.social•136 days ago

MaestroMotif is a scalable and effective algorithm for AI assisted skill design. It starts by leveraging an agent designer’s prior knowledge about a domain who defines a set of useful skills, or agents. Agents here are described on a high level in natural language.

martinklissarov.bsky.social•136 days ago

Once the skill policies are learned, MaestroMotif can adapt, zero-shot, to new instructions and solve complex tasks simply by re-combining skills, similarly to motifs in a composition. In other words, it writes a different code policy over skills which achieves a completely different task.

martinklissarov.bsky.social•136 days ago

We highlight the complexity of some of these tasks, which on average take more than a thousand steps for completion. Even methods that are trained specifically for each task are not able to make any kind of progress.

martinklissarov.bsky.social•136 days ago

Evaluations in such complex tasks is only possibly thanks to the work of dedicated fans of NetHack, who have been building and upgrading the game since 1987 (it is still an ongoing and maintained repository). We show in this figure some of the complexities of NetHack.

martinklissarov.bsky.social•136 days ago

A few years back, AI researchers (Heinrich Kuttler, @egrefen.bsky.social and @rockt.ai to name a few) foresaw the importance of such an environment and created the NetHack Learning Environment, which allows for experimenting with RL agents.

Comments

Posting Rules

Reply