Meta tags:
description=Dynalang leverages diverse types of language to solve tasks by using language to predict the future in a multimodal world model.;

Headings (most frequently used words):

in, language, learning, to, model, the, world, with, overview, how, it, works, hints, homegrid, game, manuals, messenger, instruction, following, habitat, grounded, generation, langroom, text, pretraining, citation,

Text of the page (most frequently used words):
the (58), #language (39), and (38), model (25), world (21), that (20), text (20), dynalang (20), from (16), with (15), agents (13), future (12), what (11), environment (10), can (8), observations (8), learning (7), agent (7), task (7), learns (7), tom (6), one (6), rewards (6), predict (6), for (5), named (5), saw (5), data (5), using (5), will (5), instructions (5), prediction (5), actions (5), code (4), asked (4), are (4), time (4), not (4), only (4), pretrained (4), representation (4), this (4), pretraining (4), visual (4), how (4), dynamics (4), homegrid (4), hints (4), multimodal (4), diverse (4), tweet (3), paper (3), more (3), you (3), sad (3), him (3), was (3), true (3), his (3), met (3), girl (3), but (3), they (3), sue (3), work (3), understanding (3), acting (3), like (3), rollouts (3), token (3), each (3), timestep (3), see (3), generation (3), single (3), architecture (3), modeling (3), act (3), without (3), performance (3), messenger (3), also (3), show (3), photorealistic (3), instruction (3), following (3), unified (3), game (3), learn (3), over (3), which (3), receive (3), types (3), via (3), use (3), humans (3), solve (3), tasks (3), such (3), video (3), image (3), understand (3), information (2), jessy (2), lin (2), yuqing (2), olivia (2), watkins (2), danijar (2), hafner (2), pieter (2), abbeel (2), dan (2), klein (2), anca (2), dragan (2), said (2), get (2), why (2), once (2), upon (2), samples (2), boy (2), had (2), belt (2), day (2), very (2), prompt (2), friends (2), top (2), tim (2), park (2), although (2), our (2), generate (2), tinystories (2), latent (2), space (2), models (2), offline (2), action (2), reward (2), provides (2), datasets (2), all (2), embeddings (2), general (2), improves (2), beyond (2), affect (2), langroom (2), grounded (2), perform (2), habitat (2), must (2), same (2), evaluate (2), reason (2), manuals (2), entities (2), outperforms (2), impala (2), r2d2 (2), well (2), specific (2), objective (2), kinds (2), often (2), corrections (2), where (2), addition (2), knowledge (2), might (2), describe (2), interactive (2), feedback (2), based (2), unlike (2), sequence (2), intuitively (2), enables (2), representations (2), trained (2), imagined (2), into (2), build (2), experience (2), while (2), range (2), state (2), check, out, summary, article, lin2023learning, title, author, year, 2023, eprint, 2308, 01399, archiveprefix, arxiv, citation, tree, look, don, worry, howeny, did, frustrated, rude, there, little, special, loved, wear, could, find, felt, mom, lily, played, stick, sam, kept, playing, ball, favorite, friend, put, she, young, found, dull, round, rock, picked, looked, thought, fun, took, focuses, sample, decode, observation, generations, surprisingly, coherent, still, below, quality, modern, unifying, exciting, avenue, because, decoupled, labels, capability, way, benefit, large, scale, within, pretrain, scratch, short, stories, downstream, just, predictions, observes, expects, hear, statements, about, has, seen, outputting, embodied, question, answering, able, handle, follow, natural, navigate, goal, location, scan, home, framework, viewing, test, longer, complex, requires, multi, hop, reasoning, describing, episode, combine, them, determine, messages, avoid, emma, baseline, uses, specialized, particularly, most, difficult, stage, even, though, explicit, supervision, piece, corresponds, ground, conditioned, struggle, different, worse, released, encourage, further, direction, introduce, simulate, read, providing, helpful, required, pedal, open, compost, bin, provide, doing, turn, around, observe, plates, kitchen, previous, consume, sentence, paragraph, design, consuming, frame, similar, inputs, real, stream, listening, takes, everything, right, chooses, training, policy, network, compressed, take, maximize, predicted, left, compresses, reconstruct, original, next, given, reads, should, expect, naturally, fits, paradigm, dreamerv3, continuously, collects, works, formulation, additional, capabilities, demonstrate, broad, outperforming, art, algorithms, architectures, propose, grounds, contributions, interact, need, people, relate, current, execute, simple, aim, leverage, conveys, describes, key, idea, observed, behave, situations, rewarded, perspective, unifies, powerful, self, supervised, present, predicts, traditional, acquires, rich, past, online, interaction, both, grid, worlds, navigating, scans, homes, utilizes, improve, including, descriptions, rules, helps, overview, leverages, berkeley,

Text of the page (random words):
learning to model the world with language learning to model the world with language jessy lin yuqing du olivia watkins danijar hafner pieter abbeel dan klein anca dragan uc berkeley paper code tweet tl dr dynalang leverages diverse types of language to solve tasks by using language to predict the future via a multimodal world model overview to interact with humans and act in the world agents need to understand the range of language that people use and relate it to the visual world while current agents learn to execute simple language instructions from task rewards we aim to build agents that leverage diverse language that conveys general knowledge describes the state of the world provides interactive feedback and more our key idea is that language helps agents predict the future what will be observed how the world will behave and which situations will be rewarded this perspective unifies language understanding with future prediction as a powerful self supervised learning objective we present dynalang an agent that learns a multimodal world model that predicts future text and image representations and learns to act from imagined model rollouts unlike traditional agents that use language only to predict actions dynalang acquires rich language understanding by using past language also to predict future language video and rewards in addition to learning from online interaction in an environment dynalang can be pretrained on datasets of text video or both without actions or rewards from using language hints in grid worlds to navigating photorealistic scans of homes dynalang utilizes diverse types of language to improve task performance including environment descriptions game rules and instructions contributions we propose dynalang an agent that grounds language to visual experience via future prediction we demonstrate that dynalang learns to understand diverse kinds of language to solve a broad range of tasks often outperforming state of the art rl algorithms and task specific architectures we show that the dynalang formulation enables additional capabilities language generation can be unified in the same model as well as text only pretraining without actions or task rewards how it works using language to understand the world naturally fits into a world modeling paradigm we build on dreamerv3 a model based rl agent dynalang learns continuously from experience data the agent collects while acting in an environment left the world model compresses the text and image at each timestep into a latent representation from the representation the model is trained to reconstruct the original observations predict rewards and predict the representation at the next timestep intuitively the world model learns what it should expect to see in the world given what it reads in text right dynalang chooses actions by training a policy network on top of the compressed world model representations it is trained on imagined rollouts from the world model and learns to take actions that maximize predicted rewards unlike previous multimodal models that consume text one sentence or paragraph at a time we design dynalang to model video and text as a unified sequence consuming one image frame and one text token at a time intuitively this is similar to how humans receive inputs in the real world as a single multimodal stream where listening to language takes time modeling everything as one sequence enables pretraining the model on text data like a language model and improves rl performance language hints in homegrid we introduce homegrid to evaluate agents in an environment where they receive language hints in addition to task instructions hints in homegrid simulate knowledge that agents might learn from humans or read in text providing information that is helpful but not required to solve tasks future observations describe what agents might observe in the future such as the plates are in the kitchen corrections provide interactive feedback based on what the agent is doing such as turn around dynamics describe the dynamics of the environment such as pedal to open the compost bin the homegrid environment will be released with the code to encourage further work in this direction future observations corrections dynamics even though agents do not receive explicit supervision for what observations a piece of text corresponds to dynalang learns to ground language of all types to the environment via the future prediction objective dynalang outperforms language conditioned impala and r2d2 that struggle to use different kinds of language and often do worse with language beyond instructions game manuals in messenger we evaluate on the messenger game environment to test how agents learn from longer and more complex text that requires multi hop reasoning over text and visual observations agents must reason over text manuals describing the dynamics of each episode and combine them with observations of the entities in the environment to determine which entities to get messages from and which to avoid dynalang outperforms impala and r2d2 as well as the task specific emma baseline that uses a specialized architecture to reason over text and observations particularly on the most difficult stage 3 instruction following in habitat we also show that dynalang is able to handle photorealistic visual observations and perform instruction following in habitat agents must follow natural language instructions to navigate to a goal location in a photorealistic scan of a home in dynalang instruction following can be unified in the same prediction framework by viewing it as future reward prediction grounded language generation in langroom just like language can affect agents predictions of what they will see what an agent observes can also affect what language it expects to hear e g true statements about what it has seen by outputting language in the action space in langroom we show that dynalang can generate language grounded in environment to perform embodied question answering text pretraining because world modeling with language is decoupled from learning to act with a world model dynalang can be pretrained with offline data without action or reward labels this capability provides a way for dynalang to benefit from large scale offline datasets all within a single model architecture we pretrain dynalang with text only data learning token embeddings from scratch pretraining the model on general text data tinystories 2m short stories improves downstream rl task performance on messenger beyond using pretrained t5 embeddings although our work focuses on language understanding for acting in world we can generate text from the world model like a text only language model we sample rollouts from the pretrained tinystories model in latent space and decode the token observation from the representation at each timestep model generations are surprisingly coherent although still below the quality of modern language models we see unifying language generation and acting in a single agent architecture as an exciting avenue for future work prompt one day a young boy named tim found a dull round rock he picked it up and looked at it he thought it was not very fun but he took it with him to the park at the park tim true saw a girl named sue she had model samples he met favorite friend he put it his to met a girl named sue sue saw the ball saw a stick top sam he kept playing with played with his friends and but they friends li met a girl named lily s ly saw prompt once upon a time there was a little boy named tom tom had a special belt that he loved to wear one day he could not find his belt and felt very sad tom s mom saw him and true asked why are you sad tom model samples frustrated and asked him what was rude once upon asked why are you sad tom s asked howeny i did get said don t worry tom we said to tree you look be in citation article lin2023learning title learning to model the world with language author jessy lin and yuqing du and olivia watkins and danijar hafner and pieter abbeel and dan klein and anca dragan year 2023 eprint 2308 01399 archiveprefix arxiv for more information check out the paper code and tweet summary paper code tweet