Meta tags:
description=An interactive, open-source book on modeling rational and biased agents for (PO)MDPs and reinforcement learning using probabilistic programs, with runnable WebPPL code examples.;

Headings (most frequently used words):

modeling, agents, with, probabilistic, programs, table, of, contents, citation, open, source, acknowledgments,

Text of the page (most frequently used words):
and (35), agents (16), for (12), webppl (9), #probabilistic (9), the (8), with (8), #modeling (7), programs (7), mdps (7), inference (7), planning (7), programming (5), book (5), pomdps (5), agent (5), biases (5), learning (5), this (4), evans (4), owain (4), andreas (4), from (4), create (4), models (4), reinforcement (4), their (3), future (3), library (3), language (3), agentmodels (3), salvatier (3), john (3), filan (3), daniel (3), stuhlmüller (3), bandits (3), multi (3), softmax (3), hyperbolic (3), discounting (3), examples (3), bounded (3), time (3), gridworld (3), decision (3), human (3), table (2), contents (2), work (2), institute (2), pomdp (2), https (2), org (2), your (2), own (2), games (2), using (2), noise (2), joint (2), preferences (2), procrastination (2), bandit (2), short (2), inconsistency (2), motivation (2), rationality (2), inverse (2), about (2), choice (2), environments (2), problems (2), one (2), functional (2), javascript (2), based (2), are (2), simulating (2), edit, github, interactive, editor, thank, noah, goodman, helpful, discussions, all, contributors, long, ouyang, was, supported, life, grant, 2015, 144846, oxford, humanity, viz, acknowledgments, mdp, web, markdown, code, chapters, content, open, source, misc, title, author, stuhlm, ller, year, 2017, howpublished, url, note, accessed, electronic, retrieved, bibtex, please, cite, citation, gridworlds, armed, use, quick, start, guide, schelling, coordination, tic, tac, toe, simple, natural, example, explaining, temptation, pre, commitment, assuming, optimality, leads, mistakes, heuristic, algorithms, that, assume, horizon, myopia, rewards, updates, formal, model, inconsistent, exponential, naive, sophisticated, soft, max, limited, memory, heuristics, intractability, cognitive, overview, inferring, utilities, beliefs, choices, reasoning, thomson, sampling, learn, mathematical, formalism, restaurant, hidden, state, noisy, actions, stochastic, transitions, policies, values, markov, processes, efficient, dynamic, sequential, shot, expected, utility, monty, hall, subset, automatic, bayesian, via, mcmc, gradient, variational, motivating, problem, rich, computational, introduction, more, information, project, contact, assumes, basic, experience, but, otherwise, self, contained, includes, introductions, myopic, implemented, differentiable, plan, recursively, selves, opponents, our, run, directly, browser, easy, modify, extend, describes, implements, rational, richer, which, capture,

Text of the page (random words):
modeling agents with probabilistic programs modeling agents with probabilistic programs by owain evans andreas stuhlmüller john salvatier and daniel filan modeling agents with probabilistic programs this book describes and implements models of rational agents for po mdps and reinforcement learning one motivation is to create richer models of human planning which capture human biases and bounded rationality agents are implemented as differentiable functional programs in a probabilistic programming language based on javascript agents plan by recursively simulating their future selves or by simulating their opponents in multi agent games our agents and environments run directly in the browser and are easy to modify and extend the book assumes basic programming experience but is otherwise self contained it includes short introductions to planning as inference mdps pomdps inverse reinforcement learning hyperbolic discounting myopic planning and multi agent planning for more information about this project contact owain evans or andreas stuhlmüller table of contents introduction motivating the problem of modeling human planning and inference using rich computational models probabilistic programming in webppl webppl is a functional subset of javascript with automatic bayesian inference via mcmc and gradient based variational inference agents as probabilistic programs one shot decision problems expected utility softmax choice and monty hall sequential decision problems mdps markov decision processes efficient planning with dynamic programming mdps and gridworld in webppl noisy actions softmax stochastic transitions policies q values environments with hidden state pomdps mathematical formalism for pomdps bandit and restaurant choice examples reinforcement learning to learn mdps rl for bandits thomson sampling for learning mdps reasoning about agents overview of inverse reinforcement learning inferring utilities and beliefs from choices in gridworld and bandits cognitive biases and bounded rationality soft max noise limited memory heuristics and biases motivation from intractability of pomdps time inconsistency i exponential vs hyperbolic discounting naive vs sophisticated planning time inconsistency ii formal model of time inconsistent agent gridworld and procrastination examples bounded agents myopia for rewards and updates heuristic pomdp algorithms that assume a short horizon joint inference of biases and preferences i assuming agent optimality leads to mistakes in inference procrastination and bandit examples joint inference of biases and preferences ii explaining temptation and pre commitment using softmax noise and hyperbolic discounting multi agent models schelling coordination games tic tac toe and a simple natural language example quick start guide to the webppl agents library create your own mdps and pomdps create gridworlds and k armed bandits use agents from the library and create your own citation please cite this book as owain evans andreas stuhlmüller john salvatier and daniel filan electronic modeling agents with probabilistic programs retrieved from https agentmodels org bibtex misc agentmodels title modeling agents with probabilistic programs author evans owain and stuhlm u ller andreas and salvatier john and filan daniel year 2017 howpublished url https agentmodels org note accessed open source book content markdown code for the book chapters webppl a probabilistic programming language for the web webppl agents a library for modeling mdp and pomdp agents in webppl acknowledgments we thank noah goodman for helpful discussions all webppl contributors for their work and long ouyang for webppl viz this work was supported by future of life institute grant 2015 144846 and by the future of humanity institute oxford table of contents interactive editor edit on github