Algorithmic Models

Teaching material and code examples for reinforcement learning and algorithmic models used to solve classic problems and as computational description of recorded behavior (all codes are in Matlab).

Disclaimer: I usually run through the code several times before publishing it and especially the parts that have been used in a course/workshop have been commented, but do let me know if you find any bug or have any particular question. Please consider I am not a programmer, but a cognitive neuroscientist, so these examples do not represent the best possible practice for coding.

By the way, if you think there is a better way to code the same functions or if you simply want to share your code, please feel free to comment!

Lotka–Volterra equations: basic dynamics and input-controlled parameter modulation

Lotka-Volterra equations have been first developed to simulate ecological nonlinear interactions among different species. Assuming two species affect each another in a prey-predator relationship, basic Lotka-Volterra equations describe the fluctuation in number of the population of both species as follows: \[ \frac{dx}{dt} = \alpha x – \beta xy \] \[ \frac{dy}{dt} = – \gamma y …

ON and OFF policy solutions for the “cliff task”

As an example of the different solutions to a problem emerging by adopting ON- or OFF-policy TD algorithms, I use here a task described in the new edition of the Sutton and Barto 2017 (you can download the entire book for free here : http://ufal.mff.cuni.cz/~straka/courses/npfl114/2016/sutton-bookdraft2016sep.pdf, see chapter 6). In this task the agent is required …

ON and OFF policy solutions for the “windy grid-world task”

As an example of the different strategies emerging by adopting ON- or OFF-policy TD algorithms, I use here a task described in the new edition of the Sutton and Barto 2017 (you can download the entire book for free here : http://incompleteideas.net/book/bookdraft2017nov5.pdf, see chapter 6). In this task the agent is required to navigate the environment …

Model-based and Model-free RL solving a sequential two-choice Markov decision task

In this example I replicated task and model described in Glasher et al. 2010 (available here: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2895323/ ). The task is essentially a two armed bandit with probabilistic outcomes (distribution of probabilities: 0.7-0.3), played on two levels, so that the agent has to perform 2 choices in sequence (left or right), to reach a reward, virtually …

Reinforcement Learning vs Bayesian approach

As part of the Computational Psychiatry summer (pre) course, I have discussed the differences in the approaches characterising Reinforcement learning (RL) and Bayesian models (see slides 22 onward, here: Fiore_Introduction_Copm_Psyc_July2019 ). In particular, I have presented a case in which values can be misleading, as the correct (optimal) choice selection leads to either +100 points (75%) or …

Algorithmic Models

Like this:

Lotka–Volterra equations: basic dynamics and input-controlled parameter modulation

Like this:

ON and OFF policy solutions for the “cliff task”

Like this:

ON and OFF policy solutions for the “windy grid-world task”

Like this:

Model-based and Model-free RL solving a sequential two-choice Markov decision task

Like this:

Reinforcement Learning vs Bayesian approach

Like this:

In this section

Meta

Algorithmic Models

Share this:

Like this:

Lotka–Volterra equations: basic dynamics and input-controlled parameter modulation

Share this:

Like this:

ON and OFF policy solutions for the “cliff task”

Share this:

Like this:

ON and OFF policy solutions for the “windy grid-world task”

Share this:

Like this:

Model-based and Model-free RL solving a sequential two-choice Markov decision task

Share this:

Like this:

Reinforcement Learning vs Bayesian approach

Share this:

Like this:

In this section

Meta