Fundamentals of Active Inference (Chapter 7, Session 34) August 7, 2026

All right. Hello everyone. So it's August 7th, 2026. We're here in the fourth and final session for chapter 7 of the fundamentals of active inference textbook uh group. So this is um the first chapter that we've seen action uh for the first time. So we're putting the active and active inference. Uh we've already gone over typically we go over the first half of a chapter in one session and then the second half in the next session. And of course, there's two sessions, myself and Daniel's that uh go in parallel. We've actually already gone through the content in chapter 7. It's it's kind of short as as far as the chapters go in the textbook. And really in a lot of ways, I've already said this, it's kind of an addendum to chapter six. Chapter six is where we introduced a huge amount of new uh mathematical material and so on where we really got to to work specifying the generative model and the generative process and the mathematical machinery by which we'll do that. [clears throat] Uh namely generalized filtering for the perception problem. And then all chapter 7 really does it just says well let's let's add in action to this this whole whole story. Um, so there's not a lot more to do uh if you understand chapter six and and the machinery in there. For chapter 7, I thought it would be worthwhile. There are some really wonderful examples in in chapter 7. There's three canonical ones. Um, we go through, we do mostly univariat uh inference, active inference where uh there's only one hidden state, that sort of thing. And of course, we do the multivariate uh situation later, right at the very end actually. Um, and it's it's really only a paragraph. So, 7.5 multivaried active inference in generalized coordinates. Uh, in I think because we're going to be going into chapter eight next week, which is almost like chapter 3 redux. So, if I share my screen here, I'll just share the entire thing. There we go. We come back over. We we we've kind of already gone through this procedure once where uh if I go here in chap in the introduction. Chapter [clears throat] two was very much about you know hidden state estimation perception only. Uh and then we we made assumptions in chapter two that well we're going to assume that all of the parameters in our model are well learned already and therefore we can just get on with the business of perception. And we did that. But in chapter three, we said, "Okay, excellent. But how how really are we going to deal with this issue of parameter learning?" And that's what we saw in chapter 3. We're going to do almost exactly the same kind of thing again going from chapter seven into eight where we're going to repeat what we did in seven, but we're now going to have to deal again with this issue of learning, okay, and attention. And we're going to see some more discussion of hierarchical models as well. And that's going to be six, seven, and eight. That's going to be the core of continuous state space active inference. Okay. And then we're going to move into nine and 10 which is uh the discrete side of things. And we haven't seen anything like that yet in the book. Um and that's going to mark quite a big change actually. Part two really is kind of something like part 2 a and part 2 b. Um part 2 a is where we are now very much about initially what does it mean to put action into our models. And then part 2B is going to deal with this issue of planning. Okay. And this is going to be quite a big change um because at the moment we're just selecting actions on the basis of the gradient of the variational free energy. But we'll see that there is a different way to do things in in 2A in 2B. So we're in two ages now. Chapter 7. We'll finish that off. Um there are if I go back to the questions over here there are a few questions I've began in chapter 7. I have begun going through and answering some of them. I know I've been a bit laxed with that so far. So this one here, this is a good question about you know active int often applied to navigation tasks, maze learning, physical tasks in other words, but if you want to capture, you know, a model around sort of mental states and and veilance and interiorception are there kind of modeling criteria to to what sort of modeling pattern would you like to look at? So I began answering that there. I'm not quite done just yet, but do do keep answering or asking questions and I'll I'll keep answering them. Um, right. So, effectively what I think would be useful to do and obviously if anyone has questions, please raise your hand or or whatever and we'll we'll address them. I think it would be nice. I've prepared a a notebook a code notebook to go through example 7.1. So, if I go down just quickly and then I'll show you the notebook. So this is uh you know what we're doing is we're we're instantiating univariat active inference. Okay. So the simplest possible case of actually putting action into the picture. I repeated myself. So and again so we're all on the same page. This is the picture now. Right? This is figure 7.3. I showed you this last week. The idea is that we have our generative process. we have the generative model and we have a boundary between them whereby the model can observe um things from the process and we can also issue now actions. Okay, that's the big change that we have in this chapter. Uh and that's going to stay with us forever. Uh that's going to be very much this this figures like this figures that that look like it are going to be with us for the remainder of active inference. Okay. And we, you know, a lot of the machinery here that we we've talked about in chapter six. So again, chapter six is a very lynchpin chapter. These these precision weight prediction errors, this this kind of stuff um that's going to be sort of assumed material going forward. Now, okay. Now there is I I don't I mean I'll I'll show you the equations later and I have them in the notebook. The first example 7.1 is this idea this this example of a Bayian thermostat. And this is typically how people have their first introduction to active inference per se. You know, really when we really do actually have the ability to act on our environment. And it's a very nice example because it's it's quite simple. Um again, we we did go through this last week. Um this this figure and everything we talked about where it comes from. Um there the idea very basically is that and again it's univariate. So we're just on the on the real line. we can sense some temperature distribution and we have a preference for observing a particular temperature. Okay. Um and but we not only are we able to observe, we're also able to affect actions in our environment. So the general idea is bit like this. So there is actually a creature called a hydra which is a very it's a tiny little creature that lives in the water column. Basically they they look like these little uh octopus type things. They're microscopic. Um, and they effectively do this. They're they're a little bit like um a a thermostat in the sense that they have a preference to observe certain temperature, maybe certain nutrient gradient. Um, and it can move up and down, right? That's all it can really do. But if we go back here, we're now going to be thinking about how it is that we can create an active inference idealization of this process. So I I have, as I say, I've prepared a a notebook that does this. Hopefully the inversion is color invert. I like to use color inversion, but people don't necessarily like that all the time. I can get rid of that. Uh I can go back to lights. There we go. So, just a couple things to note. Um I've implemented this very kind of step by step. I know there are some people who have actually implemented their own versions of the Beijian thermostat already, which is excellent. I think Vera, you um you gave us a repository where you had something that was at least flavored Beijian thermostat flavored. I've taken a look and I I like that, but I thought for the purposes of just getting everyone on the same page, this would be the single best example to look at in the entire book. Um I think in order to be able to begin to understand active inference as a totality, you know, not just not just the perception case, but we've got perception and action. So it's kind of the foundation for a lot of things going forward basically. Um right so and it so it's quite long you know and we're only dealing with one example here the there are actually a few problems I've noted with the uh the book as it is for instance just really quickly I'll go back to the uh equations and this is not something we need to really meditate on for too long so 7.15 and 7.16 are the specification of the model so here we go we've got our environment [clears throat] gener of process here uh for the Beijing thermostat. So we're assuming that you know the hidden states they transition according to our ordinary differential equation in in in this case and of course that's now a function of actions. The agent can issue some sort of exogenous thing into the environment that causes state transitions. So this is the the thing that uh describes that that process. And of course we have some noise on state transitions. We have some noise on observations. Again, we we've got the usual observation function that we've seen all the way through. Um, this is presented as an ordinary differential equation in continuous time. However, the actual implementation is discreet. So I just want to note that they're not identical espec the the example as it's presented in in the textbook uh uses a a simplification a discretization of this ordinary this continuous time ordinary differential equation. Technically that's different from what's here mathematically specified but that's getting into some weeds that we really don't need to get into in order to understand the conceptual core of what's going on here. I just it is worthwhile highlighting that. However, um, one other thing as well is, so this is the environment. If I look at the model, this one here. [snorts] So now we have our generative model. So what do we need? We need our state prior and we're making the LLAS approximation. So we're assuming that the belief about hidden states is extremely tightly coupled about the mean. Therefore, we only really need to care about the mean. So we can throw away the variance and we're left with just mux, our belief about the hidden state. Uh, well, okay, the the state transition. This is uh some normal distribution centered about that mean um well given by the state transition function inside our our generative model. Now and we have the same thing for our generative for our our observation likelihood not the same thing we use we use the observation mapping function for that um and we have this state transition function here. Okay, in our observation function this is exactly the same observation function as in the generative process. In general that need not be the case. And we have what's curiously known here as our forward model. So Daniel spent some time earlier this week in his session talking about the forward model. Uh kind of where this comes from. And indeed it's a kind of result of the fact that what we need to do in order to to minimize variational free energy uh is of course we can do this by perception. But now we're saying we can do it by action. But the problem is that the the the variational free energy functional itself is not a function of action or not a functional of action. Uh so here oh and in the equations in both sessions I don't want that. So I I know we've done this before but I really want to hammer it home because it is important. Nevertheless, we know that our observations, the things that come from the generative process, our observations were generated by state transitions in the generative process, which were themselves generated by actions. Okay, so in some sense, our obser our observations are kind of like functions of actions basically. We can sort of model them that way. And that's indeed the trick that we make in order to be able to say what flow how should my actions change, right? Well, how how do changes in my actions affect changes in the variational free energy? That's exactly what we we claim. So we say really my observations are functions of actions. So therefore I can do the chain rule and we can actually calculate what the gradient of the variational free energy is with respect to actions. Okay. So that's that's enough review over here in my well first of all I would like to take this opportunity to just ask for any questions and then I I think we'll launch into my notebook on example 7.1. I see that Andrew you've got some you've added a slide here on example 7.4 I believe. Um excellent as well. So we've got 7.3 and uh well not you know questions about this this autonomous and exogenous force. Um these these are great slides from Andrew. Very very good. So any questions so far before I launch into my notebook Andrew? [clears throat] >> Yeah thanks. Uh yeah and thanks for sharing those. Uh yeah, I mean the the the again it it's another sort of redux of chapter 7, but to be able to say that we understand the distinction between the exogenous forces as opposed to action as opposed to all the other variables we've been frequently seeing like hidden states and the rest um >> all floating around. Yeah. >> Yeah. Right. And so it it it's very important to be able to distinguish we have exogenous forces and then there's action. And so that's something that I'm just trying to kind of clarify a bit. I mean, Sanjie does a good job of distinguishing them, but because we're introduced to to both at the same time. It's they start to look very part and parcel of each other. Uh whereas like to some degree they're they're distinct. and >> the way we treat action uh just as sort of an innote because I'm really looking forward to seeing your notebook uh on the thermostat Frasier. Um so I I don't want to be too long-winded but the big thing here is that whenever we've added exogenous force and action together into um our uh into basically our overall simulation, right? because there's an implementation of those things in both the generative process as well as the generative model. Um the something crucial is that for the generative model I don't want to say that we're computing action almost like some kind of extra tacked on auxiliary sort of thing but but recognize that whenever we extend the generative model the agent to include exogenous forces we're looking at uh a joint space of x y and v. So our states observations and the exogenous force um we've not included a right. So even though it's the agent side that's sort of computing A and then sending it into the environment it's actually only the environment that ne actually has like the A within its generative space and all the rest that it starts to impact the environment. And that's why whenever we see that figure um it's rather interesting like in order to to compute action we have to use sensory mappings right because um we know that in the sense of action and changing the world um we know that that an agent you know commits an action in order to change the world which then it does not know the hidden states of which it does not know and so really what's happening is you act and then the real change that you get to observe are your observations Right. So, it's a direct sort of mapping of A to Y or Y to A. Um, and so that's sort of the roundabout way that we we connect action and and observation in order to aid um inference, right? Um, and so that's what that for forward model that directly relates partial derivatives of action and and sensory observations uh comes from. But then you start to notice that many of the equations start to look like many things that we're already doing for hidden state inference. Right? We could find this sort of gradient flow of action. We can have generalized action, you know, in the sense of like trying to act to throw a ball with a particular trajectory or something along those lines. Like that's a whole kind of sequence of micro actions as it were. um you know that could be subdivided into temporal derivatives and um yeah so so there's just more information that relates action and exogenous force and and all the rest in some of those slides um but yeah uh please take it away Frasier I think I spoke longer than I intended >> it is it is an important point this this kind of initial preferatory discussion we had about exogenous forces and autonomous states and so on but that's I would say you don't have to get too caught up on that with respect to the general intuition about what action does in the environment. You know, um we're going to be seeing a lot of well, we're we're now actions going to be with us forever. So, we don't uh there is quite a lot of discussion that can be had in that regard and it's not central to active inference. That's just all I wanted to say there. So, I think that's that was your your point as well. So, okay. All right. I will I think I'll leap into my implementation of um the Beijing thermostat. Uh, but I see there's some things in the chat, so I'll have to get to that uh just a bit afterwards. It's not too long, so it shouldn't take too [clears throat] long. Uh, but I shall share my screen here. This is not currently available on the uh either the the um the repo or the institute page, but I will make it so in a Google Collab notebook. This is currently I'm using Jupyter notebook here. Uh don't worry about those details if you don't know about them. So it'll be something you can just click on and you'll be able to plug and play. So so this is example 7.1 the basian thermostat. I think it's the most important example to study to get the intuitions around active inference. Um very basically yes perception changes beliefs. That's everything from chapter 1 to five. Action changes the world. So this is now a different thing that we're doing. Um this is all in just base numpy and mapplot lib as well. A couple of things. So this is not a complete and total reproduction of the code that exists for the like figure 7.6 um figure 7.5 that code exists. It's not currently publicly available. Sanjie is still kind of working with it trying to make it um uh better and and and make sure that it's all correct. So this is my version of the the the Beijian thermostat. It's not designed to be a complete reproduction. So for instance um there there's actually I think a few inconsistencies in the in in the if we go to what is it 7.16 specification of the model and also in the the process um you know for instance this specification of the state dynamics is not uh the the the if there was no action there would still be some state dynamics and he says that there um there wouldn't be that's not technically correct. Um so there's a few issues there. The most important issue I think is around this this preference prior. So in the model uh as it's presented we have uh you know in the model is the hidden state plus this uh desired state. Okay. So V is going to be this kind of preference state or point attractor. Uh this actually does not produce a point attractor at V. Um so I'm using a prediction error which is corrected I believe where we have the prediction error minus the preferred state and that's going to be our prediction error on um on on hidden states. So that you can you can get a prior on hidden states a preference prior by means of you know something like this here now um and that will serve the function that we want it to serve. So we want V to be an attractor something that was is pulling our state dynamics in some particular direction and this will do that. So as I say there are there is actually code for this but uh it's not available just yet. I'll make this available after our our session here. So we're only using numpy map plot lib really basically. So again we're this hide our creature thing in a water column and what do we have? We have our our observation function. Okay this is just as it's given in 7.16. Okay. Well, yeah. 16. Um, [clears throat] so there's a parameter in this observation function theta, right? Um, this is actually the initial or the temperature at the origin of wherever we happen to be. It's 20 here. Okay. It can be anything. I kept it as 20. Um, yeah. So, we begin at the initial temperature being 20. The preferred temperature is five. Okay. So, maybe this is degrees Celsius, maybe this is Kelvin. Who knows? doesn't really matter too much uh for our purposes. So I have you know two functions I've tried to name everything as verbosely as possible. So we have the observation function literally what does that do? It takes in a hidden state x and it gives me the temperature which is this thing here. Uh and then I also have the derivative of that. Why do I need the derivative of that? Well if we go back to uh well it's in literally equation 7. No no not quite. We need it for our prediction errors. Okay, we saw that in in chapter six. So there's there's things in here that I haven't quite explained necessarily where they come from. So the observation at you know X being 20 is some very low temperature. The observation at X being five is some higher temperature. Right? So let's visualize this. This is a little weird because the water column is kind of on its side here. Um what you're seeing is the position or depth on the X axis. So this would be at the surface right zero and then as we go to the right we're going deeper and deeper into the water column and then what you see on the y- axis this red thing this is the observation function okay so as you can see crucially it's nonlinear okay it's this curve thing this exponential curvy you know thing so as let's say I I'm at 20 I start out at 20 what do I observe here I observe something very very close to zero. I'm quite far down in the water column. It's quite cold basically. And I, as my little hide, I'd like to change that observation. I'd like to observe a temperature close to um maybe five, right? So over here, what would that be? Well, I'd like to reach a position five and that has a temperature that's one. You know, it's a bit hard to see actually on this on this figure in terms of the scale. So, I would eventually like to have sliders in the in the way that you do um Andrew in in your your notebook for chapter two. Don't quite have that just yet, but I try to have little vignettes here where to get you to think about what would happen under certain counterfactuals. Okay. So, you know, to experience the preferred temperature should the action move upward or downward numerically. Okay. So I'm calling this um a repaired notebook in the sense that there are this is this is not quite the the implementation given in the book. So again the published generative process is is this here we've seen these equations. Um the technically as I say the state dynamics would not be stationary if there were no action. Um that's that's not quite right. Doesn't really matter though for our purposes. Now the actual Beijian thermostat itself, the generative process again in 6.16 we have noise. Okay, I've reproduced this here. We got noise on our state transitions. That's this omega x. And we also have noise in our observation function. Not too crazy. Um and it is discreet in the sense that we're taking little steps. We're not able to sort of slide around. Okay, technically that is an approximation. I also have this continuous inspired version which is is not quite there yet. Uh okay. So the observation function same as it was before. We have our preference around five. The prediction errors sensory prediction error same as it has been in chapter six. Uh and this is the corrected one for this chapter. And then our our variational free energy well we can just have this be precision weighted prediction error as we have had in chapter six. And as we will continue to to to go through well began we began in chapter five with precision weighted prediction error. Technically there are a few constant terms floating around here. Um but they're not actually important with respect to the gradient the change in the variational free energy with respect to actions or hidden states. For those you only really need to care about your uh prediction errors themselves. Okay. So there are actually a few uh constants. We saw that in in chapter six but they're not relevant here. So if you're wondering why this isn't exactly the same equation as I forget which equation exactly but it is shown in chapter six that's why we don't actually need those constants so not finding it immediately okay now we I also have the gradient of the observation function that's necessary for our prediction error or our observation prediction error uh and I I have this here as well now I have the the forward model okay in terms of the approximation that's made in the book that's we only really care about the sign of the Ford model. So I've just kind of given that SA that really is just exactly the same thing as where is it 7.112 it's got that whole series of equations 7.11 a through b as well. Okay, so I have a way of stepping the generative process. I've got an environment step function takes in the current state, takes in the action from the agent and we have a time discretization as well and that gives me the next state of the environment. Okay. So this is literally F. Okay. In um 7.16. Yeah. Uh5 sorry. And I have my observation function. So that takes in an hidden state X and it gives me an observation. So this is a temperature, right? I can run those. I've also got functions to give me my prediction errors. So I take in an observation. I take in a belief about a hidden state. I might take in my autonomous state or preference as well. And I have the this will give me my respective prediction errors in that regard. I have a my precision weighted prediction error for fixed precisions as well. So the precisions are not changing. Uh and I have my my gradient of my my variational free energy with respect to my hidden state. Um and the action gradient as well. So those are those last two we come here. Those are 7.10 10 a and b right here. So really this is the only new thing in the chapter for chapter 7. Okay. Uh and I have a function to step my agent in terms of you know the next point in time. So what does an agent need? Well, it needs a current belief. It needs an action observation preference uh and parameters and such like. And we have our precisions. Precision on hidden states, precision on observations, learning rates. There's all kinds of stuff floating around there. Um, I don't want to go through step by step because going through code step by step is an incredibly painful and boring process. Um, but this will be available for you to do imminently. It's not quite there just yet. And then I have a function that is simulate which just says take the whole thing. So if I go back here, take this entire specification and run this loop. Right? So this is the sensor motor loop and you might imagine one iteration of that. So begin with a hidden state, make an action, state transition in the environment. State transition gives rise to an observation. Observation comes in, we update our belief and we go around again. That is one iteration of the sensor motor loop. Now typically it's going to take a few iterations for the agent to actually accomplish what it wants in the environment beyond which time any change in what's happening might might be be negligible. So we're typically going to have to do this for a few iterations here. And that's what this simulate function does. So I'm going to simulate everything for 30 seconds. I can do as many seconds as I want. I can choose my initial hidden states, choose my initial belief and so on and so forth. Right? So that's basically what that function does. We just have a big loop. So we have a history. Then what does that give us? It gives us a history of kind of what happened. So I end up with this this history. And crucially for us, we can inspect it. So and I I also have a just a quick check about whether the analytic derivative uh term is is agreeing with the finite difference method. You don't need to worry about that. So so okay one of the big things in in the univer case figure 7.6 is that we illustrate the difference between perception and action in terms of the effect that they have on the variational free energy. Uh, and this comes up again in figure later on, figure 7.11. So for the first five seconds, right, I'm not allowed to act. I'm only allowed to perceive. So the qu there's interesting questions around that. You know, can the hidden state change? Can the latent state change? You know, these are these are interesting questions. for the isolated perceptual demonstration you just say okay well the precision on on hidden states that can be can be zero so the pre the preference does not compete with sensory evidence so we can do that let's just run perception only so I can say the time at which I want to start my action is some very large number right so that's like way way out there basically that means that I don't act for the entire simulation so I'm going to run 10 seconds of a simulation where I don't actually act let's run that and boom we're done okay that wasn't too Not let's actually see what happened here. So what are we looking at? We're looking at again perception only. This is the the change in belief across time, right? So the initial temperature we said was well so the the initial hidden state where the agent is in the water column that's 20. Okay? So it's quite it's quite far down in the water column. But initially it believes it's at position 10. Okay? So there's quite a big difference between its belief believed position and where it really is. And you can see across time it is [clears throat] able to perceive, oh look, I I'm going to have to change my belief so as to become closer and closer to the true position. So that's that's quite nice. It's able to update its belief to get, you know, within relatively close uh distance to the real hidden state, but that's all it can do. It can't really do anything else. Likewise, if we looked at the the uh predicted sensations, so the predicted temperatures at those positions, uh what do we have? Well, we're getting a perception of something very very cold. We're close to zero because we're very far down in the water column. Initially, the predicted observation is quite high because we believe we're quite high up in the water column. Okay? And this is wrong. So you can see likewise across time we're able to update our predictive belief about what the observations should be. There's some more vignettes here about you know when certain things change what would you expect certain other things to to to be the interesting case is when we're able to put action into this picture. It would be nice. I think I should uh really have plotted the variational free energy in this situation but you'll see it later. So all right let's actually turn on action at time step five. So we're still going to only perceive for the first five time steps and then after time step five we'll be able to act as well. So the books I just have two different sets of parameters here. The book has certain set of parameters. I have some different ones you can play around with. Uh so you can have a look there. Let's run for for 30 time steps. Uh and we'll have the initial hidden state or sorry the initial yeah initial hidden state be 20 as it is in the books. All the parameters match uh except for the precisions in some cases. Yeah. Let's have a see what happens here. So I've actually run it. There we go. This is my reproduction of figure 6.7. So it's not the same thing uh literally and it's also in three different panels. It's not one panel. So this is kind of the most constructive figure. What are we looking at again? For the first 5 seconds, we're not able to act. So we have in blue the real hidden state of the environment. So this is the real uh position that the agent is in. We have in green the actions of the agent. So in in the very beginning there's no actions at all. The blue kind of dotted line horizontal. Uh this is the preferred state and in orange we have the belief. Okay. So initially you can see that the belief starts out very wrong at 10. Um, and we're able to correct that as uh time goes on, but without action, we're not fully able to correct actually. And then at time equals 5, we turn on action and we're able to do some very small wiggly little actions down here. And this results in the hidden state changing. We have this kind of transient oscillator behavior in the hidden state. And indeed, by about time equals 10, the believed hidden state and the true hidden state, they actually track with one another very nicely. at across time we're able to get to our preferred uh hidden state of five which is very nice. So you can do the same thing again. These are the the observed predicted and preferred sensory data. Um so we have you know the actual sensory observation in blue. Again we're very low in the water column. Um so we're quite cold initially and the real observation is in in orange. So at the beginning they they really don't match each other very nicely but then as action is turned on we're able to um have them track and we're getting close to our desired temperature desired observation of uh you know what 79 or something like that and this is I think this is the most interesting plot this one here this kind of tells the whole story this is the variational free energy across time okay or the fixed precision precision weighted variational free energy across time. So you can see in the beginning, you know, we're quite high and then we kind of we we do minimize when we're doing perception only and then we plateau, but we're not able to really minimize any further. Okay. However, once we turn on the ability to act, there's this initial transient behavior and we kind of go all over the place, but then we are able to minimize our variational free energy much further by means of action than we could in the uh prediction only case. And that's we saw that with figure 7.6. So, and you I talk about the transient there. Um, there's this kind of interesting funky behavior. I I then have a little bit on the forward model. Um, kind of what it does, why we chose the particular forward model that we did. And I say, well, let's choose a let's choose a different one. Let's choose a forward model of plus one, okay? As opposed to minus one. So we can run this simulation again uh just with a different forward model. Uh and you can see here we're actually getting disastrous results. So let's let's visualize that. So I've I've zoomed in here once if we have a forward model. So the good forward model that we have in the example this is in blue and the bad forward model is in orange. So what this is this is showing you the the external state uh across time uh as as a function of time. Uh but if we have an incorrect forward model, we have this whole this this very bad blow up in uh the external state. We're getting kind of deeper and deeper into the water column, further and further away from our goals. Whereas with the correct forward model, we're able to sort of slowly well I haven't run it for long enough, but we're able to get close to our preferred hidden state. So there's a there's a little bit there on on the forward model. I think I need to tend to explain that a little bit more. Give a bit more motivation. Um this is just to say that it has quite a big effect on the resulting uh you know dynamics of of in the generative process also in the generative model. Uh and then I have some experiments around changing the precision in the in well the sensory precision in the generative model. So you know because our our sensory this is the the precision weighted so this is our precision this is our prediction error this is in the gradient of the variational free energy with respect to hidden states we can change the precisions to be whatever we want they're fixed here um they could be they could be otherwise okay so let's just say well let's take a precision of 50 precision of 100 precision of 500 and do three different simulations and see what we get so we can do that and we'll visualize this M and this is quite interesting. So this is essentially the same plot that we saw before except now that we've got blue, orange and green. This is the uh the external state across time. Okay, so X star in the generative process. You can see that broadly speaking we get the same kind of behavior. We're able to get the hidden state to end up close to where we want it to be. Um but depending on the precision we sort of do that quicker or smaller. So for a precision of 500 um we're able we're not able to get there as sort of quickly as a precision of 50. Right? So there's interesting questions there to be you know levers that can be pulled to to change exactly how the uh precision weighted prediction error is uh well computed. And we have the same thing again for our sensory precision. Sorry. Yeah. Yeah. So sensory position that changes the strength and speed of the correction. And then we have uh now the same thing on action. So what how did this change the actions that we were actually able to admit into the environment? You can see that there's they're broadly kind of the same. Um these were just arbitrary choices for me 50 100 and 500 and so on. Uh that is about it and I want to add a few more things in there. This is quite rough as it is, but I think this gives you at least the intuition that is hoped for in the example. Um, so I'll make this available uh on the page. Basically, what you'll be able to do is you'll be able to just go over here to chapter 7 and it'll just be a link. I'll put it here somewhere. I think we'll put it in the repo as well. So, if there's any questions about that, please uh do fire away. But hopefully that was useful. Um, I know that other people they they're interested in in implementing I would encourage you to implement your own version of the the Bijian thermostat. Um, so hopefully maybe that can be something of a um baseline uh implementation. Oh, lots of things in the in the chat. Excellent. Beautiful. Beautiful. Let me just quickly see here. Yeah. All right. No. Uh, yeah. I uh I liked your ex I liked your version, Vera. Um uh I I did have a look. So I'll I'll hopefully give you some feedback about yours as well. So opportunity for questions. Are there in fact any questions? 7.2. Yeah, actually let's go to figure 7.11. So if there are questions, please uh please do. But I'll just show you figure 7.11. Again, this makes the point I was making a little bit nicer. This one here. So this is kind of, you know, showing you that when you know in that example where we've we've got the first we're sort of partitioning the the the simulation into perception only and then perception and action. This is kind of showing you why we got the results that we got for um the variational free energy. So in the beginning we were able to decrease it slightly but then we were able to decrease it much much more when we had the ability to act. So if we take the D form I take I think that is the D form from memory. um the free energy where we've got the divergence between our approximate posterior well the predictive prior and the posterior after observation and then we have the surprisal right so the thing is when we and I did I said this last week but if we change if we update our beliefs using perception to make this diversion to small as possible we can get that to be zero and that's excellent that means that we have beliefs that track the hidden state perfectly but that does nothing to change the surprisal. Okay, so we saw that hopefully I'm still sharing here down here where there was still remaining VF here. Okay, so we're kind of getting rid of all of the divergence between the posterior and the well the predictive prior and the posterior after observations, but there's still remaining surprisal. And why is there remaining surprisal? Well, the surprisal is as you can see a function of our observations Y. So changing our beliefs does nothing to change the observations that we're getting in. And at the very beginning we said that we assume that our observations are functions covert functions of our actions. So the thing that we have to do is we have to act to change our observations if we want to make this smaller here and that's exactly what we did when we turned on action and then we were able to reduce the VF even further. So that's the that's the well this this this diagram is a nice pictorial or you know diagrammatic representation of that process and kind of why it is that we saw this behavior here and there's there's there's other comments to be had about different decompositions of the VF. So we have the I think it's the G form here I don't remember actually uh complexity and accuracy. Um, you can talk about the the complexity as being kind of the other the the same sort of divergence that's only really minimized in the in the perception case. And then accuracy is, as you can see, involves our observations. [clears throat] Cool, cool, cool. First perception, then action, or are they completely equivalent? Aha. Well, so that's an interesting point, Jen Cuomo. Um does an agent first calibrate his perception and realize it's not enough hence decides to intervene or would he just decide to act immediately? That is an interesting question. Um there there are ways to stagger the the updates in the sense that you know you could imagine updating your belief uh for some amount of time and then only acting. But abstractly, they're not kind of different things that happen at different times other than after you've made a Beijing update. Andrew, do you want to have a have a comment there? >> Yeah, sure. Um, yeah. Um, so I' I've thought about this often. It's interesting because you can implement it in different ways. uh all the way to the degree that there have been arguments about should we be calling it an action perception loop or a perception action loop if one of them is occurring before or after the other >> perception perception perception action perception perception perception action you know you can do those gradations >> yeah for sure um so so yeah with all that said it's um I mean t the of course the way we've been learning this is that apt uh perception comes before action rather than after it's through perception that um and this somewhat addresses another question in the chat too like what is the general higher order uh kind of interpretation of what happens here like if we could boil down the the action perception loop or perception action loop down to just a few top line like how should we look at this what's the order of steps without necessarily having to uh you know have all the uh equations memorized and that be the primary way we speak of it I would say generally the process is uh an agent receives and this is only coming from the agent side. So we're going to ignore referring to the environment too much. Uh just on the agent side you the agent will receive an observation. Uh it will then use that observation to update its beliefs about hidden states. Then it will use its uh beliefs about hidden states that it's updated to kind of determine what to do next terms of action. And that that's basically updating beliefs about action. We've not actually done the action yet, right? So step four is then to actually commit the action. That's the that's the general flow. And uh it's a bit tricky because uh the chat's been a bit lively since someone brought up the discrete case as well. Action is a little bit different in the discrete case than it is in um this continuous case we've been looking at. But that four those four steps generally hold across both. we can think of it as as that. Um and so another important thing that's more conceptual is you know just as we've been seeing that um in this continuous case especially like you can imagine each one of these iterations could be occurring every hour. uh if you create some kind of monitoring system that's only active every hour like or or a thermostat that checks every couple of minutes or something along those lines or it could be a real time like system or organism that's doing this in real time very rapidly. So um I think what's important to recognize is that we you know this can be conceived of as perception and action are happening nearly simultaneously and are sort of part and parcel with each other because they're directly informing each other very rapidly step by you know one after another right just imagine um you know kind of two wheels running and so um yeah I just want to give a little bit of context on the thought of that another way of looking at it is as a programmer as someone who's building a simulation, um, how do you sort of want to drop your agent into the environment, so to speak, right? Like, is it that your agent already commits an action at the very start before observing anything or or is your agent supposed to observe before acting? That's something that really just boils down to what do you want to happen at the very first time step, like the very first blip of time, right? um because immediately the ball starts both of the wheels start rolling after that to where it it is as if it didn't matter which one we started with. Um so yeah, from a coding perspective, it's just like well, how do you want to program? Do you want one to happen before the other? Um it probably won't lead to dramatically different results if you if you do one versus the other, but then again, it's really going to depend on the stakes of the actions that the the agents commit to. So >> there was a good there was a good question just in the chat now about the costs of actions and the costs of um you know belief updates and that kind of thing. We haven't yet looked at you know okay because it is typically the case that acting in the environment has some kind of cost even minimal even me just turning my my head here to change the amount of light I'm getting right there's some metabolic cost to that obviously trivially um but yes in general there are going to be costs associated with actions and that's one thing that's that's that is a sort of motivating um reason as to why we're going to want to be a little bit more dare I say sophisticated or sophisticated has a has a particular meaning in active inference. We're going to want eventually to be a little bit more sophisticated about how we do our action selection in active inference. Okay, especially for really complex environments. Fortunately, this environment is really simple. You're just a little high moving up and down the water column to change your your perception of your temperature. So, that's kind of nice. There's not really much more that you have to add to the story. But once it is the case, and it is typically the case that actions do have costs, you know, you're an agent, maybe a self-driving car or a little, I don't know, Roomba, you've only got so much battery, okay? And you can't just do every action. You have to decide if I did that action, it's going to be this costly. If I did this other action, it's going to be that costly. I'm going to take this other action that's minimally costly. Okay? But that's getting into planning and this kind of counterfactual uh stuff. And we're going to see that in chapter 9 and 10. That's that's where that's going to enter. Uh so typically yes we do have costs associated with action but we're not we're not looking at that just yet. So >> yeah, if it's all right. Um, [clears throat] something that was in the 2022 textbook, like I I've mentioned this before, but the 2022 textbook I think was much more centered on uh sort of immediate practitioners at a broad scale, but more strongly linking it with sort of um neuroscience at times psychiatry just sort of a lot more of the the the the originary empirical work that was done that led to a lot of these developments that resulted in something like active inference. Um, and it it specifies these three different kinds and you could say that there are more than three, but it's just a nice way of conceiving of different kinds of observations that one can receive. Um, and and those would be those which are extterosceptive, interosceptive and proprioceptive. And the the shortest way of saying that is exterceptive. we can relate to the common uh taking a human being as uh our our case study here. Um the exterceptive would be what relates most commonly to the colloquial five senses. So sight, touch, uh you know, hearing or sound, um olfactory, etc. And interosceptive would sort of be whatever is sensed from within from within the body like we don't have the same kinds of feeling receptors like within our our our stomachs and and the rest but there are different kinds of signals that we can feel that relate to you know am I hungry or satiated right now um there so the ability to read one's own body and then proprioceptive is a bit similar to interosceptive but it just relates a little bit more to the notion of like being able to sense one's positioning in space with respect to one's own body. So this can play into things like reflex arcs and kind of how how a joint moves uh or rather how how you know parts of the arm can move around a joint and being able to read that that's been carried out uh simulations of that have been carried out much more in the continuous space and I know that Frasier's done uh some nice work on kind of simulating you know um robotic arms or otherwise too. Um, but so so all this is to say it brings up this philosophical question almost of like does an agent need to be able to read itself, right? Because if I'm going to figure out how to uh spend my day and I'm going to do a variety of things including, you know, work my job for 8 hours and I'm also going to get a long workout and I'm also going to play uh a sport with my friends at the end of the night and I'm also going to go to sleep very late to fit all these things in. I'm clearly coming like going to be uh kind of butdding up against the metabolic costs of doing all of that and and if I ignore the signals from my body entirely, then I might just be able to do it as long as my body doesn't itself run out of energy. But if I pay attention to the signals in my body, they're probably going to I'm going to infer the hidden state of my body is I'm very very exhausted and and and I probably learn that I probably should not be expending myself so much. So all that is to say, you know, just as Frasier said, the thermostat example, we're not necessarily getting this sort of interosceptive um um series of observations as if the as if the thermostat could get worn down and and had some way of um you know, some kind of signal informing it of itself getting worn down like it it lacks that. But yeah, and it's a good question. uh many things in active inference and and a lot of other kinds of modeling like this do have some notion of metabolism or other sorts of like um resource constraints or costs sort of embedded within the framework it itself. So um in fact variational free energy and the like um t and expected free energy which is in planning which as Fraser said we'll get to those in future chapters. Um all these do have their so their own sort of inherent aspects of metabolism. Um part of the idea is reduction of complexity. Um whenever you have a lot of complexity, there's just much more to manage, much more to compute. And by then, you've probably spent a lot of resources um trying to sort of update your beliefs with a single inference step, so to speak, right? It's a very large model that requires a lot of computation just to get from one update to the next. Um, it'd be akin to, you know, say you have a PhD student, they're working on their dissertation, but they've so strongly overthought every bit of material that they need to include and still cannot figure out a clear argument through all of it that it's excessively complex. They still have not come up with a solution of what should I be writing right now. Um, and by anything sorry, >> I was just making a joke. This is, you know, this is too familiar. So, >> Oh, yeah. Yeah. I don't I don't want to say anything too close to too close to home, Frasier. Sorry. You I mean you you you you seem uh um >> I was just being >> focused enough. So you're not the first person I would think of in that situation, but yeah. Um but yeah, so so I hope that's useful in some way that this notion of um cost uh expenditure of the agent itself uh and otherwise is is very it's very important. It's useful. you get a much clearer sense of that kind of idea and homeostasis, the idea of having these sort of different homeostats like the body, the the autonomic nervous system and all these other things that are otherwise like generally unconscious to the sort of conscious mind as it were. Um, you know, a lot of these sort of um autoregulatory systems in the in the human body, we tend to regulate our our body temperature around a certain level, right? and any kind of strong derivations from that as long as we can sense them allows us to recognize that we have a fever right now or something along those lines, right? Um so so being able to read one's own body. We're not getting much of that in Sanjieve's very engineering oriented textbook quite rightly. It's it's very much tailored for engineers and we do get a much clearer in-depth take on the maths I think that we didn't get in 2022 textbook. But I would strongly suggest >> we we we do need to proceed kind of step by step. So there's only so many things we can do at any one time. But you're right, a lot of these questions will will come up again in chapter eight where oh you know we've seen this initial sort of schema but we want to kind of relax those assumptions. We want to bring in other concerns about how how do we do parameter learning and that kind of thing. So we will get to to some of those more broader concerns. So the chat has been very lively. I've been trying to keep up with it. Uh it's proven proving difficult. So that is excellent. Um where I I guess we're at the sort of the top of the hour. Um if there are other questions now would be a good time to uh to fire them away. >> Yeah. From the >> Yeah. Yeah. Okay. >> recording soon and then people who want to ask a question after the recording can do so as well. So fire away. >> Yeah. Yeah. Of course. Of course. Of course. Of course. >> No, you can ask your question now. >> Oh, okay. Okay. Okay. So, think from the engineering perspective, you know, you asked I mean you saying engineer perspective. So I mean I think I was texting in the chat. Okay. So thing is that so from that from that way we can see the technical part I mean technical way of seeing this thing. I mean using the I mean I was trying to using this active input to you know mainly uh in the physical AI and the AI and also in human being. So the thing is that so from that we can see that okay so the from the sensor input okay so it's a operational what so it will go to tensors. So the tensor will uh try to you know probability matching to find what is the exact variable. So from the variable we will be doing the statistical in the you know policy optimization in the MDP. Then from that uh from the guessing we will have that action. So what to do with that variable you know from the from the variable. So the action will happen. So in this case when the so the thing is that in the tensor area when the confidence score is high it's I mean confidence score is high it will be great but in the unmapped area then the confidence score will be low. So at this time you know instead of I mean when the conference score is low in that time we can use the can we can we can we can stop this we can just uh use the episomical pause you know to pass the uh that particular area and we can just try to enu the user to understand what it is for instance that's like there's a I mean there could be a black dot on my wall and it could be in your in your home too. So it could be either for shooting like you know like for crowded practice but in other time it could be diagram too I mean like like a painting too. So in that time it's a confusion you know. So that's that's you can take it like a untrained uh anomaly. Okay. So in that area we can stop we can pause the >> I'm not quite sure what what your question is. There seems to be a lot of stuff there. Could you boil it down just really quickly? >> Yeah I mean you know the perception changes the belief and the you know the action changes the world. So it's I'm striking that part in the technical way to execute that part, you know, to build the systems with it, you know, build the logical way to, you know, like use it in the physical. Yeah. Yeah. Yeah. That's what I was talking about. >> I mean, it's been hard because we we've only like literally 7.1 is the first example where we have active inference. So uh but you're right in that direction. Yeah. Yeah. Yeah. I was I mean yeah >> just a just a brief reference to this but then I have no further comments afterward but yeah it's still a good it's still a good topic I think that this notion of um one to one mappings many to many mappings many to one and one to many it's very it's very relevant especially in the case of like how we understand a likelihood right because it's you know certain observations we know nothing about we've never even seen them before but poss possibly, right? So, we have no idea how they relate to hidden states until we sort of just see what happens next and we engage in parameter learning that allows for some kind of buildup of experience over time that might inform us further. So, yeah, uh you can have these situations where a model is, you know, has very good clear observations it knows it can rely upon and then other observations that are not useful. It's like, oh, I I need to go to the store. Do I think the store is open or closed? Well, I see that it's daytime outside, so that's usually a good indicator that the store is open. Uh, the grass is wet. That's not very useful for me to know if the store is open or not. Right. Even though it's it's a fair enough observation, but it has no relationship. That's great. Right. Right. Does that make sense? So, it's it's nice to boil these down to kind of easier to understand examples over time uh that that let you see like, oh, common sensically that just makes sense. Um but yeah the the the many to many mappings is so important because the agent receives only observations those are the only things it sort of knows and then even then you can have an agent that has a bad sensor right to where even the observation itself is sort of distorted in some way but anyway interesting uh topic for sure um we >> yeah and also from your last session you know like you you saying about the null yeah that's right you were getting about there won't be null value all the time you are exactly right actually the thing is that when we separate that tensor area in the tensor area there will there will definitely there will be no null value at all because the thing is that there will be always I mean there will be always a signal from uh you know sensor I'm sorry I'll stop the recording here and then I think we can continue and anyone wants to ask a question can so thank you very much everyone I'll stop the recording goodbye we'll see you guys next week for chapter eight it's gonna be good so >> okay F.F.