1 00:00:09,080 --> 00:00:14,100 Good morning everyone, my name is Dr. Igor Nikolic, an assistant professor at Delft University 2 00:00:14,100 --> 00:00:18,360 of Technology and a senior visiting research associate at the environmental change institute 3 00:00:18,360 --> 00:00:22,180 of the University of Oxford, and this where I'm speaking to you from. 4 00:00:22,180 --> 00:00:27,180 This is the next installment of the NGI MOOC and today, we will be talking about Agent 5 00:00:27,180 --> 00:00:33,239 Based Modeling, but before we start let's consider something that you'll do tonight. 6 00:00:33,239 --> 00:00:40,030 Tonight, when you go home, you will perform a seemingly insignificant and simple action. 7 00:00:40,030 --> 00:00:46,180 You will arrive at home and notice that it's dark and you will go click, turn on the light. 8 00:00:46,180 --> 00:00:50,799 And suddenly because at the same time you are doing this, millions of others are doing 9 00:00:50,799 --> 00:00:55,769 the same thing an overall pattern, overall over the world, overall emergent presence 10 00:00:55,769 --> 00:01:00,080 of humanity becomes visible on the face of the earth. 11 00:01:00,080 --> 00:01:05,540 Now nobody has told us, collectively, when to turn on the light and how to make sure 12 00:01:05,540 --> 00:01:11,820 that we become visible through space, now this is a consequence of individual actions, 13 00:01:11,820 --> 00:01:15,900 of individual choices all happening at the same time. 14 00:01:15,900 --> 00:01:18,690 And how do you see things come to be? 15 00:01:18,690 --> 00:01:20,250 Where do the systems come from? 16 00:01:20,250 --> 00:01:22,540 How come that we are collectively able to do that? 17 00:01:22,540 --> 00:01:28,000 Well, just think about it: The world that you are part of is a large complex adaptive system. 18 00:01:28,000 --> 00:01:35,600 There is a myriad of technical things, wires, transformers, switches, machines there's lots 19 00:01:35,680 --> 00:01:43,000 of people, large transport systems, energy production, laws, regulations, habits, all 20 00:01:43,040 --> 00:01:48,000 sorts of things going on at the same time, all over the place, acting and interacting 21 00:01:48,050 --> 00:01:54,600 with each other to create this beautiful blue marble with lights on them that we see. 22 00:01:54,680 --> 00:01:59,800 So how does one even go about trying to understand all these interconnected systems, that adapt 23 00:01:59,800 --> 00:02:01,470 grow and evolve? 24 00:02:01,470 --> 00:02:08,300 Well, this is the lecture on Agent Based Modeling, so this is obviously how we are going to approach this. 25 00:02:08,300 --> 00:02:10,450 So, Agent Based Modeling, what is it? 26 00:02:10,450 --> 00:02:12,200 And why should you care? 27 00:02:12,200 --> 00:02:17,920 So what we will discuss today is the following: We will talk about what an agent is, what 28 00:02:17,920 --> 00:02:22,670 agent based modeling is, I will show you a number of examples to help you understand 29 00:02:22,670 --> 00:02:27,400 when and how to use agent based modeling and I will try to give you some pointers to help 30 00:02:27,400 --> 00:02:29,980 you explore Agent Based Modeling on your own. 31 00:02:29,980 --> 00:02:33,680 So, without further a due, let's start. 32 00:02:33,680 --> 00:02:38,300 Well, Agent Based Modeling is a technique that comes from the Complex Adaptive Systems toolkit. 33 00:02:38,300 --> 00:02:42,560 It is something that has been around for quite a while, but it's only recently that computers 34 00:02:42,560 --> 00:02:47,060 have become powerful enough to really use it to its full potential. 35 00:02:47,060 --> 00:02:52,870 And one of the great thinkers in the field, Joshua Epstein, has once said: "If you did 36 00:02:52,870 --> 00:02:58,349 not grow it, you did not explain it." Now this approach is called Generative Science 37 00:02:58,349 --> 00:03:01,130 and this is what is the heart of Agent Based Modeling. 38 00:03:01,130 --> 00:03:05,910 So it is all about understanding the world phenomena of the world from the bottom up. 39 00:03:05,910 --> 00:03:10,599 It is a very multi-disciplinary approach used for understanding complex systems and it is 40 00:03:10,599 --> 00:03:14,930 really about understanding how things develop, rather than how things are. 41 00:03:14,930 --> 00:03:19,880 So it is all about the process of interaction and action and reacting individuals giving 42 00:03:19,880 --> 00:03:23,860 rise to a pattern that we observe. 43 00:03:23,860 --> 00:03:30,560 So, how this is done, well we will start by situating an autonomous heterogeneous agent, 44 00:03:30,560 --> 00:03:36,650 or entities or things, in some kind of relevant environment in space a network or whatever. 45 00:03:36,650 --> 00:03:42,720 We will let them act and react and interact following relatively simple individual rules, 46 00:03:42,720 --> 00:03:48,640 then these interactions will generate, or grow, this observed macroscopic pattern that 47 00:03:48,640 --> 00:03:49,569 we're interested in. 48 00:03:49,569 --> 00:03:54,000 For example if you're interested in how the stock market behaves, and why it is going 49 00:03:54,000 --> 00:03:59,629 up or down, one way to understand it is to look at many individual investors and decision 50 00:03:59,629 --> 00:04:05,209 makers, see how they act and react, looking at what others are doing, looking at what 51 00:04:05,209 --> 00:04:08,900 the environment is doing and understanding how the pattern comes to be. 52 00:04:08,900 --> 00:04:13,970 Alternatively, what we also can do, is if we have a collection of individuals that we 53 00:04:13,970 --> 00:04:17,099 understand very well, so we know how this firm behaves, we know how this government 54 00:04:17,099 --> 00:04:22,129 behaves, we can then explore and simulate what kind of behaviors are they capable of 55 00:04:22,129 --> 00:04:24,129 in the long run. 56 00:04:24,129 --> 00:04:26,699 So, how does this work? 57 00:04:26,699 --> 00:04:30,990 Well, we sometimes say that an Agent is a thing that does things to other things. 58 00:04:30,990 --> 00:04:33,270 So, why is this important? 59 00:04:33,270 --> 00:04:34,770 Well, just look at the image. 60 00:04:34,770 --> 00:04:40,200 You see that at the middle there is an agent, an entity or thing that knows things and does things. 61 00:04:40,279 --> 00:04:44,500 Now this thing is situated in a model in some kind of environment, where there are other agents, 62 00:04:44,500 --> 00:04:51,300 which sits inside a model that we as a modeler have chosen and drawn boundaries around. 63 00:04:51,360 --> 00:04:54,490 We have said this belongs to the system, and this does not. 64 00:04:54,490 --> 00:04:59,680 Now, looking at these agents, following inputs from others, from the environment and from 65 00:04:59,680 --> 00:05:04,460 their own behaviors, they will make decisions and they will perform actions. 66 00:05:04,460 --> 00:05:08,089 Now these actions can affect themselves, they can affect the environment around, they can 67 00:05:08,089 --> 00:05:09,839 affect other agents. 68 00:05:09,839 --> 00:05:13,969 Now there even might be many other agents that are acting, and interacting on their own, 69 00:05:13,969 --> 00:05:17,520 without having direct interaction with you, the agent. 70 00:05:17,520 --> 00:05:23,080 But because they change the environment that they are in they might indirectly affect you. 71 00:05:23,080 --> 00:05:28,500 So it is literally the thing that does things to other things that gives rise to a pattern of interaction. 72 00:05:28,589 --> 00:05:32,169 Now, what are the components of such an Agent based model? 73 00:05:32,169 --> 00:05:35,930 Well, first, clearly it is the agents, the entities, the things themselves. 74 00:05:35,930 --> 00:05:41,490 Then there are states, then there are decision rules, the actions they perform, the environment 75 00:05:41,490 --> 00:05:44,830 they are in and the time in which they exist. 76 00:05:44,830 --> 00:05:50,809 So let we discuss these in a little bit more detail and give you an insight in how it works. 77 00:05:50,809 --> 00:05:54,020 The agents themselves, as I said, they are things that do things to other things. 78 00:05:54,020 --> 00:05:58,600 This agent can be anything, really anything that you care about and you think is worth exploring. 79 00:05:58,659 --> 00:06:06,300 For example: individual people, firms, organizations, ideas, countries, technologies, bacteria, 80 00:06:06,330 --> 00:06:08,319 whatever you really care about. 81 00:06:08,319 --> 00:06:13,409 They key however is to identify a relevant entity that has a boundary, that has a clear 82 00:06:13,409 --> 00:06:18,249 inside and an outside, that has an internal state, has decision making processes and is 83 00:06:18,249 --> 00:06:20,289 capable of actions. 84 00:06:20,289 --> 00:06:22,599 So, the states then. 85 00:06:22,599 --> 00:06:25,649 The states are things that the agents knows or has. 86 00:06:25,649 --> 00:06:30,279 They could be public or private, such as my location, my color, my assets, you know, what 87 00:06:30,279 --> 00:06:32,599 is the content of my wallet for example. 88 00:06:32,599 --> 00:06:38,969 And I can tell you, being an academic, it's not a lot, but the things that I have, that 89 00:06:38,969 --> 00:06:44,169 allow me to reason on and make actions based upon it. 90 00:06:44,169 --> 00:06:45,999 So it is the things I know or have. 91 00:06:45,999 --> 00:06:51,589 Now, the decision rules, as you can see here, are the decision and transformation rules 92 00:06:51,589 --> 00:06:56,500 that take inputs and my states and convert them into some kind of actions and overall behaviors. 93 00:06:56,580 --> 00:07:00,619 These things can be static or dynamic, and for example, as you can see on the slide, 94 00:07:00,619 --> 00:07:07,240 here is a simple reasoning logic for setting a price of a product you're selling based 95 00:07:07,240 --> 00:07:10,050 on a cost plus margin mechanism. 96 00:07:10,050 --> 00:07:14,729 Now this can be very elaborate, people have done very crazy adaptive things, like genetic 97 00:07:14,729 --> 00:07:19,400 algorithms, genetic programming, all sorts of things, or they can be very simple as IF THEN statements. 98 00:07:19,409 --> 00:07:24,900 If it rains, and if I do not have an umbrella, I will have to buy an umbrella in order not to get wet. 99 00:07:25,119 --> 00:07:29,330 And then the action becomes the buying and the state is being wet, yes or no. 100 00:07:29,330 --> 00:07:36,110 Now the actions, the consequences of the decision making processes, the result of those and 101 00:07:36,110 --> 00:07:39,429 Agent will perform or not perform some action. 102 00:07:39,429 --> 00:07:43,819 Now this is important to realize, that agents are autonomous, they can choose not to act. 103 00:07:43,819 --> 00:07:48,490 Whereas you are dealing with for example object oriented programming an object will always 104 00:07:48,490 --> 00:07:50,240 do what you ask it to do. 105 00:07:50,240 --> 00:07:54,240 With agents you'll say it's your turn to act, the agent will then consider its environment, 106 00:07:54,240 --> 00:07:59,490 consider its state, make the computation that it needs to do, and then decides whether to 107 00:07:59,490 --> 00:08:00,599 act, and how to act. 108 00:08:00,599 --> 00:08:06,520 Now, this action can be based on some input of another agent, on some state, or some kind 109 00:08:06,520 --> 00:08:07,909 of internal decision rule. 110 00:08:07,909 --> 00:08:13,119 These actions can also affect other agents, they can affect their own rules, their own 111 00:08:13,119 --> 00:08:18,659 states, they can affect the environment, and it is often through this indirect interaction 112 00:08:18,659 --> 00:08:22,889 throughout the environment that the true complexity of the system arises. 113 00:08:22,889 --> 00:08:25,849 Now, the environment. 114 00:08:25,849 --> 00:08:30,490 Now the environment is a thing the agent is in, and everything that is not the agent but 115 00:08:30,490 --> 00:08:31,889 is relevant. 116 00:08:31,889 --> 00:08:36,490 Now this environment provides the agent with structure and information. 117 00:08:36,490 --> 00:08:40,880 For example if I'm an investor agent trying to decide whether to build a power plant, 118 00:08:40,880 --> 00:08:44,960 I might consider the cost of capital, the interest rate. 119 00:08:44,960 --> 00:08:48,820 This is not something I control, it's not something that is part of me, but it is something 120 00:08:48,820 --> 00:08:51,160 that the external world sets. 121 00:08:51,160 --> 00:08:56,020 So the environment affects the agent and in some cases the agent may affect it as well. 122 00:08:56,020 --> 00:08:59,880 It also provides structure, it provides space through which we interact. 123 00:08:59,880 --> 00:09:05,870 For example, an agent can be on a grid so I can have eight neighbours around me and 124 00:09:05,870 --> 00:09:12,000 I only know of those eight, or I could be in a physical space, for example on a GIS 125 00:09:12,000 --> 00:09:17,260 map so I can say well this agent is three kilometers away from me or two meters. 126 00:09:17,260 --> 00:09:21,990 I could be in a network, so I could have a network environment where I could see my first 127 00:09:21,990 --> 00:09:26,490 degree neighbours, my second degree neighbours, and so forth. 128 00:09:26,490 --> 00:09:29,420 Now finally the aspect I want to discuss is time. 129 00:09:29,420 --> 00:09:35,700 Now time is a very peculiar thing, excuse me, in Agent Based Modeling, as time takes 130 00:09:35,700 --> 00:09:42,400 place in discrete steps, in ticks and between two ticks we assume that everything happens at once. 131 00:09:42,900 --> 00:09:45,130 Now, why do we do this? 132 00:09:45,130 --> 00:09:50,890 Well, if you remember from the opening, the complexity is all about things happening at 133 00:09:50,890 --> 00:09:55,110 the same time across the space, acting and interacting all over the time. 134 00:09:55,110 --> 00:09:58,970 Now it is this parallelism of the real world that we are trying to capture when we build 135 00:09:58,970 --> 00:10:00,020 Agent based models. 136 00:10:00,020 --> 00:10:05,280 Now given that computers are sequential processing machines, we have to allow things to happen 137 00:10:05,280 --> 00:10:08,110 at the same time even if they happening one after the other. 138 00:10:08,110 --> 00:10:12,830 So we assume that between two steps, between two ticks, everything happens at once. 139 00:10:12,830 --> 00:10:18,940 So the order, or the sequence of agent interaction, is very important, because like you can probably 140 00:10:18,940 --> 00:10:23,690 imagine if a certain agent always goes first, for example is always allowed to buy first, 141 00:10:23,690 --> 00:10:27,960 it will buy the cheapest resources for example and always have an advantage. 142 00:10:27,960 --> 00:10:33,110 So we shuffle the agents every time step, and allow them to take turns to form interactions 143 00:10:33,110 --> 00:10:38,830 by this we simulate parallel action that we observe in the real word. 144 00:10:38,830 --> 00:10:42,140 Now let me give you a very simple example of how this works. 145 00:10:42,140 --> 00:10:49,080 Now as I opened with this wonderful blue marble image of our planet, what has happened there 146 00:10:49,080 --> 00:10:57,100 is individuals arriving at home experiencing the environmental change, they see dark, they 147 00:10:57,100 --> 00:11:01,780 see other agents, choosing to turn on the light or not, and based on inputs of others, 148 00:11:01,780 --> 00:11:05,910 on their own individual preferences, they decide to turn on the lights. 149 00:11:05,910 --> 00:11:10,170 Now, nobody coordinates when to turn on the lights, but because we are very similar in 150 00:11:10,170 --> 00:11:14,440 many ways as humans we will roughly do that at the same time and we will see the lights 151 00:11:14,440 --> 00:11:16,870 of the world come up. 152 00:11:16,870 --> 00:11:22,430 Now let me give you a slightly more complicated example and one that is at the so called operational 153 00:11:22,430 --> 00:11:27,500 scale, something with a short time scale, where it is all about how actions are actually performed. 154 00:11:27,880 --> 00:11:34,880 Now, what you see here is a simulation of the airbus a 380 evacuation plan. 155 00:11:35,290 --> 00:11:42,290 Now, as you might be aware, before an airplane is put into production before use it has to 156 00:11:42,680 --> 00:11:47,120 pass certain requirements in terms of how quickly it can be evacuated. 157 00:11:47,120 --> 00:11:51,620 Now, normally you would do this with many, many, many individuals, many participants, 158 00:11:51,620 --> 00:11:54,510 you will put them into the plane, tell them, fire! 159 00:11:54,510 --> 00:11:56,830 And then they have to run out as quickly as possible. 160 00:11:56,830 --> 00:12:01,040 Now this is expensive and takes a lot of time, because as you decide they are not moving 161 00:12:01,040 --> 00:12:06,270 fast enough you will have to adapt the lay-out of the seats, move the exits, things like that. 162 00:12:06,800 --> 00:12:13,160 Now, what airbus has done in this case, it has simulated human behavior, individual behavior, 163 00:12:13,160 --> 00:12:14,430 in a computer as agents. 164 00:12:14,430 --> 00:12:19,940 Now this agents follow rules that are observed when humans are in panic and try to run away 165 00:12:19,940 --> 00:12:26,820 and they can simulate time and time over to see how quickly people can evacuate the plane. 166 00:12:26,820 --> 00:12:30,730 And the emergent pattern from this interaction of bumping into each other, trying to squeeze 167 00:12:30,730 --> 00:12:36,500 through a tight spot, is the movement patterns and the evacuation time that results. 168 00:12:36,500 --> 00:12:40,010 Sorry about that. 169 00:12:40,010 --> 00:12:48,000 So, this is a very operational, short time-scale example where we try to understand the emergent 170 00:12:48,000 --> 00:12:49,840 pattern at a short scale. 171 00:12:49,840 --> 00:12:56,320 Now at the medium time-scale, at the more tactical time-scale, there is this other example 172 00:12:56,320 --> 00:12:59,960 of for example a refinery. 173 00:12:59,960 --> 00:13:05,080 This is work done by a colleague of mine, Dr. van Dam, who has looked at the purchasing 174 00:13:05,080 --> 00:13:12,080 strategy and the effect it has on the performance of refineries when the refinery is facing 175 00:13:12,680 --> 00:13:16,410 delays and disruptions in its crude oil supply. 176 00:13:16,410 --> 00:13:25,000 Now in this case the agents are multiple departments, they all have their own internal logic, 177 00:13:25,000 --> 00:13:29,810 their internal goals they are trying to achieve and they are trying to cooperate and make 178 00:13:29,810 --> 00:13:34,300 money at the same time and they are experiencing these delays as for example when there are 179 00:13:34,300 --> 00:13:36,800 disrupted shipments. 180 00:13:36,800 --> 00:13:42,310 Now the individual share again are the departments and the overall pattern is the overall performance 181 00:13:42,310 --> 00:13:45,670 and efficiency and profitability of the entire plant. 182 00:13:45,670 --> 00:13:50,100 Now, at the long-term, at the strategic level example. 183 00:13:50,100 --> 00:13:52,840 There is wonderful work done by my colleague 184 00:13:52,840 --> 00:13:57,560 Dr. Chappin who has looked at the effects of carbon tax on carbon trading and power generation 185 00:13:57,560 --> 00:13:58,930 in the Netherlands. 186 00:13:58,930 --> 00:14:04,800 So here we have the agents that are energy companies, own multiple facilities that experience 187 00:14:04,800 --> 00:14:08,970 changes in prices and changes in electricity demand, and they have to make decisions whether 188 00:14:08,970 --> 00:14:14,529 to invest, whether to divest, whether to buy one kind of fuel, or another kind of fuel, 189 00:14:14,529 --> 00:14:21,100 interact with each other and the overall emergent pattern will be various levels of CO2 emissions 190 00:14:21,100 --> 00:14:25,820 and CO2 reductions and various electricity prices. 191 00:14:25,820 --> 00:14:31,700 So this is something that happens over decades compared to the airbus example that happens in minutes. 192 00:14:32,300 --> 00:14:37,430 But you can still use an agent based model in a very useful fashion to explore how individual 193 00:14:37,430 --> 00:14:43,430 interactions, individual decision-making processes affect an overall pattern. 194 00:14:43,430 --> 00:14:47,330 And here at this final slide you can see an example of a running simulation. 195 00:14:47,330 --> 00:14:52,980 This is power plants making decisions, investing into new facilities, you see that the generation 196 00:14:52,980 --> 00:14:57,910 profile is changing, there is more electricity produced from one source than from the other, 197 00:14:57,910 --> 00:15:02,750 you'll see that the CO2 market is changing, you can see all sorts of dynamics happening 198 00:15:02,750 --> 00:15:05,520 over time as agents act and react. 199 00:15:05,520 --> 00:15:09,740 Now, with this I would like to thank you for your attention and I hope I have given you 200 00:15:09,740 --> 00:15:14,110 a basic understanding of what an agent based model is and how you can use it. 201 00:15:14,110 --> 00:15:19,160 Now, to wrap up I would like to point at several resources that might be useful for you to 202 00:15:19,160 --> 00:15:20,779 continue studying this topic. 203 00:15:20,779 --> 00:15:26,160 Ohh yes, and of course please post any questions you might have on our discussion forum. 204 00:15:26,160 --> 00:15:30,750 So, further reading: First, I encourage you to download Netlogo. 205 00:15:30,750 --> 00:15:35,130 Netlogo is a free and open source Agent Based Modeling platform that is very well suited 206 00:15:35,130 --> 00:15:38,339 for teaching and studying agent based models. 207 00:15:38,339 --> 00:15:39,730 There is the link where you can get it. 208 00:15:39,730 --> 00:15:42,300 Secondly, the book by Joshua M. Epstein, 209 00:15:42,300 --> 00:15:47,730 Generative Social Science, that talks about how does one explore social phenomena 210 00:15:47,730 --> 00:15:48,510 from the bottom up. 211 00:15:48,510 --> 00:15:54,029 I have already mentioned Joshua Epstein, but I do strongly suggest reading that book. 212 00:15:54,029 --> 00:16:00,480 Furthermore, there are two open courseware courses that I offer online. 213 00:16:00,480 --> 00:16:07,060 The first one, SPM4530, deals with complex systems and a short introduction into agent 214 00:16:07,060 --> 00:16:13,279 based modeling and the other course, SPM9555, deals with advanced agent based modeling and 215 00:16:13,279 --> 00:16:17,089 it takes you in great detail in how to build complex models. 216 00:16:17,089 --> 00:16:21,529 Finally, when you're ready to dive even deeper there is a book called Agent Based Modeling 217 00:16:21,529 --> 00:16:28,150 of Socio-Technical Systems, edited by me, my colleagues Dr. van Dam and Dr. Lukszo and 218 00:16:28,150 --> 00:16:32,310 there it is online, you can check it out to dive into this topic much deeper. 219 00:16:32,310 --> 00:16:38,700 Now, I wish you a lot of fun and enjoyment studying Agent Based Modelings and good luck.