WEBVTT 1 00:00:01.220 --> 00:00:03.240 Hi and welcome to this AI in C# 2 00:00:03.240 --> 00:00:05.540 video on the Microsoft Agent Framework. 3 00:00:06.140 --> 00:00:08.000 Today we're going to look into something called 4 00:00:08.000 --> 00:00:12.340 AG-UI, which is the acronym for Agent User 5 00:00:12.340 --> 00:00:13.660 Interaction Protocol. 6 00:00:14.680 --> 00:00:16.760 When I first saw this name I actually 7 00:00:16.760 --> 00:00:18.640 thought it was something to do with Autogen 8 00:00:18.640 --> 00:00:22.480 and their drag and drop features that have 9 00:00:22.480 --> 00:00:24.440 been there for a long time, but it's 10 00:00:24.440 --> 00:00:27.060 actually a new protocol that is very, very 11 00:00:27.060 --> 00:00:27.400 new. 12 00:00:28.940 --> 00:00:33.260 Came back in May, I think, of 2025, 13 00:00:33.620 --> 00:00:35.420 so it's less than six months old. 14 00:00:37.360 --> 00:00:40.100 So when I saw that and found out 15 00:00:40.100 --> 00:00:42.660 it's a protocol, I said, oh, not another 16 00:00:42.660 --> 00:00:42.980 protocol. 17 00:00:43.160 --> 00:00:46.120 We have MCP, we have AG-UI, we have 18 00:00:46.120 --> 00:00:47.500 all these different things. 19 00:00:48.300 --> 00:00:52.500 But the more I looked into it, it 20 00:00:52.500 --> 00:00:56.820 really solves an actual problem in terms of 21 00:00:56.820 --> 00:01:00.700 the UI part of AI, and it does 22 00:01:00.700 --> 00:01:04.000 it in quite an elegant way and a 23 00:01:04.000 --> 00:01:06.520 standard way that I really hope this protocol 24 00:01:06.520 --> 00:01:08.820 will kick on. 25 00:01:09.320 --> 00:01:12.040 Microsoft is embracing it, so that's a really 26 00:01:12.040 --> 00:01:12.680 good sign. 27 00:01:14.120 --> 00:01:16.380 But what is it? 28 00:01:18.380 --> 00:01:22.140 When we build our small console apps here 29 00:01:22.140 --> 00:01:25.380 in this series, we don't have a problem. 30 00:01:25.580 --> 00:01:28.640 We just have some consoles and we can 31 00:01:28.640 --> 00:01:32.360 do normal calls, we can do streaming, we 32 00:01:32.360 --> 00:01:34.800 can do tools and so on, but everything 33 00:01:34.800 --> 00:01:35.920 happens in one process. 34 00:01:36.500 --> 00:01:40.320 In real life, that's not often the case. 35 00:01:40.580 --> 00:01:42.060 Of course, it can still be done, but 36 00:01:42.060 --> 00:01:43.720 often we have a frontend and we have 37 00:01:43.720 --> 00:01:44.140 a backend. 38 00:01:44.840 --> 00:01:47.500 And the frontend would be the chatbot or 39 00:01:47.500 --> 00:01:50.740 whatever we do, and the server have the 40 00:01:50.740 --> 00:01:51.780 backend agent. 41 00:01:52.820 --> 00:01:57.060 Of course, there's also backend systems and automated 42 00:01:57.060 --> 00:02:00.560 behind the scenes things without frontends, and this 43 00:02:00.560 --> 00:02:01.800 protocol is not for that. 44 00:02:02.620 --> 00:02:05.780 But if you have this frontend, backend system, 45 00:02:06.620 --> 00:02:09.320 and you've tried it in real life, you 46 00:02:09.320 --> 00:02:11.440 know that it's quite simple, of course, to 47 00:02:11.440 --> 00:02:13.660 have a backend and you do a request 48 00:02:13.660 --> 00:02:17.820 response in order to get some information. 49 00:02:18.000 --> 00:02:20.160 So you could have the user answer a 50 00:02:20.160 --> 00:02:22.140 question and get a response back. 51 00:02:22.580 --> 00:02:23.280 No big deal. 52 00:02:25.360 --> 00:02:27.960 It's also easy just to couple a bunch 53 00:02:27.960 --> 00:02:32.540 of MCP servers, own tools and such onto 54 00:02:32.540 --> 00:02:34.860 the backend server and have them be able 55 00:02:34.860 --> 00:02:38.700 to be called so the user can just 56 00:02:38.700 --> 00:02:41.820 ask and the server can use its own 57 00:02:41.820 --> 00:02:42.600 server tools. 58 00:02:44.160 --> 00:02:48.500 But where it becomes tricky is when we 59 00:02:48.500 --> 00:02:52.040 need to be live chat GPT and do 60 00:02:52.040 --> 00:02:55.440 a stream request and get a streaming response. 61 00:02:56.060 --> 00:02:58.280 Because how do you do that? 62 00:02:58.400 --> 00:02:59.400 It is possible. 63 00:02:59.600 --> 00:03:02.120 I have actually done it in a sample 64 00:03:02.120 --> 00:03:04.260 application and also in a real application. 65 00:03:05.360 --> 00:03:09.100 But if you sit with some React frontend 66 00:03:09.100 --> 00:03:12.340 and a C-sharp backend, how do you 67 00:03:12.340 --> 00:03:15.060 begin to send server-side events, which it 68 00:03:15.060 --> 00:03:15.400 is? 69 00:03:16.500 --> 00:03:19.280 And how do you make it scalable? 70 00:03:19.520 --> 00:03:21.620 How do you make it easy to work 71 00:03:21.620 --> 00:03:21.880 with? 72 00:03:23.000 --> 00:03:25.640 So it can be done, but it's a 73 00:03:25.640 --> 00:03:25.980 pain. 74 00:03:28.250 --> 00:03:31.210 Another thing is, which I have actually never 75 00:03:31.210 --> 00:03:34.070 thought about, what if the frontend actually have 76 00:03:34.070 --> 00:03:37.290 some capabilities that you want to expose as 77 00:03:37.290 --> 00:03:38.350 tools to the backend? 78 00:03:39.870 --> 00:03:43.530 So just like we have server tools, what 79 00:03:43.530 --> 00:03:46.330 if the frontend could be changed the colour 80 00:03:46.330 --> 00:03:49.950 of something or how something looks, and we 81 00:03:49.950 --> 00:03:51.690 wanted AI to do that? 82 00:03:52.130 --> 00:03:55.530 I actually have really no idea without this 83 00:03:55.530 --> 00:03:57.410 protocol, how you could do that. 84 00:03:57.870 --> 00:04:02.290 But that is what we can do in 85 00:04:02.290 --> 00:04:04.690 a very, very elegant way. 86 00:04:05.130 --> 00:04:07.290 And let's go in and see some code 87 00:04:07.290 --> 00:04:09.990 because there's actually not too much to it. 88 00:04:12.760 --> 00:04:15.839 So here we are in the system and 89 00:04:15.839 --> 00:04:19.200 we are working with this agent user interaction 90 00:04:19.200 --> 00:04:21.040 and we have a client and a server. 91 00:04:22.460 --> 00:04:24.980 So for that reason, I have gone up 92 00:04:24.980 --> 00:04:27.400 and used the multiple startup projects. 93 00:04:27.820 --> 00:04:29.620 So I run both the server and the 94 00:04:29.620 --> 00:04:29.980 client. 95 00:04:32.480 --> 00:04:35.100 And I will go to the server, but 96 00:04:35.100 --> 00:04:36.820 I will just start the code and set 97 00:04:36.820 --> 00:04:38.520 a breakpoint on the client here. 98 00:04:41.180 --> 00:04:44.580 So let me run it and then we 99 00:04:44.580 --> 00:04:47.640 go quickly to the server and see what 100 00:04:47.640 --> 00:04:48.340 that's about. 101 00:04:49.920 --> 00:04:53.660 And here we have the client UI. 102 00:04:54.720 --> 00:04:57.380 Of course, I'm still using consoles, but this 103 00:04:57.380 --> 00:05:00.120 could just have easily been a Blazor app 104 00:05:00.120 --> 00:05:04.380 that would have this problem of talking with 105 00:05:04.380 --> 00:05:05.260 a backend server. 106 00:05:06.600 --> 00:05:08.920 So what happens is we start with the 107 00:05:08.920 --> 00:05:10.480 normal business as usual. 108 00:05:10.720 --> 00:05:13.740 We just make a client and we make 109 00:05:13.740 --> 00:05:14.400 an agent. 110 00:05:14.980 --> 00:05:16.820 And in this case, I make an agent 111 00:05:16.820 --> 00:05:19.460 with one tool call that is called GetWeather. 112 00:05:20.840 --> 00:05:24.560 And then we make a normal web application. 113 00:05:24.960 --> 00:05:28.800 So had this been a standard ASP.NET 114 00:05:28.800 --> 00:05:31.500 Core web UI, we have had this out 115 00:05:31.500 --> 00:05:31.960 of the box. 116 00:05:32.580 --> 00:05:34.080 And the only thing we need to really 117 00:05:34.080 --> 00:05:40.220 do is we need to add a AG-UI 118 00:05:40.220 --> 00:05:44.280 as a service, and we need to map 119 00:05:44.280 --> 00:05:45.680 it to an endpoint. 120 00:05:45.820 --> 00:05:47.740 In our case, just the root of this. 121 00:05:48.660 --> 00:05:50.180 And then we run the system. 122 00:05:51.440 --> 00:05:56.160 And if we do that, we will see 123 00:05:57.400 --> 00:06:00.380 that we are just running as something here, 124 00:06:00.620 --> 00:06:02.580 and we have a local host here. 125 00:06:03.060 --> 00:06:05.040 In real life, it'll of course be HTTPS 126 00:06:05.040 --> 00:06:08.980 and over the internet instead of just a 127 00:06:08.980 --> 00:06:10.920 local host here. 128 00:06:11.620 --> 00:06:13.960 So you can see the server is nothing 129 00:06:13.960 --> 00:06:18.660 other than instantiating an agent and exposing it. 130 00:06:21.010 --> 00:06:25.770 The client is also fairly simple in that 131 00:06:25.770 --> 00:06:28.710 we just make an HTTP client because we 132 00:06:28.710 --> 00:06:30.750 need to talk with the server. 133 00:06:31.470 --> 00:06:34.850 And we do that over the URL we 134 00:06:34.850 --> 00:06:35.370 have chosen. 135 00:06:36.490 --> 00:06:39.890 Then I make something with the colour of 136 00:06:39.890 --> 00:06:40.570 the text here. 137 00:06:40.710 --> 00:06:41.990 Right now you can see it's white. 138 00:06:42.550 --> 00:06:46.430 And then I make an AG-UI chat client. 139 00:06:46.750 --> 00:06:48.410 So instead of a normal chat client, I 140 00:06:48.410 --> 00:06:49.770 make an AGUI client. 141 00:06:50.150 --> 00:06:52.710 And all it needs is an HTTP client 142 00:06:52.710 --> 00:06:54.130 and the server URL. 143 00:06:55.170 --> 00:06:56.910 And once I have such a client, I 144 00:06:56.910 --> 00:07:00.050 can do sort of the same as I 145 00:07:00.050 --> 00:07:05.770 normally do using the agent part, create AI 146 00:07:05.770 --> 00:07:06.190 agent. 147 00:07:06.930 --> 00:07:09.030 And here I can give it tools, which 148 00:07:09.030 --> 00:07:11.450 will be the local tools. 149 00:07:11.710 --> 00:07:13.550 We also gave tools over here. 150 00:07:14.290 --> 00:07:18.590 But suddenly we can give this proxy of 151 00:07:18.590 --> 00:07:21.950 an agent tools that will be handed on 152 00:07:21.950 --> 00:07:24.310 to the server, which is quite a cool 153 00:07:24.310 --> 00:07:24.890 concept. 154 00:07:27.240 --> 00:07:31.180 This agent, however, is not a fully blown 155 00:07:31.180 --> 00:07:31.680 agent. 156 00:07:31.840 --> 00:07:33.900 For example, if we give it instructions, it 157 00:07:33.900 --> 00:07:35.520 will not follow those instructions. 158 00:07:36.600 --> 00:07:38.760 And that's because it's just a proxy. 159 00:07:39.160 --> 00:07:41.160 We can give it tools, but the rest 160 00:07:41.160 --> 00:07:44.960 we cannot really give because it won't follow 161 00:07:44.960 --> 00:07:45.500 that along. 162 00:07:46.000 --> 00:07:47.180 So it's a kind of a shell of 163 00:07:47.180 --> 00:07:47.700 an agent. 164 00:07:48.880 --> 00:07:50.660 And for that reason, for example, if we 165 00:07:50.660 --> 00:07:53.500 want to make an instruction, we need to 166 00:07:53.500 --> 00:07:54.760 give it ourself here. 167 00:07:55.280 --> 00:07:57.580 And in the same manner, it cannot work 168 00:07:57.580 --> 00:07:58.940 with the threads. 169 00:08:00.400 --> 00:08:02.360 So we also need to give that on 170 00:08:02.360 --> 00:08:02.880 our own. 171 00:08:03.840 --> 00:08:05.980 This might be because it's so new a 172 00:08:05.980 --> 00:08:07.900 protocol, and we might at some point be 173 00:08:07.900 --> 00:08:11.240 able to give the instructions and the threads 174 00:08:11.240 --> 00:08:14.200 like normal and behind the scenes, everything would 175 00:08:14.200 --> 00:08:14.580 work. 176 00:08:15.080 --> 00:08:16.820 But for now, we need to do a 177 00:08:16.820 --> 00:08:19.860 little extra here, but trust me, it's worth 178 00:08:19.860 --> 00:08:21.720 doing compared to what we get. 179 00:08:23.440 --> 00:08:26.480 Because what we get, and it can be 180 00:08:26.480 --> 00:08:29.080 perhaps a little difficult to understand if you 181 00:08:29.080 --> 00:08:32.340 haven't had this pain of how in the 182 00:08:32.340 --> 00:08:34.799 world do I get a client to talk 183 00:08:34.799 --> 00:08:36.960 to a server and stream the data back? 184 00:08:37.539 --> 00:08:40.440 Again, can be done, but trust me, it's 185 00:08:40.440 --> 00:08:41.340 damn annoying. 186 00:08:42.520 --> 00:08:45.280 But in this case, we're just going in 187 00:08:45.280 --> 00:08:46.440 and making an agent. 188 00:08:46.580 --> 00:08:50.510 So if I say hi, it can come 189 00:08:50.510 --> 00:08:50.770 back. 190 00:08:51.010 --> 00:08:55.570 And if we noticed there is no API 191 00:08:55.570 --> 00:08:58.410 keys or endpoints or anything here, there's just 192 00:08:58.410 --> 00:08:59.030 the URL. 193 00:08:59.930 --> 00:09:03.390 And the URL can probably also be protected 194 00:09:03.390 --> 00:09:04.950 by normal HTTP. 195 00:09:06.050 --> 00:09:08.610 But in our case, we don't need it. 196 00:09:09.590 --> 00:09:12.630 It's just an endpoint for our client to 197 00:09:12.630 --> 00:09:13.410 work with. 198 00:09:17.160 --> 00:09:20.640 So what we do is a little like 199 00:09:20.640 --> 00:09:21.320 we normally do. 200 00:09:21.580 --> 00:09:23.380 The only thing is we keep track of 201 00:09:23.380 --> 00:09:24.020 all the messages. 202 00:09:24.060 --> 00:09:26.520 Instead of in a thread, we just keep 203 00:09:26.520 --> 00:09:26.760 them. 204 00:09:27.320 --> 00:09:29.660 But then we can suddenly use streaming here. 205 00:09:30.880 --> 00:09:34.380 And streaming, yes, in one process is easy, 206 00:09:34.500 --> 00:09:36.300 just back to console. 207 00:09:36.760 --> 00:09:40.880 But here, it's actually going over HTTP and 208 00:09:40.880 --> 00:09:43.480 sending the server events back. 209 00:09:44.120 --> 00:09:46.860 So if we look behind the scenes, we 210 00:09:46.860 --> 00:09:50.360 can actually see that there is some text 211 00:09:50.360 --> 00:09:53.180 event streams going on back and forward when 212 00:09:53.180 --> 00:09:54.040 I wrote hello. 213 00:09:57.610 --> 00:10:00.950 So okay, we can say then, is that 214 00:10:00.950 --> 00:10:02.170 the only thing? 215 00:10:02.690 --> 00:10:06.950 No, it's also able to expose the different 216 00:10:06.950 --> 00:10:08.530 things that happens on the server. 217 00:10:09.590 --> 00:10:11.930 And it does that using the different content 218 00:10:11.930 --> 00:10:13.810 back from the updates here. 219 00:10:14.890 --> 00:10:17.310 So it can give us back the text. 220 00:10:17.630 --> 00:10:18.970 That's what we're seeing here. 221 00:10:19.490 --> 00:10:22.130 It can give us back any function call, 222 00:10:22.290 --> 00:10:25.290 which we haven't made yet, any results of 223 00:10:25.290 --> 00:10:27.790 those function calls, and any errors that might 224 00:10:27.790 --> 00:10:28.030 be. 225 00:10:29.110 --> 00:10:32.310 And again, we are able to give a 226 00:10:32.310 --> 00:10:36.630 local tool, again, something I would never have 227 00:10:36.630 --> 00:10:39.170 thought about how to do in real life 228 00:10:39.170 --> 00:10:41.590 because I would just have said that's impossible. 229 00:10:42.110 --> 00:10:43.810 But we can actually give this tool and 230 00:10:43.810 --> 00:10:47.410 on the fly when we call here, that 231 00:10:47.410 --> 00:10:51.330 tool is being passed onto the server, given 232 00:10:51.330 --> 00:10:53.170 as if it was something that could happen 233 00:10:53.170 --> 00:10:54.010 on the server. 234 00:10:55.010 --> 00:10:58.090 When the AI wants to call it, it 235 00:10:58.090 --> 00:11:04.170 communicates back to the client, executes the thing, 236 00:11:04.390 --> 00:11:05.990 which is in our case, just change the 237 00:11:05.990 --> 00:11:10.050 colour, and go on its merry way to 238 00:11:10.050 --> 00:11:10.770 work. 239 00:11:11.510 --> 00:11:13.810 So let's see some of this in some 240 00:11:13.810 --> 00:11:15.210 code here. 241 00:11:15.210 --> 00:11:18.210 So I could, for example, go in and 242 00:11:18.210 --> 00:11:24.010 say, what is the weather like in Paris? 243 00:11:26.160 --> 00:11:32.280 And if I do that, it should say 244 00:11:32.280 --> 00:11:33.960 like, not link, sorry. 245 00:11:35.660 --> 00:11:37.480 And now it knows it's just a tool 246 00:11:37.480 --> 00:11:39.640 call and not try to find a link 247 00:11:39.640 --> 00:11:41.460 to the weather in Paris. 248 00:11:42.780 --> 00:11:44.960 So what it did, it made a tool 249 00:11:44.960 --> 00:11:48.760 call on the server, got the response back, 250 00:11:50.160 --> 00:11:54.360 and we could see on the client, it 251 00:11:54.360 --> 00:11:56.680 actually gave information on what it was doing. 252 00:11:57.300 --> 00:11:59.440 So we can make a GUI that indicates, 253 00:11:59.860 --> 00:12:01.600 hey, we are calling this tool with this 254 00:12:01.600 --> 00:12:02.200 information. 255 00:12:02.460 --> 00:12:05.780 Again, something that was very, very difficult to 256 00:12:05.780 --> 00:12:08.260 send using these streaming events. 257 00:12:08.360 --> 00:12:10.120 But this is packed in, so we can 258 00:12:10.120 --> 00:12:11.640 do it very, very simple. 259 00:12:12.720 --> 00:12:14.060 And we get the result back. 260 00:12:14.140 --> 00:12:16.400 And of course, that result from the tool 261 00:12:16.400 --> 00:12:18.100 can be used. 262 00:12:18.580 --> 00:12:21.900 So we wouldn't need to write this out, 263 00:12:22.220 --> 00:12:23.780 but if we want to make a GUI 264 00:12:23.780 --> 00:12:26.320 that is very interactive and telling exactly what's 265 00:12:26.320 --> 00:12:28.940 going on, we can do it compared to 266 00:12:29.820 --> 00:12:32.300 in the past, where this was really a 267 00:12:32.300 --> 00:12:32.500 pain. 268 00:12:34.380 --> 00:12:37.900 In the same manner, we can go in 269 00:12:37.900 --> 00:12:41.200 and then call a local tool and pass 270 00:12:41.200 --> 00:12:42.020 it onto the server. 271 00:12:42.620 --> 00:12:45.680 So we could, for example, say, change the 272 00:12:45.680 --> 00:12:48.540 colour to green. 273 00:12:55.230 --> 00:12:56.470 And now it did a tool call. 274 00:12:56.650 --> 00:12:59.470 It passed the text onto the server. 275 00:13:00.930 --> 00:13:04.050 The server found out it got that extra 276 00:13:04.050 --> 00:13:04.450 tool. 277 00:13:05.090 --> 00:13:08.770 It passed that request back to the client 278 00:13:08.770 --> 00:13:11.970 where it ran this piece of code down 279 00:13:11.970 --> 00:13:12.170 here. 280 00:13:12.290 --> 00:13:14.930 We can set a breakpoint and say, change 281 00:13:14.930 --> 00:13:17.410 this colour to yellow. 282 00:13:21.520 --> 00:13:25.280 So this is happening over streaming on the 283 00:13:25.280 --> 00:13:26.720 fly from a server. 284 00:13:28.060 --> 00:13:30.120 Again, I would have no clue how to 285 00:13:30.120 --> 00:13:31.640 make this with the normal HTTP. 286 00:13:33.460 --> 00:13:35.060 But it works. 287 00:13:35.200 --> 00:13:38.140 We can suddenly give client-side tools. 288 00:13:38.520 --> 00:13:42.520 I could see desktop apps doing something directly 289 00:13:42.520 --> 00:13:47.280 in a system while the AI is remote. 290 00:13:48.580 --> 00:13:52.060 It is kind of incredible, in my mind, 291 00:13:52.140 --> 00:13:53.240 what this can do. 292 00:13:54.740 --> 00:13:57.340 And the only thing that are the drawbacks 293 00:13:57.340 --> 00:13:59.160 is we need to keep track of the 294 00:13:59.160 --> 00:14:03.940 entire conversation, and we need to make the 295 00:14:03.940 --> 00:14:06.240 system call upfront ourself. 296 00:14:06.700 --> 00:14:09.620 The rest happens quite automatically. 297 00:14:10.480 --> 00:14:13.360 And again, we just get these events back 298 00:14:13.360 --> 00:14:15.880 so we, on the fly, can update a 299 00:14:15.880 --> 00:14:18.040 GUI showing how it's going. 300 00:14:20.060 --> 00:14:25.000 So, so much pain and things I have 301 00:14:25.000 --> 00:14:28.420 thought about needing to do and not done 302 00:14:28.420 --> 00:14:32.520 because it was hard can suddenly be expressed 303 00:14:32.520 --> 00:14:34.460 in a very, very simple way. 304 00:14:35.020 --> 00:14:38.380 And again, we don't have any credentials here, 305 00:14:38.560 --> 00:14:40.600 which is also incredible. 306 00:14:41.680 --> 00:14:43.780 So as you can hear, I'm really a 307 00:14:43.780 --> 00:14:46.680 fan of this protocol, and I really hope 308 00:14:46.680 --> 00:14:49.660 it gains traction because this will make the 309 00:14:49.660 --> 00:14:52.080 life of us who also needs to build 310 00:14:52.080 --> 00:14:55.140 a GUI on top of an AI much 311 00:14:55.140 --> 00:14:55.540 simpler. 312 00:14:57.060 --> 00:14:58.860 So with that, thank you for attending this 313 00:14:58.860 --> 00:15:00.400 one, and see you in the next one.