WEBVTT

1
00:00:01.200 --> 00:00:03.580
In this lecture, I'm going to show you

2
00:00:03.580 --> 00:00:06.140
something called tool calling middleware.

3
00:00:07.400 --> 00:00:10.080
And for that, I've taken a copy of

4
00:00:10.080 --> 00:00:12.740
the creating tools, the first tool video we

5
00:00:12.740 --> 00:00:16.720
saw, where we had three different tools for

6
00:00:16.720 --> 00:00:20.660
getting person, getting person, and change colour.

7
00:00:20.660 --> 00:00:23.620
And what I have done over in this

8
00:00:23.620 --> 00:00:30.280
version is I have removed these information printouts

9
00:00:30.280 --> 00:00:33.480
that tools are being called, because that is

10
00:00:33.480 --> 00:00:34.580
too cumbersome to do.

11
00:00:34.940 --> 00:00:37.120
We have middleware to do that for us.

12
00:00:38.740 --> 00:00:43.220
And middleware is a method called middleware, or

13
00:00:43.220 --> 00:00:45.000
whatever you call it, you can choose.

14
00:00:45.720 --> 00:00:48.340
And it takes an input of the agent

15
00:00:48.340 --> 00:00:51.560
that are using the middleware, or about to

16
00:00:51.560 --> 00:00:54.740
do the tool call, context about the tool

17
00:00:54.740 --> 00:00:58.260
call, and next objects.

18
00:00:58.700 --> 00:01:01.320
If you know middleware, you normally invoke these

19
00:01:01.320 --> 00:01:04.480
if you want to proceed, and not invoke

20
00:01:04.480 --> 00:01:07.860
it if you don't, and then some cancellations.

21
00:01:09.160 --> 00:01:11.220
We can set a breakpoint in this.

22
00:01:11.820 --> 00:01:15.040
In order to enable this middleware, we have

23
00:01:15.040 --> 00:01:17.860
changed our agent slightly, and that we have

24
00:01:17.860 --> 00:01:22.580
set as builder on the agent, use middleware

25
00:01:22.580 --> 00:01:27.820
.build. If we do this, the agent goes

26
00:01:27.820 --> 00:01:29.960
from a chat client agent to an AI

27
00:01:29.960 --> 00:01:32.880
agent, which is a more generic agent.

28
00:01:33.480 --> 00:01:37.240
This has some negative effects later on, but

29
00:01:37.240 --> 00:01:39.200
for now, it doesn't matter for us.

30
00:01:43.260 --> 00:01:47.500
And the negative thing is that structured output

31
00:01:47.500 --> 00:01:50.380
becomes a bit more cumbersome to do, so

32
00:01:50.380 --> 00:01:52.880
it's not a total loss, but it is

33
00:01:52.880 --> 00:01:54.420
a little annoying that it changes.

34
00:01:56.240 --> 00:02:00.120
But if we do this, we can now

35
00:02:00.120 --> 00:02:07.240
run, and we can ask about the different

36
00:02:07.240 --> 00:02:08.000
people's colours.

37
00:02:08.600 --> 00:02:12.960
So if we ask about Mona's favourite colour,

38
00:02:14.360 --> 00:02:18.200
we hit this breakpoint, so we get that

39
00:02:18.200 --> 00:02:20.460
we have an agent, and we get some

40
00:02:20.460 --> 00:02:23.500
context about what we are about to call

41
00:02:23.500 --> 00:02:23.740
here.

42
00:02:24.200 --> 00:02:28.780
In this case, it's the tool called getPerson,

43
00:02:29.520 --> 00:02:34.420
and the argument is this case, as we

44
00:02:34.420 --> 00:02:35.400
expect, Mona.

45
00:02:36.260 --> 00:02:38.580
So here, I'm just building up a small

46
00:02:38.580 --> 00:02:42.160
little output message, so I don't need to

47
00:02:42.160 --> 00:02:43.740
put it into every single tool.

48
00:02:44.200 --> 00:02:47.740
And then I'm calling next to let the

49
00:02:47.740 --> 00:02:49.620
tool be called.

50
00:02:51.140 --> 00:02:52.980
You can do the same for Ben.

51
00:02:57.300 --> 00:03:00.520
We will hit this, and our argument will

52
00:03:00.520 --> 00:03:03.360
instead be Ben down here.

53
00:03:06.500 --> 00:03:08.580
And finally, we can do it for John,

54
00:03:12.780 --> 00:03:14.860
and see that his colour is red.

55
00:03:16.120 --> 00:03:20.800
So that is the best way of telling

56
00:03:20.800 --> 00:03:21.540
your system.

57
00:03:21.880 --> 00:03:23.880
Right now, we are doing consoles, but it

58
00:03:23.880 --> 00:03:25.660
could be to a lock that this and

59
00:03:25.660 --> 00:03:27.440
this tool was called with this and this

60
00:03:27.440 --> 00:03:31.260
information, instead of putting them into the tools

61
00:03:31.260 --> 00:03:31.640
themselves.

62
00:03:35.400 --> 00:03:36.900
But we can do more than this in

63
00:03:36.900 --> 00:03:37.120
here.

64
00:03:37.320 --> 00:03:42.740
We can also manipulate and even refuse to

65
00:03:42.740 --> 00:03:43.380
call the tool.

66
00:03:43.860 --> 00:03:45.680
So I just put in a bit of

67
00:03:45.680 --> 00:03:47.960
code here, and I will set a few

68
00:03:47.960 --> 00:03:48.820
breakpoints here.

69
00:03:49.640 --> 00:03:52.120
We'll see in one second what these do.

70
00:03:54.540 --> 00:03:58.060
So we'll start up again, and we'll again

71
00:03:58.060 --> 00:04:00.820
ask for Mona's favourite colour.

72
00:04:01.700 --> 00:04:03.300
And Mona is not involved in any of

73
00:04:03.300 --> 00:04:05.140
the new code I put in, so she

74
00:04:05.140 --> 00:04:08.380
can still go through without any problems.

75
00:04:12.140 --> 00:04:17.839
If we take Ben, we actually have set

76
00:04:17.839 --> 00:04:20.680
up that if it's getPerson, and the argument

77
00:04:20.680 --> 00:04:25.500
value is Ben, we want to change on

78
00:04:25.500 --> 00:04:26.800
the fly what's going on.

79
00:04:27.160 --> 00:04:30.060
So we will hard code here that his

80
00:04:30.060 --> 00:04:31.920
favourite colour is orange with a hint of

81
00:04:31.920 --> 00:04:35.060
black stripes, which is certainly not the real

82
00:04:35.060 --> 00:04:36.160
one, which was green.

83
00:04:37.100 --> 00:04:39.300
So we can on the fly change things

84
00:04:39.300 --> 00:04:39.540
here.

85
00:04:39.640 --> 00:04:42.900
We can just return what the tool should

86
00:04:42.900 --> 00:04:43.500
give back.

87
00:04:43.780 --> 00:04:46.360
We can on the fly change some of

88
00:04:46.360 --> 00:04:49.220
the argument values and the names.

89
00:04:49.440 --> 00:04:52.200
So you can inspect if things look slightly

90
00:04:52.200 --> 00:04:54.680
wrong, and you can on the fly change

91
00:04:54.680 --> 00:04:54.920
it.

92
00:04:55.120 --> 00:04:56.000
You can do that.

93
00:04:56.000 --> 00:05:00.560
So you can essentially just intercept and change

94
00:05:00.560 --> 00:05:04.540
whatever you want in the tool code, because

95
00:05:04.540 --> 00:05:09.240
if we do this, it now says, okay,

96
00:05:09.740 --> 00:05:11.020
we went a bit too long.

97
00:05:19.000 --> 00:05:21.400
So now it comes back and says Ben's

98
00:05:21.400 --> 00:05:23.400
favourite colour is orange with a hint of

99
00:05:23.400 --> 00:05:24.060
black stripes.

100
00:05:24.840 --> 00:05:30.000
So before the internal timeout of the tool

101
00:05:30.000 --> 00:05:32.760
calling inside Agent Framework happened, and it said,

102
00:05:32.940 --> 00:05:34.560
oh, something must be wrong.

103
00:05:34.700 --> 00:05:35.700
It didn't come back.

104
00:05:38.680 --> 00:05:42.020
What happens if we ask about John's favourite

105
00:05:42.020 --> 00:05:42.320
colour?

106
00:05:43.640 --> 00:05:47.080
In this case, I put in that John's

107
00:05:47.080 --> 00:05:50.420
data is secret, so we can either throw

108
00:05:50.420 --> 00:05:53.620
back null, or we can throw back an

109
00:05:53.620 --> 00:05:54.060
exception.

110
00:05:55.580 --> 00:05:59.880
And what we will see happen is that

111
00:05:59.880 --> 00:06:01.480
it says it can't get it.

112
00:06:01.580 --> 00:06:04.800
And this was not because of the exception

113
00:06:04.800 --> 00:06:06.440
or the timeout.

114
00:06:07.040 --> 00:06:09.840
We can try again to see what happens.

115
00:06:11.440 --> 00:06:13.020
What is John's favourite colour?

116
00:06:14.120 --> 00:06:16.360
You will often see it tries it twice.

117
00:06:16.360 --> 00:06:19.820
This is a retry mechanism inside the Agent

118
00:06:19.820 --> 00:06:24.180
Framework that it says, oh, something could have

119
00:06:24.180 --> 00:06:25.480
had a hiccup.

120
00:06:25.680 --> 00:06:26.660
Let me do it twice.

121
00:06:27.240 --> 00:06:30.240
But it won't be let through because I'm

122
00:06:30.240 --> 00:06:31.600
throwing the exception every time.

123
00:06:33.920 --> 00:06:36.460
So this is how tool calling works.

124
00:06:36.580 --> 00:06:40.060
There are other tool middlewares that you add

125
00:06:40.060 --> 00:06:43.180
in the same manner, which we'll cover in

126
00:06:43.180 --> 00:06:44.580
later sessions.

127
00:06:46.360 --> 00:06:46.640
Thank you.