WEBVTT

1
00:00:01.100 --> 00:00:03.440
Hi, and welcome to this AI in C#

2
00:00:03.440 --> 00:00:05.960
video on the Microsoft Agent Framework.

3
00:00:06.560 --> 00:00:10.020
Today we're going to look into OpenTelemetry, where

4
00:00:10.020 --> 00:00:12.460
we'll be looking into the console and application

5
00:00:12.460 --> 00:00:13.120
insights.

6
00:00:14.260 --> 00:00:16.820
And in a separate video, we're going to

7
00:00:16.820 --> 00:00:19.800
look at it on .NET Aspire's dashboard as

8
00:00:19.800 --> 00:00:20.160
well.

9
00:00:21.340 --> 00:00:23.660
A little disclaimer, I'm not too good at

10
00:00:23.660 --> 00:00:28.860
OpenTelemetry, so those of you who are experts

11
00:00:28.860 --> 00:00:31.920
in it, sorry in advance, but at least

12
00:00:31.920 --> 00:00:33.420
this is how you get going.

13
00:00:34.700 --> 00:00:38.940
Because OpenTelemetry have a standard, more or less,

14
00:00:39.240 --> 00:00:40.000
for AI.

15
00:00:40.820 --> 00:00:45.160
It's called gen underscore AI, and the Agent

16
00:00:45.160 --> 00:00:48.840
Framework follows this whenever you set up OpenTelemetry.

17
00:00:49.140 --> 00:00:50.340
So let's go in and see that.

18
00:00:53.680 --> 00:00:55.720
So here we are in the system, and

19
00:00:55.720 --> 00:00:59.460
we are in the telemetry example here.

20
00:01:00.440 --> 00:01:03.959
And what we need to do is, you

21
00:01:03.959 --> 00:01:11.280
can see, these already have OpenTelemetry as a sub-nuget

22
00:01:11.280 --> 00:01:13.040
packages, so we don't need to put anything

23
00:01:13.040 --> 00:01:14.160
extra in here.

24
00:01:15.140 --> 00:01:17.300
But what we need to do is we

25
00:01:17.300 --> 00:01:19.600
need to make a source, a trace provider,

26
00:01:21.040 --> 00:01:24.800
in order to trace whatever happens with every

27
00:01:24.800 --> 00:01:25.100
AI.

28
00:01:25.880 --> 00:01:27.120
For that, we need to give it a

29
00:01:27.120 --> 00:01:29.020
name, so we just give it, in my

30
00:01:29.020 --> 00:01:31.060
case, AI source here.

31
00:01:32.140 --> 00:01:35.580
And then we are going to use OpenTelemetry,

32
00:01:35.960 --> 00:01:38.560
where we add a source, and in my

33
00:01:38.560 --> 00:01:41.660
case, we also add a console exporter, so

34
00:01:41.660 --> 00:01:47.040
we can export the various data out to

35
00:01:47.040 --> 00:01:47.500
the console.

36
00:01:48.840 --> 00:01:51.680
And then I have an application inside set

37
00:01:51.680 --> 00:01:51.880
up.

38
00:01:52.820 --> 00:01:55.240
So I have an application inside set up

39
00:01:55.240 --> 00:01:55.460
here.

40
00:01:57.270 --> 00:02:00.230
Quickly grab this before we move on.

41
00:02:03.340 --> 00:02:06.420
And I'm using one here, and I have

42
00:02:06.420 --> 00:02:09.780
an instrumentation key that is in my user

43
00:02:09.780 --> 00:02:10.200
secrets.

44
00:02:11.660 --> 00:02:13.080
If you don't have that, you can still

45
00:02:13.080 --> 00:02:15.800
run the application, but it won't log to

46
00:02:15.800 --> 00:02:16.340
Azure then.

47
00:02:17.540 --> 00:02:19.400
Then we build this trace provider.

48
00:02:21.040 --> 00:02:25.100
So we have something that can be traced

49
00:02:25.100 --> 00:02:28.700
and logged to two different, or exported to

50
00:02:28.700 --> 00:02:29.480
two different sources.

51
00:02:29.980 --> 00:02:31.840
And the only thing we need to do,

52
00:02:32.060 --> 00:02:35.000
rather than the normal things we do, is

53
00:02:35.000 --> 00:02:37.960
we need to just say, as builder, add

54
00:02:37.960 --> 00:02:40.720
the middleware of OpenTelemetry, where we give the

55
00:02:40.720 --> 00:02:41.220
source name.

56
00:02:41.700 --> 00:02:44.180
And then we need to choose if we

57
00:02:44.180 --> 00:02:47.380
want to enable sensitive data or not, meaning

58
00:02:47.380 --> 00:02:50.380
should the actual answers back and forward be

59
00:02:50.380 --> 00:02:52.580
there, or should it only be something about

60
00:02:52.580 --> 00:02:55.920
what is the model, and how many tokens

61
00:02:55.920 --> 00:02:58.300
was used, and such things like that, which

62
00:02:58.300 --> 00:02:59.100
is non-sensitive.

63
00:03:00.640 --> 00:03:03.520
Microsoft recommends that you don't put this on

64
00:03:03.520 --> 00:03:08.440
in production, of course, but they can't stop

65
00:03:08.440 --> 00:03:09.480
you in doing it.

66
00:03:09.580 --> 00:03:13.940
So if it's for training or anything, you

67
00:03:13.940 --> 00:03:14.800
might do this.

68
00:03:14.800 --> 00:03:18.000
But beyond that, I'm doing nothing special.

69
00:03:18.420 --> 00:03:19.900
The only thing I'm doing here is my

70
00:03:19.900 --> 00:03:22.860
responses are coming out in yellow here.

71
00:03:24.400 --> 00:03:27.360
And we might actually go to red, so

72
00:03:27.360 --> 00:03:29.020
it's even easier to see where they are,

73
00:03:29.120 --> 00:03:33.820
because as I have the telemetry console I

74
00:03:33.820 --> 00:03:36.260
exported here, a lot of extra data will

75
00:03:36.260 --> 00:03:39.400
come out, despite our three texts here.

76
00:03:40.080 --> 00:03:42.720
So I'm telling it my name.

77
00:03:42.720 --> 00:03:45.700
I'm asking for the capital of France, and

78
00:03:45.700 --> 00:03:49.160
I'm asking again what my name was, just

79
00:03:49.160 --> 00:03:49.980
in a normal thread.

80
00:03:51.580 --> 00:03:55.940
So let's set a breakpoint and get this

81
00:03:55.940 --> 00:03:56.540
going.

82
00:04:02.770 --> 00:04:08.130
We are on the wrong project here, so

83
00:04:08.130 --> 00:04:09.810
let's jump to that one instead.

84
00:04:11.410 --> 00:04:15.050
And let's get this side-by-side like

85
00:04:15.050 --> 00:04:15.390
we do.

86
00:04:17.620 --> 00:04:22.290
So we set up the trace, and in

87
00:04:22.290 --> 00:04:25.050
my case the Azure Monitor as well, or

88
00:04:25.050 --> 00:04:26.790
in the application inside that is called as

89
00:04:26.790 --> 00:04:26.950
well.

90
00:04:27.530 --> 00:04:28.330
So that's ready.

91
00:04:28.570 --> 00:04:31.030
We build our normal agents and hook it

92
00:04:31.030 --> 00:04:31.270
up.

93
00:04:31.930 --> 00:04:35.270
And in my case, I'm using the extra

94
00:04:35.270 --> 00:04:40.550
data out, and then I'm making a thread

95
00:04:40.550 --> 00:04:43.210
and do the first request.

96
00:04:43.210 --> 00:04:45.050
And as you saw, a lot of things

97
00:04:45.050 --> 00:04:47.350
came out, and then our answer.

98
00:04:48.790 --> 00:04:51.450
Second request, a lot of things came out,

99
00:04:52.030 --> 00:04:53.870
our answer, and the third one.

100
00:04:55.070 --> 00:04:56.070
So what's in here?

101
00:04:57.210 --> 00:04:59.810
So we see a bunch of trace IDs,

102
00:04:59.970 --> 00:05:03.150
band ID, all kinds of things of what

103
00:05:03.150 --> 00:05:05.350
this was that happened.

104
00:05:05.490 --> 00:05:07.550
In this case, it was an invoke agent

105
00:05:07.550 --> 00:05:11.750
event, and it had some tags about it.

106
00:05:11.750 --> 00:05:12.370
It was a chat.

107
00:05:12.450 --> 00:05:16.130
It was the GPT model that I used.

108
00:05:16.210 --> 00:05:18.510
It was OpenAI from this address.

109
00:05:19.870 --> 00:05:22.850
And in my case, the input message came

110
00:05:22.850 --> 00:05:26.510
in, and the output message came out as

111
00:05:26.510 --> 00:05:26.710
well.

112
00:05:27.330 --> 00:05:29.430
We had the agent and an ID for

113
00:05:29.430 --> 00:05:29.890
the agent.

114
00:05:31.670 --> 00:05:33.250
And then a little more down here with

115
00:05:33.250 --> 00:05:34.370
some of the same data.

116
00:05:36.630 --> 00:05:40.210
And essentially the same with some different input

117
00:05:40.210 --> 00:05:41.830
and output in this case.

118
00:05:43.370 --> 00:05:46.470
And this is what we get also sent

119
00:05:46.470 --> 00:05:47.590
up into Azure.

120
00:05:48.810 --> 00:05:50.350
And what we can do up in Azure,

121
00:05:50.950 --> 00:05:53.210
we can go in.

122
00:05:53.990 --> 00:05:56.370
Azure don't really have our application inside, don't

123
00:05:56.370 --> 00:05:58.450
really have anything as far as I know

124
00:05:58.450 --> 00:06:03.610
for showing GNI data, but we can make

125
00:06:03.610 --> 00:06:05.770
our own logs and our own workbooks if

126
00:06:05.770 --> 00:06:06.390
we want to.

127
00:06:09.570 --> 00:06:12.570
So all the data goes into a table

128
00:06:12.570 --> 00:06:15.650
called dependencies, and most of it go into

129
00:06:15.650 --> 00:06:18.710
something called custom dimensions, which is a JSON.

130
00:06:18.990 --> 00:06:22.130
So I'm just projecting that out using KQL.

131
00:06:23.550 --> 00:06:28.090
And if I run this, I need to

132
00:06:28.090 --> 00:06:31.070
not stand on, so it thinks it's only

133
00:06:31.070 --> 00:06:32.930
a sub layer part of it.

134
00:06:33.670 --> 00:06:36.010
We should be able to see our agent

135
00:06:36.010 --> 00:06:39.890
here where we asked, what is my name?

136
00:06:41.470 --> 00:06:43.390
So that was the last one.

137
00:06:43.970 --> 00:06:46.390
So this comes in descending order.

138
00:06:47.650 --> 00:06:49.830
The second one was the thing about France.

139
00:06:51.090 --> 00:06:57.880
And had we not set the sensitive data,

140
00:06:58.100 --> 00:07:00.120
input and output data would have been empty,

141
00:07:00.520 --> 00:07:01.860
and the rest would have been there.

142
00:07:03.300 --> 00:07:07.520
So I will put this in the description

143
00:07:07.520 --> 00:07:10.680
of the video, so you can grab this

144
00:07:10.680 --> 00:07:12.880
and use it in your own logs.

145
00:07:13.500 --> 00:07:18.600
Or you can use, since OpenTelemetry is open,

146
00:07:18.660 --> 00:07:20.980
as the name implies, you can use it

147
00:07:20.980 --> 00:07:22.140
in other places.

148
00:07:24.720 --> 00:07:26.480
But this is how you do it up

149
00:07:26.480 --> 00:07:28.000
in Application Insights.

150
00:07:28.100 --> 00:07:31.700
In Azure Finder, you have direct integration, and

151
00:07:31.700 --> 00:07:36.760
in Aspire Dashboards, you can get them there

152
00:07:36.760 --> 00:07:37.180
as well.

153
00:07:38.720 --> 00:07:42.100
But that was actually everything for this one.

154
00:07:43.100 --> 00:07:44.640
It's fairly simple to set up.

155
00:07:44.960 --> 00:07:47.600
You can, of course, add logging with normal

156
00:07:47.600 --> 00:07:51.120
logging and so on, but that's normal OpenTelemetry

157
00:07:51.120 --> 00:07:53.620
and not really anything to do with AI

158
00:07:53.620 --> 00:07:54.240
as such.

159
00:07:55.940 --> 00:07:57.640
So with that, see you on the next

160
00:07:57.640 --> 00:07:57.820
one.