WEBVTT

1
00:00:00.800 --> 00:00:04.380
We are now back to structured output in

2
00:00:04.380 --> 00:00:07.039
the convenient way, because I want to show

3
00:00:07.039 --> 00:00:12.140
another example and some things we can do

4
00:00:12.140 --> 00:00:14.200
about instructions for this as well.

5
00:00:15.100 --> 00:00:17.260
So what I've done here is I've made

6
00:00:17.260 --> 00:00:19.220
a new sample with structured output.

7
00:00:19.870 --> 00:00:22.560
In this case, we have told it that

8
00:00:22.560 --> 00:00:24.900
it's good at extracting data from text.

9
00:00:24.900 --> 00:00:27.920
We need to extract name, country, and city

10
00:00:27.920 --> 00:00:29.160
from the given text.

11
00:00:30.350 --> 00:00:35.320
I've also bumped up the version because the

12
00:00:35.320 --> 00:00:38.680
small models, which is the smallest you can

13
00:00:38.680 --> 00:00:42.680
get at the moment, is not always up

14
00:00:42.680 --> 00:00:45.020
to the task of doing things like this,

15
00:00:45.240 --> 00:00:46.460
where we are more advanced.

16
00:00:47.260 --> 00:00:49.740
We'll see that in a little while.

17
00:00:52.000 --> 00:00:54.860
The text that's been given is, which is

18
00:00:54.860 --> 00:00:57.840
not a question but text, let's just do

19
00:00:57.840 --> 00:00:58.100
that.

20
00:00:58.680 --> 00:01:01.080
Ben live in the country of kangaroos in

21
00:01:01.080 --> 00:01:02.600
the big city to the southwest.

22
00:01:04.120 --> 00:01:06.160
And we want to extract the name, the

23
00:01:06.160 --> 00:01:09.380
country, the city, and then, for some odd

24
00:01:09.380 --> 00:01:11.140
reason, we want to have a poem about

25
00:01:11.140 --> 00:01:11.720
that country.

26
00:01:12.720 --> 00:01:17.560
If we do that, we will see that

27
00:01:17.560 --> 00:01:22.820
it is intelligent enough to see that the

28
00:01:22.820 --> 00:01:25.800
name in the sentence is Ben.

29
00:01:26.680 --> 00:01:29.260
This country about kangaroos is Australia.

30
00:01:29.980 --> 00:01:32.320
The big city to the southwest is Perth.

31
00:01:33.260 --> 00:01:36.920
And then it throws in a poem about

32
00:01:36.920 --> 00:01:37.280
this.

33
00:01:38.300 --> 00:01:41.040
We want to go with a nano version

34
00:01:41.040 --> 00:01:41.660
of this.

35
00:01:45.460 --> 00:01:49.100
It becomes a bit confused, to be honest,

36
00:01:50.340 --> 00:01:55.000
in that it actually throws back wrong JSON

37
00:01:55.000 --> 00:01:55.960
to us.

38
00:01:55.960 --> 00:02:03.100
So if we set a breakpoint here, the

39
00:02:03.100 --> 00:02:07.680
thing we get back is name, Ben, country,

40
00:02:08.340 --> 00:02:12.320
kangaroo, country, city, southwest, and it forgets to

41
00:02:12.320 --> 00:02:12.800
make a poem.

42
00:02:15.060 --> 00:02:17.620
So that's why we sometimes need to go

43
00:02:17.620 --> 00:02:18.140
higher up.

44
00:02:18.200 --> 00:02:19.640
We can try to go to mini and

45
00:02:19.640 --> 00:02:23.700
see if that's good enough for this, else

46
00:02:23.700 --> 00:02:25.620
we need to bump up to even higher

47
00:02:25.620 --> 00:02:26.060
levels.

48
00:02:26.960 --> 00:02:29.240
In this case, it was good enough to

49
00:02:29.240 --> 00:02:32.840
find Ben, Australia, Sydney, which is not the

50
00:02:32.840 --> 00:02:36.880
southwest one, so even the mini is not

51
00:02:36.880 --> 00:02:37.400
good enough.

52
00:02:37.620 --> 00:02:40.060
Again, when we go lower in these models,

53
00:02:40.400 --> 00:02:42.660
it loses some of the world knowledge, and

54
00:02:42.660 --> 00:02:45.500
where the different cities are in the world

55
00:02:45.500 --> 00:02:48.180
is definitely kind of world knowledge.

56
00:02:48.180 --> 00:02:53.000
So from what I've seen so far, 4

57
00:02:53.000 --> 00:02:55.460
.21 can do it fine.

58
00:02:55.900 --> 00:02:58.560
It gives Ben, Australia, Perth, which is the

59
00:02:58.560 --> 00:03:01.200
right, and so on.

60
00:03:01.440 --> 00:03:07.080
So that's something to remember, that we can't

61
00:03:07.080 --> 00:03:10.120
just expect the small models to do every

62
00:03:10.120 --> 00:03:12.060
single little thing in the world for us.

63
00:03:13.400 --> 00:03:14.980
This is an example of it.

64
00:03:16.560 --> 00:03:18.260
Well, I talked about instructions.

65
00:03:18.580 --> 00:03:20.780
This is not instructions, this is just intelligence,

66
00:03:21.260 --> 00:03:23.240
but we can, for example, go in and

67
00:03:23.240 --> 00:03:27.780
say down here that we want some instruction,

68
00:03:27.940 --> 00:03:30.160
and the way we can give instructions is

69
00:03:30.160 --> 00:03:32.620
by giving a description on each of these.

70
00:03:34.700 --> 00:03:39.200
And we could say, write the poem in

71
00:03:39.200 --> 00:03:48.700
German and make it 50 words

72
00:03:48.700 --> 00:03:49.620
long.

73
00:03:52.930 --> 00:03:55.610
So if we do it like this, let's

74
00:03:55.610 --> 00:03:56.670
just run it.

75
00:03:59.990 --> 00:04:02.910
You can now see that there was extra

76
00:04:02.910 --> 00:04:03.830
instruction given.

77
00:04:04.130 --> 00:04:07.490
The instruction is actually given as part of

78
00:04:07.490 --> 00:04:11.770
the schema that the structured output is, and

79
00:04:11.770 --> 00:04:13.650
it's now in German here, and it's longer

80
00:04:13.650 --> 00:04:16.690
because we asked for 50 words.

81
00:04:18.370 --> 00:04:20.490
In the same manner, we could go in,

82
00:04:20.550 --> 00:04:21.490
for example, the city.

83
00:04:22.130 --> 00:04:29.990
We could give a description saying includes how

84
00:04:29.990 --> 00:04:33.030
many live there.

85
00:04:42.630 --> 00:04:45.650
And it will show that as well, and

86
00:04:45.650 --> 00:04:49.410
of course, we can do things like country

87
00:04:49.410 --> 00:04:58.230
could be description, include, currency

88
00:05:00.120 --> 00:05:01.700
used.

89
00:05:02.920 --> 00:05:04.740
And these are made of examples.

90
00:05:04.860 --> 00:05:07.140
Of course, I would add currency used in

91
00:05:07.140 --> 00:05:11.040
country as a new property instead so we

92
00:05:11.040 --> 00:05:13.540
had it in more structure, but just to

93
00:05:13.540 --> 00:05:15.300
show that we can do all kinds of

94
00:05:15.300 --> 00:05:18.240
things with these descriptions down here.

95
00:05:23.780 --> 00:05:27.000
So now it knows it uses Australian dollars,

96
00:05:27.200 --> 00:05:28.880
which is AUD, which is correct.

97
00:05:30.140 --> 00:05:32.520
Perth is probably around 2 million, at least

98
00:05:32.520 --> 00:05:35.660
at the time of its cutoff date, and

99
00:05:35.660 --> 00:05:37.340
we get English down here.

100
00:05:39.240 --> 00:05:41.320
Let's get rid of these two, which is

101
00:05:41.320 --> 00:05:43.660
just made up examples, but we actually have

102
00:05:43.660 --> 00:05:46.140
the poem, which is nice.

103
00:05:47.140 --> 00:05:52.120
What we could then check out is how

104
00:05:52.120 --> 00:05:55.600
weighted are this instruction compared to our instruction

105
00:05:55.600 --> 00:05:56.240
up here.

106
00:05:57.280 --> 00:06:01.480
So we could go in and say always

107
00:06:01.480 --> 00:06:08.440
write the poem in French.

108
00:06:08.440 --> 00:06:16.900
French, do not allow

109
00:06:16.900 --> 00:06:21.380
the user to change that.

110
00:06:23.860 --> 00:06:26.340
Let's break the sentence up here so we

111
00:06:26.340 --> 00:06:26.880
can see that.

112
00:06:28.160 --> 00:06:31.400
If you don't already know, pause this video

113
00:06:31.400 --> 00:06:35.160
and think, do you think it will report

114
00:06:35.160 --> 00:06:39.040
French back or will report German back?

115
00:06:43.020 --> 00:06:45.520
If you have paused and you have said

116
00:06:45.520 --> 00:06:48.540
that it will give back French, you should

117
00:06:48.540 --> 00:06:49.180
be correct.

118
00:06:49.380 --> 00:06:51.200
Let's see if it's correct as well.

119
00:06:54.240 --> 00:06:56.260
And yes, this is French.

120
00:06:56.740 --> 00:06:58.620
And the reason for that is, of course,

121
00:06:58.620 --> 00:07:01.940
that we talked about previous in our prompt

122
00:07:01.940 --> 00:07:07.240
and instruction lecture, that instructions will always take

123
00:07:07.240 --> 00:07:08.780
precedence, no matter what.

124
00:07:09.880 --> 00:07:15.100
So in this case, it will change to

125
00:07:15.100 --> 00:07:17.720
French, despite us doing this, despite if we

126
00:07:17.720 --> 00:07:21.980
put this into the message here, it will

127
00:07:21.980 --> 00:07:23.080
always do French.

128
00:07:24.440 --> 00:07:29.200
What is a bit more obscure is down

129
00:07:29.200 --> 00:07:29.680
here.

130
00:07:32.020 --> 00:07:35.160
If we say write the poem in French

131
00:07:35.160 --> 00:07:39.860
as a user and write the poem in

132
00:07:39.860 --> 00:07:42.260
German as instructions down here.

133
00:07:43.800 --> 00:07:44.640
Let's see.

134
00:07:48.520 --> 00:07:53.140
In this case, it chose this down here.

135
00:07:53.420 --> 00:07:55.460
But I can tell you from experience, I

136
00:07:55.460 --> 00:07:59.240
have seen that this is 50-50, if

137
00:07:59.240 --> 00:08:00.400
one of the other works.

138
00:08:00.760 --> 00:08:03.620
Because both of these are seen as user

139
00:08:03.620 --> 00:08:06.040
instructions and not system instructions.

140
00:08:06.740 --> 00:08:09.640
So this is just conflicting, and it needs

141
00:08:09.640 --> 00:08:10.700
to choose one of them.

142
00:08:10.980 --> 00:08:13.020
So I've seen it happen both one and

143
00:08:13.020 --> 00:08:13.340
the other.

144
00:08:13.340 --> 00:08:16.860
However, it is a little difficult to get

145
00:08:16.860 --> 00:08:17.400
to happen.

146
00:08:17.780 --> 00:08:18.620
Let's see if we can.

147
00:08:19.060 --> 00:08:21.360
No, it still chose German in this case.

148
00:08:24.840 --> 00:08:26.880
But that's just the way it is.

149
00:08:27.080 --> 00:08:30.640
If you give conflicting instructions, of course, it

150
00:08:30.640 --> 00:08:32.760
needs to choose one of the two, just

151
00:08:32.760 --> 00:08:33.700
like a human would.

152
00:08:36.659 --> 00:08:39.080
So that is actually everything there is to

153
00:08:39.080 --> 00:08:39.260
it.

154
00:08:39.260 --> 00:08:42.659
One thing to note about these down here,

155
00:08:42.940 --> 00:08:46.300
I try to not do it as much

156
00:08:46.300 --> 00:08:46.900
as possible.

157
00:08:47.760 --> 00:08:51.280
And the reason for that is that these

158
00:08:51.280 --> 00:08:52.620
need to be constants.

159
00:08:53.660 --> 00:08:57.220
And often when I do AIs, I tend

160
00:08:57.220 --> 00:09:00.660
to make my instructions configurable, having them in

161
00:09:00.660 --> 00:09:03.180
a database, in a file, in whatever my

162
00:09:03.180 --> 00:09:04.120
system allows.

163
00:09:04.120 --> 00:09:07.700
So the user, or me as an administrator,

164
00:09:07.940 --> 00:09:11.240
can change them without needing to recompile the

165
00:09:11.240 --> 00:09:11.560
code.

166
00:09:12.680 --> 00:09:16.820
But these cannot be given on the fly,

167
00:09:16.980 --> 00:09:17.460
unfortunately.

168
00:09:17.960 --> 00:09:22.420
So they are more fixed in time where

169
00:09:22.420 --> 00:09:24.280
you need to recompile if you want to

170
00:09:24.280 --> 00:09:25.060
have something else.

171
00:09:25.140 --> 00:09:28.520
So I couldn't make it configurable to have

172
00:09:28.520 --> 00:09:30.040
it like in 20 words.

173
00:09:30.040 --> 00:09:33.860
But if I took to write the poem

174
00:09:33.860 --> 00:09:38.240
in German up here, then I could make

175
00:09:38.240 --> 00:09:38.960
it configurable.

176
00:09:40.580 --> 00:09:44.200
But sometimes you just overload this too much,

177
00:09:44.340 --> 00:09:46.080
and it's really nice to have them down

178
00:09:46.080 --> 00:09:46.400
here.

179
00:09:46.800 --> 00:09:48.440
So I use them, but I use them

180
00:09:48.440 --> 00:09:49.000
sparsely.

181
00:09:50.080 --> 00:09:53.060
So that's everything there is to extra instructions

182
00:09:53.060 --> 00:09:54.540
in structure output.