WEBVTT 00:00.600 --> 00:02.700 -: Hey guys, gonna take you through the worksheet 00:02.700 --> 00:05.310 that you can use to apply the prompt engineering principles. 00:05.310 --> 00:08.100 So this is something I use as a checklist. 00:08.100 --> 00:09.870 I start with my naive prompt, 00:09.870 --> 00:13.770 something that is relatively simple that I'm happy with, 00:13.770 --> 00:16.830 it gets decent results, and then I start to iterate. 00:16.830 --> 00:18.615 And this is only really important to do 00:18.615 --> 00:21.390 if you're gonna be using this prompt again and again. 00:21.390 --> 00:22.980 If it's just a throwaway thing, 00:22.980 --> 00:25.320 then you know you're doing it for one task, 00:25.320 --> 00:28.170 just talk to chatGPT, do a couple of iterations. 00:28.170 --> 00:30.150 This is really, if it's a serious prompt 00:30.150 --> 00:31.860 that you're gonna use multiple times 00:31.860 --> 00:33.120 and you need to optimize it. 00:33.120 --> 00:35.160 So start with the naive prompt. 00:35.160 --> 00:38.040 And in this case, the prompt is, 00:38.040 --> 00:41.370 can I have a list of product names? 00:41.370 --> 00:43.560 In this case, the prompt is, can I have a list 00:43.560 --> 00:46.860 of product names for a pair of shoes that fit any shoe size? 00:46.860 --> 00:49.050 So I've come up with this amazing product, 00:49.050 --> 00:51.090 a pair of shoes that can fit any shoe size, 00:51.090 --> 00:54.030 and I'm trying to brainstorm product names. 00:54.030 --> 00:55.470 And see here that there are a couple 00:55.470 --> 00:57.300 of things wrong with this prompt. 00:57.300 --> 01:00.990 First of all, it gives us this boilerplate text 01:00.990 --> 01:02.970 at the beginning that we don't really want. 01:02.970 --> 01:07.110 Then we have an arbitrating list, like it's numbered by 10, 01:07.110 --> 01:09.420 but sometimes you get comma separated, 01:09.420 --> 01:12.390 sometimes you get a different number of names. 01:12.390 --> 01:13.500 It really just depends. 01:13.500 --> 01:15.690 And this isn't really reliable enough 01:15.690 --> 01:18.270 if you're gonna be using this again and again. 01:18.270 --> 01:20.280 So if I'm just coming up with this one product name, 01:20.280 --> 01:21.180 that's one thing. 01:21.180 --> 01:23.370 But if I'm gonna be doing it for lots of different products, 01:23.370 --> 01:25.470 that's another and I need to make the prompt 01:25.470 --> 01:26.760 more robust, right? 01:26.760 --> 01:31.140 So let me copy and paste the response in there 01:31.140 --> 01:32.580 and that usually this is what I do. 01:32.580 --> 01:34.440 I keep track of what the responses are 01:34.440 --> 01:36.810 so I can see how they've changed. 01:36.810 --> 01:39.390 The next thing I've done here is I've just structured 01:39.390 --> 01:42.540 the prompt a little bit and in this case I've asked it 01:42.540 --> 01:46.230 for product names still, but I've given these seed words 01:46.230 --> 01:49.170 and seed words doesn't mean anything in particular. 01:49.170 --> 01:54.170 It's a magic word or anything for chatGPT, 01:54.270 --> 01:57.570 but it's just really a way for me to give a couple of words 01:57.570 --> 01:59.910 that I like and that it can go off, 01:59.910 --> 02:03.690 but I'm still leaving it open for creativity. 02:03.690 --> 02:05.190 So let's see what we get back. 02:06.570 --> 02:08.160 You can see it's recognized 02:08.160 --> 02:09.990 that I've given it some seed words 02:09.990 --> 02:13.410 and it's given me names that are closer to those seed words. 02:13.410 --> 02:15.480 So you can see I've had omni-fit in here 02:15.480 --> 02:18.420 that they came back with OmniFit Walkers. 02:18.420 --> 02:20.580 Just gonna paste that in. And that's already looking better. 02:20.580 --> 02:22.380 It's given me some direction. 02:22.380 --> 02:25.680 I can specify by changing the seed words, 02:25.680 --> 02:27.237 what type of names I like, 02:27.237 --> 02:30.054 and it's gonna gimme names of this sort. 02:30.054 --> 02:32.130 All right, so next thing I wanted to do 02:32.130 --> 02:33.630 is specify the format. 02:33.630 --> 02:36.930 And in this case it's unclear because I've just put a colon 02:36.930 --> 02:38.640 and it could put whatever it wants afterwards. 02:38.640 --> 02:40.170 So it's still giving me the numbers. 02:40.170 --> 02:41.430 But I want something like this. 02:41.430 --> 02:43.830 I want a comma separated list. 02:43.830 --> 02:47.492 What I've done here, I created a new session 02:47.492 --> 02:52.492 and I've just put in a previous example 02:52.680 --> 02:54.540 and I've shown the comma separated list 02:54.540 --> 02:56.370 in that previous example. 02:56.370 --> 02:57.930 You can see that we're still 02:57.930 --> 03:00.330 not really following the format. 03:00.330 --> 03:03.297 So we're gonna have to continue to improve that 03:03.297 --> 03:05.493 by adding another example. 03:06.390 --> 03:08.370 So I'm just gonna copy that in. 03:08.370 --> 03:10.350 And sometimes it works, sometimes it doesn't. 03:10.350 --> 03:12.660 The other path we can take here is rather than just 03:12.660 --> 03:15.300 showing it, we can actually specify and say, 03:15.300 --> 03:18.000 give me a comma separated list. 03:18.000 --> 03:20.310 But lemme just add this other example 03:20.310 --> 03:23.733 because that's the third principle is adding examples. 03:25.080 --> 03:27.750 And I find that the more examples you add, 03:27.750 --> 03:29.943 the better the results in general. 03:34.830 --> 03:36.153 Okay, in this case, it has decreased 03:36.153 --> 03:38.820 the number of names here, 03:38.820 --> 03:42.030 but it hasn't given me this specific 03:42.030 --> 03:44.880 comma separated list that I asked for. 03:44.880 --> 03:47.580 And so what I'm gonna do is I'm gonna edit that, 03:47.580 --> 03:51.730 say list product names separated, say brainstorm. 03:59.160 --> 04:04.160 And here we go. That's much. 04:04.830 --> 04:08.040 So what we're gonna do is we're gonna copy this prompt 04:08.040 --> 04:09.120 and we're gonna change it, 04:09.120 --> 04:13.930 but based on the previous plan that we had, again, 04:13.930 --> 04:16.533 we're gonna add that next one planning to as well. 04:18.173 --> 04:21.050 You can see that we iterate through this work time. Okay? 04:23.400 --> 04:27.450 So now we have our first really decent response 04:27.450 --> 04:30.930 that's more reliable and we don't wanna go over the top 04:30.930 --> 04:34.320 with examples because one, it takes some time 04:34.320 --> 04:37.740 and some costs for me to add to come up with new examples. 04:37.740 --> 04:42.030 But the other thing is that it's gonna take up token space. 04:42.030 --> 04:45.420 So if we're gonna put this into a product name generator 04:45.420 --> 04:48.900 type product, then if we add too many examples, 04:48.900 --> 04:50.910 that's gonna cost us more tokens when we're 04:50.910 --> 04:52.520 running it through the API. 04:52.520 --> 04:55.440 So that's another good reason to not, you know, 04:55.440 --> 04:56.850 add too many examples. 04:56.850 --> 04:58.320 The other thing is it can constrain 04:58.320 --> 05:00.060 the creativity a little bit. 05:00.060 --> 05:04.770 You can see here I have a lot of Apple type product names. 05:04.770 --> 05:07.950 It's iShake, iSmoothie, iShake mini. 05:07.950 --> 05:11.100 This is reminiscent of how Steve Jobs would name it. 05:11.100 --> 05:13.770 And it has learned a little bit from that. 05:13.770 --> 05:17.250 But if I put a lot of examples with the I in front of it, 05:17.250 --> 05:19.740 if I add additional examples, then it might not, 05:19.740 --> 05:22.290 it might just always provide that 05:22.290 --> 05:24.270 and it wouldn't come up with these more interesting names, 05:24.270 --> 05:25.773 like Versa Shoe or AnyFit. 05:26.730 --> 05:27.750 Okay, cool. 05:27.750 --> 05:31.738 Then the next thing we want to do is just see 05:31.738 --> 05:36.510 how many examples we need to give it for it to start 05:36.510 --> 05:41.180 very specifically using the I in front of every name. 05:41.180 --> 05:44.043 If we add a couple more products in here, 05:46.080 --> 05:48.960 we tried with two examples and it didn't give the name, 05:48.960 --> 05:53.960 but with I, we can say refrigerator that dispenses beer. 05:57.030 --> 06:02.030 And say like bar, say iBarFridge. 06:02.450 --> 06:03.617 iFridgeBeer, and iDrink Fridge. 06:12.030 --> 06:16.680 Right, now let me see if it now comes back with, 06:16.680 --> 06:17.700 yeah, here we go. 06:17.700 --> 06:22.353 By adding an additional example, we're starting to get, 06:23.190 --> 06:26.940 you know, more of the names are coming back with I thing. 06:26.940 --> 06:29.940 As you can see, it's starting to constrain the creativity 06:29.940 --> 06:32.880 and we want to test to see how often that happens. 06:32.880 --> 06:37.580 So if we run it out again, two outta three again, 06:37.580 --> 06:42.180 we run that again, it's two outta three again. 06:42.180 --> 06:43.143 Run that again. 06:45.450 --> 06:48.760 Okay, see now it's given us a bigger list of names 06:50.040 --> 06:52.350 and actually only one outta three. 06:52.350 --> 06:54.870 So we can start to test this. 06:54.870 --> 06:57.240 You see, this is the level where you start 06:57.240 --> 06:59.100 to get into production when you're working 06:59.100 --> 07:00.270 with prompt engineering. 07:00.270 --> 07:03.960 And it comes as something where it's really important 07:03.960 --> 07:06.270 to test how often it fails. 07:06.270 --> 07:08.160 And the failure case here could be, 07:08.160 --> 07:11.190 if we always want the name to be Steve Jobs style, 07:11.190 --> 07:13.650 then we might need to add more examples in order 07:13.650 --> 07:15.480 to make sure it always does this. 07:15.480 --> 07:18.159 We can see that we had just added one more example 07:18.159 --> 07:20.370 and it wasn't more reliable. 07:20.370 --> 07:23.340 We can run that again, but it wasn't reliable enough. 07:23.340 --> 07:26.661 And here we go, we just have one with the Steve Jobs, 07:26.661 --> 07:29.100 iFitShoe had the Steve Jobs style. 07:29.100 --> 07:31.770 So that's an important part of this. 07:31.770 --> 07:34.770 And when you start to use a prompt in production, 07:34.770 --> 07:36.960 like whether it's something that you're using daily 07:36.960 --> 07:38.790 for your work or something you're gonna 07:38.790 --> 07:40.140 incorporate into a product, 07:40.140 --> 07:43.560 this is the step that starts to get really important. 07:43.560 --> 07:45.510 Evaluating the quality. 07:45.510 --> 07:50.373 Lemme just go back and take the typical example here. 07:52.590 --> 07:54.750 Alright, so we're getting much, much closer. 07:54.750 --> 07:57.960 You can see we're working through and we can also test now 07:57.960 --> 07:59.280 whether this wording makes sense. 07:59.280 --> 08:00.897 We can test adding more examples 08:00.897 --> 08:04.050 and we can test changing the product data style 08:04.050 --> 08:05.610 as well if we want. 08:05.610 --> 08:08.850 Now the final one is dividing labor. 08:08.850 --> 08:11.850 And this is where we start to build a system 08:11.850 --> 08:15.570 or a chain rather than specifically one prompt. 08:15.570 --> 08:17.910 And we usually can't do everything at one prompt. 08:17.910 --> 08:19.830 Like I have a list of names now, 08:19.830 --> 08:21.390 but I'm not sure which ones are good. 08:21.390 --> 08:23.010 Normally a human would do that 08:23.010 --> 08:26.160 and we've already come up with three good names here. 08:26.160 --> 08:29.253 But yeah, we're going to put this extra prompt here. 08:29.253 --> 08:30.960 This is a follow one 08:30.960 --> 08:34.140 and it's gonna rate the prompts, the names for us, you know, 08:34.140 --> 08:37.560 it's come up with the product names from before 08:37.560 --> 08:39.840 and then it's rated them based on catchiness, 08:39.840 --> 08:41.910 uniqueness, and simplicity. 08:41.910 --> 08:44.010 So again, this is another prompt, 08:44.010 --> 08:45.510 this is a follow up prompt. 08:45.510 --> 08:47.340 We want to be testing this too, right? 08:47.340 --> 08:49.290 So one of the things we wanna test for 08:49.290 --> 08:51.330 is it rating them consistently. 08:51.330 --> 08:54.660 So iFitShoe was rated fours across the board. 08:54.660 --> 08:55.713 Let's run it again. 08:59.310 --> 09:02.130 4, 3, 4. Not fully consistent. 09:02.130 --> 09:05.910 4, 3, 4. 4, 3, 4. That is interesting. 09:05.910 --> 09:07.920 It does seem relatively consistent 09:07.920 --> 09:09.870 that it's rating at 4, 3, 4. 09:09.870 --> 09:12.930 That's close enough for me to use this scale. 09:12.930 --> 09:15.870 If we were getting really different results every time, 09:15.870 --> 09:19.230 that's proof that their AI doesn't actually know how 09:19.230 --> 09:20.880 to rate them consistently. 09:20.880 --> 09:22.980 And I find that with GT4, 09:22.980 --> 09:25.740 it is a pretty consistent rate too, whereas as GT3.5, 09:25.740 --> 09:26.883 you cannot find. 09:27.930 --> 09:30.030 And to copy this across. 09:30.030 --> 09:32.730 And this is the final response here. 09:32.730 --> 09:33.840 And then based on the ratings 09:33.840 --> 09:36.000 we could choose which one is best and it looks like 09:36.000 --> 09:39.870 Omni Fit Walk is the best one based on these ratings. 09:39.870 --> 09:41.970 Cool. Hopefully that gives you an idea of how to work 09:41.970 --> 09:44.760 through these principles 09:44.760 --> 09:47.460 and how I use them on a day-to-day basis. 09:47.460 --> 09:48.780 I start with that naive prompt 09:48.780 --> 09:51.150 and 80% of the time that's fine. 09:51.150 --> 09:53.310 But then if I'm gonna be using the prompt in production, 09:53.310 --> 09:56.610 I start to work down the list until I get it really robust. 09:56.610 --> 09:58.830 And I might be doing this actually with code 09:58.830 --> 10:02.100 rather than in chatGPT specifically. 10:02.100 --> 10:04.410 You know, you don't have to be technical to do this process, 10:04.410 --> 10:07.740 it's just a case of retrying, retrying, testing what works, 10:07.740 --> 10:09.840 and making changes and then seeing 10:09.840 --> 10:13.530 what gives you reliable results that you actually want. 10:13.530 --> 10:15.840 And that's the important thing with this type of work 10:15.840 --> 10:17.550 because if you know the right way to ask, 10:17.550 --> 10:19.650 you can get it to do pretty much anything.