‫Welcome back.
‫So let's now talk a little bit
‫about read performance in MongoDB,
‫why something called indexes are so important,
‫and how we can actually create them ourselves.
‫And I want to start this demonstration about indexes
‫by firing off a simple query on all our tours.
‫So let's come here to get all tours
‫and I will also filter
‫for a price less than 1,000.
‫Okay and so let's see.
‫Yeah, so we get three results back, all right.
‫And that's important to keep in mind
‫for what I'm gonna show you next which is
‫that we can actually also get a couple of statistics
‫about the query itself.
‫So let's go here basically to the handler function.
‫And so this remember right now in the handler factory.
‫Right, so it's this,
‫I think it's at the bottom.
‫Yeah so it's this, get all factory function
‫that will create this handler that is called
‫for the query that we just performed.
‫And so here on the query,
‫I'll actually now add an explain method.
‫Okay so after the query we will then call explain all right.
‫And so let's take a look at that.
‫And so we now get a completely different result,
‫which is basically these statistics.
‫Now there's a lot of stuff in here.
‫But what I'm really interested
‫in is these execution statistics, okay.
‫So you can see here that the number of documents
‫that were returned were three.
‫And so that's exactly the result that we got before.
‫So before doing the explain right.
‫But what's really important to note here is
‫that the number of documents
‫that were examined is nine, okay.
‫And so this means that MongoDB had to examine,
‫so basically to scan all of the nine documents
‫in order to find the correct three ones.
‫So the three ones that match the query okay.
‫And so that's not efficient at all right?
‫Now of course at this scale
‫with only nine documents it makes absolutely no difference.
‫But if we had hundreds of thousands,
‫or even millions of documents here,
‫then this would significantly affect
‫the read performance of this query.
‫So again, it's not going to be the case in this application,
‫but it might be in an app that you will build someday.
‫And so you really need to learn about indexes.
‫Because with indexes, we will be able
‫to kind of solve this problem.
‫So we can create indexes on specific fields in a collection.
‫For example Mongo automatically creates an index
‫on the ID field by default.
‫And let's actually see that in Compass okay.
‫For example in all tours, we have here the indexes tab.
‫So up until this point we only ever were here
‫in the documents tab,
‫but as you see we have a lot of different stuff here
‫and one of them is the indexes.
‫And so again you see that by default we have an ID index.
‫And this ID index is then basically an ordered list
‫of all the IDs that get stored somewhere
‫outside of the collection okay.
‫And you can actually see a size here,
‫that it has 36 kilobytes, this index all right.
‫And this index is extremely useful.
‫Because whenever documents are queried
‫by the ID MongoDB will search that ordered index
‫instead of searching through the whole collection
‫and look at all the documents one by one,
‫which is of course much slower.
‫So again without an index Mongo has to look
‫at each document one by one.
‫But with an index on the field that we are querying for,
‫this process becomes much more efficient.
‫So that is pretty smart, right?
‫And of course, we can set our own indexes
‫on fields that we query very often.
‫So let's actually do that with the price field
‫that we just queried for before
‫because I believe that it is one of the most important
‫that people will query for, okay.
‫And so this is how it works.
‫So we need to go to the tour model right.
‫And then let's do it right here after this inner declaration
‫and we say tourschema.index okay.
‫And then an object with the name of the field
‫and remember how I said we were gonna set the index
‫on the price and then either a one or a minus one.
‫And a one means that we're sorting the price index
‫in an ascending order,
‫while the minus one stands for descending order okay.
‫And there are actually other types of indexes as well,
‫like for text or for geospatial data,
‫but we will see that a bit later.
‫Okay so let's give it a save now
‫and try our query again.
‫And actually I will do it a couple of times here
‫to make sure that the index is really set.
‫Because sometimes it is not set right away.
‫But let's now take a look here.
‫And so indeed we get still our number of returned at three
‫but this time the number of documents that were examined,
‫so that were scanned, were also only three.
‫And so that proves that with this index,
‫we basically achieved exactly what we wanted.
‫So before we had to scan through all of the nine documents
‫and now the engine only needs to scan the three documents
‫that are actually also returned.
‫And again because their prices
‫are now ordered in that index.
‫And so that makes it much easier and much faster
‫for the MongoDB engine to find them.
‫And so this is of course a huge performance gain.
‫Let's now also check out Compass here,
‫and actually let's reload the entire database,
‫and now it should actually be here
‫but for some reason it's not.
‫Let's maybe try to reload the documents.
‫Maybe it appears then.
‫Not really, let's also analyze the schema,
‫and that's something that we're gonna talk
‫about a bit later.
‫But still as you see we still only have these two indexes.
‫But that doesn't matter at all because we already know
‫that the index is actually working right?
‫So it's completely normal
‫that sometimes this takes some time to update.
‫Now another thing that you might notice here is
‫how this ID index that we talked
‫about earlier says unique here okay.
‫And so unique is also a property
‫that we can give to indexes.
‫And this is actually the reason
‫why the IDs have always to be unique.
‫So simply because the index
‫of the ID has this unique property.
‫You probably also noticed that there is an index
‫for the name here right?
‫But we didn't actually create that manually ourselves right?
‫So can you guess why it is here?
‫Well it is because in our schema definition,
‫we set the name field to be unique.
‫And so what Mongos then does behind the scenes
‫in order to ensure the uniqueness of this field is
‫to create a unique index for it, all right.
‫And so because of that, not only the ID
‫but also the name always have to be unique.
‫Okay and this is great already.
‫So when all we ever do is to just query
‫for one single field alone,
‫then a single field index is perfect
‫because remember the index that we just set
‫before is called a single field index.
‫Not sure if I mentioned it back then but I think I did.
‫But anyway, if we sometimes query for that field
‫but combined with another one,
‫then it's actually more efficient
‫to create a compound index.
‫So one with two fields and not just one.
‫So let's create a query for that just to illustrate it.
‫And so another field that I think is going
‫to be queried for all the time is the ratings average.
‫So ratings average, I think that's how it's called,
‫and let's say that we want greater or equal than 4.7 okay.
‫Let's send that query.
‫And let's see how many results we get.
‫Where is that?
‫Yeah here.
‫So the number of results,
‫so the number of documents that are returned,
‫so that match this query is two.
‫But we still had to examine three documents.
‫And again, at this scale of course,
‫that doesn't make any difference.
‫But as you understand, this is just an example.
‫And so now we want to fix the situation as well
‫and for that we're gonna use a compound index.
‫So let's go back here.
‫Comment this one out.
‫Or actually first duplicate it and then comment.
‫And so it's actually very simple.
‫All we need to do is to add here the other field.
‫So ratings average and let's put this one
‫in the ascending order.
‫Or actually, that's the descending order all right.
‫So let's give this a save.
‫Let's go back here.
‫And again, I'm going to do it a couple of times here okay.
‫And let's see our results.
‫And so now we get the result that we wanted.
‫So only two documents were scanned in order
‫to find the two documents that we were actually looking for.
‫Perfect and actually this compound index
‫that we just created is also going to work when the query
‫for just one of these two fields here individually,
‫so price or ratings average.
‫So when we create a compound index like this,
‫we do not have to then create one individual
‫for each of the fields as well okay.
‫I just want to check out how it looks in Compass now.
‫But well it still looks the same.
‫But again, not really important.
‫One thing that we can still see here
‫and which is pretty interesting is
‫that actually the size of these indexes.
‫So 72 kilobytes is actually way bigger
‫than the total size of all the documents combined,
‫which is only 14 kilobyte right?
‫And so basically these indexes that we create
‫to search the documents take up a lot more space
‫than the documents themselves.
‫But again that's just because we're operating
‫on such a small scale in this example.
‫And so that's not really relevant okay.
‫But it's still important to talk about this
‫because actually this leads me to our next question.
‫And that question is,
‫how do we decide which field we actually need to index?
‫And why don't we set indexes on all the fields?
‫Well we kind of used the strategy that I used
‫to set the indexes on the price and on the average rating.
‫So basically we need to carefully study the access patterns
‫of our application in order to figure out
‫which fields are queried the most
‫and then set the indexes for these fields.
‫For example, I'm not setting an index here on the group size
‫because I don't really believe
‫that many people will query for that parameter,
‫and so I don't need to create an index there.
‫Because we really do not want to overdo it with indexes.
‫So we don't want to blindly set indexes on all the fields
‫and then hope for the best basically.
‫And the reason for that is
‫that each index actually uses resources,
‫so as you can actually see here right.
‫And also, each index needs to be updated each time
‫that the underlying collection is updated.
‫So if you have a collection with a high write-read ratio,
‫so a collection that is mostly written to,
‫then it would make absolutely no sense
‫to create an index on any field in this collection
‫because the cost of always updating the index
‫and keeping it in memory clearly outweighs the benefit
‫of having the index in the first place
‫if we rarely have searches,
‫so have queries, for that collection.
‫So in summary, when deciding whether
‫to index a certain field or not,
‫we must kind of balance the frequency
‫of queries using that exact field
‫with the cost of maintaining this index,
‫and also with the read-write pattern of the resource.
‫However, just like it is with data modeling,
‫there are not really hard rules here.
‫So it's all a bit fuzzy
‫and you really need some experimentation
‫and also experience to get it right, all right.
‫But whatever you do, please don't just ignore indexing.
‫Because even if it's not perfect,
‫it will always have a huge benefit for your application.
‫All right, and that's actually all I had
‫to tell you about indexes.
‫There's just one more index that I actually want to set here
‫which is for the tour slug okay.
‫Because later on we will actually want
‫to use the unique slug to query for tours.
‫So meaning that the slug will then probably
‫become the most queried field.
‫And so it makes all the sense
‫to also have an index for that one.
‫So tourschema.index
‫and slug one.
‫And most times the one or minus one is not that important.
‫Okay now I'm really curious
‫to try to see this here in Compass.
‫So what I'm gonna do is to try to connect
‫to the database again here.
‫And so I can just get that here from the most recent ones.
‫Click, connect, and then we should get connected
‫to the same database.
‫So natours, tours, let's come to our indexes.
‫And now here we go.
‫So now we actually have our indexes right here.
‫So let's make this window bigger
‫and take a look at what we got.
‫So we have our slug here all right.
‫We have the price, which is the first one that we set.
‫And then we also have the price and ratings average,
‫which is compound and you also see here
‫that this one here is ascending and this one is descending
‫because it has the minus one.
‫And another thing you might notice is
‫that actually we no longer
‫have this price index in our code.
‫But it's still here.
‫And so it's not enough to remove the index from our code,
‫we really need to delete it from the database itself right.
‫So remember we had it here in the beginning
‫and then we commented it out
‫and created this new compound index,
‫but again the index is stand still here.
‫But since we actually no longer need it,
‫we can just go ahead and delete it okay.
‫Now we need to type the name just to make sure
‫that we do not do any mistakes.
‫And so here we go, great.
‫So that is the power of indexes.
‫They really can make our read performance
‫on databases much, much better.
‫And so in your own applications
‫you should really never ignore them.
‫And before we finish let's actually take back
‫that explain method that we added here
‫in our handler function.
‫And actually just as a reference,
‫I will leave it here like this okay.
‫Give it a save.
‫Back in post menu.
‫Let's try that now.
‫And indeed, it's back to working.
‫Okay and now that's really it.