How to Index Location Search Results With MongoDB Indexes

[00:00 - 00:25] In this lesson, we'll begin with some slides before we make a small change in our database that's going to have a pretty big importance when it comes to having our app be built in a larger scale. At this moment in time, we've introduced the capability to have our client query for listing documents for different locations with all the features we wanted to introduce, filtering, pagination, searching, etc.

[00:26 - 00:37] Our database at this moment in time only contains a handful of listings that we 've prepared as mock data. So when we query our database based on location data, it returns results fairly quickly.

[00:38 - 00:58] However, imagine if we had tens of millions of listings in our database. If we try to search all the listings in, let's say, New York, our server will take probably several minutes to scan through our entire listings collection and pick every single document that has the region of New York.

[00:59 - 01:07] Here's an analogy. Imagine you had a book with 1 million random pictures in it, and someone told you to find all the pictures with a cat.

[01:08 - 01:28] It would take you a pretty long time to check each and every picture. But if there was an index in the beginning of the book that categorized all the pictures based on the object, along with their page number, you would have a much easier time finding all the pictures that evolve a cat.

[01:29 - 01:40] So what we need in our use case is an index for our listings collection. MongoDB indexes support the efficient execution of queries in a collection.

[01:41 - 02:03] Without indexes, Mongo must perform a collection scan, or that is to say, a scan for every document in a collection to select the documents that match the query statement . If an appropriate index exists for a query, Mongo can use the index to limit the number of documents it must inspect.

[02:04 - 02:25] Indexes are important when it comes to the world of databases and are just data structures that essentially allow for the faster access of data. We won't spend too much time diving into the details of how indexes work, but the takeaway is without an index, Mongo will perform a scan for every document in a collection to satisfy a query.

[02:26 - 02:30] If there is millions of documents, so be it. It will go through every single one.

[02:31 - 02:47] When we define an index for a certain field, we tell the database, or in this case, MongoDB, to create a different data structure for the values of that field for every document in the collection. And this data structure isn't shown to us.

[02:48 - 03:03] When we provide a query to find the documents based on certain values of that certain field, Mongo doesn't do a collection scan and go through every document, but instead gets the documents we're looking for in a more time-effective manner. How?

[03:04 - 03:25] That's because the data structure it prepared is now easier to traverse to find the documents in question. Now as to what the algorithm or data structure is, the documentation in Mongo actually doesn't spend too much time talking about this, but it does note that the B-tree data structure is used.

[03:26 - 03:48] B-trees are self-balancing data structures that maintain sorted data and allow for fairly quick searches and insertions of data. Traversing through a B-tree data structure is often recognized to be done in logarithmic time, which is more time-effective, especially as the data set grows when compared with linear time, which is what a collection scan will do.

[03:49 - 04:17] Now this is getting to the nitty-gritty stuff of how Mongo helps facilitate quick searches, and we don't need to understand everything we're talking about here, but the key takeaway is that specifying indexes helps speed up the amount of time needed to query for documents in large collections. Here's a diagram from the MongoDB documentation that illustrates a query that selects and orders matching documents using an index.

[04:18 - 04:45] Here it's saying we want to find all the documents in a user's collection where the score of the user is less than 30, and we want these results to be sorted in ascending order of the score. In this case, an index has been specified for the score field, and by doing so, Mongo creates a data structure for all the values of the score field for all the documents in this collection.

[04:46 - 04:59] Then it performs a search faster than linear time to get all the documents that match the intended query. Fundamentally indexes in MongoDB are similar to indexes in other database systems.

[05:00 - 05:18] MongoDB defines indexes at the collection level and supports indexes on any field or subfield of the documents in a collection. MongoDB automatically creates a unique index on the underscore ID field during the creation of a collection.

[05:19 - 05:37] Since the underscore ID field index is given a unique property, Mongo rejects duplicate values for this indexed field. So the underscore ID index prevents clients from inserting two documents with the same value for this particular ID field.

[05:38 - 05:56] Mongo also offers many other different index types, such as geospatial, text, etc. We're not going to spend much time talking about those, but do read the MongoDB documentation on indexes that we're going to share in the lesson manuscript if you're interested in reading more.

[05:57 - 06:14] From what we've talked about, we've gathered that MongoDB supports the creation of indexes on a single field of a document. Mongo also supports the creation of indexes on multiple fields, and this is known as compound indexes.

[06:15 - 06:35] The order of fields listed in a compound index has significance. For instance, if a compound index consists of a user ID 1, score negative 1, the index sorts first by user ID, and then within each user ID value, it does a further sort by the score.

[06:36 - 06:52] With all that said, let's go back to our use case. We've mentioned that if a certain location is searched in our app at this moment in time, Mongo will conduct a collection scan and go through all the documents in the listings collection that satisfy the query we have in mind.

[06:53 - 07:07] This isn't a huge issue now since we don't have a lot of listings, but it could be an issue if we start to have thousands or millions of listings in our app. When it comes to searching for listings in a certain location, what are the fields we're actually querying for?

[07:08 - 07:22] If you recall, we make a query for listings that match at the most three fields , country, admin, and city. At the minimum, we provide a query that contains the country when a location is requested.

[07:23 - 07:39] If our geocoder picks up the administration area and city, we include those fields in our query as well. So at this stage, when all three fields exist, when Mongo makes a search, it isn't looking for documents that satisfy a single field, but all of these three.

[07:40 - 07:55] With that said, let's visit the Atlas dashboard of our MongoDB cluster and prepare indexes for the three fields we have in mind. We'll visit the listings collection and click on the indexes tab to provide a new index.

[07:56 - 08:05] Here we'll already see the default underscore ID unique index automatically created by Mongo DB. We'll go ahead and create our own index.

[08:06 - 08:15] So we'll first click on create index. Since we're going to create an index for multiple fields, we'll create a compound index.

[08:16 - 08:33] The entity holds the most importance in our location searches, followed by admin and followed by city. So with that said, we'll define a compound index that sorts our listings documents by country first, then admin and lastly by city.

[08:34 - 08:46] And we'll provide all of them with a value of one, which helps prepare the sorted indexes in ascending alphabetical order. And we'll select the create button to create our new index.

[08:47 - 08:55] It would ask us that we should confirm that we want to create an index in this collection, main.listings, with this following options. Yes, we do.

[08:56 - 09:09] So we'll confirm. And now a compound index has been created for the country admin and city fields for the documents in our listings collection.

[09:10 - 09:22] If we now had millions of listings in our database, a query that utilizes our new index for location based searching could potentially take seconds as opposed to minutes to complete.