explain() method: MongoDB

We’ve seen the working of explain() method briefly in previous video tutorials – in this video tutorial lets dive in to learn more about this useful method, which helps us analyze and optimize our MongoDB commands.

explain-method-mongodb

Related Read:
Create and Insert Documents: MongoDB
index creation: MongoDB

temp: database name
name: collection name

Inserting 1000 Documents

1
2
3
4
5
6
7
8
MongoDB shell version: 2.6.1
connecting to: test
 
> use temp
switched to db temp
 
> for(i = 1; i < = 1000; i++) db.name.insert({a: i, b: i, c: i});
WriteResult({ "nInserted" : 1 })

Here we inserted 1000 documents using for loop.

Basic Cursor

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
> db.name.find({a: 5})
{ "_id" : ObjectId("53dcd070340ac74e061d7215"), "a" : 5, "b" : 5, "c" : 5 }
 
> db.name.find({a: 5}).explain()
{
        "cursor" : "BasicCursor",
        "isMultiKey" : false,
        "n" : 1,
        "nscannedObjects" : 1000,
        "nscanned" : 1000,
        "nscannedObjectsAllPlans" : 1000,
        "nscannedAllPlans" : 1000,
        "scanAndOrder" : false,
        "indexOnly" : false,
        "nYields" : 7,
        "nChunkSkips" : 0,
        "millis" : 1,
        "server" : "Satish-PC:27017",
        "filterSet" : false
}

Here MongoDB server is returning a basic cursor, as we do not have index on field “a”. Also note that, it’s scanning 1000 documents in the “name” collection and scanning 1000 index keys in “system.indexes” collection, where the default key is “_id”.

explain() method: MongoDB



YouTube Link: https://www.youtube.com/watch?v=q59nF4ziW-w [Watch the Video In Full Screen.]



Creating Index on fields “a” and “b”

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
> db.name.ensureIndex({a: 1, b: 1});
{
        "createdCollectionAutomatically" : false,
        "numIndexesBefore" : 1,
        "numIndexesAfter" : 2,
        "ok" : 1
}
 
> db.name.find({a: 5}).explain()
{
        "cursor" : "BtreeCursor a_1_b_1",
        "isMultiKey" : false,
        "n" : 1,
        "nscannedObjects" : 1,
        "nscanned" : 1,
        "nscannedObjectsAllPlans" : 1,
        "nscannedAllPlans" : 1,
        "scanAndOrder" : false,
        "indexOnly" : false,
        "nYields" : 0,
        "nChunkSkips" : 0,
        "millis" : 0,
        "indexBounds" : {
                "a" : [
                        [
                                5,
                                5
                        ]
                ],
                "b" : [
                        [
                                {
                                        "$minElement" : 1
                                },
                                {
                                        "$maxElement" : 1
                                }
                        ]
                ]
        },
        "server" : "Satish-PC:27017",
        "filterSet" : false
}

Here we create index on key “a” and “b” – it’s called compound key – after which the explain() method shows that MongoDB server is returning a Btree cursor, meaning it’s making use of index key to execute the query/command.

Also not the number of objects scanned in the “name” collection as well as in the “system.indexes” collection.

indexOnly: true

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
> db.name.find({a: 5})
{ "_id" : ObjectId("53dcd070340ac74e061d7215"), "a" : 5, "b" : 5, "c" : 5 }
 
> db.name.find({a: 5}, {a: 1, b: 1, _id: 0})
{ "a" : 5, "b" : 5 }
 
> db.name.find({a: 5}, {a: 1, b: 1, _id: 0}).explain()
{
        "cursor" : "BtreeCursor a_1_b_1",
        "isMultiKey" : false,
        "n" : 1,
        "nscannedObjects" : 0,
        "nscanned" : 1,
        "nscannedObjectsAllPlans" : 0,
        "nscannedAllPlans" : 1,
        "scanAndOrder" : false,
        "indexOnly" : true,
        "nYields" : 0,
        "nChunkSkips" : 0,
        "millis" : 0,
        "indexBounds" : {
                "a" : [
                        [
                                5,
                                5
                        ]
                ],
                "b" : [
                        [
                                {
                                        "$minElement" : 1
                                },
                                {
                                        "$maxElement" : 1
                                }
                        ]
                ]
        },
        "server" : "Satish-PC:27017",
        "filterSet" : false
}

Covered Index or indexOnly
If our command is requesting for values present inside the index, mongo engine will fetch that information from the “system.indexes” collection itself and does not go to the collection where the entire document is present – in our case “name” collection. Since index is on keys “a” and “b” in our case, if we query for “a” and “b” values only, it fetches those values directly from the index information and will not go to “name” collection. Hence the speed of execution of the command is faster, and this type of commands are very efficient in creating optimized web applications.

Note from above explain() result, the number of objects scanned in the “name” collection is 0, and the number of scans made on indexes in only 1.

Multi-key Indexes and Arrays: MongoDB

We have learnt the basics of multi-key indexes in MongoDB. Lets look at an example to demonstrate the multi-key indexing on arrays.

arrays-multi-key-index-mongodb

foo: database name
name: collection name

Insert a document

1
2
3
4
5
6
7
8
MongoDB shell version: 2.6.1
connecting to: test
 
> use foo
switched to db foo
 
> db.name.insert({a: 1, b: 2, c: 3});
WriteResult({ "nInserted" : 1 })

Here we insert {a: 1, b: 2, c: 3} into “name” collection.

Multi-key Indexes and Arrays: MongoDB



YouTube Link: https://www.youtube.com/watch?v=VGHSmjVmnzs [Watch the Video In Full Screen.]



Basic Cursor

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
> db.name.find({a: 1, b: 2})
{ "_id" : ObjectId("53d8982b79142c385cddc607"), "a" : 1, "b" : 2, "c" : 3 }
 
> db.name.find({a: 1, b: 2}).explain()
{
        "cursor" : "BasicCursor",
        "isMultiKey" : false,
        "n" : 1,
        "nscannedObjects" : 1,
        "nscanned" : 1,
        "nscannedObjectsAllPlans" : 1,
        "nscannedAllPlans" : 1,
        "scanAndOrder" : false,
        "indexOnly" : false,
        "nYields" : 0,
        "nChunkSkips" : 0,
        "millis" : 0,
        "server" : "Satish-PC:27017",
        "filterSet" : false
}

We find() the document using fields “a” and “b” and the query/command returns a basic cursor, as we do not have indexing on them.

Related Read: index creation: MongoDB

Lets create index on a and b

1
2
3
4
5
6
7
> db.name.ensureIndex({a: 1, b: 1});
{
        "createdCollectionAutomatically" : false,
        "numIndexesBefore" : 1,
        "numIndexesAfter" : 2,
        "ok" : 1
}

Previous there was only 1 index i.e., on “_id” Now there are 2 indexes – “_id” and “{a: 1, b: 1}”

Btree Cursor with multi-key as false

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
> db.name.find({a: 1, b: 2}).explain()
{
        "cursor" : "BtreeCursor a_1_b_1",
        "isMultiKey" : false,
        "n" : 1,
        "nscannedObjects" : 1,
        "nscanned" : 1,
        "nscannedObjectsAllPlans" : 1,
        "nscannedAllPlans" : 1,
        "scanAndOrder" : false,
        "indexOnly" : false,
        "nYields" : 0,
        "nChunkSkips" : 0,
        "millis" : 0,
        "indexBounds" : {
                "a" : [
                        [
                                1,
                                1
                        ]
                ],
                "b" : [
                        [
                                2,
                                2
                        ]
                ]
        },
        "server" : "Satish-PC:27017",
        "filterSet" : false
}

After creating the index on “a” and “b”, chain explain() method on the same command, and it shows you that, now it returns a Btree Cursor.

Lets insert another document

1
2
> db.name.insert({a: [0, 1, 2], b: 2, c: 3});
WriteResult({ "nInserted" : 1 })

Lets insert an array as value to field “a” and scalar values to “b” and “c”.

Btree Cursor with Multi-key true

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
> db.name.find({a: 1, b: 2})
{ "_id" : ObjectId("53d8982b79142c385cddc607"), 
  "a" : 1, "b" : 2, "c" : 3 }
{ "_id" : ObjectId("53d8986f79142c385cddc608"), 
  "a" : [ 0, 1, 2 ], "b" : 2, "c": 3 }
 
> db.name.find({a: 1, b: 2}).explain()
{
        "cursor" : "BtreeCursor a_1_b_1",
        "isMultiKey" : true,
        "n" : 2,
        "nscannedObjects" : 2,
        "nscanned" : 2,
        "nscannedObjectsAllPlans" : 2,
        "nscannedAllPlans" : 2,
        "scanAndOrder" : false,
        "indexOnly" : false,
        "nYields" : 0,
        "nChunkSkips" : 0,
        "millis" : 0,
        "indexBounds" : {
                "a" : [
                        [
                                1,
                                1
                        ]
                ],
                "b" : [
                        [
                                2,
                                2
                        ]
                ]
        },
        "server" : "Satish-PC:27017",
        "filterSet" : false
}

Now append explain() method to our command, it shows us that it returns a Btree Cursor and multi-key as true. MongoDB engine need to match every element of the array present in field “a” with the scalar value of field “b”. Hence it uses Multi-Key indexing.

Multi-Key Condition in MongoDB

1
2
3
4
5
6
7
8
9
> db.name.insert({a: [0, 1, 2], b: [3, 4], c: 3});
WriteResult({
        "nInserted" : 0,
        "writeError" : {
                "code" : 10088,
                "errmsg" : "insertDocument :: caused by :: 10088 cannot 
                            index parallel arrays [b] [a]"
        }
})

It’s difficult to match every combination of the array elements present inside both “a” and “b” fields. If both keys/indexes has its value as an array, then it gets complicated. Thus, mongoDB doesn’t allow both keys to be arrays. Either one of them must be a scalar value.

Multi-key Index: MongoDB

Lets learn about multikey indexes and how they are efficient in MongoDB.

multi key index mongodb

database name: daily
collections: gadgets, users

Insert documents into gadgets collection

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
> use daily
switched to db daily
 
> db.gadgets.insert({_id: 1, gadget: "Nexus 5"});
WriteResult({ "nInserted" : 1 })
 
> db.gadgets.insert({_id: 2, gadget: "iPhone"});
WriteResult({ "nInserted" : 1 })
 
> db.gadgets.insert({_id: 3, gadget: "iPad"});
WriteResult({ "nInserted" : 1 })
 
> db.gadgets.insert({_id: 4, gadget: "Nexus 7"});
WriteResult({ "nInserted" : 1 })
 
> db.gadgets.find()
{ "_id" : 1, "gadget" : "Nexus 5" }
{ "_id" : 2, "gadget" : "iPhone" }
{ "_id" : 3, "gadget" : "iPad" }
{ "_id" : 4, "gadget" : "Nexus 7" }

Here we have 4 documents with “_id” as 1, 2, 3, 4.

Insert documents into users collection

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
> db.users.insert({name: "Satish", 
                   products: [db.gadgets.find()[0]._id,
                              db.gadgets.find()[3]._id]});
WriteResult({ "nInserted" : 1 })
 
> db.users.insert({name: "Kiran", 
                   products: [db.gadgets.find()[0]._id, 
                              db.gadgets.find()[3]._id, 
                              db.gadgets.find()[2]._id]});
WriteResult({ "nInserted" : 1 })
 
> db.users.insert({name: "Jyothi", products: [1, 2]});
WriteResult({ "nInserted" : 1 })
 
> db.users.find().pretty()
{
        "_id" : ObjectId("53c7a30efd7d3c9597ca2593"),
        "name" : "Satish",
        "products" : [
                1,
                4
        ]
}
{
        "_id" : ObjectId("53c7a337fd7d3c9597ca2594"),
        "name" : "Kiran",
        "products" : [
                1,
                4,
                3
        ]
}
{
        "_id" : ObjectId("53c7a34efd7d3c9597ca2595"),
        "name" : "Jyothi",
        "products" : [
                1,
                2
        ]
}

Here we insert documents into “users” collection and embed the “_id” value of “gadgets” collection as array elements of “products” key.

Multi-key Index: MongoDB



YouTube Link: https://www.youtube.com/watch?v=9C1x-xrN_qQ [Watch the Video In Full Screen.]



Command to fetch documents with array element 1 and 4

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
> db.users.find({products: {$all: [1, 4]}}).pretty();
{
        "_id" : ObjectId("53c7a30efd7d3c9597ca2593"),
        "name" : "Satish",
        "products" : [
                1,
                4
        ]
}
{
        "_id" : ObjectId("53c7a337fd7d3c9597ca2594"),
        "name" : "Kiran",
        "products" : [
                1,
                4,
                3
        ]
}

Both these documents have array values 1 and 4 in them.

Related Read: Comparison Operators: MongoDB

system.indexes content

1
2
3
4
5
6
7
8
> show collections
gadgets
system.indexes
users
 
> db.system.indexes.find()
{ "v" : 1, "key" : { "_id" : 1 }, "name" : "_id_", "ns" : "daily.gadgets" }
{ "v" : 1, "key" : { "_id" : 1 }, "name" : "_id_", "ns" : "daily.users" }

Here we have only 1 key and it’s on “_id”.

Basic Cursor

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
> db.users.find({products: {$all: [1, 4]}}).explain();
{
        "cursor" : "BasicCursor",
        "isMultiKey" : false,
        "n" : 2,
        "nscannedObjects" : 3,
        "nscanned" : 3,
        "nscannedObjectsAllPlans" : 3,
        "nscannedAllPlans" : 3,
        "scanAndOrder" : false,
        "indexOnly" : false,
        "nYields" : 0,
        "nChunkSkips" : 0,
        "millis" : 0,
        "server" : "Satish-PC:27017",
        "filterSet" : false
}

If we chain explain() method to our command, we can know some details about the command and also evaluate about its efficiency. The above command doesn’t have multi-key enabled and it’s a Basic Cursor.

Lets create index on products field

1
2
3
4
5
6
7
> db.users.ensureIndex({"products": 1});
{
        "createdCollectionAutomatically" : false,
        "numIndexesBefore" : 1,
        "numIndexesAfter" : 2,
        "ok" : 1
}

This creates index on field “products”.

Now the system.indexes content

1
2
3
4
5
6
> db.system.indexes.find()
{ "v" : 1, "key" : { "_id" : 1 }, "name" : "_id_", "ns" : "daily.gadgets" }
{ "v" : 1, "key" : { "_id" : 1 }, "name" : "_id_", "ns" : "daily.users" }
{ "v" : 1, "key" : { "products" : 1 }, 
                     "name" : "products_1", 
                     "ns" : "daily.users" }

So now we have 2 keys, “_id” and “products”.

BTree Cursor

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
> db.users.find({products: {$all: [1, 4]}}).explain();
{
        "cursor" : "BtreeCursor products_1",
        "isMultiKey" : true,
        "n" : 2,
        "nscannedObjects" : 3,
        "nscanned" : 3,
        "nscannedObjectsAllPlans" : 3,
        "nscannedAllPlans" : 8,
        "scanAndOrder" : false,
        "indexOnly" : false,
        "nYields" : 0,
        "nChunkSkips" : 0,
        "millis" : 0,
        "indexBounds" : {
                "products" : [
                        [
                                1,
                                1
                        ]
                ]
        },
        "server" : "Satish-PC:27017",
        "filterSet" : false
}

Now if we run the same command chained with explain() method, we could see that its a Btree Cursor and multi-key index is true.

Note: Btree Cursors are faster and efficient than Basic Cursors.