mongodb - Why Mongo query for null filters in FETCH after performing IXSCAN -

according mongo documentation,

the { item : null } query matches documents either contain item field value null or not contain item field.

i can't find documentation this, far can tell, both cases (value null or field missing) stored in index null.

so if db.orders.createindex({item: 1}) , db.orders.find({item: null}), expect ixscan find documents either contain item field value null or not contain item field, , documents.

so why db.orders.find({item: null}).explain() perform filter: {item: {$eq: null}} in fetch stage after performs ixscan? possible documents need filtered out?

{     "queryplanner" : {         "plannerversion" : 1,         "namespace" : "temp.orders",         "indexfilterset" : false,         "parsedquery" : {             "item" : {                 "$eq" : null             }         },         "winningplan" : {             "stage" : "fetch",             "filter" : {                 "item" : {                     "$eq" : null                 }             },             "inputstage" : {                 "stage" : "ixscan",                 "keypattern" : {                     "item" : 1                 },                 "indexname" : "item_1",                 "ismultikey" : false,                 "isunique" : false,                 "issparse" : false,                 "ispartial" : false,                 "indexversion" : 1,                 "direction" : "forward",                 "indexbounds" : {                     "item" : [                         "[null, null]"                     ]                 }             }         },         "rejectedplans" : [ ]     },     "serverinfo" : {         "host" : "andys-macbook-pro-2.local",         "port" : 27017,         "version" : "3.2.8",         "gitversion" : "ed70e33130c977bda0024c125b56d159573dbaf0"     },     "ok" : 1 } 

i thought maybe undefined values indexed null, simple experimentation rules out:

> db.orders.createindex({item: 1}) {     "createdcollectionautomatically" : true,     "numindexesbefore" : 1,     "numindexesafter" : 2,     "ok" : 1 } > db.orders.insert({item: undefined}) writeresult({ "ninserted" : 1 }) > db.orders.find({item: {$type: 6}}).explain() {     "queryplanner" : {         "plannerversion" : 1,         "namespace" : "temp.orders",         "indexfilterset" : false,         "parsedquery" : {             "item" : {                 "$type" : 6             }         },         "winningplan" : {             "stage" : "fetch",             "filter" : {                 "item" : {                     "$type" : 6                 }             },             "inputstage" : {                 "stage" : "ixscan",                 "keypattern" : {                     "item" : 1                 },                 "indexname" : "item_1",                 "ismultikey" : false,                 "isunique" : false,                 "issparse" : false,                 "ispartial" : false,                 "indexversion" : 1,                 "direction" : "forward",                 "indexbounds" : {                     "item" : [                         "[undefined, undefined]"                     ]                 }             }         },         "rejectedplans" : [ ]     },     "serverinfo" : {         "host" : "andys-macbook-pro-2.local",         "port" : 27017,         "version" : "3.2.8",         "gitversion" : "ed70e33130c977bda0024c125b56d159573dbaf0"     },     "ok" : 1 } 

the semantics null equality match predicate (e.g. {"a.b": null}) complicated enough because field contain subdocuments index scan alone isn't enough provide correct result.

according https://jira.mongodb.org/browse/server-18653?focusedcommentid=931817&page=com.atlassian.jira.plugin.system.issuetabpanels%3acomment-tabpanel#comment-931817,

version 2.6.0 of server changed semantics of null equality match predicate, such document {a: []} no longer considered match query predicate {"a.b": null} (in prior versions of server, document considered match predicate). documented in 2.6 compatibility notes, under "null comparison" section.

for index key pattern {"a.b": 1}, document {a: []} generates index key {"": null}. other documents {a: null} , empty document {} generate index key {"": null}. result, if query predicate {"a.b": null} uses index, query system cannot tell index key {"": null} whether or not associated document matches predicate. result, inexact_fetch bounds assigned instead of exact bounds, , hence fetch stage added query execution tree.

additional explanation:

1. the document {} generates index key {"": null} index key pattern {"a.b": 1}.
2. the document {a: []} generates index key {"": null} index key pattern {"a.b": 1}.
3. the document {} matches query {"a.b": null}.
4. the document {a: []} not match query {"a.b": null}.

therefore, query {"a.b": null} answered index key pattern {"a.b": 1} must fetch document , re-check predicate, in order ensure document {} included in result set , document {a: []} not included in result set.

wiki