-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore: mql specs #92
Draft
kmruiz
wants to merge
22
commits into
main
Choose a base branch
from
chore/mql-specs
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+435
−0
Draft
chore: mql specs #92
Changes from 2 commits
Commits
Show all changes
22 commits
Select commit
Hold shift + click to select a range
41be40b
chore: draft of mql-query spec
kmruiz 86ead3e
chore: add serialization
kmruiz 23dc67f
Update packages/mongodb-mql-model/src/docs/md/mql-query/mql-query.md
kmruiz 2f535e1
chore: specify that components are a sorted list
kmruiz d02dc8a
chore: specify that a component can be more than once in a node
kmruiz 50f04e7
chore: fix typo
kmruiz f6f6fda
chore: typo
kmruiz 655314f
chore: add the query to the table
kmruiz 7d7f45f
chore: initial bson-type spec
kmruiz 04c22dd
chore: type assignability table
kmruiz b9c4b7f
chore: small typos
kmruiz 922d0eb
chore: java type mapping table
kmruiz 4beefa3
chore: fixing typo
kmruiz aa6c360
Update packages/mongodb-mql-model/src/docs/md/bson-type/bson-type.md
kmruiz b17b90c
chore: simplify BsonString explanation and drop links to code
kmruiz 42f7a90
chore: add an example for the superset
kmruiz af3c6b1
chore: fix typos
kmruiz 2529027
Merge branch 'main' into chore/mql-specs
kmruiz 3f0ac52
chore: definition of the computed types
kmruiz 097e94b
Merge branch 'main' into chore/mql-specs
kmruiz 39cb569
chore: components
kmruiz 52657dc
chore: iteration 2
kmruiz File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
145 changes: 145 additions & 0 deletions
145
packages/mongodb-mql-model/src/docs/md/mql-query/mql-query.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,145 @@ | ||
# MQL Query | ||
----------- | ||
|
||
## Abstract | ||
|
||
This specification documents the structure of a MongoDB Query from a mixed perspective of both | ||
the original source code and the target server that might run the query. It is primarily aimed | ||
to provide developers of dialects and linters a common and flexible structure for code processing. | ||
|
||
## META | ||
|
||
The keywords "MUST", "MUST NOT", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY" | ||
and "OPTIONAL" in this document are to be interpreted as described in [RFC 2119](https://www.ietf.org/rfc/rfc2119.txt). | ||
|
||
## Specification | ||
|
||
A MongoDB Query (**query** from now on), is a single execution unit, written in any of the supported dialects, | ||
that MAY be consumed by a valid MongoDB Cluster. A query SHOULD contain all the semantics specific to the source dialect | ||
so it can be tailored back to the original source code. A query MAY be unsupported by a specific target MongoDB Cluster. | ||
|
||
### Query validation and support | ||
|
||
A query MAY be valid for a target MongoDB Cluster if the MongoDB Cluster can consume the query once it | ||
is translated to a consumable dialect by the target Cluster. However, a query MAY be unsupported by the | ||
cluster if it doesn't have the capabilities to fulfill the query request. | ||
|
||
For example, let's consider the following query, in pseudocode: | ||
|
||
```java | ||
collection.aggregate(AtlasSearch(text().eq("baby one more time"))) | ||
``` | ||
|
||
| Cluster | Is Valid | Is Supported | | ||
|--------------------------------|------------|----------------| | ||
| MongoDB Community 7.0 | ✅ | 🔴 | | ||
| MongoDB Enterprise 8.0 | ✅ | 🔴 | | ||
| MongoDB Atlas 8.0 w/o Search | ✅ | 🔴 | | ||
| MongoDB Atlas 8.0 with Search | ✅ | ✅ | | ||
|
||
For the purpose of this specification and project, we will only allow the `mongosh` dialect as a | ||
consumable dialect for a MongoDB Cluster. | ||
|
||
### Query equivalence | ||
|
||
We will consider two queries equivalent, independently of the query structure, if the following conditions | ||
apply: | ||
|
||
* They MUST be **valid** by the same set of target clusters. | ||
* They MUST be **supported** by the same set of target clusters. | ||
* They MUST return the same subset of results for the same input data set. | ||
* They MAY be sourced from the same dialect. | ||
* They MAY lead to equivalent **execution plans** for the same target cluster. | ||
|
||
We will consider two execution plans equivalent if the cluster query planner lead to the same list | ||
lerouxb marked this conversation as resolved.
Show resolved
Hide resolved
|
||
of operations. | ||
|
||
Let's consider a different use case. For the following two queries in the `Java Driver` dialect: | ||
|
||
```java | ||
collection.find(eq("bookName", myBookName)) | ||
collection.aggregate(matches(eq("bookName", myBookName))) | ||
``` | ||
|
||
We will test two different target clusters: | ||
|
||
* MongoDB Community 8.0, from a development environment, does not have an index on bookName for the target collection. | ||
* MongoDB Atlas 8.0, production environment, does have an index on bookName for the target collection. | ||
|
||
In the development environment, we copy the data from the production environment once every week. For this example, | ||
we will consider that the data sets are exactly the same on both clusters. | ||
|
||
| Cluster Environment | Is Valid | Is Supported | Same Results | Same Dialect | Same Execution Plan | | ||
|---------------------------------|-------------|----------------|----------------|----------------|-----------------------| | ||
| MongoDB Development | ✅ | ✅ | ✅ | ✅ | 🔴 | | ||
| MongoDB Production | ✅ | ✅ | ✅ | ✅ | 🔴 | | ||
addaleax marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
**✅ They are equivalent.** | ||
|
||
And now, finally, let's assume the same environment, but with the query written in two dialects, | ||
`mongosh` and `Java Driver`: | ||
|
||
```java | ||
collection.find(eq("bookName", myBookName)) | ||
``` | ||
```js | ||
collection.find({"bookName": myBookName}) | ||
``` | ||
|
||
| Cluster Environment | Is Valid | Is Supported | Same Results | Same Dialect | Same Execution Plan | | ||
|---------------------------------|-------------|----------------|----------------|----------------|-----------------------| | ||
| MongoDB Development | ✅ | ✅ | ✅ | 🔴 | 🔴 | | ||
| MongoDB Production | ✅ | ✅ | ✅ | 🔴 | 🔴 | | ||
|
||
**✅ They are equivalent.** | ||
|
||
|
||
### Query Nodes | ||
|
||
A [MQL Node or a Node for short](/packages/mongodb-mql-model/src/main/kotlin/com/mongodb/jbplugin/mql/Node.kt) | ||
represents a set of semantic properties of a MongoDB query or a subset of it. Nodes MUST NOT be specific to | ||
a single source dialect, but MAY contain semantics that are relevant for processing. | ||
|
||
Nodes MUST contain a single reference to the original source code, in any of the valid dialects. Multiple | ||
nodes MAY contain a reference to the same original source code. That reference is called the **source** | ||
of the Node. For example, let's consider this query, written in the **Java Driver dialect** and how it is referenced by a Node. | ||
|
||
```java | ||
collection.find(eq("_id", 123456)).first(); | ||
// ^ ^ | ||
// +----------------------------------------+ | ||
// Node(source) | ||
``` | ||
|
||
A Node MAY contain parent nodes and children nodes, through specific **components**. A Node that | ||
lerouxb marked this conversation as resolved.
Show resolved
Hide resolved
|
||
doesn't contain any parent node, but contains children nodes is called the **root** node, and | ||
represents the whole query. | ||
|
||
Nodes MAY have additional components that contain metadata for that node. Components MAY have | ||
references to other Nodes and other components. | ||
kmruiz marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
Nodes with components MAY build a tree like structure, resembling an Abstract Syntax Tree. Nodes MUST | ||
NOT refer to themselves either directly or through one of it's childrens, avoiding circular references. | ||
kmruiz marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
### MQL Serialization | ||
|
||
A query MUST be serializable to readable text. The serialization format is independent of the | ||
dialects used for parsing it. A serialized query SHOULD look like this: | ||
|
||
```kt | ||
Node( | ||
source=collection.find(eq("_id", 123456)).first(), | ||
components=[ | ||
// list of components | ||
] | ||
) | ||
``` | ||
|
||
The serialization format MAY ignore printing the source of the query, but MUST print all the components | ||
attached to each of the nodes of the query. In that case, a short form on the syntax MAY be used: | ||
|
||
```kt | ||
Node([ | ||
// list of components | ||
]) | ||
``` |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
😆
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.