From 81bd60dad58fc915e0a1349e6590492a7f8fa953 Mon Sep 17 00:00:00 2001 From: linehanMatt Date: Tue, 26 Dec 2023 13:12:06 -0500 Subject: [PATCH 1/7] added sections 10.7, 10.7.1, 10.5.2, 10.6, 10.6.1, 10.6.2, and REFRESH_SCHEDULE in 10.3.2. --- nql.md | 139 +++++++++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 130 insertions(+), 9 deletions(-) diff --git a/nql.md b/nql.md index f84776a..07bb1b8 100644 --- a/nql.md +++ b/nql.md @@ -104,13 +104,13 @@ NQL ***SHALL*** support the following keywords. - - Setting a price (CPM) to 0 ***SHALL*** explicitly filter out access rules with a price, thereby only querying data that is freely available. - - Omitting a CPM filter ***SHALL*** apply no filter, allowing targeting of data at any price. This is particularly useful for `EXPLAIN` statements to understand available data, but not recommended for `CREATE MATERIALIZED VIEW` statements due to potential high costs. -#### 5.1.2 `_access_rule_id` (NOT YET IMPLEMENTED) +#### 5.1.2 `_access_rule_id` - **Definition**: Identifier for the Access Rule that governs the query's permissions. - **Data Type**: String or Numeric ID - **Usage**: Typically used in the `WHERE` clause to specify which Access Rule to apply for the query. Can also appear in the `SELECT` clause for debugging or auditing. - **Constraints**: Must match an existing Access Rule ID. -#### 5.1.3 `_source_company_id` (NOT YET IMPLEMENTED) +#### 5.1.3 `_source_company_id` - **Definition**: Identifier for the company or entity providing the Access Rule. - **Data Type**: String or Numeric ID - **Usage**: Used in the `WHERE` clause to specify data shared by a particular provider. Can also be used in the `SELECT` clause for output. @@ -186,23 +186,24 @@ Using `EXPLAIN `, a forecast can be generated. This forecast estimates bo ### 10.3 CREATING MATERIALIZED VIEWS -#### 10.4.1 Materialized Views +#### 10.3.1 Materialized Views In databases that support them, a materialized view is a database object that stores the result of a query physically. It provides a way to cache expensive query results and improve query performance by reading from this pre-computed result set, which can be refreshed periodically or on-demand. Similarly, in NQL, creating a materialized view creates a new, unique dataset within the Narrative Data Collaboration Platform that can be further queried or actioned on downstream. Creating a materialized view effectively creates a new dataset with a unique name. Such datasets cannot ingest data from other sources. Data purchase costs may or may not be incurred when executing a query that materializes data as a new dataset, depending on the underlying query's access rules. -#### 10.4.2 Materialized View Syntax +#### 10.3.2 Materialized View Syntax ```sql CREATE MATERIALIZED VIEW "" [ DISPLAY_NAME = '' ] [ DESCRIPTION = '' ] -[ EXPIRE = { } // Supported syntax: "expire_when > P60", default retains all data. +[ EXPIRE = { } // Supported syntax: "expire_when > P60", default retains all data. ] [ STATUS = { 'active' | 'updating' | 'draft' } ] [ TAGS = ( '_nio_materialized_view', '', ... ) ] [ WRITE_MODE = { 'append' | 'overwrite' } ] [ EXTENDED_STATS = { 'all' | 'none' } ] -[ PARTITIONED_BY , ] +[ PARTITIONED_BY , ] +[ REFRESH_SCHEDULE = { '@hourly' | '@daily' | '@weekly' | '@monthly' | '@once' | cron expression} ] AS SELECT FROM ``` @@ -257,16 +258,28 @@ The following parameters apply to the dataset that is generated by the `CREATE M - Type: expression - Default: Narrative sample partition is always present; users can add additional partitions. +- `REFRESH_SCHEDULE`: Defines the frequency of updates for the materialized view. + - Allowed Values: + - '@hourly' + - '@daily' + - '@weekly' + - '@monthly' + - '@once' + - cron expressions + - Type: enum | cron + - + - Default: '@once' + -## 10.5 Specialized Functions in NQL +### 10.5 Specialized Functions in NQL NQL also supports a variety of specialized functions or User-Defined Functions (UDFs) to cater to specific use-cases. -### 10.5.1 `ADDRESS_HASHES()` +#### 10.5.1 `ADDRESS_HASHES()` The `ADDRESS_HASHES()` function generates libpostal address hashes from an unstructured address string. This is especially useful for conducting fuzzy address comparisons where exact string matching isn't sufficient. By hashing the addresses and then joining two input lists based on these hashes, users can find approximate address matches with high efficiency. -#### Example: +##### Example: ```sql CREATE MATERIALIZED VIEW "address_hashes_sample_v2" AS @@ -290,6 +303,114 @@ SELECT ) as address_hashes, ... ``` +#### 10.5.2 `country_code_3_to_2()` + +The country_code_3_to_2('column') function takes in a column of ISO 3166-1 alpha-3 country code(s) and converts them to The ISO 3166-1 alpha-2 country code(s). The function is useful for matching the standard output for the `iso_3166_1_country` Rosetta Stone attribute, expressed as ISO 3166-1 alpha-2 country codes. + +##### Example: + +```sql +CREATE MATERIALIZED VIEW "country_code_sample" AS +SELECT + country_code_3_to_2(my_dataset.country_code) as two_letter_codes +FROM + company_date.my_dataset +... +``` + +### 10.6 Querying via Access Rules + +NQL supports querying internal or external access rules directly, in addition to the Rosetta Stone attribute catalog and dataset ids, through `_access_rule_id`. + +#### 10.6.1 Querying Internal Access Rules + +An Access Rule ID, defined in section 5.1.2 as an identifier for the Access Rule that governs the query's permission, is added after the company identifier. When querying data in your own company seat, an access rule id always follows `company_data`. + +##### Example + +```sql +SELECT pd.hashed_emails +FROM company_data.access_rule_for_private_deal pd +``` + +#### 10.6.2 Querying External Access Rules + +An Access Rule ID, defined in section 5.1.2 as an identifier for the Access Rule that governs the query's permission, is added after the company identifier. When querying data in an external company seat, an access rule id always follows `_source_company_id`. + +##### Example + +```sql +SELECT teams.baseball_teams +FROM _source_company_id.access_rule_unique_name_1 teams +``` + +### 10.7 Embedded Namespaces in NQL + +#### 10.7.1 `_rosetta_stone` + + NQL supports Attribute querying via the Rosetta Embedded Namespace is facilitated by `_rosetta_stone`, a direct method to query Rosetta Stone attributes. `_rosetta_stone` acts as an attribute reference within the dataset or access rule identifier. In case of an absence of mappings or an incorrect attribute reference, the query will return an error. + + ##### Basic Usage + + ```sql + SELECT ds_identifier._rosetta_stone."attribute_name" AS alias_name + FROM dataset_source + ``` + + - **`ds_identifier`**: Alias or identifier for the dataset. + - **`attribute_name`**: The name of the Rosetta Stone attribute to be mapped. + - **`alias_name`**: An optional alias for the selected attribute. + +##### Example with Single Dataset + + ```sql + SELECT ds_123._rosetta_stone."event_timestamp" AS event_time + FROM company_data.ds_123 AS ds_123 + ``` + +##### Example Joining Multiple Datasets + + ```sql + SELECT + ds_123._rosetta_stone."attribute_1" AS attribute_from_a, + ds_456._rosetta_stone."attribute_2" AS attribute_from_b, + ds_123.email, + ds_456.username + FROM + company_data.ds_123 AS ds_123 + JOIN + company_data.ds_456 AS ds_456 + ON + ds_123.user_id = ds_456.user_id + ``` + + In this example: + + - The first Rosetta Stone attribute (**`attribute_1`**) is being pulled from dataset **`ds_123`**. + - The second Rosetta Stone attribute (**`attribute_2`**) is being pulled from dataset **`ds_456`**. + +##### Example of Filtering with Rosetta Attributes + + ```sql + SELECT + ds_123._rosetta_stone."unique_id"."value" AS id + FROM + company_data.ds_123 AS ds_123 + WHERE + ds_123.id = 123 + ``` + + Here, the **`WHERE`** clause uses the Rosetta attribute **`unique_id.value`** from dataset **`ds_123`** for filtering. + +##### Example of Nested Properties + For nested properties, the same dot notation is used within the **`_rosetta_stone`** namespace. + + ```sql + SELECT + ds_123._rosetta_stone."nested"."attribute" AS nested_attribute + FROM + company_data.ds_123 AS ds_123 + ``` ## 12. Example Queries From 36bc7b2cdc70219ecffeca07129910379ef87860 Mon Sep 17 00:00:00 2001 From: linehanMatt Date: Tue, 26 Dec 2023 13:34:22 -0500 Subject: [PATCH 2/7] added sections 10.7, 10.7.1, 10.5.2, 10.6, 10.6.1, 10.6.2, and REFRESH_SCHEDULE in 10.3.2. --- nql.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/nql.md b/nql.md index 07bb1b8..483b62a 100644 --- a/nql.md +++ b/nql.md @@ -267,8 +267,7 @@ The following parameters apply to the dataset that is generated by the `CREATE M - '@once' - cron expressions - Type: enum | cron - - - - Default: '@once' + - Default: '@once' ### 10.5 Specialized Functions in NQL From cca08fd33ed440c7aa9e800b7b18779bf6923bb7 Mon Sep 17 00:00:00 2001 From: linehanMatt Date: Tue, 26 Dec 2023 14:35:10 -0500 Subject: [PATCH 3/7] cleaned up explanation of company ids vs slugs and access rule ids vs names. --- nql.md | 30 +++++++++++++++++++++--------- 1 file changed, 21 insertions(+), 9 deletions(-) diff --git a/nql.md b/nql.md index 483b62a..bce61fa 100644 --- a/nql.md +++ b/nql.md @@ -104,18 +104,30 @@ NQL ***SHALL*** support the following keywords. - - Setting a price (CPM) to 0 ***SHALL*** explicitly filter out access rules with a price, thereby only querying data that is freely available. - - Omitting a CPM filter ***SHALL*** apply no filter, allowing targeting of data at any price. This is particularly useful for `EXPLAIN` statements to understand available data, but not recommended for `CREATE MATERIALIZED VIEW` statements due to potential high costs. -#### 5.1.2 `_access_rule_id` +#### 5.1.2 `_access_rule_id` (NOT YET IMPLEMENTED) - **Definition**: Identifier for the Access Rule that governs the query's permissions. - **Data Type**: String or Numeric ID - **Usage**: Typically used in the `WHERE` clause to specify which Access Rule to apply for the query. Can also appear in the `SELECT` clause for debugging or auditing. - **Constraints**: Must match an existing Access Rule ID. -#### 5.1.3 `_source_company_id` +#### 5.1.3 `_source_company_id` (NOT YET IMPLEMENTED) - **Definition**: Identifier for the company or entity providing the Access Rule. - **Data Type**: String or Numeric ID - **Usage**: Used in the `WHERE` clause to specify data shared by a particular provider. Can also be used in the `SELECT` clause for output. - **Constraints**: Must match an existing provider ID. +#### 5.1.4 `_source_company_slug` +- **Definition**: A unique identifier for a company usually created off of a company's name. +- **Data Type**: String +- **Usage**: Used in the `WHERE` clause to specify data shared by a particular provider. Can also be used in the `SELECT` clause for output. +- **Constraints**: Must be globally unique and only contain alphanumeric characters. + +#### 5.1.5 `_access_rule_name` +- **Definition**: a unique, human-readable name for a specific access rule in a company seat. +- **Data Type**: String +- **Usage**: Typically used in the `WHERE` clause to specify which Access Rule to apply for the query. Can also appear in the `SELECT` clause for debugging or auditing. +- **Constraints**: Must be unique within a company seat and only contain alphanumeric characters. + ### 5.2 Identifier Quoting and Referencing - **Quoting Rule**: Identifiers not starting with a lowercase letter [a-z] ***MUST*** be enclosed in double quotation marks (`"`). @@ -317,13 +329,13 @@ FROM ... ``` -### 10.6 Querying via Access Rules +### 10.6 Querying an Access Rule Directly -NQL supports querying internal or external access rules directly, in addition to the Rosetta Stone attribute catalog and dataset ids, through `_access_rule_id`. +An access rule has two identifiers: a `name` and an `id`. NQL supports querying internal or external access rules directly, in addition to the Rosetta Stone attribute catalog and dataset ids, through `_access_rule_name` and not `access_rule_id`. #### 10.6.1 Querying Internal Access Rules -An Access Rule ID, defined in section 5.1.2 as an identifier for the Access Rule that governs the query's permission, is added after the company identifier. When querying data in your own company seat, an access rule id always follows `company_data`. +An access rule name, defined in section 5.1.5 as a unique, human-readable name for a specific access rule in a company seat, is added after the company identifier. When querying data in your own company seat, an access rule name always follows `company_data`. ##### Example @@ -334,7 +346,7 @@ FROM company_data.access_rule_for_private_deal pd #### 10.6.2 Querying External Access Rules -An Access Rule ID, defined in section 5.1.2 as an identifier for the Access Rule that governs the query's permission, is added after the company identifier. When querying data in an external company seat, an access rule id always follows `_source_company_id`. +An access rule name, defined in section 5.1.5 as a unique, human-readable name for a specific access rule in a company seat, is added after the company identifier. When querying data in an external company seat, an access rule name always follows `_source_company_slug`. ##### Example @@ -347,7 +359,7 @@ FROM _source_company_id.access_rule_unique_name_1 teams #### 10.7.1 `_rosetta_stone` - NQL supports Attribute querying via the Rosetta Embedded Namespace is facilitated by `_rosetta_stone`, a direct method to query Rosetta Stone attributes. `_rosetta_stone` acts as an attribute reference within the dataset or access rule identifier. In case of an absence of mappings or an incorrect attribute reference, the query will return an error. + NQL supports Attribute querying via the Rosetta Embedded Namespace is facilitated by `_rosetta_stone`, a direct method to query Rosetta Stone attributes. `_rosetta_stone` acts as an attribute reference within the dataset or access rule. `_rosetta_stone` must follow either a dataset's `unique_name`, a dataset's `id`, or an access rule's `name`. In case of an absence of mappings or an incorrect attribute reference, the query will return an error. ##### Basic Usage @@ -356,8 +368,8 @@ FROM _source_company_id.access_rule_unique_name_1 teams FROM dataset_source ``` - - **`ds_identifier`**: Alias or identifier for the dataset. - - **`attribute_name`**: The name of the Rosetta Stone attribute to be mapped. + - **`ds_identifier`**: Alias or identifier for the dataset. A dataset can be referenced by its `id` or `unique_name`. + - **`attribute_name`**: The name of the Rosetta Stone attribute that is being selected. - **`alias_name`**: An optional alias for the selected attribute. ##### Example with Single Dataset From 46e4dc0a4c9788fd8a614ea87fd8ea79031f9581 Mon Sep 17 00:00:00 2001 From: linehanMatt Date: Tue, 26 Dec 2023 15:31:39 -0500 Subject: [PATCH 4/7] changed company slug and access rule name --- nql.md | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-) diff --git a/nql.md b/nql.md index bce61fa..da14283 100644 --- a/nql.md +++ b/nql.md @@ -116,13 +116,13 @@ NQL ***SHALL*** support the following keywords. - **Usage**: Used in the `WHERE` clause to specify data shared by a particular provider. Can also be used in the `SELECT` clause for output. - **Constraints**: Must match an existing provider ID. -#### 5.1.4 `_source_company_slug` +#### 5.1.4 `_source_company_slug` (NOT YET IMPLEMENTED) - **Definition**: A unique identifier for a company usually created off of a company's name. - **Data Type**: String - **Usage**: Used in the `WHERE` clause to specify data shared by a particular provider. Can also be used in the `SELECT` clause for output. - **Constraints**: Must be globally unique and only contain alphanumeric characters. -#### 5.1.5 `_access_rule_name` +#### 5.1.5 `_access_rule_name` (NOT YET IMPLEMENTED) - **Definition**: a unique, human-readable name for a specific access rule in a company seat. - **Data Type**: String - **Usage**: Typically used in the `WHERE` clause to specify which Access Rule to apply for the query. Can also appear in the `SELECT` clause for debugging or auditing. @@ -331,7 +331,7 @@ FROM ### 10.6 Querying an Access Rule Directly -An access rule has two identifiers: a `name` and an `id`. NQL supports querying internal or external access rules directly, in addition to the Rosetta Stone attribute catalog and dataset ids, through `_access_rule_name` and not `access_rule_id`. +An access rule has two identifiers: a `name` and an `id`. NQL supports querying internal or external access rules directly, in addition to the Rosetta Stone attribute catalog and dataset ids, through `access_rule_name` and not `access_rule_id`. #### 10.6.1 Querying Internal Access Rules @@ -346,7 +346,7 @@ FROM company_data.access_rule_for_private_deal pd #### 10.6.2 Querying External Access Rules -An access rule name, defined in section 5.1.5 as a unique, human-readable name for a specific access rule in a company seat, is added after the company identifier. When querying data in an external company seat, an access rule name always follows `_source_company_slug`. +An access rule name, defined in section 5.1.5 as a unique, human-readable name for a specific access rule in a company seat, is added after the company identifier. When querying data in an external company seat, an access rule name always follows `company_slug`. ##### Example @@ -482,6 +482,20 @@ WHERE # CHANGE LOG +## Update 2023-12-26 +### Section 5 - Reserved Fields + +- `company_slug` as an identifier to target specific companies. company_slug is used in the `WHERE` clause to specify data shared by a particular provider. Can also be used in the `SELECT` clause for output. +- `access_rule_name` as an identifier to target specific access rules. _access_rule_name is used in the `WHERE` clause to specify which Access Rule to apply for the query. Can also appear in the `SELECT` clause for debugging or auditing. + +### Section 10 - CREATING MATERIALIZED VIEWS + +- Updated to include `REFRESH_SCHEDULE`, which defines the frequency of updates for the materialized view. +- Included the new country_code_3_to_2() UDF. +- Updated to explain querying access rules directly. Internal access rules are targeted using `access_rule_name` and the 'company_data' identifier and external access rules are targeted using `company_slug` and `access_rule_name` +- Created a new section explaining the Rosetta Embedded Namespace. The Rosetta Embedded Namespace is facilitated by `_rosetta_stone`, a direct method to query Rosetta Stone attributes. + + ## Update 2023-11-05 ### Section 2 - Scope - Revised to highlight NQL's integration with the Narrative platform. From d9a246892f17796d6a6470a065e30c39118fa754 Mon Sep 17 00:00:00 2001 From: linehanMatt Date: Tue, 26 Dec 2023 15:35:15 -0500 Subject: [PATCH 5/7] changed company slug and access rule name --- nql.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/nql.md b/nql.md index da14283..a93a0bf 100644 --- a/nql.md +++ b/nql.md @@ -485,14 +485,14 @@ WHERE ## Update 2023-12-26 ### Section 5 - Reserved Fields -- `company_slug` as an identifier to target specific companies. company_slug is used in the `WHERE` clause to specify data shared by a particular provider. Can also be used in the `SELECT` clause for output. -- `access_rule_name` as an identifier to target specific access rules. _access_rule_name is used in the `WHERE` clause to specify which Access Rule to apply for the query. Can also appear in the `SELECT` clause for debugging or auditing. +- `_source_company_slug` as an identifier to target specific companies. company_slug is used in the `WHERE` clause to specify data shared by a particular provider. Can also be used in the `SELECT` clause for output. +- `_access_rule_name` as an identifier to target specific access rules. `_access_rule_name` is used in the `WHERE` clause to specify which Access Rule to apply for the query. Can also appear in the `SELECT` clause for debugging or auditing. ### Section 10 - CREATING MATERIALIZED VIEWS - Updated to include `REFRESH_SCHEDULE`, which defines the frequency of updates for the materialized view. - Included the new country_code_3_to_2() UDF. -- Updated to explain querying access rules directly. Internal access rules are targeted using `access_rule_name` and the 'company_data' identifier and external access rules are targeted using `company_slug` and `access_rule_name` +- Updated to explain querying access rules directly. Internal access rules are targeted using `access_rule_name` and the `company_data` identifier and external access rules are targeted using `company_slug` and `access_rule_name` - Created a new section explaining the Rosetta Embedded Namespace. The Rosetta Embedded Namespace is facilitated by `_rosetta_stone`, a direct method to query Rosetta Stone attributes. From 813313bac5fdd8419a18d9256756aea09f3101f8 Mon Sep 17 00:00:00 2001 From: linehanMatt Date: Tue, 26 Dec 2023 15:42:36 -0500 Subject: [PATCH 6/7] changed company slug and access rule name --- nql.md | 4 ---- 1 file changed, 4 deletions(-) diff --git a/nql.md b/nql.md index a93a0bf..bd6e5b1 100644 --- a/nql.md +++ b/nql.md @@ -483,10 +483,6 @@ WHERE # CHANGE LOG ## Update 2023-12-26 -### Section 5 - Reserved Fields - -- `_source_company_slug` as an identifier to target specific companies. company_slug is used in the `WHERE` clause to specify data shared by a particular provider. Can also be used in the `SELECT` clause for output. -- `_access_rule_name` as an identifier to target specific access rules. `_access_rule_name` is used in the `WHERE` clause to specify which Access Rule to apply for the query. Can also appear in the `SELECT` clause for debugging or auditing. ### Section 10 - CREATING MATERIALIZED VIEWS From 6a32391995b22fe27ccd582efef6078d44a64325 Mon Sep 17 00:00:00 2001 From: linehanMatt Date: Thu, 28 Dec 2023 10:20:39 -0500 Subject: [PATCH 7/7] updated wording and removed updates to section five --- nql.md | 57 +++++++++++++++++++++++++-------------------------------- 1 file changed, 25 insertions(+), 32 deletions(-) diff --git a/nql.md b/nql.md index bd6e5b1..69d47bd 100644 --- a/nql.md +++ b/nql.md @@ -116,18 +116,6 @@ NQL ***SHALL*** support the following keywords. - **Usage**: Used in the `WHERE` clause to specify data shared by a particular provider. Can also be used in the `SELECT` clause for output. - **Constraints**: Must match an existing provider ID. -#### 5.1.4 `_source_company_slug` (NOT YET IMPLEMENTED) -- **Definition**: A unique identifier for a company usually created off of a company's name. -- **Data Type**: String -- **Usage**: Used in the `WHERE` clause to specify data shared by a particular provider. Can also be used in the `SELECT` clause for output. -- **Constraints**: Must be globally unique and only contain alphanumeric characters. - -#### 5.1.5 `_access_rule_name` (NOT YET IMPLEMENTED) -- **Definition**: a unique, human-readable name for a specific access rule in a company seat. -- **Data Type**: String -- **Usage**: Typically used in the `WHERE` clause to specify which Access Rule to apply for the query. Can also appear in the `SELECT` clause for debugging or auditing. -- **Constraints**: Must be unique within a company seat and only contain alphanumeric characters. - ### 5.2 Identifier Quoting and Referencing - **Quoting Rule**: Identifiers not starting with a lowercase letter [a-z] ***MUST*** be enclosed in double quotation marks (`"`). @@ -316,7 +304,7 @@ SELECT ``` #### 10.5.2 `country_code_3_to_2()` -The country_code_3_to_2('column') function takes in a column of ISO 3166-1 alpha-3 country code(s) and converts them to The ISO 3166-1 alpha-2 country code(s). The function is useful for matching the standard output for the `iso_3166_1_country` Rosetta Stone attribute, expressed as ISO 3166-1 alpha-2 country codes. +The country_code_3_to_2('column') function takes in a single column of ISO 3166-1 alpha-3 country code(s) and converts it to ISO 3166-1 alpha-2 country code(s). The function is useful for matching the standard output of the `iso_3166_1_country` Rosetta Stone attribute, expressed as ISO 3166-1 alpha-2 country codes. ##### Example: @@ -325,17 +313,19 @@ CREATE MATERIALIZED VIEW "country_code_sample" AS SELECT country_code_3_to_2(my_dataset.country_code) as two_letter_codes FROM - company_date.my_dataset + company_data.my_dataset ... ``` ### 10.6 Querying an Access Rule Directly -An access rule has two identifiers: a `name` and an `id`. NQL supports querying internal or external access rules directly, in addition to the Rosetta Stone attribute catalog and dataset ids, through `access_rule_name` and not `access_rule_id`. +An access rule has two identifiers: an `access_rule_name` and an `access_rule_id`. Access rule names are human readable and must be created explicitly, while access rule ids are created automatically during the initial set up for each access rule. NQL supports querying access rules directly through `access_rule_name` and not `access_rule_id`. + +NQL supports querying internal access rules (access rules on datasets in the same company seat) or external access rules (access rules on datasets in a different company seat) directly. Querying an access rule is the third method of querying datasets in NQL, in addition to the Rosetta Stone attribute catalog and dataset ids. #### 10.6.1 Querying Internal Access Rules -An access rule name, defined in section 5.1.5 as a unique, human-readable name for a specific access rule in a company seat, is added after the company identifier. When querying data in your own company seat, an access rule name always follows `company_data`. +An access rule name is added after the company identifier. When querying data in your own company seat, an access rule name always follows `company_data`. ##### Example @@ -346,20 +336,20 @@ FROM company_data.access_rule_for_private_deal pd #### 10.6.2 Querying External Access Rules -An access rule name, defined in section 5.1.5 as a unique, human-readable name for a specific access rule in a company seat, is added after the company identifier. When querying data in an external company seat, an access rule name always follows `company_slug`. +An access rule name is added after the company identifier. When querying data in your own company seat, an access rule name always follows `company_slug`. ##### Example ```sql SELECT teams.baseball_teams -FROM _source_company_id.access_rule_unique_name_1 teams +FROM company_slug.access_rule_unique_name_1 teams ``` ### 10.7 Embedded Namespaces in NQL #### 10.7.1 `_rosetta_stone` - NQL supports Attribute querying via the Rosetta Embedded Namespace is facilitated by `_rosetta_stone`, a direct method to query Rosetta Stone attributes. `_rosetta_stone` acts as an attribute reference within the dataset or access rule. `_rosetta_stone` must follow either a dataset's `unique_name`, a dataset's `id`, or an access rule's `name`. In case of an absence of mappings or an incorrect attribute reference, the query will return an error. + NQL supports attribute querying via the Rosetta Embedded Namespace. This namespace is facilitated by `_rosetta_stone`, a direct method to query Rosetta Stone attributes. `_rosetta_stone` acts as an attribute reference within the dataset or access rule. `_rosetta_stone` must follow either a dataset's `unique_name`, a dataset's `id`, or an access rule's `name`. In case of an absence of mappings or an incorrect attribute reference, the query will return an error. ##### Basic Usage @@ -400,6 +390,16 @@ FROM _source_company_id.access_rule_unique_name_1 teams - The first Rosetta Stone attribute (**`attribute_1`**) is being pulled from dataset **`ds_123`**. - The second Rosetta Stone attribute (**`attribute_2`**) is being pulled from dataset **`ds_456`**. +##### Example of Nested Properties + For nested properties, the same dot notation is used within the **`_rosetta_stone`** namespace. + + ```sql + SELECT + ds_123._rosetta_stone."nested"."attribute" AS nested_attribute + FROM + company_data.ds_123 AS ds_123 + ``` + ##### Example of Filtering with Rosetta Attributes ```sql @@ -413,15 +413,6 @@ FROM _source_company_id.access_rule_unique_name_1 teams Here, the **`WHERE`** clause uses the Rosetta attribute **`unique_id.value`** from dataset **`ds_123`** for filtering. -##### Example of Nested Properties - For nested properties, the same dot notation is used within the **`_rosetta_stone`** namespace. - - ```sql - SELECT - ds_123._rosetta_stone."nested"."attribute" AS nested_attribute - FROM - company_data.ds_123 AS ds_123 - ``` ## 12. Example Queries @@ -486,10 +477,12 @@ WHERE ### Section 10 - CREATING MATERIALIZED VIEWS -- Updated to include `REFRESH_SCHEDULE`, which defines the frequency of updates for the materialized view. -- Included the new country_code_3_to_2() UDF. -- Updated to explain querying access rules directly. Internal access rules are targeted using `access_rule_name` and the `company_data` identifier and external access rules are targeted using `company_slug` and `access_rule_name` -- Created a new section explaining the Rosetta Embedded Namespace. The Rosetta Embedded Namespace is facilitated by `_rosetta_stone`, a direct method to query Rosetta Stone attributes. +- Create Materialized View syntax was updated to include `REFRESH_SCHEDULE`, which defines the frequency of updates for the materialized view. +- The UDF section includes a new function: `country_code_3_to_2()`. +- NQL supports targeting access rules directly. + - Internal access rules are targeted using `access_rule_name` and the `company_data` identifier. + - External access rules are targeted using `company_slug` and `access_rule_name`. +- Introduction of the Rosetta Embedded Namespace as a way to query attributes from specific access rules or datasets. The Rosetta Embedded Namespace is facilitated by `_rosetta_stone`. ## Update 2023-11-05