Docs: Document BigQuery connector credentials and configuration (#3326)

* Docs: Document BigQuery connector credentials and configuration

* project_id description

---------

Co-authored-by: e.sevastyanov <[email protected]>

docs/docs/deploy/credentials/bigquery.md

@@ -0,0 +1,64 @@
+---
+title: BigQuery 
+description: Connect to data in BigQuery
+sidebar_label: BigQuery
+sidebar_position: 50
+---
+
+<!-- WARNING: There are links to this page in source code. If you move it, find and replace the links and consider adding a redirect in docusaurus.config.js. -->
+
+## How to configure credentials in Rill
+
+How you configure access to BigQuery depends on whether you are developing a project locally using `rill start` or are setting up a deployment using `rill deploy`.
+
+### Configure credentials for local development
+
+When developing a project locally, Rill uses the credentials configured in your local environment using the Google Cloud CLI (`gcloud`). Follow these steps to configure it:
+
+1. Open a terminal window and run `gcloud auth list` to check if you already have the Google Cloud CLI installed and authenticated. 
+
+2. If it did not print information about your user, follow the steps on [Install the Google Cloud CLI](https://cloud.google.com/sdk/docs/install-sdk). Make sure to run `gcloud init` after installation as described in the tutorial.
+
+You have now configured Google Cloud access from your local environment. Rill will detect and use your credentials next time you try to ingest a source.
+
+### Configure credentials for deployments on Rill Cloud
+
+When deploying a project to Rill Cloud, Rill requires you to explicitly provide a JSON key file for a Google Cloud service account with access to BigQuery used in your project. 
+
+When you first deploy a project using `rill deploy`, you will be prompted to provide credentials for the remote sources in your project that require authentication.
+
+If you subsequently add sources that require new credentials (or if you input the wrong credentials during the initial deploy), you can update the credentials used by Rill Cloud by running:
+```
+rill env configure
+```
+Note that you must `cd` into the Git repository that your project was deployed from before running `rill env configure`.
+
+## How to create a service account using the Google Cloud Console
+
+Here is a step-by-step guide on how to create a Google Cloud service account with access to BigQuery:
+
+1. Navigate to the [Service Accounts page](https://console.cloud.google.com/iam-admin/serviceaccounts) under "IAM & Admin" in the Google Cloud Console.
+
+2. Click the "Create Service Account" button at the top of the page.
+
+3. In the "Create Service Account" window, enter a name for the service account, then click "Create and continue".
+
+4. In the "Role" field, search for and select the following [BigQuery roles](https://cloud.google.com/bigquery/docs/access-control): 
+   - [roles/bigquery.dataViewer](https://cloud.google.com/bigquery/docs/access-control#bigquery.dataViewer) (Lowest-level resources: Table, View)
+     - provides the ability to read data and metadata from the project's datasets/dataset's tables/table or view
+   - [roles/bigquery.readSessionUser](https://cloud.google.com/bigquery/docs/access-control#bigquery.readSessionUser) (Lowest-level resources: Project)
+     - provides the ability to create and use read sessions that can be used to read data from BigQuery managed tables using the Storage API (to read data from BigQuery at high speeds). The role does not provide any other permissions related to BigQuery datasets, tables, or other resources.
+   - [roles/bigquery.jobUser](https://cloud.google.com/bigquery/docs/access-control#bigquery.jobUser) (Lowest-level resources: Project)
+     - provides permissions to run BigQuery-specific jobs (including queries), within the project and respecting limits set by roles above
+
+   Click "Continue", then click "Done".
+
+   **Note**: BigQuery has storage and compute [separated](https://cloud.google.com/blog/products/bigquery/separation-of-storage-and-compute-in-bigquery) from each other so the lowest-level resource where compute-specific roles are granted is a project, while lowest-level for data-specific roles is table/view.
+
+5. On the "Service Accounts" page, locate the service account you just created and click on the three dots on the right-hand side. Select "Manage Keys" from the dropdown menu.
+
+6. On the "Keys" page, click the "Add key" button and select "Create new key".
+
+7. Choose the "JSON" key type and click "Create".
+
+8. Download and save the JSON key file to a secure location on your computer.

docs/docs/deploy/credentials/credentials.md

@@ -16,3 +16,4 @@ For instructions on how to create a service account and set credentials in Rill
 - [Postgres](./postgres.md)
 - [MotherDuck](./motherduck.md)
 - [Amazon Athena](./athena.md)
+- [BigQuery](./bigquery.md)

docs/docs/reference/project-files/sources.md

@@ -46,6 +46,9 @@ In your Rill project directory, create a `<source_name>.yaml` file in the `sourc
 **`workgroup`**
 - Optionally sets a workgroup for Athena connector. The workgroup is also used to determine an output location. A workgroup may override `output_location` if [Override client-side settings](https://docs.aws.amazon.com/athena/latest/ug/workgroups-settings-override.html) is turned on for the workgroup.  
 
+**`project_id`**
+- Sets a project id to be used to run BigQuery [jobs](https://cloud.google.com/bigquery/docs/jobs-overview) (mandatory for BiqQuery connection)
+
 **`glob.max_total_size`**
  — Applicable if the URI is a glob pattern. The max allowed total size (in bytes) of all objects matching the glob pattern.
   - default value is _`10737418240 (10GB)`_