Skip to content

Commit

Permalink
Update module-09-6-data-lineage-with-dataproc.md (#62)
Browse files Browse the repository at this point in the history
  • Loading branch information
anagha-google authored Jan 16, 2025
1 parent c7ed1d0 commit 3ee661d
Showing 1 changed file with 3 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -149,12 +149,14 @@ gcloud dataproc clusters create $DPGCE_CLUSTER_NM \
--num-workers 2 \
--worker-machine-type n1-standard-4 \
--worker-boot-disk-size 500 \
--image-version 2.1.22-debian11 \
--image-version 2.1.75-debian11 \
--optional-components JUPYTER \
--dataproc-metastore projects/$PROJECT_ID/locations/$LOCATION/services/$DPMS_NM \
--properties 'dataproc:dataproc.lineage.enabled=true' \
--impersonate-service-account $UMSA_FQN \
--scopes https://www.googleapis.com/auth/cloud-platform
# 2.2 does not support open lineage
```
This should take 1-2 minutes (or less) to complete.
Expand Down

0 comments on commit 3ee661d

Please sign in to comment.