Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add spark-scala-quickstart #148

Closed
wants to merge 2 commits into from
Closed

add spark-scala-quickstart #148

wants to merge 2 commits into from

Conversation

NiloFreitas
Copy link

Dataproc - Spark Scala Quickstart is an effort to assist in the creation of Spark jobs written in Scala to run on Dataproc.
It provides different pre-implemented Spark jobs and technical guides to run them on GCP.
It is all based on the on the WordCount ETL example with common sources and sinks (Kafka, GCS, BigQuery, etc).
It demonstrates how to run Spark jobs using Dataproc Submit, Serverless, Workflow and how to orchestrate them with Cloud Composer.

@NiloFreitas
Copy link
Author

#149

@davidrabinowitz
Copy link
Member

Hi @NiloFreitas thanks for the quickstart. It seems that many other files were added by mistake, among them existing notebooks, codelabs, etc. Can you please verify that only the relevant files are in the PR?

@NiloFreitas
Copy link
Author

NiloFreitas commented Mar 21, 2022

Hi @davidrabinowitz . What files do you mean? I could not find what you referring to.
My pull request contains 1 commit of the quickstart guide. It is composed of several scala and python code, but not notebooks and codelabs. All code in the PR was written by me.

@dedeco
Copy link

dedeco commented Apr 12, 2022

This code can helps a lot! ;-)

@NiloFreitas NiloFreitas closed this Nov 3, 2022
@NiloFreitas NiloFreitas deleted the spark-scala-quickstart branch November 3, 2022 19:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants