Spark on Google Cloud

These instructions show how to deploy a Spark application on Google Cloud Platform.

1. Compile your application as an uber .jar file and upload it to a Google Cloud Storage bucket.
2. In the Dataproc console select Create cluster.
3. Once you have created the cluster, in the Jobs console select Submit job:
   • Set Job type to Spark
   • In the Main class or jar field provide full class name of your application (including the package)
   • In the Jar files field provide the link to your uploaded .jar file in the form gs://<bucket-name>/<jar-path>

It takes ~10 mins to spin up the cluster before your application is executed.