Skip to content

Cloud Tools for Eclipse Technical Design

Jump to bottom
Elliotte Rusty Harold edited this page Mar 30, 2016 · 28 revisions

Cloud Tools for Eclipse (C4E) supports development of Java servlet applications on the Google Cloud Platform (GCP) inside the Eclipse Integrated Development Environment (IDE) version 4.5 and later. It enables you to build web applications and sites that run on top of:

  • App Engine Standard
  • App Engine Flex
  • Google Compute Engine (GCE)
  • Google Container Environment (GKE)

Each of these is a separate Eclipse Project nature.

Build environment

All code will be hosted in the open on Github. Design docs, issue trackers, and the like will also be hosted on Github.

Build will be based on Maven and tycho.

Eclipse settings will be checked into the repo for easy development and import.

Travis will be used for continuous integration and testing.

Java 7 or later will be assumed.

File encoding is UTF-8.

The Google Java Style Guide is followed.

Official plugin releases will be signed by Google and can be installed from Google's Eclipse download site.

Eclipse integration

The plugin will be based on the Eclipse Web Tools Platform (WTP).

There will be a single feature and and multiple plugins. Note that here "features" and "plugins" are technical terms that have almost the reverse of their normal meanings. A "feature" is a group of bundled functionality installed as a unit. Each feature contains multiple "plugins" that provide specific functionality such as authorization, deployment, problem reports, etc.

GCP integration

For most interactions with the Google Cloud Platform, C4E will shell out to the gcloud tool rather than invoking network services directly. The details are hidden from the user.

The latest version of gcloud will be bundled with C4E. For simplicity the plugin version will match the version of gcloud (currently 94.0).

New application wizards

Each wizard will provide a fully configured, ultra-simple Eclipse project.

App Engine Standard

The wizard generates a new project containing these files and directories:

  • src directory
    • HelloWorldServlet.java that responds only to GET requests with a constant message
  • test directory
    • HelloWorldServletTest.java
  • war directory
    • WEB-INF directory
      • appengine-web.xml AppEngine configuration file
      • lib directory containing any jars the servlet requires
    • index.html with a link that points to the servlet.
  • web.xml servlet configuration file
  • README file
  • Maven pom.xml file

In addition, servlet.jar, junit4.jar, and any other required libraries not bundled with the JDK will be added to the project classpath. The project will be tagged with an App Engine Standard nature. The project is set to build with Java 7 (the latest version supported by App Engine Standard).

App Engine Flexible

The sample for the Flexible environment is almost identical to the sample for the standard environment except:

  • The project will be tagged with an App Engine Flexible nature.
  • app.yaml replaces appengine-web.xml
  • The javac source and target are set to Java 8

Load existing projects

If M2Eclipse is installed in Eclipse, then an existing Maven based project can be imported into Eclipse in the usual way ("File > Import > Existing Maven Projects"). The project will be tagged with the matching nature (standard, flex, GCE, GKE).

Run your web application on your local workstation for manual testing

The run command from gcloud is

dev_appserver.py --port 8080 war

This integrates into Eclipse WTP via the com.google.gcp.eclipse.appengine.localserver/gcloud_serverdef.xml file.

WTP provides functionality such as changing the default port and specifying the install location of gcloud.

Switching from a run to debug environment is also provided by Eclipse, with no additional work on our part.

Code validations for problems specific to the App Engine Standard environment

C4E will reference a white list of Java classes available in the App Engine Standard environment. It checks that only Google App Engine supported JRE types are used by a CompilationUnit, and no classes are imported from com.google.appengine.repackaged.

It will be possible for the user to exclude classes and packages from these checks; e.g. testing packages that are not deployed to the server.

TBD: canonical whitelist location for programmatic use

Authentication and Authorization

The ide-login library provides IDE agnostic functionality for authorizing code to deploy, monitor, and manage GCP resources.

C4E delegates this to gcloud auth login. gcloud launches a browser window that requests these credentials:

  • Know who you are on Google
  • View your email address
  • View and manage the files in your Google Drive
  • View and manage your applications deployed on Google App Engine

Assuming the user consents, the server sends an oAuth 2.0 access token back. gcloud stores it on local disk and presents it with future deployments.

TBD: if we deploy before authing, does gcloud run this flow automatically?

TBD: UI. Where do we put the "login" button/menu item/ etc.?

Deploy web application

Prerequisites:

In order to deploy, several things must first be done in the DevConsole outside of Eclipse. In particular,

  1. The environment you're deploying to must be enabled in the Google Developer Console.

  2. Billing should be turned on. (Required for GCE, GKE, and App Engine Flex).

  3. Create a project with name and ID in the Developer console.

Deployment Details

Deployment will be managed via WTP's org.eclipse.jst.server.generic.antpublisher. We supply an ant script named gcp.xml that contains targets to deploy and undeploy the application. WTP executes these targets at user request.

These targets invoke gcloud preview app deploy with the correct arguments to locate the user's authorization credentials and upload the artifacts to Google servers.

These targets will then invoke gcloud preview app deploy. Here we run into a problem. gcloud does not support deployment of either war directories or appengine-web.xml files. It needs an app.yaml file instead. Unless this is fixed, we "stage" the application to generate an app.yaml from our appengine-web.xml file. The maven gcloud plugin has code for doing this we can imitate.

Wrapping gcloud

Many operations, deployment and authorization in particular, amount to GUI layers over the gcloud tool. How exactly we invoke gcloud is an open question. There are three basic approaches we can take:

  1. Ad hoc, write each call into the relevant code as we go, mostly through java.lang.Runtime.exec().

  2. Hide gcloud behind a class or classes that provides a more abstract API. Build this class as needed and refactor as we go.

  3. Build on top of the gcloud maven plugin.

  4. Use the gcloud-java library.

Number 3 isn't ready. Number 1 seems too ugly and brittle. Number 4 seems to be missing features we need, most notably deployment, so I'm inclined to push for option 2. I don't feel that we have a strong enough grasp of the requirements for a Java API for managing GCP resources to design more than that at this point in time.

As we learn more and refactor, we can consider breaking this piece out into a separate project or perhaps as a contribution to gcloud-java. This requires making sure that dependencies are one-way. That is, our abstraction layer should not depend on eclipse, SWT, and so forth.

TBD: How do we support progress reporting, especially if the operation takes a while?

Analytics

When the plugin is first installed, the user is asked to opt in to usage tracking.

If and only if they agree, then the plugin sends non-personally identifiable pings for usage of these features to the C4E Analytics account:

  • installations
  • uninstalls
  • crashes + associated stack trace
  • deploys
  • local emulation runs

At no time do we include or send the project ID, the project name, the Gaia ID, the username, or any other such personally identifiable information. We merely track the total number of events. The only things we track that are not simple counters are exception stack traces from unanticipated exceptions.

The Analytics account ID is not stored in the source repository on Github. Instead it is supplied when the official artifact is built.

C4E communicates with Google Analytics via the Analytics Measurement Protocol. This is demonstrated in AnalyticsPingManager in the old plugin.

End User Documentation

Contributor Docs

Clone this wiki locally