Add ability to configure the external firewall

This change also allows complete access between all hosts in the same nodeset

README.md

@@ -10,7 +10,7 @@
 
 Beaker library to use the Google hypervisor
 
-# How to use this wizardry
+# How to use this module
 
 This is a gem that allows you to use hosts with [Google Compute](https://cloud.google.com/compute) hypervisor with [Beaker](https://github.com/voxpupuli/beaker).
 
@@ -46,6 +46,7 @@ The behavior of this library can be configured using either the beaker host conf
 | gce_zone             | true             |                                  | The zone to place compute instances in. The region is calculated from the zone name.                                                                                                                                                                                                          |
 | gce_network          | false            | Default                          | The name of the network to attach to instances. If the project uses the default network, this and `gce_subnetwork` can be left empty.                                                                                                                                                         |
 | gce_subnetwork       | false            | Default                          | The name of the subnetwork to attach to the instances network interface. If the Default network is not used, this must be supplied.                                                                                                                                                           |
+| gce_ports            | false            | `[ ]`                            | A comma separated list of ports to add to the external firewall. Each port is specified in the format `number/protocol` where protocol is one of `tcp`, `udp`, `icmp`, `esp`, `ah`, `ipip`, or `sctp`. **NOTE:** Port `22/tcp` is required for beaker to function and is automatically added to the firewall. |
 | gce_ssh_private_key  | false            | $HOME/.ssh/google_compute_engine | The file path of the private key to use to connect to instances. If using the key created by the gcloud tool, this can be left blank.                                                                                                                                                         |
 | gce_ssh_public_key   | false            | <gce_ssh_private_key>.pub        | The file path of the public key to upload to the instance. If left blank, attempt to use the file at `gce_ssh_private_key` with a `.pub` extension.                                                                                                                                           |
 | gce_machine_type     | false            | e2-standard-4                    | The machine type to use for the instance. If the `BEAKER_gce_machine_type` environment variable is set, it will be used for all hosts.                                                                                                                                                        |
@@ -55,14 +56,22 @@ The behavior of this library can be configured using either the beaker host conf
 
 All the variables in the list can be set in the Beaker host configuration file, or the ones starting with `gce_` can be overridden by environment variables in the form `BEAKER_gce_...`. i.e. To override the `gce_machine_type` setting in the environment, set `BEAKER_gce_machine_type`.
 
+## Networking
+
+Each run of beaker creates a pair of firewalls to protect the hosts, and internal host-to-host firewall, and an external firewall between the hosts and the internet.
+
+The internal firewall allows all communication between hosts in the node set, while keeping them isolated from any other Beaker jobs that may be running in the same environment. This firewall is attached to all hosts in the test set, and allows all `tcp`, `udp`, and `icmp` traffic.
+
+The external firewall allows outside communication from the internet into the hosts in the test. Due to constrains of the Beaker system, the firewall accepts any source IP (`0.0.0.0/0`) and the SSH port (`22/tcp`) is always allowed. Other ports may be added by using the `gce_ports` configuration option or the `BEAKER_gce_ports` environment variable as a comma separated list of `port/proto` values where `port` is the port number or range, and proto is one of `tcp`, `udp`, `icmp`, `esp`, `ah`, `ipip`, or `sctp`. The port number is only used for `tcp`, `udp`, and `sctp` protocols. Any value, such as `-1` can be used for the other protocols.
+
 # Cleanup
 
 In cases where the beaker process is killed before finishing, it may leave resources in GCP. These resources will need to be manually deleted.
 
 | Resource Type | Name Pattern        | Count                                       |
 | ------------- | ------------------- | ------------------------------------------- |
-| Firewall      | `beaker-<number>-*` | 1                                           |
-| Instance      | `beaker-*`          | One or more depending on test configuration |
+| Firewall      | `beaker-<number>-*` | 2                                           |
+| Instance      | `beaker-<number>-*` | One or more depending on test configuration |
 
 # Contributing
 

lib/beaker/hypervisor/google_compute.rb

@@ -79,27 +79,34 @@ def initialize(google_hosts, options)
       @options = options
       @logger = options[:logger]
       @hosts = google_hosts
-      @firewall = ''
+      @external_firewall_name = ''
+      @internal_firewall_name = ''
       @gce_helper = GoogleComputeHelper.new(options)
     end
 
     # Create and configure virtual machines in the Google Compute Engine,
     # including their associated disks and firewall rules
     def provision
       start = Time.now
-      test_group_identifier = "beaker-#{start.to_i}-"
+      test_group_identifier = "beaker-#{start.to_i}"
 
       # set firewall to open pe ports
       network = @gce_helper.get_network
 
-      @firewall = test_group_identifier + generate_host_name
+      @external_firewall_name = test_group_identifier + '-external'
 
-      @gce_helper.create_firewall(@firewall, network)
-
-      @logger.debug("Created Google Compute firewall #{@firewall}")
+      # Always allow ssh from anywhere as it's needed for Beaker to run
+      @gce_helper.create_firewall(@external_firewall_name, network, allow: @options[:gce_ports] + ['22/tcp'], source_ranges: ['0.0.0.0/0'], target_tags: [test_group_identifier])
+
+      @logger.debug("Created External Google Compute firewall #{@external_firewall_name}")
 
+      # Create a firewall that opens everything between all the hosts in this test group
+      @internal_firewall_name = test_group_identifier + '-internal'
+      internal_ports = ['1-65535/tcp', '1-65535/udp', '-1/icmp']
+      @gce_helper.create_firewall(@internal_firewall_name, network, allow: internal_ports, source_tags: [test_group_identifier], target_tags: [test_group_identifier])
+      @logger.debug("Created test group Google Compute firewall #{@internal_firewall_name}")
+
       @hosts.each do |host|
-
         machine_type_name = ENV.fetch('BEAKER_gce_machine_type', host['gce_machine_type'])
         raise "Must provide a machine type name in 'gce_machine_type'." if machine_type_name.nil?
         # Get the GCE machine type object for this host
@@ -142,7 +149,7 @@ def provision
           raise('You must specify either :image or :family')
         end
 
-        unique_host_id = test_group_identifier + generate_host_name
+        unique_host_id = test_group_identifier + '-' + generate_host_name
 
         boot_size = host['volume_size'] || img.disk_size_gb
 
@@ -162,6 +169,9 @@ def provision
         @logger.debug("Created Google Compute instance for #{host.name}: #{host['vmhostname']}")
         instance = @gce_helper.get_instance(host['vmhostname'])
 
+        @gce_helper.add_instance_tag(host['vmhostname'], test_group_identifier)
+        @logger.debug("Added network tag #{test_group_identifier} to instance")
+
         # Make sure we have a non root/Adminsitor user to log in as
         if host['user'] == "root" || host['user'] == "Administrator" || host['user'].empty?
           initial_user = 'google_compute'
@@ -210,9 +220,6 @@ def provision
 
         host['ip'] = instance.network_interfaces[0].access_configs[0].nat_ip
 
-        # Add the new host to the firewall
-        @gce_helper.add_firewall_tag(@firewall, host['vmhostname'])
-
         if host['disable_root_ssh'] == true
           @logger.info('Not enabling root ssh as disable_root_ssh is true')
         else
@@ -235,7 +242,8 @@ def provision
     # Shutdown and destroy virtual machines in the Google Compute Engine,
     # including their associated disks and firewall rules
     def cleanup
-      @gce_helper.delete_firewall(@firewall)
+      @gce_helper.delete_firewall(@external_firewall_name)
+      @gce_helper.delete_firewall(@internal_firewall_name)
 
       @hosts.each do |host|
         # TODO: Delete any other disks attached during the instance creation

lib/beaker/hypervisor/google_compute_helper.rb

@@ -24,6 +24,8 @@ class GoogleComputeError < StandardError
   DEFAULT_MACHINE_TYPE = 'e2-standard-4'
   DEFAULT_NETWORK_NAME = 'default'
 
+  VALID_PROTOS = ['tcp', 'udp', 'icmp', 'esp', 'ah', 'ipip', 'sctp']
+
   GCP_AUTH_SCOPE = [
     Google::Apis::ComputeV1::AUTH_COMPUTE,
     Google::Apis::OsloginV1::AUTH_CLOUD_PLATFORM_READ_ONLY,
@@ -48,8 +50,18 @@ def initialize(options)
     @options[:gce_network] = ENV.fetch('BEAKER_gce_network', DEFAULT_NETWORK_NAME)
     @options[:gce_subnetwork] = ENV.fetch('BEAKER_gce_subnetwork', nil)
 
+    @configure_ports = ENV.fetch('BEAKER_gce_ports', "").strip
+    # Split the ports based on commas, removing any empty values
+    @options[:gce_ports] = @configure_ports.split(/\s*?,\s*/).reject { |s| s.empty? }
+
     raise 'You must specify a gce_project for Google Compute Engine instances!' unless @options[:gce_project]
 
+    @options[:gce_ports].each do |port|
+      parts = port.split('/', 2)
+      raise "Invalid format for port #{port}. Should be 'port/proto'" unless parts.length == 2
+      proto = parts[1]
+      raise "Invalid value '#{proto}' for protocol in '#{port}'. Must be one of '#{VALID_PROTOS.join("', '")}'" unless VALID_PROTOS.include? proto
+    end
     authorizer = authenticate
     @compute = ::Google::Apis::ComputeV1::ComputeService.new
     @compute.authorization = authorizer
@@ -288,46 +300,161 @@ def list_firewalls
   # @param [::Google::Apis::ComputeV1::Network] network The Google Compute networkin which to create
   # the firewall
   #
+  # @param [Array<String>] allow List of ports to allow through the firewall. One of 'allow' or 'deny' must be specified, but not both.
+  #
+  # @param [Array<String>] deny List of ports to deny through the firewall. One of 'allow' or 'deny' must be specified, but not both.
+  #
+  # @param [Array<String>] source_ranges List of ranges in CIDR format to accept through the firewall. If neither 'source_ranges' 
+  # or 'source_tags' is specified, GCP adds a default 'source_range' of '0.0.0.0/0' (allow all)
+  #
+  # @param [Array<String>] source_tags List of network tags to accept through the firewall. If neither 'source_ranges' 
+  # or 'source_tags' is specified, GCP adds a default 'source_range' of '0.0.0.0/0' (allow all)
+  #
+  # @param [Array<String>] target_ranges List of ranges in CIDR format to apply this firewall. If neither 'target_ranges' 
+  # or 'target_tags' is specified, the firewall applies to all hosts in the VPC
+  #
+  # @param [Array<String>] target_tags List of network tags to apply this firewall. If neither 'target_ranges' 
+  # or 'target_tags' is specified, the firewall applies to all hosts in the VPC
+  #
   # @return [Google::Apis::ComputeV1::Operation]
   #
   # @raise [Google::Apis::ServerError] An error occurred on the server and the request can be retried
   # @raise [Google::Apis::ClientError] The request is invalid and should not be retried without modification
   # @raise [Google::Apis::AuthorizationError] Authorization is required
-  def create_firewall(name, network)
+  def create_firewall(name, network, allow: [], deny: [], source_ranges: [], source_tags: [], target_ranges: [], target_tags: [])
+    # require 'pry-byebug'
+    # binding.pry
+    allowed = []
+    allow.each do |port|
+      parts = port.split('/', 2)
+      # allowed << ::Google::Apis::ComputeV1::Firewall::Allowed.new(ip_protocol: parts[1], ports: [parts[0]])
+      if parts[1] == 'tcp' || parts[1] == 'udp' || parts[1] == 'sctp' then
+        allowed << ::Google::Apis::ComputeV1::Firewall::Allowed.new(ip_protocol: parts[1], ports: [parts[0]])
+      else
+        allowed << ::Google::Apis::ComputeV1::Firewall::Allowed.new(ip_protocol: parts[1])
+      end
+    end
+    denied = []
+    deny.each do |port|
+      parts = port.split('/', 2)
+      if parts[1] == 'tcp' || parts[1] == 'udp' || parts[1] == 'sctp' then
+        denied << ::Google::Apis::ComputeV1::Firewall::Denied.new(ip_protocol: parts[1], ports: [parts[0]])
+      else
+        denied << ::Google::Apis::ComputeV1::Firewall::Denied.new(ip_protocol: parts[1])
+      end
+    end
+
     firewall_object = ::Google::Apis::ComputeV1::Firewall.new(
       name: name,
-      allowed: [
-        ::Google::Apis::ComputeV1::Firewall::Allowed.new(ip_protocol: 'tcp',
-                                                         ports: ['443', '8140', '61613', '8080', '8081', '22']),
-      ],
+      direction: 'INGRESS',
       network: network.self_link,
-      # TODO: Is there a better way to do this?
-      sourceRanges: ['0.0.0.0/0'], # Allow from anywhere
+      allowed: allowed,
+      denied: denied,
+      source_ranges: source_ranges,
+      source_tags: source_tags,
+      target_ranges: target_ranges,
+      target_tags: target_tags,
     )
     operation = @compute.insert_firewall(@options[:gce_project], firewall_object)
     @compute.wait_global_operation(@options[:gce_project], operation.name)
   end
 
+  ##
+  # Get the named firewall
+  #
+  # @param [String] name The name of the firewall
+  #
+  # @return [Google::Apis::ComputeV1::Firewall]
+  #
+  # @raise [Google::Apis::ServerError] An error occurred on the server and the request can be retried
+  # @raise [Google::Apis::ClientError] The request is invalid and should not be retried without modification
+  # @raise [Google::Apis::AuthorizationError] Authorization is required
+  def get_firewall(name)
+    firewall = @compute.get_firewall(@options[:gce_project], name)
+  end
+
+  ##
+  # Add a source range to the firewall.
+  #
+  # @param [String] name The name of the firewall
+  #
+  # @param [String] range The IP range in CIDR format to add to the firewall
+  #
+  # @return [Google::Apis::ComputeV1::Operation]
+  #
+  # @raise [Google::Apis::ServerError] An error occurred on the server and the request can be retried
+  # @raise [Google::Apis::ClientError] The request is invalid and should not be retried without modification
+  # @raise [Google::Apis::AuthorizationError] Authorization is required
+  def add_firewall_source_range(name, range)
+    firewall = get_firewall(name)
+    firewall.source_ranges = [] if firewall.source_ranges.nil?
+    firewall.source_ranges << range
+    operation = @compute.patch_firewall(@options[:gce_project], name, firewall)
+    @compute.wait_global_operation(@options[:gce_project], operation.name)
+  end
+
+  ##
+  # Add an allowed port to the firewall
+  #
+  # @param [String] name The name of the firewall
+  #
+  # @param [String] port The port number to open on the firewall
+  #
+  # @param [String] proto The protocol of the port. This should be 'tcp' or 'udp'
+  #
+  # @return [Google::Apis::ComputeV1::Operation]
+  #
+  # @raise [Google::Apis::ServerError] An error occurred on the server and the request can be retried
+  # @raise [Google::Apis::ClientError] The request is invalid and should not be retried without modification
+  # @raise [Google::Apis::AuthorizationError] Authorization is required
+  def add_firewall_port(name, port, proto)
+    firewall = get_firewall(name)
+    firewall.allowed = [] if firewall.allowed.nil?
+    firewall.allowed << ::Google::Apis::ComputeV1::Firewall::Allowed.new(ip_protocol: proto, ports: [port])
+    operation = @compute.patch_firewall(@options[:gce_project], name, firewall)
+    @compute.wait_global_operation(@options[:gce_project], operation.name)
+  end
+
   ##
   # Add a taget_tag to an existing firewall
   #
   # @param [String] the name of the firewall to update
   #
-  # @ param [String] tag The tag to add to the firewall
+  # @param [String] tag The target tag to add to the firewall
   #
   # @return [Google::Apis::ComputeV1::Operation]
   #
   # @raise [Google::Apis::ServerError] An error occurred on the server and the request can be retried
   # @raise [Google::Apis::ClientError] The request is invalid and should not be retried without modification
   # @raise [Google::Apis::AuthorizationError] Authorization is required
-  def add_firewall_tag(name, tag)
+  def add_firewall_target_tag(name, tag)
     firewall = @compute.get_firewall(@options[:gce_project], name)
     firewall.target_tags = [] if firewall.target_tags.nil?
     firewall.target_tags << tag
     operation = @compute.patch_firewall(@options[:gce_project], name, firewall)
     @compute.wait_global_operation(@options[:gce_project], operation.name)
   end
 
+  ##
+  # Add a source_tag to an existing firewall
+  #
+  # @param [String] the name of the firewall to update
+  #
+  # @param [String] tag The source tag to add to the firewall
+  #
+  # @return [Google::Apis::ComputeV1::Operation]
+  #
+  # @raise [Google::Apis::ServerError] An error occurred on the server and the request can be retried
+  # @raise [Google::Apis::ClientError] The request is invalid and should not be retried without modification
+  # @raise [Google::Apis::AuthorizationError] Authorization is required
+  def add_firewall_source_tag(name, tag)
+    firewall = @compute.get_firewall(@options[:gce_project], name)
+    firewall.source_tags = [] if firewall.source_tags.nil?
+    firewall.source_tags << tag
+    operation = @compute.patch_firewall(@options[:gce_project], name, firewall)
+    @compute.wait_global_operation(@options[:gce_project], operation.name)
+  end
+
   ##
   # Create a Google Compute disk
   #
@@ -418,6 +545,26 @@ def get_instance(name)
     @compute.get_instance(@options[:gce_project], @options[:gce_zone], name)
   end
 
+  ##
+  # Add a tag to a Google Compute Instance
+  #
+  # @param [String] name The name of the instance
+  #
+  # @param [String] tag The tag to add to the instance
+  #
+  # @return [Google::Apis::ComputeV1::Operation]
+  #
+  # @raise [Google::Apis::ServerError] An error occurred on the server and the request can be retried
+  # @raise [Google::Apis::ClientError] The request is invalid and should not be retried without modification
+  # @raise [Google::Apis::AuthorizationError] Authorization is required
+  def add_instance_tag(name, tag)
+    instance = get_instance(name)
+    tags = instance.tags
+    tags.items << tag
+    operation = @compute.set_instance_tags(@options[:gce_project], @options[:gce_zone], name, tags)
+    @compute.wait_zone_operation(@options[:gce_project], @options[:gce_zone], operation.name)
+  end
+
   ##
   # Set key/value metadata pairs to a Google Compute instance
   #