Skip to content

Latest commit

 

History

History
256 lines (182 loc) · 11.5 KB

kubernetes-resources.md

File metadata and controls

256 lines (182 loc) · 11.5 KB

Kubernetes resource validation example

This is an example of how PSRule can be used to validate Kubernetes resources to match an internal metadata and configuration standard.

This scenario covers the following:

  • Defining a basic rule.
  • Configuring custom binding.
  • Using a type precondition.
  • Running rules using YAML input.

In this scenario we will use a YAML file:

  • resources.yaml - A Kubernetes manifest containing deployments and services.

Define rules

To validate our Kubernetes resources, we need to define some rules. Rules are defined by using the Rule keyword in a file ending with the .Rule.ps1 extension.

Our business rules for configuration Kubernetes resources can be defined with the following dot points:

  • The following recommended labels will be used on all services and deployments:
    • app.kubernetes.io/name - the name of the application/ service.
    • app.kubernetes.io/version - the version of the service.
    • app.kubernetes.io/component - identifies the type of component, valid options are web, api, database and gateway
  • For web or api deployments, a minimum of two (2) replicas must be used.
  • Deployments must use container images with a specific version tag, and not latest.
  • Deployments must declare minimum and maximum memory/ CPU resources.

In the example below:

  • We use metadata.Name directly after the Rule keyword to name the rule definition. Each rule must be named uniquely.
  • The # Description: comment is used to add additional metadata interpreted by PSRule.
  • One or more conditions are defined within the curly braces { }.
  • The rule definition is saved within a file named kubernetes.Rule.ps1.
# Description: Must have the app.kubernetes.io/name label
Rule 'metadata.Name' {
    # Rule conditions go here
}

Check that the label exists

In the next step, we define one or more conditions.

Conditions can be:

  • Any valid PowerShell that returns a true (pass) when the condition is met or false (fail) when the condition is not met.
  • More than one condition can be defined, if any condition returns false then the whole rule fails.

PSRule includes several convenience keywords such as AllOf, AnyOf, Exists, Match, TypeOf and Within that make conditions faster to define, easier to understand and troubleshoot. However, use of these keywords is optional.

In the example below:

  • We use the Exists keyword to check that the resource has the app.kubernetes.io/name label set.
    • By default, PSRule will step through nested properties separated by a .. i.e. labels is a property of metadata.
    • Kubernetes supports and recommends label namespaces, which often use . in their name. PSRule supports this by enclosing the field name (app.kubernetes.io/name) in apostrophes (') so that app.kubernetes.io/name is checked instead of app.
# Description: Must have the app.kubernetes.io/name label
Rule 'metadata.Name' {
    Exists "metadata.labels.'app.kubernetes.io/name'"
}

We have also defined something similar for the version and component labels.

In the example below:

  • Double apostrophes ('') are used to enclose app.kubernetes.io/name because the field name uses ' at the start and end of the string instead of " in the previous example.
  • The Within keyword is used to validate that the app.kubernetes.io/component only uses one of four (4) allowed values.
# Description: Must have the app.kubernetes.io/version label
Rule 'metadata.Version' {
    Exists 'metadata.labels.''app.kubernetes.io/version'''
}

# Description: Must have the app.kubernetes.io/component label
Rule 'metadata.Component' {
    Exists 'metadata.labels.''app.kubernetes.io/component'''
    Within 'metadata.labels.''app.kubernetes.io/component''' 'web', 'api', 'database', 'gateway' -CaseSensitive
}

Use custom binding

Before processing rules, PSRule binds TargetName and TargetType properties to the pipeline object. These properties are used for filtering and displaying results.

The default properties that PSRule binds are different from how Kubernetes resources are structured. Kubernetes uses:

  • metadata.name to store the name of a resource.
  • kind to store the type of resource.

The default bindings can be updated by providing custom property names or a custom script. To change binding property names set the Binding.TargetName and Binding.TargetType configuration options.

The following example shows how to set the options using a YAML configuration file:

  • TargetName is bound to metadata.name
  • TargetType is bound to kind
binding:
  targetName:
  - metadata.name
  targetType:
  - kind

Alternatively, these options can be set at runtime using the hashtable syntax.

$option = New-PSRuleOption -Option @{ 'Binding.TargetName' = 'metadata.name'; 'Binding.TargetType' = 'kind' };

These options will be passed to Invoke-PSRule using the -Option parameter in a later step.

Define preconditions

Currently the metadata.Name rule defined in a previous step will be executed for any type of object. Kubernetes has many types of built-in resource such as Services, Deployments, Namespaces, Pods and ClusterRoles.

By defining a precondition, we can ensure that the rule is only processed for Services or Deployments to match our business rules.

PSRule supports two types of preconditions, either type (-Type) or script block (-If).

  • Type preconditions are one or more type names that PSRule compares to the TargetType binding, where:
    • One of the type names names equal TargetType the rule will be processed.
    • None of the type names equal TargetType the rule be skipped.
  • Script block preconditions is a PowerShell script block that returns true or false, where:
    • True - Continue processing the rule.
    • False - Skip processing the rule.

Preconditions are evaluated once per rule for each object.

In the example below:

  • We update our metadata.Name rule to use the -Type parameter to specify a type precondition of either Deployment or Service.
  • In a previous step, TypeName was bound to the kind property which will be Deployment or Service for these resource types.
# Description: Must have the app.kubernetes.io/name label
Rule 'metadata.Name' -Type 'Deployment', 'Service' {
    Exists "metadata.labels.'app.kubernetes.io/name'"
}

Using a type precondition satisfies our business rules and will deliver faster performance then using a script block. An example using a script block precondition is also shown below.

# Description: Must have the app.kubernetes.io/name label
Rule 'metadata.Name' -If { $TargetObject.kind -eq 'Deployment' -or $TargetObject.kind -eq 'Service' } {
    Exists "metadata.labels.'app.kubernetes.io/name'"
}

Complete remaining rules

The remaining rule definitions from our defined business rules are included below. Each follows a similar pattern and builds on the previous sections.

In the example below:

  • The built-in variable $TargetObject is used to get the current pipeline object.
    • Built-in keywords like Exists automatically default to $TargetObject, but can be piped alternative input as shown in the rule definition named deployment.ResourcesSet.
# Description: Deployments use a minimum of 2 replicas
Rule 'deployment.HasMinimumReplicas' -Type 'Deployment' {
    Exists 'spec.replicas'
    $TargetObject.spec.replicas -ge 2
}

# Description: Deployments use specific tags
Rule 'deployment.NotLatestImage' -Type 'Deployment' {
    foreach ($container in $TargetObject.spec.template.spec.containers) {
        $container.image -like '*:*' -and
        $container.image -notlike '*:latest'
    }
}

# Description: Resource requirements are set for each container
Rule 'deployment.ResourcesSet' -Type 'Deployment' {
    foreach ($container in $TargetObject.spec.template.spec.containers) {
        $container | Exists 'resources.requests.cpu'
        $container | Exists 'resources.requests.memory'
        $container | Exists 'resources.limits.cpu'
        $container | Exists 'resources.limits.memory'
    }
}

Execute rules

With some rules defined, the next step is to execute them. For this example, we'll use Invoke-PSRule to get the result for each rule. The Test-PSRuleTarget cmdlet can be used if only a true or false is required.

In our example we are using the YAML format to store Kubernetes resources. PSRule has built-in support for YAML so we can import these files directly from disk or process output from a command such as kubectl.

In the examples below:

  • Get-Content is piped to Invoke-PSRule with the -Raw switch. The -Raw switch reads the full contents of the rule as a string. In contrast, without the -Raw switch, each line would be passed to Invoke-PSRule individually as separate objects.
  • The -Format parameter informs PSRule that the string is YAML and it should convert the string into structured objects.
  • The -Option parameter sets the location of the configuration file that contains custom binding options.
  • kubectl is called with the -o yaml to output resources as YAML.
  • kubectl is piped to Out-String to convert the multi-line output to a single string.
  • The -ObjectPath parameter is used with the output from kubectl. This is required because the output from kubectl is a collection of resources instead of individual resources. Specifically -ObjectPath items gets the resources from the items property of the output.
# Validate resources from file
Get-Content .\resources.yaml -Raw | Invoke-PSRule -Format Yaml -Option .\PSRule.yaml;
# Validate resources directly from kubectl output
kubectl get services -o yaml | Out-String | Invoke-PSRule -Format Yaml -Option .\PSRule.yaml -ObjectPath items;

For this example, we limited the output to failed results with the following command:

Get-Content docs/scenarios/kubernetes-resources/resources.yaml -Raw | Invoke-PSRule -Path docs/scenarios/kubernetes-resources -Format Yaml -Option docs/scenarios/kubernetes-resources/PSRule.yaml -Outcome Fail;

The resulting output is:

   TargetName: app1-cache

RuleName                            Outcome    Message
--------                            -------    -------
deployment.HasMinimumReplicas       Fail       Deployments use a minimum of 2 replicas
deployment.NotLatestImage           Fail       Deployments use specific tags
deployment.ResourcesSet             Fail       Resource requirements are set for each container


   TargetName: app1-cache-service

RuleName                            Outcome    Message
--------                            -------    -------
metadata.Name                       Fail       Must have the app.kubernetes.io/name label
metadata.Version                    Fail       Must have the app.kubernetes.io/version label
metadata.Component                  Fail       Must have the app.kubernetes.io/component label


   TargetName: app1-ui

RuleName                            Outcome    Message
--------                            -------    -------
metadata.Version                    Fail       Must have the app.kubernetes.io/version label

More information