Skip to content

Commit

Permalink
Implement implicit parameter resolution.
Browse files Browse the repository at this point in the history
This is the implementation of
[TEP-0023](https://github.com/tektoncd/community/blob/main/teps/0023-implicit-mapping.md).

This adds information to the defaulting context to allow parameters to
be passed down the stack during evaluation. This functionality is gated
behind the alpha feature flag.

Additionally, Tasks are now allowed to accept more params than are
actually used. This should generally be safe, since extra params that
are provided should not affect the behavior of the Task itself.
  • Loading branch information
wlynch authored and tekton-robot committed Sep 15, 2021
1 parent b47ed53 commit f597c92
Show file tree
Hide file tree
Showing 18 changed files with 910 additions and 38 deletions.
2 changes: 1 addition & 1 deletion config/200-role.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ rules:
- apiGroups: [""]
resources: ["configmaps"]
verbs: ["get"]
resourceNames: ["config-logging", "config-observability", "config-leader-election"]
resourceNames: ["config-logging", "config-observability", "config-leader-election", "feature-flags"]
- apiGroups: [""]
resources: ["secrets"]
verbs: ["list", "watch"]
Expand Down
2 changes: 2 additions & 0 deletions config/webhook.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,8 @@ spec:
value: config-observability
- name: CONFIG_LEADERELECTION_NAME
value: config-leader-election
- name: CONFIG_FEATURE_FLAGS_NAME
value: feature-flags
- name: WEBHOOK_SERVICE_NAME
value: tekton-pipelines-webhook
- name: WEBHOOK_SECRET_NAME
Expand Down
69 changes: 69 additions & 0 deletions docs/pipelineruns.md
Original file line number Diff line number Diff line change
Expand Up @@ -246,6 +246,75 @@ case is when your CI system autogenerates `PipelineRuns` and it has `Parameters`
provide to all `PipelineRuns`. Because you can pass in extra `Parameters`, you don't have to
go through the complexity of checking each `Pipeline` and providing only the required params.

#### Implicit Parameters

**([alpha only](https://github.com/tektoncd/pipeline/blob/main/docs/install.md#alpha-features))**

When using an inlined spec, parameters from the parent `PipelineRun` will be
available to any inlined specs without needing to be explicitly defined. This
allows authors to simplify specs by automatically propagating top-level
parameters down to other inlined resources.

```yaml
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
generateName: echo-
spec:
params:
- name: MESSAGE
value: "Good Morning!"
pipelineSpec:
tasks:
- name: echo-message
taskSpec:
steps:
- name: echo
image: ubuntu
script: |
#!/usr/bin/env bash
echo "$(params.MESSAGE)"
```

On creation, this will resolve to a fully-formed spec and will be returned back
to clients to avoid ambiguity:

```yaml
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
generateName: echo-
spec:
params:
- name: MESSAGE
value: Good Morning!
pipelineSpec:
params:
- name: MESSAGE
type: string
tasks:
- name: echo-message
params:
- name: MESSAGE
value: $(params.MESSAGE)
taskSpec:
params:
- name: MESSAGE
type: string
spec: null
steps:
- name: echo
image: ubuntu
script: |
#!/usr/bin/env bash
echo "$(params.MESSAGE)"
```

Note that all implicit Parameters will be passed through to inlined resources
(i.e. PipelineRun -> Pipeline -> Tasks) even if they are not used.
Extra parameters passed this way should generally be safe (since they aren't
actually used), but may result in more verbose specs being returned by the API.

### Specifying custom `ServiceAccount` credentials

You can execute the `Pipeline` in your `PipelineRun` with a specific set of credentials by
Expand Down
63 changes: 63 additions & 0 deletions docs/taskruns.md
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,69 @@ spec:

**Note:** If a parameter does not have an implicit default value, you must explicitly set its value.

#### Implicit Parameters

**([alpha only](https://github.com/tektoncd/pipeline/blob/main/docs/install.md#alpha-features))**

When using an inlined `taskSpec`, parameters from the parent `TaskRun` will be
available to the `Task` without needing to be explicitly defined.

```yaml
apiVersion: tekton.dev/v1beta1
kind: TaskRun
metadata:
generateName: hello-
spec:
params:
- name: message
value: "hello world!"
taskSpec:
# There are no explicit params defined here.
# They are derived from the TaskRun params above.
steps:
- name: default
image: ubuntu
script: |
echo $(params.message)
```

On creation, this will resolve to a fully-formed spec and will be returned back
to clients to avoid ambiguity:

```yaml
apiVersion: tekton.dev/v1beta1
kind: TaskRun
metadata:
generateName: hello-
spec:
params:
- name: message
value: "hello world!"
taskSpec:
params:
- name: message
type: string
steps:
- name: default
image: ubuntu
script: |
echo $(params.message)
```

Note that all implicit Parameters will be passed through to inlined resource,
even if they are not used. Extra parameters passed this way should generally
be safe (since they aren't actually used), but may result in more verbose specs
being returned by the API.

#### Extra Parameters

**([alpha only](https://github.com/tektoncd/pipeline/blob/main/docs/install.md#alpha-features))**

You can pass in extra `Parameters` if needed depending on your use cases. An example use
case is when your CI system autogenerates `TaskRuns` and it has `Parameters` it wants to
provide to all `TaskRuns`. Because you can pass in extra `Parameters`, you don't have to
go through the complexity of checking each `Task` and providing only the required params.

### Specifying `Resources`

If a `Task` requires [`Resources`](tasks.md#specifying-resources) (that is, `inputs` and `outputs`) you must
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
generateName: echo-
spec:
pipelineSpec:
tasks:
- name: echo-message
taskSpec:
steps:
- name: echo
image: ubuntu
script: |
#!/usr/bin/env bash
echo "$(params.MESSAGE)"
params:
- name: MESSAGE
value: "Good Morning!"
15 changes: 15 additions & 0 deletions examples/v1beta1/taskruns/alpha/implicit-params.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
apiVersion: tekton.dev/v1beta1
kind: TaskRun
metadata:
generateName: hello-
spec:
params:
- name: message
value: "hello world!"
taskSpec:
# There are no explicit params defined here. They are derived from the TaskRun.
steps:
- name: default
image: ubuntu
script: |
echo $(params.message)
172 changes: 172 additions & 0 deletions pkg/apis/pipeline/v1beta1/param_context.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,172 @@
// Copyright 2021 The Tekton Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package v1beta1

import (
"context"
"fmt"

"github.com/tektoncd/pipeline/pkg/apis/config"
)

// paramCtxKey is the unique identifier for referencing param information from
// a context.Context. See [context.Context.Value](https://pkg.go.dev/context#Context)
// for more details.
var paramCtxKey struct{}

// paramCtxVal is the data type stored in the param context.
// This maps param names -> ParamSpec.
type paramCtxVal map[string]ParamSpec

// AddContextParams adds the given Params to the param context. This only
// preserves the fields included in ParamSpec - Name and Type.
func AddContextParams(ctx context.Context, in []Param) context.Context {
if in == nil {
return ctx
}

if config.FromContextOrDefaults(ctx).FeatureFlags.EnableAPIFields != "alpha" {
return ctx
}

out := paramCtxVal{}
// Copy map to ensure that contexts are unique.
v := ctx.Value(paramCtxKey)
if v != nil {
for n, cps := range v.(paramCtxVal) {
out[n] = cps
}
}
for _, p := range in {
// The user may have omitted type data. Fill this in in to normalize data.
if v := p.Value; v.Type == "" {
if len(v.ArrayVal) > 0 {
p.Value.Type = ParamTypeArray
}
if v.StringVal != "" {
p.Value.Type = ParamTypeString
}
}
out[p.Name] = ParamSpec{
Name: p.Name,
Type: p.Value.Type,
}
}
return context.WithValue(ctx, paramCtxKey, out)
}

// AddContextParamSpec adds the given ParamSpecs to the param context.
func AddContextParamSpec(ctx context.Context, in []ParamSpec) context.Context {
if in == nil {
return ctx
}

if config.FromContextOrDefaults(ctx).FeatureFlags.EnableAPIFields != "alpha" {
return ctx
}

out := paramCtxVal{}
// Copy map to ensure that contexts are unique.
v := ctx.Value(paramCtxKey)
if v != nil {
for n, ps := range v.(paramCtxVal) {
out[n] = ps
}
}
for _, p := range in {
cps := ParamSpec{
Name: p.Name,
Type: p.Type,
Description: p.Description,
Default: p.Default,
}
out[p.Name] = cps
}
return context.WithValue(ctx, paramCtxKey, out)
}

// GetContextParams returns the current context parameters overlayed with a
// given set of params. Overrides should generally be the current layer you
// are trying to evaluate. Any context params not in the overrides will default
// to a generic pass-through param of the given type (i.e. $(params.name) or
// $(params.name[*])).
func GetContextParams(ctx context.Context, overlays ...Param) []Param {
pv := paramCtxVal{}
v := ctx.Value(paramCtxKey)
if v == nil && len(overlays) == 0 {
return nil
}
if v != nil {
pv = v.(paramCtxVal)
}
out := make([]Param, 0, len(pv))

// Overlays take precedence over any context params. Keep track of
// these and automatically add them to the output.
overrideSet := make(map[string]Param, len(overlays))
for _, p := range overlays {
overrideSet[p.Name] = p
out = append(out, p)
}

// Include the rest of the context params.
for _, ps := range pv {
// Don't do anything for any overlay params - these are already
// included.
if _, ok := overrideSet[ps.Name]; ok {
continue
}

// If there is no overlay, pass through the param to the next level.
// e.g. for strings $(params.name), for arrays $(params.name[*]).
p := Param{
Name: ps.Name,
}
if ps.Type == ParamTypeString {
p.Value = ArrayOrString{
Type: ParamTypeString,
StringVal: fmt.Sprintf("$(params.%s)", ps.Name),
}
} else {
p.Value = ArrayOrString{
Type: ParamTypeArray,
ArrayVal: []string{fmt.Sprintf("$(params.%s[*])", ps.Name)},
}
}
out = append(out, p)
}

return out
}

// GetContextParamSpecs returns the current context ParamSpecs.
func GetContextParamSpecs(ctx context.Context) []ParamSpec {
v := ctx.Value(paramCtxKey)
if v == nil {
return nil
}

pv := v.(paramCtxVal)
out := make([]ParamSpec, 0, len(pv))
for _, ps := range pv {
out = append(out, ParamSpec{
Name: ps.Name,
Type: ps.Type,
Description: ps.Description,
Default: ps.Default,
})
}
return out
}
Loading

0 comments on commit f597c92

Please sign in to comment.