Provider functions: if object parameter with map field has an unknown value on any entry in the nested map, the entire parameter is unknown

[jason-johnson]

I have a provider which computes names for resources. I try to have this computation work at plan time if at all possible. One issue is that there are times that a random component would be needed in the computed name. I'm using terraform_data in my acceptance test to simulate random_string (but with a known value). The currently published version of the provider does not allow any parameters to be unknown but I am working on a version which allows them to be unknown and simply computes the name in cases where it can and returns unknown where it relies on unknown data.

The issue is that the configuration for this function is an object with nested maps and map of map fields which configure how the name computation should be done. In the version I'm working on, I try to handle values which can be unknown and I set the configuration parameter to allow unknown values. Unfortunately, even with just a single element of one of the nested maps set to terraform_data.test.output, the entire argument is passed to the function as unknown. I thought the issue might have to do with the struct I was converting to but when inspecting in the debugger, I see that the configuration argument is already unknown. I checked in the call stack further into the plugin framework code and it seems this unknown value is passed from the terraform server directly (the Unmarshall function gets unknown already). From reading the documentation and various issues on the subject, I was under the impression that everything would be known except this exact value which is unknown.

Apologies if the linked code is a bit "busy" but I have linked directly to the relevant lines. The line in the test is the only relevant one which puts this "unknown at plan time" value into the configuration. If I change that value to be a string instead of referencing the terraform_data entry then everything is known at plan time. The linked location in the provider is the area one can stop with the debugger to verify that the second parameter (configuration) is unknown.

Module version

github.com/hashicorp/terraform-plugin-framework v1.13.0

Relevant provider source code

Function test (relevant line)

Function error location

Terraform Configuration Files

In the test (there is only one to make it easier to see the issue)

Expected Behavior

I expected all parameters to be defined, but a single value in the variables nested map to be UnKnown at plan time.

Actual Behavior

The entire configuration parameters value is set to unknown.

Steps to Reproduce

If the test is run, as is, it gets a null deference crash because I didn't handle the "should be impossible" case of configuration being unknown. However, if a breakpoint is set in the provider function code at the location shown, it is possible to inspect req.Arguments and verify that configuration is unknown.

[austinvalle]

Hey there @jason-johnson 👋🏻 , thanks for reporting the issue with a detailed repro, very helpful!

I believe your understanding is correct about functions accepting unknown values (where a map with an unknown element value should not be fully unknown), but I'm thinking the behavior you're observing might be coming from the data source data.namep_configuration.example.configuration.

For data sources, if any of the configuration is unknown, the entire data source will not be read until apply. I'm not 100% sure if that should be resulting in a fully unknown object when referencing it in a function, I'd need to dig a little deeper to find where that's occurring.

---

I wrote a quick little test function to try and verify without the data source, and I'm able to define a map of objects, where one of the inner element types are unknown, then I receive a partially known value as expected:

func (f MapFunction) Definition(ctx context.Context, req function.DefinitionRequest, resp *function.DefinitionResponse) {
	resp.Definition = function.Definition{
		Parameters: []function.Parameter{
			function.MapParameter{
				AllowUnknownValues: true,
				ElementType: types.ObjectType{
					AttrTypes: map[string]attr.Type{
						"key1": types.StringType,
						"key2": types.BoolType,
					},
				},
				Name: "map_param",
			},
		},
		Return: function.MapReturn{
			ElementType: types.ObjectType{
				AttrTypes: map[string]attr.Type{
					"key1": types.StringType,
					"key2": types.BoolType,
				},
			},
		},
	}
}

func (f MapFunction) Run(ctx context.Context, req function.RunRequest, resp *function.RunResponse) {
	var arg types.Map
	resp.Error = req.Arguments.Get(ctx, &arg)

	fmt.Println(arg.String())

	resp.Error = function.ConcatFuncErrors(resp.Error, resp.Result.Set(ctx, arg))
}

Running a test with this config

resource "terraform_data" "test" {
	input = "value1"
}
output "test" {
	value = provider::framework::map({
		"a" = {
			"key1" = terraform_data.test.output, // will be unknown during plan!
			"key2" = true,
		},
		"b" = {
			"key1" = "value2",
			"key2" = false,
		},
	})
}

I can print out a partially known value via the Run function

{"a":{"key1":<unknown>,"key2":true},"b":{"key1":"value2","key2":false}}

Is it possible for you to test without the data source to confirm?

[jason-johnson]

Hi @austinvalle, thanks for the response. So I have checked here and it seems you are correct that the unknown value was coming from the data source. But now I have a new problem: by manually creating the configuration the variables map has the expected behaviour (i.e. only unknown values are unknown)... but now all the other data source values (e.g. types) are unknown despite having no dependencies at all. I find this quite surprising because if I just turn off AllowUnknownValues then suddenly the data sources work at plan time.

UPDATE: Ok, so I changed the code just to not fail if the req.Arguments.Get call fails and now the tests actually pass. Which means that the function must be getting called a few times during the plan. I can live with that.

[austinvalle]

UPDATE: Ok, so I changed the code just to not fail if the req.Arguments.Get call fails and now the tests actually pass. Which means that the function must be getting called a few times during the plan. I can live with that.

Ah yes, that does make sense. When Terraform is evaluating the configuration (during say, the terraform validate portion of the plan) it's going to execute your function since it indicates that the parameter configurations can be unknown. It might be evaluating the configuration before the data source is read, which would mean that the entire configurations value is unknown at that point.

Since Terraform is passing you an unknown value, your function should be allowed to return an unknown value in response if you don't have enough data to successfully complete your logic (like when the entire configurations value is unknown). Returning early without setting the result would actually achieve that exactly, since we pre-populate Result with an unknown result.

Eventually, if you keep returning unknown result data, Terraform will call your function with a fully known configurations value, in which case you must respond with a fully known result, otherwise you'll get an error. So I believe your current understanding is correct 👍🏻