[BUG]Host' or ':authority' must be a 'SignedHeader' in the AWS Authorization.

[dmnlk]

What is the bug?

We are performing a migration based on PR233.
This is a service that used IAM role authentication within a container on AWS Fargate.

After the migration, when executing a request, the following error occurs:
Host' or ':authority' must be a 'SignedHeader' in the AWS Authorization.

The code is as follows:

$client = new \GuzzleHttp\Client($opensearchConfig);
$credentials = $this->getAwsCredentials();
$signer = new SignatureV4('es', 'ap-northeast-1');
$client = new SigningClientDecorator($client, $credentials, $signer);
$guzzleHttpFactory = new HttpFactory();
$serializer = new SmartSerializer();
$requestFactory = new RequestFactory(
            $guzzleHttpFactory,
            $guzzleHttpFactory,
            $guzzleHttpFactory,
            $serializer
        );

$transport = (new OpenSearch\TransportFactory())
            ->setHttpClient($client)
            ->setRequestFactory($requestFactory)
            ->create();

$endpointFactory = new EndpointFactory();
$opensearchClient = new Client($transport, $endpointFactory, []);
$opensearchClient->cluster()->health();

After changing the code as shown below, a 400 Bad Request is returned
SigningClientDecorator.php

    public function sendRequest(RequestInterface $request): ResponseInterface
    {
        $request = $request->withHeader('x-amz-content-sha256', hash('sha256', (string) $request->getBody()));
        $request = $request->withHeader('Host',  $request->getUri()->getHost()); // add Host header
        $request = $this->signer->signRequest($request, $this->credentials);

        return $this->inner->sendRequest($request);
    }

Since it works without any issues on the local environment with OpenSearch, I believe there might be a problem with the request process to AWS OpenSearch.

https://github.com/opensearch-project/opensearch-php/blob/main/USER_GUIDE.md#setsigv4credentialprovider-for-aws-opensearch-service

This page is not yet updated to support the new code, so I am proceeding with the migration through trial and error.
If the migration is successful, I plan to create a PR to update this user guide.

How can one reproduce the bug?

use openseach in AWS

What is the expected behavior?

A clear and concise description of what you expected to happen.
migrate psr interface

What is your host/environment?

Operating system, version.
debian php

Do you have any screenshots?

If applicable, add screenshots to help explain your problem.

Do you have any additional context?

Add any other context about the problem.

[dblock]

@dmnlk likely a bug I got this working, but your code looks correct too.

I have been modifying https://github.com/dblock/opensearch-php-client-demo and it does work without a custom transport for me.

$client = (new \OpenSearch\ClientBuilder())
   ->setHosts([getenv("ENDPOINT")])
   ->setSigV4Region(getenv("AWS_REGION"))    
   ->setSigV4CredentialProvider(true)
   ->build();

and the who thing with Psr, working code in https://github.com/dblock/opensearch-php-client-demo/blob/2.4.0/index.php. I used the existing \OpenSearch\Aws\SigningClientDecorator instead and added the Host header myself.

[dblock]

@kimpepper I am not sure the SigningClientDecorator should be adding the Host header. I don't quite understand in which order these things are called, but in my example the signing client decorator is not getting any of the headers we set when creating the PSR client. So the gist of the problem is that \OpenSearch\Aws\SigningClientDecorator needs to be receiving the request with a Host header in it. Do you know how to fix it?

[dmnlk]

@dblock
I also tried your code sample, and I was able to access my AWS OpenSearch environment. The main difference might be that I am using the Symfony client instead of Guzzle. I will continue my investigation as well.

[dmnlk]

@dblock
I tried it with Guzzle, and it worked.
https://github.com/dmnlk/opensearch-php-client-demo/blob/2.4.0-guzzle/index.php

Upon investigation, I found that in the sendRequest method of the SigningClientDecorator, neither the URL nor the host could be retrieved when calling $request->getUri()->getHost().

Getting it from environment variables introduces too much dependency, so I believe it would be better to retrieve it from within the request object. I will continue investigating, and if I manage to fix it, I will submit a PR request.

[kimpepper]

I've added a test to show how it works.

[dblock]

My proposed solution in #253. Try it out @dmnlk?

[kimpepper]

The test in #252 shows that the Host can be retrieved. I wonder how you are configuring the client in order for it not to be?

I don't think we should just re-add the header in the SigningClientDecorator without understanding what is going on.

[dblock]

Let's discuss in #253. I couldn't get the header from the client in my sample.

[kimpepper]

Confirmed that you need to set the default Host header when using the Symfony HTTP Client. See the tests in #256

[dblock]

@kimpepper Unfortunately it works with a mock implementation but doesn't work with the real one. The Host header is not available in the decorator from the request, even though it's passed in the client constructor options. I tried both Guzzle and Symfony and they behave the same way, $request->getUri() returns the relative URI, not the full URI.

If you have an AWS OpenSearch you can try it with https://github.com/dblock/opensearch-php-client-demo/tree/2.4.0 (composer run demo-guzzle, composer run demo-symfony, composer run demo).

Maybe I missed something or you have a better idea?

If the above is correct, the \OpenSearch\Aws\SigningClientDecorator is constructed with the client though, we could subclass it for Guzzle and Symfony into a GuzzleSigningClientDecorator and SymfonySigningClientDecorator and fetch the host from the strongly typed client instead of passing it as optional headers. I would probably still like to keep the header-based implementation in this PR for future extensions.

[kimpepper]

Ah I see. Yes the problem is the Host header is only added to the request after it is signed. OK. Let's stick with #253