CloudFront cache error in Next.js with SSR when navigating via SPA (client-side)

[marlonconstante]

Before opening, please confirm:

• I have checked to see if my question is addressed in the FAQ.
• I have searched for duplicate or closed issues.
• I have read the guide for submitting bug reports.
• I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
• I have removed any sensitive information from my code snippets and submission.

App Id

d1k1f08ve7vlp8

AWS Region

us-east-1

Amplify Hosting feature

SSR

Describe the bug

I have identified an issue related to the CloudFront cache managed by Amplify, which occurs when requesting data for a page after navigating via Single-Page Application (SPA) using the Next.js Link component.
The problem arises when CloudFront finds the data in its cache and returns an empty, invalid JSON response.
This response does not correspond to the actual data returned by the getServerSideProps method on the Next.js server.

Expected behavior

After navigating via SPA, requests to the /__next/data/**/*.json endpoints should return the correct and up-to-date data from the server, even when CloudFront is involved.

Reproduction steps

1. Open https://master.d1k1f08ve7vlp8.amplifyapp.com/sdp/account?site_domain=timgames.appsclub.com.br link.
2. Click on the login button (next/link component).
3. Go back to previous page.
4. Click the login button again.
5. Observe the requests made by Next.js to the /__next/data/**/*.json endpoints which retrieve data for the page from the server.
6. And note that when CloudFront encounters cached data (indicated by "Hit from CloudFront" in the response), empty and invalid JSON is returned as the response.

Build Settings

version: 1
frontend:
  phases:
    preBuild:
      commands:
        - yarn install
    build:
      commands:
        - yarn build
  artifacts:
    baseDirectory: .next
    files:
      - '**/*'
  cache:
    paths:
      - node_modules/**/*

Log output

Additional information

Using Next.js version 13.2.4 published on Amplify Hosting with the web compute platform.

[dallegrezzaRFK]

Hi all,

I'm experiencing the same issue and have been investigating solutions for the past week. Symptoms are nearly identical. Ultimately, "After navigating via SPA, requests to the /__next/data/**/*.json endpoints should return the correct and up-to-date data from the server, even when CloudFront is involved." says it all.

Using Next.js 13.2.4 and a standard`amplify.yml file:

version: 1
frontend:
  phases:
    preBuild:
      commands:
        - npm ci
    build:
      commands:
        - npm run build
  artifacts:
    baseDirectory: .next
    files:
      - "**/*"
  cache:
    paths:
      - node_modules/**/*

The root cause seems to be pretty straight-forward, as mentioned by @hloriii in this comment on #3509, which is also an SSR issue. Sort of two issues together:

1. Documented issue with CloudFront due to lack of support for cache headers Next.js wants to use on SSR'd pages
   • Since my SSR'd page is generating dynamic content, I really don't want to try to cache hyper aggressively
   • The root cause is ultimately, if the clients request never makes it to the actual server, then getServerSideProps is never going to run, and a 200 OK response with an empty JSON body ({}) is returned by CloudFront, which will almost certainly cause your page to break as it's missing data to hydrate/render your page.
   • ✅A hard refresh or directly navigating to the SSR'd page always works, because the server is actually running getServerSideProps
   • ❌Unsure how to comprehend the logic CloudFront uses to determine whether or not there is a cache hit or miss, but client-side navigation using the <Link /> component is hit or miss. I can click the link and have it work multiple times in a row, then it fails.
2. getServerSideProps Using the <Link/> component in production means, by default, when you hover the link, next will try to prefetch the resource

When you request this page on client-side page transitions through next/link or next/router, Next.js sends an API request to the server, which runs getServerSideProps

Some of my requests contain a request header of Purpose: prefetch. This doesn't seem to correlate to whether or not the request works or CF will allow the request to make it to the server or not.

Here is a typical end-user experience:

All requests are client-side, by clicking a navbar link 5 times. The first two returned data. Interestingly enough, the first request was a cache miss, the second a hit. But they both still returned data 🤷‍♂️

The 3rd, 4th, and 5th all do not behave as expected, returning an empty object from the .json endpoint. I diffed the request headers, hopefully it will help the AWS team out:

• The 3rd request contains two additional headers Purpose: prefetch and X-Middleware-Prefetch: 1.
  • Response header shows X-Cache: Hit from cloudfront
• The 4th request does not contain the Purpose: prefetch header
  • Response header shows X-Cache: Hit from cloudfront
• The 5th request does contain the Purpose: prefetch header
  • Response headers show X-Cache: Miss from cloudfront

Very confusing.

Attempted fixes

1. Disable prefetch on my <Link/>s
2. Add a customHeaders.yml, the one defined here.

I can't share more details on my project, but I can share that the issue is definitely specific to what the author mentioned, a CloudFront cache error in Next.js applications that attempt to navigate client-side to a page that uses getServerSideProps. There is guidance in the Amplify-hosting FAQ section, but it doesn't seem to help.

@hloriii is there a better solution for this issue? It's kind of a deal breaker for many Next.js users. Is there a way to opt out of CF until this is fixed?

[ghost]

Hi @dallegrezzaRFK and @marlonconstante 👋🏽 thanks for raising this issue and providing these details on the behavior you're experiencing. We are currently looking into this to find a solution since the suggested workarounds are not helpful in this case.

@dallegrezzaRFK we do not have a way to opt out of CloudFront but we are working on finding a solution for this.

[ghost]

Hi @marlonconstante and @dallegrezzaRFK 👋🏽 apologies for the delay here. I've tried reproducing your behavior with a test application by following these steps:

1. navigate to a next/link component and observe the response (this was a miss from CloudFront and contained expected json data)
2. go back to previous page
3. navigate to the next/link component again and observe the response (this was a hit from CloudFront and still returned all the appropriate data)

I did not observe any empty json response in my attempts to reproduce this behavior. However, we can clearly see this occurring for your application so we are prioritizing this issue internally to gain a better understanding. We will provide an update shortly.

[ghost]

Hi @dallegrezzaRFK we need some additional details from you to dive deeper on this issue. Could you provide your Amplify app ID?

We also need two request IDs, one for a request that behaves as expected and one that does not. The request ID is stored under the X-Amz-Cf-Id header. Once we have those IDs we can investigate further for your app.

[Jay2113]

Hi everyone 👋 , we are tracking this as a bug here: #3704. I'll recommend subscribing to it for any updates. Closing as duplicate.

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.

[github-actions]

This issue has been automatically locked.