Possible export bug introduced between 9.3.0 + 9.3.3

[erikdstock]

Hello folks - I have an open PR at uglycaucus/uglycauc-us#12 that is attempting to do some large data transformation in the next build+static export pipeline rather than sending and parsing it at request time. I'm having 2 overlapping issues that are difficult to debug together:

• zeit deploy previews are failing, period. The main error I get at next 9.3.0 is Error: A routes-manifest.json couldn't be found. This could be due to a failure during the build

• yarn dev works fine at [email protected], but at higher patches (9.3.3-5 especially) yarn build (which is next build && next export) hangs and my computer's fans spin up at Automatically optimizing pages. These builds also time out on zeit, which is using the command next build && next export.

Notably, the export step does include parsing and transforming some large CSVs into data on the client, but the performance hit since 9.3.0 is striking.

[ijjk]

Hi, for the missing routes-manifest.json bug it looks like you are defining an output directory of out which is not what this field is meant for. It's meant to define a custom location of your .next build output which is where the routes-manifest.json is output. We automatically detect if you are doing a next export and will use those files without you having to specify the folder in your project settings.

The build stalling on the automatically optimizing pages phase appears to be due to you attempting to render pages with 50 MB of page data. This means each page's HTML and JSON files with be at least 50 MB since you are returning that much data from getStaticProps. This doesn't appear to be related to the Next.js version and appears to be related to how much data is attempting to be rendered. If this doesn't seem to be the case feel to reply with additional information and we can investigate some more

[erikdstock]

Thanks! The page data is large and painful to parse on the client for sure (the point of this PR) but I don't think it approaches 50MB - here is the output from static export on 9.3.0, which only takes ~15s:

yarn build
yarn run v1.22.4
$ next build && next export
Enabled States:  nevada
> Using external babel configuration
> Location: "/Users/eriks/dev/personal/uglycauc-us/.babelrc"
Creating an optimized production build

Compiled successfully.

Automatically optimizing pages

Page                                                           Size     First Load
┌ ○ /                                                          1.85 kB     85.2 kB
├   /_app                                                      3.67 kB     74.2 kB
├ ○ /about                                                     846 B       84.2 kB
└ ● /states/[name]                                             63.4 kB      147 kB
    └ /states/nevada
+ shared by all                                                74.2 kB
  ├ static/pages/_app.js                                       3.67 kB
  ├ chunks/0958d27b1e6e71850542c0a9e9a7ec8d1c05dbdc.0163ca.js  9.84 kB
  ├ chunks/489ddbdd13066f80ee0b4fd87cf0c67494262e96.17c7d7.js  13 kB
  ├ chunks/framework.724b47.js                                 40.5 kB
  ├ runtime/main.2c1c95.js                                     6.48 kB
  └ runtime/webpack.b65cab.js                                  746 B

λ  (Server)  server-side renders at runtime (uses getInitialProps or getServerSideProps)
○  (Static)  automatically rendered as static HTML (uses no initial props)
●  (SSG)     automatically generated as static HTML + JSON (uses getStaticProps)

Enabled States:  nevada
> using build directory: /Users/eriks/dev/personal/uglycauc-us/.next
  copying "static build" directory
> No "exportPathMap" found in "next.config.js". Generating map from "./pages"
  launching 7 workers
  copying "public" directory
Exporting (4/4)

Export successful
✨  Done in 16.93s.

[ijjk]

The data file being loaded and returned from props is 50MB on the PR mentioned above, did the way the data is being generated change between the versions you have tried?

[erikdstock]

Ah, you're right - 52M 😬 . No changes between these two builds, just next 9.3.0 vs .5, and on .0 we get the manifest error vs .5 we have it hang.

It does build successfully locally on 9.3.0, though, and I've updated the config to have no override for the build directory and am still getting the manifest error.:

13:30:16.598  
└ ● /states/[name]                                             63.4 kB      147 kB
13:30:16.598  
    └ /states/nevada
13:30:16.598  
+ shared by all                                                74.2 kB
...
Enabled States:  nevada
13:30:18.443  
> using build directory: /zeit/1c99a6b0/.next
13:30:18.445  
  copying "static build" directory
13:30:18.448  
> No "exportPathMap" found in "next.config.js". Generating map from "./pages"
13:30:18.448  
  launching 3 workers
13:30:18.449  
Exporting (0/4)
13:30:18.449  
  copying "public" directory
...
13:30:18.657  
Export successful
13:30:18.667  
Error: A routes-manifest.json couldn't be found. This could be due to a failure during the build
13:30:18.667  
    at Object.getRoutesManifest (/zeit/1bc6bc026129a8fb/.build-utils/.builder/node_modules/@now/next/dist/index.js:6113:15)

[ijjk]

Is that the same project? I don't see a build:static script in your package.json on the PR shared above

[erikdstock]

while stepping through the above i caught this inconsistency as part of my work to flatten and simplify the repo in this PR, so i changed it back to build:static. the current commit there (uglycaucus/uglycauc-us@04ca632) is pinned to 9.3.0 and appears to work, but i agree about the large json size.