Building takes forever and fails with 95% emittingnpm ERR! code ELIFECYCLE

[stefanwouldgo]

First of all I'm not a 100% sure how to start a production build, couldn't find docs to that effect. But after looking at the package.json, I tried 'npm run build' and it runs for a really long time, fetching stuff, and elm making three times (about 20 minutes I guess), and then it fails:

`stefan@xpsgiraffeguix ~/elm-pages-starter$ npm run build

[email protected] build /home/stefan/elm-pages-starter
elm-pages build

Dependencies ready!
Success! Compiled 12 modules.

Main ───> /tmp/2021020-11200-2q3n7y.8hgjb.js

Fetching https://cbeci.org/api/csv
Fetching https://api.blockchain.info/charts/total-bitcoins?timespan=100years&format=csv&cors=true
Dependencies ready!
Success! Compiled 12 modules.

Main ───> /tmp/2021020-11200-i1flf6.f11pa.js

Dependencies ready!
Success! Compiled 12 modules.

Main ───> /tmp/2021020-11200-1lz326e.bz0kf.js

95% emittingnpm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] build: elm-pages build
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR! /home/stefan/.npm/_logs/2021-01-20T04_04_01_987Z-debug.log`

In develop mode, every reload took forever and the page is extremely slow on this project since I started including StaticHttp Requests. Any ideas?

[dillonkearns]

There were some performance problems in an older version of elm-pages that are fixed in the latest version, so make sure you're on elm-pages 1.5.4 of the NPM package and 7.0.0 of the Elm package.

Could you try doing that StaticHttp request starting from a fresh clone of the repo (and just outputting some raw data in the view)? That will clarify what's going on. I suspect that you're most likely on an older version, but either way having a minimal app will reduce some variables.

[stefanwouldgo]

Thanks for the quick response! I'm on the latest versions of both packages already. Could this maybe be a problem with the node version? I'm using v10.22.1 on guix on WSL2.

I can try to remove most of the rest of my app, the StaticHttp request is pretty involved already.

[stefanwouldgo]

OK, these are actually two issues: I tried just cloning the original repo and doing nothing but
npm install and
npm run build. The build is very fast here, but it fails just the same. Exactly the same output. Maybe this is a problem with npm.
npm start does work in both cases (it's just slow with my app).

[stefanwouldgo]

Could you try doing that StaticHttp request starting from a fresh clone of the repo (and just outputting some raw data in the view)?

That request is pretty involved as I said and does need quite a bit from my original Elm app (including Model, Msg, update, init), but I managed to not use everything else (so nothing from my app gets viewed at all), which brought the generated js files (in tmp) down from 1.2M to 975K.

It still takes a long time to build and it still crashes, but that was to be expected, I guess, since building the pure repo alone does crash, too.

The npm log it mentions in the crash doesn't seem to contain anything helpful:

0 info it worked if it ends with ok
1 verbose cli [ '/gnu/store/wsmdqpz8gv2xlxcvbcp8p8pm4zszkdid-node-10.22.1/bin/node',
1 verbose cli   '/home/stefan/.guix-profile/bin/npm',
1 verbose cli   'run',
1 verbose cli   'build' ]
2 info using [email protected]
3 info using [email protected]
4 verbose run-script [ 'prebuild', 'build', 'postbuild' ]
5 info lifecycle [email protected]~prebuild: [email protected]
6 info lifecycle [email protected]~build: [email protected]
7 verbose lifecycle [email protected]~build: unsafe-perm in lifecycle true
8 verbose lifecycle [email protected]~build: PATH: /gnu/store/wsmdqpz8gv2xlxcvbcp8p8pm4zszkdid-node-10.22.1/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/home/stefan/starter2/node_modules/.bin:/home/stefan/.emacs.d/bin/:/home/stefan/.nix-profile/bin:/run/setuid-programs:/home/stefan/.config/guix/current/bin:/home/stefan/.guix-profile/bin:/home/stefan/.guix-profile/sbin:/run/current-system/profile/bin:/run/current-system/profile/sbin
9 verbose lifecycle [email protected]~build: CWD: /home/stefan/starter2
10 silly lifecycle [email protected]~build: Args: [ '-c', 'elm-pages build' ]
11 silly lifecycle [email protected]~build: Returned: code: 1  signal: null
12 info lifecycle [email protected]~build: Failed to exec build script
13 verbose stack Error: [email protected] build: `elm-pages build`
13 verbose stack Exit status 1
13 verbose stack     at EventEmitter.<anonymous> (/gnu/store/wsmdqpz8gv2xlxcvbcp8p8pm4zszkdid-node-10.22.1/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:332:16)
13 verbose stack     at EventEmitter.emit (events.js:198:13)
13 verbose stack     at ChildProcess.<anonymous> (/gnu/store/wsmdqpz8gv2xlxcvbcp8p8pm4zszkdid-node-10.22.1/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14)
13 verbose stack     at ChildProcess.emit (events.js:198:13)
13 verbose stack     at maybeClose (internal/child_process.js:982:16)
13 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:259:5)
14 verbose pkgid [email protected]
15 verbose cwd /home/stefan/starter2
16 verbose Linux 4.19.104-microsoft-standard
17 verbose argv "/gnu/store/wsmdqpz8gv2xlxcvbcp8p8pm4zszkdid-node-10.22.1/bin/node" "/home/stefan/.guix-profile/bin/npm" "run" "build"
18 verbose node v10.22.1
19 verbose npm  v6.14.11
20 error code ELIFECYCLE
21 error errno 1
22 error [email protected] build: `elm-pages build`
22 error Exit status 1
23 error Failed at the [email protected] build script.
23 error This is probably not a problem with npm. There is likely additional logging output above.
24 verbose exit [ 1, true ]

[stefanwouldgo]

I tried again today with a native Ubuntu. Different outcome, but not too great either. Build takes about 12 minutes, and ALMOST finishes, but then this happens:

98% after emitting[prerender-spa-plugin] Unable to prerender all routes!
100% [ Error: [prerender-spa-plugin] Unable to prerender all routes!
      at PrerendererInstance.initialize.then.then.then.then.then.then.then.then.catch.err (/home/stefan/elm-pages/node_modules/prerender-spa-plugin/es6/index.js:147:33) ]
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] build: `elm-pages build`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/stefan/.npm/_logs/2021-01-22T10_29_12_803Z-debug.log

[stefanwouldgo]

Oops, this same error happens on a freshly cloned copy of elm-pages-starter as well (after about 30s build). So the long build is my app's fault, but the errors seem to depend on the npm version or something else. Btw:
npm is already the newest version (6.14.4+ds-1ubuntu2).

[stefanwouldgo]

I just updated the entire distribution (ubuntu studio focal), and now the error is gone (nb: neither node nor npm was updated) and I just get a warning:

WARNING in asset size limit: The following asset(s) exceed the recommended size limit (244 KiB).
This can impact web performance.
Assets: 
  images/author/dillon.jpg (325 KiB)

WARNING in webpack performance recommendations: 
You can limit the size of your bundles by using import() or require.ensure to lazy load some parts of your application.
For more info visit https://webpack.js.org/guides/code-splitting/

The fascinating thing is that this only worked in the pure elm-pages-starter repo. My complete app still throws the same error as above. Maybe this is just a timeout or something like that?

[dillonkearns]

We did some performance benchmarking to find bottlenecks, discussed in this thread: dillonkearns/elm-pages#137. The main finding was that rendering images in puppeteer, and using the headless browser in general, was causing a bottleneck for the number of pages it could build. And it was sometimes causing errors.

I made some performance improvements in a recent release, but the build is still using puppeteer under the hood to pre-render pages. So it's possible that that's a source of issues in your app. I built the beta build step to address those issues. It's not ready for the main release quite yet because I'm working on the recommended workflow for installing NPM packages to use with the new approach (which is no longer based on webpack).

But it's stable and I'm using it to build elm-pages.com, elm-radio.com, and several other sites. So you could give it a try if you're not relying on NPM packages in your JS bundle right now: https://github.com/dillonkearns/elm-pages/blob/master/docs/7.0.0-elm-package-upgrade-guide.md#2---beta-build-command.

[stefanwouldgo]

Thanks, I'll try that ASAP. Dillon Kearns <[email protected]> schrieb am Fr., 22. Jan. 2021, 16:30:

…

We did some performance benchmarking to find bottlenecks, discussed in this thread: dillonkearns/elm-pages#137 <dillonkearns/elm-pages#137>. The main finding was that rendering images in puppeteer, and using the headless browser in general, was causing a bottleneck for the number of pages it could build. And it was sometimes causing errors. I made some performance improvements in a recent release, but the build is still using puppeteer under the hood to pre-render pages. So it's possible that that's a source of issues in your app. I built the beta build step to address those issues. It's not ready for the main release quite yet because I'm working on the recommended workflow for installing NPM packages to use with the new approach (which is no longer based on webpack). But it's stable and I'm using it to build elm-pages.com, elm-radio.com, and several other sites. So you could give it a try if you're not relying on NPM packages in your JS bundle right now: https://github.com/dillonkearns/elm-pages/blob/master/docs/7.0.0-elm-package-upgrade-guide.md#2---beta-build-command . — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#26 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFQNEGGMCUJI7BTQUJZFHLS3GKZBANCNFSM4WKZMM7A> .

[stefanwouldgo]

Running elm-pages-beta crashes immediately for me:

stefan@xpsgiraffeguix ~/elm-pages-starter$ node_modules/.bin/elm-pages-beta
events.js:174
      throw er; // Unhandled 'error' event
      ^

Error: spawn elm-optimize-level-2 ENOENT
    at Process.ChildProcess._handle.onexit (internal/child_process.js:240:19)
    at onErrorNT (internal/child_process.js:415:16)
    at process._tickCallback (internal/process/next_tick.js:63:19)
Emitted 'error' event at:
    at Process.ChildProcess._handle.onexit (internal/child_process.js:246:12)
    at onErrorNT (internal/child_process.js:415:16)
    at process._tickCallback (internal/process/next_tick.js:63:19)

[dillonkearns]

That sounds like a platform-specific spawn issue. I know that spawn can cause problems, which is why the cross-spawn NPM package was created. I think that would probably fix the problem there. I'll try that out when I have some cycles.

[stefanwouldgo]

Unfortunately, the same is happening on my native Ubuntu installation.

[dillonkearns]

Thanks for checking. Could you try this:

npm install --save-dev elm-optimize-level-2
npx elm-pages-beta

I think the current beta build script is expecting elm-optimize-level-2 to be on the PATH.

[stefanwouldgo]

Thanks for checking. Could you try this:

On guix/WSL, I seem to have too old a node version for this:

npm WARN notsup Unsupported engine for [email protected]: wanted: {"node":">=12"} (current: {"node":"10.22.1","npm":"6.14.11"})
npm WARN notsup Not compatible with your version of node/npm: [email protected]
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules/watchpack-chokidar2/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

+ [email protected]
updated 2 packages and audited 1389 packages in 11.26s

And the same problem on Ubuntu:
npm WARN notsup Unsupported engine for [email protected]: wanted: {"node":">=12"} (current: {"node":"10.19.0","npm":"6.14.4"})

I'll see if I can get node >= 12 running ...

[stefanwouldgo]

Using node 12 from snap on Ubuntu, the beta build is working! Thank you for your help! It still takes more than 10 minutes, but maybe that's because I use unoptimizedRequests for getting csv data.

[stefanwouldgo]

BTW, is there a develop mode for the beta build?

[dillonkearns]

BTW, is there a develop mode for the beta build?

Not yet, but there definitely will be before it goes out of beta!

I'm glad that it's working now on Node 12! I still wouldn't expect it to take 10 minutes. I wonder where the bottleneck is. If you're able to share some code then I could see if I can reproduce it and debug it a bit.

I'd also be curious how the performance is through regular HTTP and parsing in vanilla Elm for comparison. Does that run in a few seconds or so?

[stefanwouldgo]

The build has actually become pretty fast now, on the order of 10 seconds. When it was more than 10 minutes, I used to do a lot of calculation in the StaticHttp request, as in the request definition in this file: https://github.com/netpositive-money/elm-pages/blob/064239ae78afecb737af6866b6c312511650b5cb/src/Calculator.elm

In regular http, this took a few seconds including downloading the data before the page got displayed.

Now I've moved the data processing beyond the request barrier into the view proper, which isn't ideal (because it seems there's no way for doing proper memoization in elm today), but it's fast enough when I cut the data down to monthly.