FATAL ERROR: v8::ToLocalChecked Empty MaybeLocal

[emrebuzkiran]

hi guys
when i try to run this code on my terminal ->ts-node .\trimMainnet.ts
im getting this error message
FATAL ERROR: v8::ToLocalChecked Empty MaybeLocal
----- Native stack trace -----

1: 00007FF6014E6E7B node::SetCppgcReference+16075
2: 00007FF60145D996 v8::base::CPU::num_virtual_address_bits+79190
3: 00007FF60145FCAC node::OnFatalError+252
4: 00007FF601ED6DD3 v8::api_internal::ToLocalEmpty+83
5: 00007FF601452E69 v8::base::CPU::num_virtual_address_bits+35369
6: 00007FF601E83F0E v8::SharedValueConveyor::SharedValueConveyor+416270
7: 00007FF601E83B0A v8::SharedValueConveyor::SharedValueConveyor+415242
8: 00007FF601E83DCF v8::SharedValueConveyor::SharedValueConveyor+415951
9: 00007FF601E83C40 v8::SharedValueConveyor::SharedValueConveyor+415552
10: 00007FF601F7EEAE v8::PropertyDescriptor::writable+676878
11: 00007FF5820E6EFE

----- JavaScript stack trace -----

1: readFileSync (node:fs:453:20)
2: trimMainnetJson (C:\Users***\raydium-swap\v2\raydium-sdk-swap-example-typescript\src\trimMainnet.ts:10:49)
3: C:\Users***\raydium-swap\v2\raydium-sdk-swap-example-typescript\src\trimMainnet.ts:28:1
4: Module._compile (node:internal/modules/cjs/loader:1376:14)
5: m._compile (C:\Users**\AppData\Roaming\npm\node_modules\ts-node\dist\index.js:857:29)
6: Module._extensions..js (node:internal/modules/cjs/loader:1435:10)
7: require.extensions. (C:\Users***\AppData\Roaming\npm\node_modules\ts-node\dist\index.js:859:16)
8: Module.load (node:internal/modules/cjs/loader:1207:32)
9: Module._load (node:internal/modules/cjs/loader:1023:12)
10: executeUserEntryPoint (node:internal/modules/run_main:135:12)

[emrebuzkiran]

Issue Update (Solution)
Hey everyone,

I found the cause of this issue, and here’s how to fix it.

Cause of the Issue
The error occurs because the mainnet.json file keeps growing larger over time, eventually becoming too big for fs.readFileSync().
When trying to load a huge JSON file all at once into memory, it can cause V8 to run out of memory, leading to the v8::ToLocalChecked Empty MaybeLocal error.

Solution: Use a Streaming JSON Parser
Instead of reading the entire file at once, use a streaming JSON parser like stream-json, which processes the file piece by piece to avoid memory overload.

How to Implement a Streaming JSON Parser
If your JSON file structure looks like this:
{ "name": "Raydium Mainnet Liquidity Pools", "official": [ ... ], "unOfficial": [ ... ] }

You can stream process the file like this:
`
import fs from 'fs';
import { pipeline } from 'stream';
import { parser } from 'stream-json';
import { streamObject } from 'stream-json/streamers/StreamObject';

const readStream = fs.createReadStream('../mainnet.json', { encoding: 'utf-8' });

pipeline(
readStream,
parser(),
streamObject(), // Streams JSON as key-value pairs
async function (source) {
for await (const { key, value } of source) {
if (key === 'official' || key === 'unOfficial') {
console.log(Processing: ${key}, Number of Pools: ${value.length});
// Process the data in chunks instead of loading everything at once
}
}
},
(err) => {
if (err) {
console.error('Error while processing file:', err);
}
}
);
`