Cannot parse from a stream in the browser #443

psaffrey · 2024-11-16T13:51:46Z

Describe the bug

I want to allow a user to upload a gzipped csv file and parse it in a stream (using pipeThrough / pipeTo). This works in Node, but not in the browser, where I get:

file-upload.html:52 Uncaught (in promise) TypeError: Failed to execute 'pipeTo' on 'ReadableStream': parameter 1 is not of type 'WritableStream'.
    at HTMLInputElement.<anonymous> (file-upload.html:52:18)

Line 52 is the pipeTo line from the example below, so it seems to think that the parse object is not a WritableStream.

To Reproduce

Dump the code below into a file, open it in a browser and then try uploading a .tsv.gz file. I tested in Chrome and Firefox. The same code works in Node, although you have to swap pipeThrough and pipeTo for pipe. I'll admit that I find the differences between Node and the browser for streaming a bit confusing so I may have made some obvious error here.

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>File Upload</title>
</head>
<body>
    <div class="upload-container">
        <form class="upload-form">
            <input type="file" id="fileInput" name="file">
        </form>
    </div>

    <script type="module">
        import { parse } from 'https://cdn.jsdelivr.net/npm/[email protected]/+esm'

        document.querySelector('#fileInput').addEventListener('change', (e) => {

            e.preventDefault();
            const file = document.getElementById('fileInput').files[0];

            // turn the file into a stream
            const fileStream = file.stream()

            // create a gunzip transform stream
            const unzipper = new DecompressionStream('gzip')    

            const records = [];

            // create a csv parser writeable stream
            const parser = parse({
              delimiter: '\t'
            });            
 
            parser.on('readable', function(){
              let record;
              while ((record = parser.read()) !== null) {
                records.push(record);
              }
            });
            parser.on('error', function(err){
              console.error(err.message);
            });
            parser.on('end', function(){
                console.log(records)
            })

            console.log("piping!")
            // pipe the file stream through the gunzip stream and into the csv parse stream
            fileStream
                .pipeThrough(unzipper)
                .pipeTo(parser)
        });
    </script>
</body>
</html>

** Additional Context **

I wonder if this issue is related, although I'm not trying to parse the complex CSVs listed in that issue.

I also have a similar issue raised for Papaparse.

The text was updated successfully, but these errors were encountered:

wdavidw · 2024-11-16T16:02:11Z

You need to reference the browser ESM or IIFE distributions.

wdavidw · 2024-11-16T16:02:52Z

you can also look at the demo folders.

psaffrey-biomodal · 2024-11-21T13:07:18Z

Thanks very much for the rapid response. Unfortunately, this doesn't seem to fix the error. Here's what I did:

Pulled down the node-csv git repository and ran the example in the demo/browser directory
Created a myparser.html file with the code I linked above in the same directory so that it's hosted by Express
Modified the import line to be import { parse } from '/lib/esm/parse/index.js';

But I still get the same error:

myparser.html:53 Uncaught (in promise) TypeError: Failed to execute 'pipeTo' on 'ReadableStream': parameter 1 is not of type 'WritableStream'.
    at HTMLInputElement.<anonymous> (myparser.html:53:18)

Once I've figured out the underlying problem here, I'd also like to use a CDN if possible. Would you be open to a PR on the documentation suggesting how a CDN can be used?

wdavidw · 2024-11-21T15:13:53Z

I noticed your URL referencing https://cdn.jsdelivr.net/npm/[email protected]/+esm. I have no idea how/by who this was set. Please update the documentation to provide guidance.

I just tried the following commands successfully.

mkdir csv-443
cd csv-443
# Download the source code
npm install csv-parse
# Create an html file
cat <<'HTML' >index.html
<script type="module">
  import { parse } from '/node_modules/csv-parse/dist/esm/sync.js';
  console.log('ok1', parse)
  const data = parse("a,b,c\n1,2,3\n4,5,6", { columns: true });
  document.body.innerHTML = `<pre>${ JSON.stringify(data, 0, 2) }</pre>`
</script>
HTML
# Start an HTML server
npm install http-server
npx http-server -p 3000 .

psaffrey-biomodal · 2024-11-21T17:21:35Z

The URL is a CDN reference - I put that in there because I want the page I'm creating to be standalone, rather than hosted alongside some node modules.

It's not that I can't import the module at all - the example you've sent works for me too. The problem is that I can't use it in a stream. So if you do the same set of commands, but instead create this code in the <script> tag:

<script type="module">
  import { parse } from '/node_modules/csv-parse/dist/esm/index.js';

  const readableStream = new ReadableStream({
    start(controller) {
      const csv_string = "a,b,c\n1,2,3\n4,5,6";
      controller.enqueue(csv_string)
    }
  });

  // create a csv parser writeable stream
  const parser = parse({
    delimiter: '\t'
  });

  parser.on('readable', function(){
    let record;
    while ((record = parser.read()) !== null) {
      records.push(record);
    }
  });
  parser.on('error', function(err){
    console.error(err.message);
  });
  parser.on('end', function(){
    console.log(records)
  })

  readableStream.pipeTo(parser);
</script>

Then it gives the error:

(index):56 Uncaught (in promise) TypeError: Failed to execute 'pipeTo' on 'ReadableStream': parameter 1 is not of type 'WritableStream'.
    at (index):56:18

I can also use the sync version (by replacing the import with import { parse } from '/node_modules/csv-parse/dist/esm/sync.js';) and then I get this error:

sync.js:3270 Uncaught TypeError: buf.slice is not a function
    at Object.parse (sync.js:3270:38)
    at parse (sync.js:3737:23)
    at (index):13:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cannot parse from a stream in the browser #443

Cannot parse from a stream in the browser #443

psaffrey commented Nov 16, 2024

wdavidw commented Nov 16, 2024

wdavidw commented Nov 16, 2024

psaffrey-biomodal commented Nov 21, 2024

wdavidw commented Nov 21, 2024

psaffrey-biomodal commented Nov 21, 2024

Cannot parse from a stream in the browser #443

Cannot parse from a stream in the browser #443

Comments

psaffrey commented Nov 16, 2024

wdavidw commented Nov 16, 2024

wdavidw commented Nov 16, 2024

psaffrey-biomodal commented Nov 21, 2024

wdavidw commented Nov 21, 2024

psaffrey-biomodal commented Nov 21, 2024