javascript - How to set limits on the number of lines read by stream nodejs -




i reading long csv text , streaming client. need able assure each stream passed on client has kind of line break mark associated it. problem when stream data, strips line breaks. there way can assure line breaks not stripped or replaced characters can recognize line breaks on client side 'while' streaming?

also, there no guarantee each buffer in stream end @ line break point example if have below data

10,20,30 40,50,60 

the next buffer may start 50 instead of 40. how know each buffer should concatenated or without line break?

server side

app.get('/test/api', (req, res) => {     let stream = fs.createreadstream('./resources/onescsv.csv');     stream = byline.createstream(stream);     stream.on('data', (data)=>{         // console.log(data.tostring());         res.write(data + '||||')     });     // stream.pipe(res);     stream.on('end', () => res.end());  }); 

client side

let count = 0;     $(document).ready(()=>{         let lastofprevious;         let firstofcurrent;         let xhr = new xmlhttprequest();         xhr.open('get', '/test/api', true);         xhr.onprogress = function (){             console.log('line count');             let incomingbuffer = xhr.responsetext;             console.log(incomingbuffer)     };     xhr.send() }); 

thanks.

i believe streaming big csv data server client, in client not able figure out line breaks in streamed csv data.

very easy soultion using builtin node module 'readline' allow read stream data line line. have added below working code snippet illustrate this.

stream_server.js

const http = require('http'); const fs = require('fs'); var server = http.createserver(function (req, res) {     let stream = fs.createreadstream(__dirname + '/tmp.csv');     stream.pipe(res); }); server.listen(8000) 

stream_client.js

const fs = require('fs'); const http = require('http'); const readline = require('readline');  var options = {     host: 'localhost',     port: 8000   };  http.get(options, function(res) {     console.log(`got response:  ${res.statuscode}`);     let readstream = readline.createinterface({         input: res,         output: process.stdout,         terminal: false     });     readstream.on('line', function(line) {         console.log(`reading -->  ${line}`);     });  }).on('error', function(e) {     console.log(`got error  ${e.message}`); }); 

hope solves problem ..

i have kept above part pertaining initial understanding of quesion, modified phrasing of question looks adding delimiter each line of csv in server response.

well if define express route below add delimiter each line of csv file

app.get('/test/api', (req, res) => {     let stream = fs.createreadstream('./tmp.csv');     let readstream = readline.createinterface({         input: stream,         output: process.stdout,         terminal: false     });     readstream.on('line', function(line) {         console.log(`reading -->  ${line}`);         res.write(line + ' |||| ')     });     stream.on('end', () => res.end()); }); 




wiki

Comments

Popular posts from this blog

Asterisk AGI Python Script to Dialplan does not work -

python - Read npy file directly from S3 StreamingBody -

kotlin - Out-projected type in generic interface prohibits the use of metod with generic parameter -