javascript - How to set limits on the number of lines read by stream nodejs -
i reading long csv text , streaming client. need able assure each stream passed on client has kind of line break mark associated it. problem when stream data, strips line breaks. there way can assure line breaks not stripped or replaced characters can recognize line breaks on client side 'while' streaming?
also, there no guarantee each buffer in stream end @ line break point example if have below data
10,20,30 40,50,60
the next buffer may start 50 instead of 40. how know each buffer should concatenated or without line break?
server side
app.get('/test/api', (req, res) => { let stream = fs.createreadstream('./resources/onescsv.csv'); stream = byline.createstream(stream); stream.on('data', (data)=>{ // console.log(data.tostring()); res.write(data + '||||') }); // stream.pipe(res); stream.on('end', () => res.end()); });
client side
let count = 0; $(document).ready(()=>{ let lastofprevious; let firstofcurrent; let xhr = new xmlhttprequest(); xhr.open('get', '/test/api', true); xhr.onprogress = function (){ console.log('line count'); let incomingbuffer = xhr.responsetext; console.log(incomingbuffer) }; xhr.send() });
thanks.
i believe streaming big csv data server client, in client not able figure out line breaks in streamed csv data.
very easy soultion using builtin node module 'readline' allow read stream data line line. have added below working code snippet illustrate this.
stream_server.js
const http = require('http'); const fs = require('fs'); var server = http.createserver(function (req, res) { let stream = fs.createreadstream(__dirname + '/tmp.csv'); stream.pipe(res); }); server.listen(8000)
stream_client.js
const fs = require('fs'); const http = require('http'); const readline = require('readline'); var options = { host: 'localhost', port: 8000 }; http.get(options, function(res) { console.log(`got response: ${res.statuscode}`); let readstream = readline.createinterface({ input: res, output: process.stdout, terminal: false }); readstream.on('line', function(line) { console.log(`reading --> ${line}`); }); }).on('error', function(e) { console.log(`got error ${e.message}`); });
hope solves problem ..
i have kept above part pertaining initial understanding of quesion, modified phrasing of question looks adding delimiter each line of csv in server response.
well if define express route below add delimiter each line of csv file
app.get('/test/api', (req, res) => { let stream = fs.createreadstream('./tmp.csv'); let readstream = readline.createinterface({ input: stream, output: process.stdout, terminal: false }); readstream.on('line', function(line) { console.log(`reading --> ${line}`); res.write(line + ' |||| ') }); stream.on('end', () => res.end()); });
wiki
Comments
Post a Comment