java - Kafka consumer.poll call not returning kafka ConsumerRecords -

-



i'm new kafka technology..i'm working on poc need send producerrecord<string, paymnt> kafka topic paymnt pojo..i able publish record & see messages being delivered kafka topic..

d:\kafka\kafka_2.11-0.11.0.0\bin\windows>kafka-run-class.bat kafka.tools.getoffsetshell --broker-list localhost:9092 --topic test --time -1 test:2:0 test:1:0 test:0:4

however on consumer side i'm not able retrieve same record..when debug consumer code,i see thread call blocking on consumer.poll()

consumer class

public class consumer {    public static void main(string args[]) throws ioexception {     properties props = new properties();     kafkaconsumer<string, paymnt> consumer = null;     props.put("bootstrap.servers", "localhost:9092");     props.put("batch.size", 16384);     props.put("buffer.memory", 33554432);     props.put("key.deserializer", "org.apache.kafka.common.serialization.stringdeserializer");     props.put("value.deserializer", "com.org.kafkapro.paymentdeserializer");     props.put("enable.auto.commit", "false");     props.put("group.id", "test-consumer-group");     try {         consumer =new kafkaconsumer<string, paymnt>(props);         consumer.subscribe(arrays.aslist("test"));             while(true){              consumerrecords<string, paymnt> records = consumer.poll(200);             (consumerrecord<string,paymnt> record : records)             {                 system.out.println(record.value().tostring());             }              consumer.commitasync();             }      }     catch(exception ex){         ex.printstacktrace();     }finally{         consumer.commitsync();         consumer.close();     } } }

paymentdeserliazer class 

 package com.org.kafkapro;     import java.io.bytearrayinputstream;     import java.io.ioexception;     import java.io.objectinput;     import java.io.objectinputstream;     import java.util.map;      import org.apache.kafka.common.serialization.deserializer;      public class paymentdeserializer implements deserializer<paymnt> {          public paymentdeserializer(){          }          public void close() {             // todo auto-generated method stub          }          public void configure(map<string, ?> arg0, boolean arg1) {             // todo auto-generated method stub          }          public paymnt deserialize(string arg0, byte[] arg1) {             // todo auto-generated method stub             bytearrayinputstream bis = new bytearrayinputstream(arg1);             objectinputstream in = null;             paymnt h2 = null;             try {                 in = new objectinputstream(bis);             } catch (ioexception e1) {                 // todo auto-generated catch block                 e1.printstacktrace();             }             try {                  h2 = (paymnt) in.readobject();             } catch (classnotfoundexception e) {                 // todo auto-generated catch block                 e.printstacktrace();             } catch (ioexception e) {                 // todo auto-generated catch block                 e.printstacktrace();             }             return h2;         }  }

paymnt class

public class paymnt  {  //fields,getters & setters         }

serializer

    public class paymentserializer implements serializer<paymnt> {           public paymentserializer(){          }          public void close() {             // todo auto-generated method stub          }          public void configure(map<string, ?> arg0, boolean arg1) {             // todo auto-generated method stub          }          public byte[] serialize(string arg0, paymnt payment) {             // todo auto-generated method stub             try {                 bytearrayoutputstream baos = new bytearrayoutputstream();                 objectoutputstream oos = new objectoutputstream(baos);                 oos.writeobject(payment);                 oos.close();                 byte[] b= baos.tobytearray();                 return b;             } catch (ioexception e) {                 return new byte[0];             }          }  }

appreciate help.thank you





wiki

Comments

Post a Comment

Popular posts from this blog

python - Read npy file directly from S3 StreamingBody -

-
i have many .npy (numpy) , .json files saved in s3, need download , load these files. for json files downloading files , loading dictionary using following code: obj = s3.object(bucket, key) response = obj.get() body = response['body'].read().decode('utf-8') data = json.loads(body) that way avoid writing .json file disk , reading it. now want same .npy files, equivalent? currently downloading file, saving temp file on disk , loading np.load avoid if possible. following https://stackoverflow.com/a/28196540/3509999 : obj = s3.object(bucket, key) io.bytesio(obj.get()["body"].read()) f: # rewind file f.seek(0) arr = np.load(f) wiki
Read more

python - Minimize function with Scipy minimize -

-
trying obtain d value (integer) std_diff objective function minimal, using scipy minimize. my code: def std_diff(d): return std(diff(df['bn'].values,d)); scipy.optimize import minimize b=(3,) res = minimize(std_diff,(1,), method='slsqp', bounds = b) **df['bn'].values** out[72]: array([ 2, 2, 2, 2, 3, 2, 7, 5, 7, 11, 8, 2, 11, 7, 15, 8, 7, 12, 21, 19, 32, 35, 40, 35, 21, 19, 25, 20, 40, 80, 99], dtype=int64) error is"indexerror: many indices array " in case not use bounds : res = minimize(std_diff,(1,), method='slsqp'), error: > in _minimize_slsqp(func, x0, args, jac, bounds, constraints, maxiter, > ftol, iprint, disp, eps, callback, **unknown_options) > 368 fx = float(np.asarray(func(x))) > 369 except: > --> 370 raise valueerror("objective function must return scalar") > 371 # compute constraints > 3
Read more

kotlin - Out-projected type in generic interface prohibits the use of metod with generic parameter -

-
i start using kotlin , defined interface this: interface aadapter<vh : recyclerview.viewholder> { fun oncreateaviewholder(parent: viewgroup): vh fun onbindaviewholder(v: vh, position: int) } and when try use on code: class klasa ( private val adapter: aadapter<*> ) { fun dosth(){ //... val vh = this.adapter.oncreateaviewholder(parent) //on below line error adapter.onbindaviewholder(v, position) //... } } i error out-projected type 'aadapter<*>' prohibits use of 'public abstract fun onbindaviewholder(v: t, position: int): unit i tied add "in" or "out" definition i'm confused. how allow it. replace aadapter<*> aadapter<recyclerview.viewholder> . class klasa ( private val adapter: aadapter<recyclerview.viewholder> ) { // ... because have declared aadapter has type parameter vh type subclass of recyclerview.
Read more