solr - SolrCloud Indexing issue -

-



we facing 1 indexing issue in solr cloud.
solr version: 6.3.0
solr cloud configuration:

3 node zookeeper 3 solr instance. 1 collection 3 shards , 2 replicas. indexing 10000 document indexed docs shard1=3393, shard2=3351, shard3=3256

testing: down leader replica when indexing running. connection refuse exception generated continuously in sendupdatestream method of class concurrentupdatesolrclient in log. display 10000 records indexed, while execute docs (:) query, solr return 6607 documents other 3393 documents not return in result.

example: gave dataimport command machine1:8091, machine1:8091 solr instance responsible document routing have indexed of document in local index , of them send dedicated shards.

suppose down machine2:8092 solr instance, contains 1 leader replica of shard1.so select machine3:8093 leader replica.

data import screen display 10000 documents indexed/processed. while searching 6607 document indexed. , while searching in shard1 0 documents indexed 3393 document of shard1 not indexed. , connection refused exception generated.

from debugging code, come know that,

in concurrentupdatesolrclient class document send dedicated shard , replica solrj request , added in runner queue , these request run scheduler later on.

when leader goes down, runner queue may have many requests point old leader , down, try send request old leader , got connection refuse exception. because have not modified pending request though new leader selected shard runner queue holding client request point old leader not new leader. there no code handling issue, if exception generated concurrentupdatesolrclient’s sendinputstream(), not handle caller.

we have tried solve issue modifying concurrentupdatesolrclient class. added code modify current client request when ioexception generated in concurrentupdatesolrclient’s sendinputstream method , resubmit current request new leader node.

by modifying code able indexed failed document but, documents still missing getting 9850 document while searching.

note: schema file have 140, of text_general type , 5 copy fields 1 of metadata field contains metadata information of document , 1 of full_text contains text file of 10 500 mb.





wiki

Comments

Post a Comment

Popular posts from this blog

Asterisk AGI Python Script to Dialplan does not work -

-
i'm stuck project have check database of sip extension incoming caller id , dial sip extension. below receive when run script on asterisk cli. -- registered sip '9125' @ 192.168.1.102:50142 > saved useragent "zoiper rv2.8.40" peer 9125 [2017-08-22 13:53:33] notice[5095]: chan_sip.c:24558 handle_response_peerpoke: peer '9125' reachable. (17ms / 2000ms) == using sip rtp tos bits 184 == using sip rtp cos mark 5 -- executing [0112617769@from-trunk:1] answer("sip/obitrunk1-0000000d", "") in new stack > 0x1ca50d0 -- probation passed - setting rtp source address 192.168.1.98:16834 -- executing [0112617769@from-trunk:2] agi("sip/obitrunk1-0000000d", "test.py") in new stack -- launched agi script /var/lib/asterisk/agi-bin/test.py args: ['/var/lib/asterisk/agi-bin/test.py'] env line: agi_request: test.py env line: agi_channel: sip/obitrunk1-0000000d env line: agi_language: en env line: a
Read more

python - Read npy file directly from S3 StreamingBody -

-
i have many .npy (numpy) , .json files saved in s3, need download , load these files. for json files downloading files , loading dictionary using following code: obj = s3.object(bucket, key) response = obj.get() body = response['body'].read().decode('utf-8') data = json.loads(body) that way avoid writing .json file disk , reading it. now want same .npy files, equivalent? currently downloading file, saving temp file on disk , loading np.load avoid if possible. following https://stackoverflow.com/a/28196540/3509999 : obj = s3.object(bucket, key) io.bytesio(obj.get()["body"].read()) f: # rewind file f.seek(0) arr = np.load(f) wiki
Read more

kotlin - Out-projected type in generic interface prohibits the use of metod with generic parameter -

-
i start using kotlin , defined interface this: interface aadapter<vh : recyclerview.viewholder> { fun oncreateaviewholder(parent: viewgroup): vh fun onbindaviewholder(v: vh, position: int) } and when try use on code: class klasa ( private val adapter: aadapter<*> ) { fun dosth(){ //... val vh = this.adapter.oncreateaviewholder(parent) //on below line error adapter.onbindaviewholder(v, position) //... } } i error out-projected type 'aadapter<*>' prohibits use of 'public abstract fun onbindaviewholder(v: t, position: int): unit i tied add "in" or "out" definition i'm confused. how allow it. replace aadapter<*> aadapter<recyclerview.viewholder> . class klasa ( private val adapter: aadapter<recyclerview.viewholder> ) { // ... because have declared aadapter has type parameter vh type subclass of recyclerview.
Read more