machine learning - Why does MITIE get stuck on segment classifier? -

-



i'm building model using mitie training dataset of 1,400 sentences, between 3-10 words long, paired around 120 intents. model training stuck @ part ii: train segment classifier. i've let run 14 hours before terminating.

my machines has 2.4 ghz intel core i7 , 8 gb 1600 mhz ddr3 , segment classifier uses available memory (around 7gb), relying on compressed memory, , @ end of last session activity monitor showed 32gb used , 27gb compressed. , segment classifier has never completed.

my current output below:

info:rasa_nlu.model:starting train component nlp_mitie info:rasa_nlu.model:finished training component. info:rasa_nlu.model:starting train component tokenizer_mitie info:rasa_nlu.model:finished training component. info:rasa_nlu.model:starting train component ner_mitie training recognize 20 labels: 'pet', 'room_number', 'broken_things', '@sys.ignore', 'climate', 'facility', 'gym', 'medicine', 'item', 'exercise_equipment ', 'service', 'number', 'electronic_device', 'charger', 'toiletries', 'time', 'date', 'facility_hours', 'cost_inquiry', 'tv channel' part i: train segmenter words in dictionary: 200000 num features: 271  training c:           20 epsilon:     0.01 num threads: 1 cache size:  5 max iterations: 2000 loss per missed segment:  3 c: 20   loss: 3         0.669591 c: 35   loss: 3         0.690058 c: 20   loss: 4.5       0.701754 c: 5   loss: 3  0.616959 c: 20   loss: 1.5       0.634503 c: 28.3003   loss: 5.74942      0.71345 c: 25.9529   loss: 5.72171      0.707602 c: 27.7407   loss: 5.97907      0.707602 c: 30.2561   loss: 5.61669      0.701754 c: 27.747   loss: 5.66612       0.710526 c: 28.9754   loss: 5.82319      0.707602 best c: 28.3003 best loss: 5.74942 num feats in chunker model: 4095 train: precision, recall, f1-score: 0.805851 0.885965 0.844011 part i: elapsed time: 180 seconds.  part ii: train segment classifier training num training samples: 415

i understand issue caused redundant labels (as explained here); however, of labels unique. understanding training shouldn't take long or use memory. i've seen others posting similar issues no solution provided yet. causing high memory usage , insane training time? how fixed?





wiki

Comments

Post a Comment

Popular posts from this blog

Asterisk AGI Python Script to Dialplan does not work -

-
i'm stuck project have check database of sip extension incoming caller id , dial sip extension. below receive when run script on asterisk cli. -- registered sip '9125' @ 192.168.1.102:50142 > saved useragent "zoiper rv2.8.40" peer 9125 [2017-08-22 13:53:33] notice[5095]: chan_sip.c:24558 handle_response_peerpoke: peer '9125' reachable. (17ms / 2000ms) == using sip rtp tos bits 184 == using sip rtp cos mark 5 -- executing [0112617769@from-trunk:1] answer("sip/obitrunk1-0000000d", "") in new stack > 0x1ca50d0 -- probation passed - setting rtp source address 192.168.1.98:16834 -- executing [0112617769@from-trunk:2] agi("sip/obitrunk1-0000000d", "test.py") in new stack -- launched agi script /var/lib/asterisk/agi-bin/test.py args: ['/var/lib/asterisk/agi-bin/test.py'] env line: agi_request: test.py env line: agi_channel: sip/obitrunk1-0000000d env line: agi_language: en env line: a
Read more

python - Read npy file directly from S3 StreamingBody -

-
i have many .npy (numpy) , .json files saved in s3, need download , load these files. for json files downloading files , loading dictionary using following code: obj = s3.object(bucket, key) response = obj.get() body = response['body'].read().decode('utf-8') data = json.loads(body) that way avoid writing .json file disk , reading it. now want same .npy files, equivalent? currently downloading file, saving temp file on disk , loading np.load avoid if possible. following https://stackoverflow.com/a/28196540/3509999 : obj = s3.object(bucket, key) io.bytesio(obj.get()["body"].read()) f: # rewind file f.seek(0) arr = np.load(f) wiki
Read more

kotlin - Out-projected type in generic interface prohibits the use of metod with generic parameter -

-
i start using kotlin , defined interface this: interface aadapter<vh : recyclerview.viewholder> { fun oncreateaviewholder(parent: viewgroup): vh fun onbindaviewholder(v: vh, position: int) } and when try use on code: class klasa ( private val adapter: aadapter<*> ) { fun dosth(){ //... val vh = this.adapter.oncreateaviewholder(parent) //on below line error adapter.onbindaviewholder(v, position) //... } } i error out-projected type 'aadapter<*>' prohibits use of 'public abstract fun onbindaviewholder(v: t, position: int): unit i tied add "in" or "out" definition i'm confused. how allow it. replace aadapter<*> aadapter<recyclerview.viewholder> . class klasa ( private val adapter: aadapter<recyclerview.viewholder> ) { // ... because have declared aadapter has type parameter vh type subclass of recyclerview.
Read more