How do I handle a large number of files as an input to a build when using VSTS? -

-



to set expectations, i'm new build tooling. we're using hosted agent we're open other options.

we've got local application kicks off build using vsts api. hosted build tasks involve get sources step github repo local file system in vso. next step need copy on large number of files (upwards of 10000 files), building solution, , running tests.

the problem cloned github repo in file system in visual studio online, , 10000 input files on local machine. seems bit much, since plan on doing ci , may have many builds being kicked off per day.

what best way move input files cloned repo can build it? should using hosted agent this? or best on our local system? i've looked in vso docs haven't found answer there. i'm not sure if asking right questions here.

there ways handle situation, can follow way closest situations.

option 1. add large files github repo

if local files related code of github repo, should add files same repo required files cloned in get sources step, can build directly without copy files step.

option 2. manage large files in git repo, , add git repo submodule github repo

if local large files used other code, can manage large files in separate repo, , treat submodule github repo git submodule add <url separate repo>. , in vsts build definition, select checkout submodules in sources step. large files can used directly when build github code.

enter image description here

option 3. use private agent on local machine

if don’t want add large files in github repo or separate git repo reasons, can use private agent instead. build run time may not improve obviously, because changed run time different between copying local files server , copying local files same local machine.





wiki

Comments

Post a Comment

Popular posts from this blog

Asterisk AGI Python Script to Dialplan does not work -

-
i'm stuck project have check database of sip extension incoming caller id , dial sip extension. below receive when run script on asterisk cli. -- registered sip '9125' @ 192.168.1.102:50142 > saved useragent "zoiper rv2.8.40" peer 9125 [2017-08-22 13:53:33] notice[5095]: chan_sip.c:24558 handle_response_peerpoke: peer '9125' reachable. (17ms / 2000ms) == using sip rtp tos bits 184 == using sip rtp cos mark 5 -- executing [0112617769@from-trunk:1] answer("sip/obitrunk1-0000000d", "") in new stack > 0x1ca50d0 -- probation passed - setting rtp source address 192.168.1.98:16834 -- executing [0112617769@from-trunk:2] agi("sip/obitrunk1-0000000d", "test.py") in new stack -- launched agi script /var/lib/asterisk/agi-bin/test.py args: ['/var/lib/asterisk/agi-bin/test.py'] env line: agi_request: test.py env line: agi_channel: sip/obitrunk1-0000000d env line: agi_language: en env line: a
Read more

python - Read npy file directly from S3 StreamingBody -

-
i have many .npy (numpy) , .json files saved in s3, need download , load these files. for json files downloading files , loading dictionary using following code: obj = s3.object(bucket, key) response = obj.get() body = response['body'].read().decode('utf-8') data = json.loads(body) that way avoid writing .json file disk , reading it. now want same .npy files, equivalent? currently downloading file, saving temp file on disk , loading np.load avoid if possible. following https://stackoverflow.com/a/28196540/3509999 : obj = s3.object(bucket, key) io.bytesio(obj.get()["body"].read()) f: # rewind file f.seek(0) arr = np.load(f) wiki
Read more

kotlin - Out-projected type in generic interface prohibits the use of metod with generic parameter -

-
i start using kotlin , defined interface this: interface aadapter<vh : recyclerview.viewholder> { fun oncreateaviewholder(parent: viewgroup): vh fun onbindaviewholder(v: vh, position: int) } and when try use on code: class klasa ( private val adapter: aadapter<*> ) { fun dosth(){ //... val vh = this.adapter.oncreateaviewholder(parent) //on below line error adapter.onbindaviewholder(v, position) //... } } i error out-projected type 'aadapter<*>' prohibits use of 'public abstract fun onbindaviewholder(v: t, position: int): unit i tied add "in" or "out" definition i'm confused. how allow it. replace aadapter<*> aadapter<recyclerview.viewholder> . class klasa ( private val adapter: aadapter<recyclerview.viewholder> ) { // ... because have declared aadapter has type parameter vh type subclass of recyclerview.
Read more