Overheads

Posted in General by Anjan Purkayastha Thu Jun 23 2016 18:48:06 GMT+0000 (Coordinated Universal Time)·6·Viewed 629 times

As a test of runtimes, I ran one of my dockerized apps without an input file. The app has been written to exit immediately, in the event an input file is not specified, with an error message. On my local machine the app behaves as expected on being run without an input file - immediate exit with a help message. However on CGC, without an input file, the time it took to complete was 3 minutes. Of this , 2m and 14s was spent in the queue and 46 s was spent in running. My question is why did the app run for 46s before exiting? Exit should have been immediate. What is the overhead time being spent in? TIA for any insight. Anjan
Erik Lehnert
June 23, 2016

Hi Anjan,

At least part of the overhead is spent building the Docker image within the instance. This can take varying amounts of time, depending on what people include in their Docker image. For example, if an image included a large number of reference files (which we don't recommend), building the image in the instance could take some time.

Best,
Erik

Anjan Purkayastha
June 23, 2016

That answers my question. Thanks Eric!

Brandi Davis-Dusenbery
June 23, 2016

Thanks for the good question Anjan, the

Brandi Davis-Dusenbery
June 23, 2016

Sorry .... just wanted to add that the system also does a number of checks to try to ensure that a task could be run prior to starting it --- for example: does the user who started the task have permissions on the files, is there sufficient funding etc. If there is interest in what happens before you click the 'run' button we could put together a blog post or similar.

Anjan Purkayastha
June 24, 2016

Brandi,
This is an important part of the CGC's workflow for users to understand, if only to be assured that the requisite checks are in place to ensure that docker images are uploaded and run as expected. So, at least, a high level description of what happens when one submits a job, whether in the form of a blog or a frequently asked question, will be useful information for users.
Thanks!
Anjan

Brandi Davis-Dusenbery
June 26, 2016

Anjan, Thanks! We'll let you know when we post something.

  
Markdown is allowed