{"_id":"599d5d7b03a64b000fde6547","project":"55faf11ba62ba1170021a9a7","version":{"_id":"55faf11ba62ba1170021a9aa","project":"55faf11ba62ba1170021a9a7","__v":40,"createdAt":"2015-09-17T16:58:03.490Z","releaseDate":"2015-09-17T16:58:03.490Z","categories":["55faf11ca62ba1170021a9ab","55faf8f4d0e22017005b8272","55faf91aa62ba1170021a9b5","55faf929a8a7770d00c2c0bd","55faf932a8a7770d00c2c0bf","55faf94b17b9d00d00969f47","55faf958d0e22017005b8274","55faf95fa8a7770d00c2c0c0","55faf96917b9d00d00969f48","55faf970a8a7770d00c2c0c1","55faf98c825d5f19001fa3a6","55faf99aa62ba1170021a9b8","55faf99fa62ba1170021a9b9","55faf9aa17b9d00d00969f49","55faf9b6a8a7770d00c2c0c3","55faf9bda62ba1170021a9ba","5604570090ee490d00440551","5637e8b2fbe1c50d008cb078","5649bb624fa1460d00780add","5671974d1b6b730d008b4823","5671979d60c8e70d006c9760","568e8eef70ca1f0d0035808e","56d0a2081ecc471500f1795e","56d4a0adde40c70b00823ea3","56d96b03dd90610b00270849","56fbb83d8f21c817002af880","573c811bee2b3b2200422be1","576bc92afb62dd20001cda85","5771811e27a5c20e00030dcd","5785191af3a10c0e009b75b0","57bdf84d5d48411900cd8dc0","57ff5c5dc135231700aed806","5804caf792398f0f00e77521","58458b4fba4f1c0f009692bb","586d3c287c6b5b2300c05055","58ef66d88646742f009a0216","58f5d52d7891630f00fe4e77","59a555bccdbd85001bfb1442","5a2a81f688574d001e9934f5","5b080c8d7833b20003ddbb6f"],"is_deprecated":false,"is_hidden":false,"is_beta":true,"is_stable":true,"codename":"","version_clean":"1.0.0","version":"1.0"},"category":{"_id":"586d3c287c6b5b2300c05055","version":"55faf11ba62ba1170021a9aa","__v":0,"project":"55faf11ba62ba1170021a9a7","sync":{"url":"","isSync":false},"reference":false,"createdAt":"2017-01-04T18:17:12.683Z","from_sync":false,"order":21,"slug":"task-execution","title":"TASK EXECUTION"},"user":"575e85ac41c8ba0e00259a44","githubsync":"","__v":0,"parentDoc":null,"updates":[],"next":{"pages":[],"description":""},"createdAt":"2017-08-23T10:48:27.448Z","link_external":false,"link_url":"","sync_unique":"","hidden":false,"api":{"results":{"codes":[]},"settings":"","auth":"required","params":[],"url":""},"isReference":false,"order":7,"body":"## Overview\nLearn how the CGC uses [AWS EC2 Spot Instances](https://aws.amazon.com/ec2/spot/) and how this strategy could help you reduce the cost of running your tasks.\n\n## About Spot Instances\nThe CGC uses two types of Amazon EC2 pricing for instances: On-Demand and Spot. On-Demand instances are purchased at a fixed rate, while the price of Spot Instances varies according to supply and demand. \n\nSpot Instances allow bidding on spare AWS EC2 computing capacity, meaning you can use the instance as long as your bidding price is higher than the market price.\n\nBecause the user always pays the current market price for the Spot Instance and Spot Instances cost a fraction of the price for On-Demand instances, using Spot Instances can significantly lower the cost of running your analysis.\n\n## How the CGC uses Spot Instances\nThe CGC uses On-Demand instances by default, meaning you pay a fixed hourly price (charged on a per second basis) for using the compute resources.\n[block:image]\n{\n  \"images\": [\n    {\n      \"image\": [\n        \"https://files.readme.io/9fa2bf5-AWS-EC2-4.png\",\n        \"AWS-EC2-4.png\",\n        433,\n        260,\n        \"#373737\"\n      ],\n      \"caption\": \"Types of AWS EC2 instances used on the CGC\"\n    }\n  ]\n}\n[/block]\nWhen using Spot Instances, you pay the market price of the Spot Instance, as long as this is lower than your bid price. The CGC strategy is to bid the On-Demand instance price for Spot Instances. This ensures you are never charged more than the On-Demand instance price for using a Spot Instance.\n\n## Analysis costs when using Spot Instances\nUsing Spot Instances can significantly reduce the cost of your analysis. Running an app using Spot Instances can cost as little as a quarter of the On-Demand price.\n \nThe [AWS Spot Instance bid advisor](https://aws.amazon.com/ec2/spot/bid-advisor/) allows you to see projected savings and a history of outbidding frequency for specific Spot Instances to help you in your decision. Some instances get outbid more frequently than others and the longer a job runs for, the more likely it is to be terminated. \n\nAWS EC2 will terminate your Spot Instance if the bid price becomes lower than the market price for that instance. In this case interrupted jobs need to be restarted, so your analysis will taken longer to complete and the analysis cost might increase as well. See the section on Spot Instance termination to learn how this could impact the cost of your analysis\n\n### The impact of Spot Instance termination\nBecause there is a limited number of  Spot Instances, the market price of Spot Instances also increases when demand increases. If the market price becomes higher than what we bid for the Spot Instance, the Spot Instance gets terminated and the the task will continue running on an On-Demand instance(s). If spot instance termination occurs during the first hour of running the task on the instance, you will not be charged for using the Spot Instance. However, if the Spot Instance is terminated at any point after the first 60 minutes, you will be charged for the entire number of seconds the instance was running.\n\nThe job(s) running on the instance at the time of termination will be interrupted and have to be run again from the beginning. The jobs will be restarted on an equivalent On-Demand instance to minimise time wasted in completing your task.\n\nRestarting jobs on another instance will inevitably prolong task execution time and add to the cost of running that job. The cost of re-running is greatest for long jobs (in the order of hours) that get interrupted close to completion. The possibility that a Spot Instance is terminated is why they are not recommended for running long, time-critical jobs.","excerpt":"","slug":"about-spot-instances","type":"basic","title":"About Spot Instances"}

About Spot Instances


## Overview Learn how the CGC uses [AWS EC2 Spot Instances](https://aws.amazon.com/ec2/spot/) and how this strategy could help you reduce the cost of running your tasks. ## About Spot Instances The CGC uses two types of Amazon EC2 pricing for instances: On-Demand and Spot. On-Demand instances are purchased at a fixed rate, while the price of Spot Instances varies according to supply and demand.  Spot Instances allow bidding on spare AWS EC2 computing capacity, meaning you can use the instance as long as your bidding price is higher than the market price. Because the user always pays the current market price for the Spot Instance and Spot Instances cost a fraction of the price for On-Demand instances, using Spot Instances can significantly lower the cost of running your analysis. ## How the CGC uses Spot Instances The CGC uses On-Demand instances by default, meaning you pay a fixed hourly price (charged on a per second basis) for using the compute resources. [block:image] { "images": [ { "image": [ "https://files.readme.io/9fa2bf5-AWS-EC2-4.png", "AWS-EC2-4.png", 433, 260, "#373737" ], "caption": "Types of AWS EC2 instances used on the CGC" } ] } [/block] When using Spot Instances, you pay the market price of the Spot Instance, as long as this is lower than your bid price. The CGC strategy is to bid the On-Demand instance price for Spot Instances. This ensures you are never charged more than the On-Demand instance price for using a Spot Instance. ## Analysis costs when using Spot Instances Using Spot Instances can significantly reduce the cost of your analysis. Running an app using Spot Instances can cost as little as a quarter of the On-Demand price.   The [AWS Spot Instance bid advisor](https://aws.amazon.com/ec2/spot/bid-advisor/) allows you to see projected savings and a history of outbidding frequency for specific Spot Instances to help you in your decision. Some instances get outbid more frequently than others and the longer a job runs for, the more likely it is to be terminated.  AWS EC2 will terminate your Spot Instance if the bid price becomes lower than the market price for that instance. In this case interrupted jobs need to be restarted, so your analysis will taken longer to complete and the analysis cost might increase as well. See the section on Spot Instance termination to learn how this could impact the cost of your analysis ### The impact of Spot Instance termination Because there is a limited number of  Spot Instances, the market price of Spot Instances also increases when demand increases. If the market price becomes higher than what we bid for the Spot Instance, the Spot Instance gets terminated and the the task will continue running on an On-Demand instance(s). If spot instance termination occurs during the first hour of running the task on the instance, you will not be charged for using the Spot Instance. However, if the Spot Instance is terminated at any point after the first 60 minutes, you will be charged for the entire number of seconds the instance was running. The job(s) running on the instance at the time of termination will be interrupted and have to be run again from the beginning. The jobs will be restarted on an equivalent On-Demand instance to minimise time wasted in completing your task. Restarting jobs on another instance will inevitably prolong task execution time and add to the cost of running that job. The cost of re-running is greatest for long jobs (in the order of hours) that get interrupted close to completion. The possibility that a Spot Instance is terminated is why they are not recommended for running long, time-critical jobs.