{"_id":"59fcb70c067f8d00286140da","project":"55faf11ba62ba1170021a9a7","version":{"_id":"55faf11ba62ba1170021a9aa","project":"55faf11ba62ba1170021a9a7","__v":39,"createdAt":"2015-09-17T16:58:03.490Z","releaseDate":"2015-09-17T16:58:03.490Z","categories":["55faf11ca62ba1170021a9ab","55faf8f4d0e22017005b8272","55faf91aa62ba1170021a9b5","55faf929a8a7770d00c2c0bd","55faf932a8a7770d00c2c0bf","55faf94b17b9d00d00969f47","55faf958d0e22017005b8274","55faf95fa8a7770d00c2c0c0","55faf96917b9d00d00969f48","55faf970a8a7770d00c2c0c1","55faf98c825d5f19001fa3a6","55faf99aa62ba1170021a9b8","55faf99fa62ba1170021a9b9","55faf9aa17b9d00d00969f49","55faf9b6a8a7770d00c2c0c3","55faf9bda62ba1170021a9ba","5604570090ee490d00440551","5637e8b2fbe1c50d008cb078","5649bb624fa1460d00780add","5671974d1b6b730d008b4823","5671979d60c8e70d006c9760","568e8eef70ca1f0d0035808e","56d0a2081ecc471500f1795e","56d4a0adde40c70b00823ea3","56d96b03dd90610b00270849","56fbb83d8f21c817002af880","573c811bee2b3b2200422be1","576bc92afb62dd20001cda85","5771811e27a5c20e00030dcd","5785191af3a10c0e009b75b0","57bdf84d5d48411900cd8dc0","57ff5c5dc135231700aed806","5804caf792398f0f00e77521","58458b4fba4f1c0f009692bb","586d3c287c6b5b2300c05055","58ef66d88646742f009a0216","58f5d52d7891630f00fe4e77","59a555bccdbd85001bfb1442","5a2a81f688574d001e9934f5"],"is_deprecated":false,"is_hidden":false,"is_beta":true,"is_stable":true,"codename":"","version_clean":"1.0.0","version":"1.0"},"category":{"_id":"57bdf84d5d48411900cd8dc0","version":"55faf11ba62ba1170021a9aa","__v":0,"project":"55faf11ba62ba1170021a9a7","sync":{"url":"","isSync":false},"reference":false,"createdAt":"2016-08-24T19:41:01.302Z","from_sync":false,"order":26,"slug":"api-hub","title":"API Hub"},"user":"575e85ac41c8ba0e00259a44","__v":0,"parentDoc":null,"updates":[],"next":{"pages":[],"description":""},"createdAt":"2017-11-03T18:35:56.693Z","link_external":false,"link_url":"","githubsync":"","sync_unique":"","hidden":false,"api":{"results":{"codes":[]},"settings":"","auth":"required","params":[],"url":""},"isReference":false,"order":12,"body":"##Overview\n\nThis guide walks you through everything you need to get started with the Seven Bridges API Java client. \n\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n##Authentication and Configuration\n\nTo start, you need to authenticate with the API by passing your authentication token and the API endpoint you will be interacting with.\n\nYou can find your authentication token at https://igor.sbgenomics.com/developers.\n\nThe API endpoint is https://api.sbgenomics.com/v2.\n\nFor more information about the API, including details of the available parameters for each API call, you should check the API documentation before using this library at http://docs.sevenbridges.com/docs/the-api.\n\nOnce you have obtained your authentication token from one of the URLs listed above, you can use the `ClientBuilder` class to initialize the API client:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"ClientBuilder builder = Clients.builder();\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\nIf you do not pass any other parameters, the builder will automatically attempt to find your API key values in a number of default/conventional locations and then use the discovered values.\n\nLocations will be each be checked in the following order:\n\n1. The classpath of the java application which uses this client library. \n2. Environment variables `SB_API_ENDPOINT` and `SB_AUTH_TOKEN`.\n3. Common configuration file with the defined parameters located at:\n  * Linux, Mac OS X: `$HOME/.sevenbridges/credentials`\n  * MS Windows: `%UserProfile%\\.sevenbridges\\credentials`\n\nThis is a special credentials file that may be shared with other Seven Bridges client applications, so you should specify only these two keys in it (if any). If you have more than one profile, you can store the information for each profile in it.  For example:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"[default]\\nauth_token = 7eab0822exAMPle23d7d14c2eEXampLE\\napi_url = https://cgc-api.sbgenomics.com/v2\\n\\n[my_other_profile]\\nauth_token = 1e43fEXampLEa5523dfd14exAMPle3e5\\napi_url = https://cgc-api.sbgenomics.com/v2/\",\n      \"language\": \"json\"\n    }\n  ]\n}\n[/block]\n4. Library-specific configuration file `sevenbridges.properties` which contains the same keys as above, located at:\n  * Linux, Mac OS X: `$HOME/.sevenbridges/sevenbridges-java/`\n  * MS Windows: `%UserProfile%/.sevenbridges/sevenbridges-java/`\n\nIf the above-mentioned methods do not satisfy your needs, you can also specify your credentials explicitly when you build a client.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"static Client initClient() {\\n  ClientBuilder builder = Clients.builder();\\n  builder.setBaseUrl(\\\"https://cgc-api.sbgenomics.com/v2\\\");    \\n  builder.setAuthenticationScheme(AuthenticationScheme.AUTH_TOKEN);\\n  builder.setApiKey(ApiKeys.builder()\\n    .setSecret(\\\"39bexAMPle2742338EXampLEe3dde8a7\\\")\\n    .build());\\n  return builder.build();\\n}\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\n\n[block:callout]\n{\n  \"type\": \"warning\",\n  \"body\": \"Keep your authentication token safe! It encodes all your credentials on the Platform. Generally, we recommend storing the token in a configuration file, which will then be stored in your home folder rather than in the code itself. This prevents the authentication token from being committed to source code repositories.\"\n}\n[/block]\nAfter the credentials are loaded, you can use a helper method to initialize the client and obtain the `User` object which holds your user information, such as your name and contact details.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"Client client = initClient();\\nUser user = client.getCurrentUser();\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\nTo obtain your details simply use the appropriate `get()` method. For email address, it would be:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"user.getEmail();\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\nFor convenience, some code samples contain logging commands. To use logging, invoke a `Logger` instance and pass the current class as a parameter.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"private static final Logger log = LoggerFactory.getLogger(ThisClass.class);\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n##Manage projects\n\nProjects are the core building blocks of our Platform. All the analyses you design and run take place inside a project. Learn more about [Projects on the CGC](doc:projects-on-the-cgc). Here are some basic methods for dealing with them.\n\n###List all the projects that you are a member of\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"ProjectList projects = user.getProjects();\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\nYou can then iterate through the list of projects and get their information, including names and IDs. A project is referenced through its ID, so you will need to know it in order to work with your project further.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"Iterator<Project> projectIterator = projects.iterator();\\nlog.info(\\\"Project names: \\\");\\nwhile (projectIterator.hasNext()) {\\n  Project currentProject = projectIterator.next();\\n  log.info(\\\" name: {} - id: {}\\\", \\n  currentProject.getName(),  \\n  currentProject.getId());\\n}\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\nThe ID for a project consists of your username and the project's name, e.g `my-username/the-first-project`.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"wantedProject = user.getProjectById(String.format(\\\"%s/my-best-project\\\", myUsername));\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\nEach project also has a name, a description string indicating its use, a type, some tags, a `billing_group` identifier representing the billing group that is attached to the project and the `href`. The property `href` is a URL on the server that uniquely identifies the resource in question. All resources have this attribute. Each of the above attributes can be obtained using the relevant `get` method.\n\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n###Obtain your billing group ID\n\nTo create a new project, you need to provide its name and the billing group ID. \n\nThe billing group ID designates which funding resource to charge for the analyses you run in the project you're about to create. Learn more about [billing groups](doc:payments#section-billing-groups).\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"BillingGroupList billingGroups = user.getBillingGroups();\\nfirstBillingGroup = billingGroups.iterator().next().getId();\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\nNow you can create a new project. \n\n<div align=\"right\"<a href=\"#top\">top</a></div>\n\n###Create a project\n\nUse the billing group ID obtained above to create a that the project will be assigned an ID which consists of your username and the project’s [shortname](doc:the-api#section-project-short-names), which is created from the name you gave it through `setName()` (e.g. `rfranklin/new-test-project`).  The human readable name you set can be changed afterwards, but the project ID remains unchanged throughout the life of the project.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"Project newProject = client.instantiate(Project.class);\\n    newProject.setName(\\\"New test project\\\")\\n        .setDescription(\\\"This is a project created through V2 API\\\")\\n        .setBillingGroupId(user.getBillingGroups().iterator().next().getId());\\n    log.info(\\\"Created new project with name '{}' and project id '{}'\\\", newProject.getName(), newProject.getId());\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\n<div align=\"right\"<a href=\"#top\">top</a></div>\n\n##Manage project members\n\nWork with project members in your projects. If you are the project admin, you can specify the exact permissions of each project member.\n\n###Add project members and assign permissions\n\nTo add other users as members of your projects, you need to know their usernames.\n\nThe `read` permission is assigned by default to each project member and cannot be taken away. Other permissions are modifiable. Learn more about [setting project member permissions](doc:set-permissions).\n\nFirst we instantiate a new project member and then provide the username of the person we want to add and set the necessary permissions. After that we add the user to the desired project.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"\\nMember newMember = client.instantiate(Member.class);\\nnewMember\\n  // must be an existing user!\\n  .setUsername(\\\"annemarie.jones\\\")\\n  .setPermissions(Members.getDefaultPermissions());\\ncurrentProject.addMember(newMember);\\n\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n###List project members\n\nIterate through a list of project members to see who has which permissions.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"MemberList members = currentProject.getMembers();\\nIterator<Member> memberIterator = members.iterator();\\nSystem.out.println(\\\"Members of the project \\\" + currentProject.getId());\\nwhile (memberIterator.hasNext()) {\\n  Member currMember = memberIterator.next();\\n  log.info(\\\" Username : {} Permissions : {}\\\",\\n  currMember.getUsername(),  \\n  currMember.getPermissions());\\n}\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n###Modify project member permissions\n\nIn the following example, let's give our user the right to modify (the `write` permission) and download (the `copy` permission) files from our common project. \n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"Map<String, Boolean> permissions = newMember.getPermissions();\\npermissions.put(\\\"write\\\", true);\\npermissions.put(\\\"copy\\\", true);\\nnewMember.setPermissions(permissions);\\nnewMember.save();\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\nNow, if you want to see the updated list of members and permissions, remember to reload it.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"members.reload();\\nfor (Member member : members) {\\n  log.info(\\\" Username : {} Permissions : {}\\\", member.getUsername(),   \\n  member.getPermissions());\\n}\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n###Remove a project member\n\nUse the following to remove a member from your project.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"currentProject.removeMember(newMember.getUsername());\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\n<div align=\"right\"<a href=\"#top\">top</a></div>\n\n##Manage files on the Platform\n\nFiles are an integral part of each analysis. You can either use the files that are already publicly available on the Platform or upload your own.\n\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n###List files in a project\n\nYou can check what files are currently in your project by iterating through a list of files.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"\\nProject myProject = user.getProjectById(String.format(\\\"%s/source-project\\\", user.getUsername()));\\nlog.info(\\\"In project {} there are {} files\\\", myProject.getName(), myProject.getFiles().getSize());\\nFileList filesInMyProject = myProject.getFiles();\\nfor (File file : filesInMyProject) {\\n  log.info(\\\" File {} with name {} and size {}B\\\", file.getId(), file.getName(), file.getSize());\\n}\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n###Copy public files into your project\n\nIf you are starting from an empty project, one way to get going is to copy some of the public files into your project. You can utilize the file tags to find the files you need. The tags are keywords or strings that make it easier to identify and organize files you’ve imported from public datasets. \n\nLearn more about [public files](doc:file-repositories) or [tagging your files](doc:tag-your-files).\n\nLet’s say you want to copy all the files that are related to human genome version 19. They will be tagged with the “hg19” tag.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"\\nProject destinationProject = user.getProjectById(String.format(\\\"%s/destination-project\\\", user.getUsername()));\\nlog.info(\\\" There are {} files in destination project before copy\\\", destinationProject.getFiles().getSize());\\nlog.info(\\\" There are {} files in destination project before copy\\\", destinationProject.getFiles().getSize());\\nFileList publicFiles = user.getPublicFiles(Files.criteria().withTag(\\\"hg19\\\"));\\nfor (File file : publicFiles) {\\n  file.copy(destinationProject, file.getName() + \\\"_copy\\\");\\n}\\nlog.info(\\\" There are {} files in destination project after copy\\\", destinationProject.getFiles().getSize());\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n###Add tags to files\n\nTags can be applied to mark files in any way you find useful. Let’s say you decided you will not use files from a certain project anymore, but do not want to delete them until someone else has checked out your project. You can then use tags to mark the files as ready for deletion.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"FileList destinationFiles = destinationProject.getFiles();\\nIterator<File> iterator = destinationFiles.iterator();\\nwhile (iterator.hasNext()) {\\n  File next = iterator.next();\\n  next.setTags(Collections.singleton(\\\"for_deletion\\\"));\\n  next.save();\\n}\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n###Edit metadata for a file\n\nAll files have associated metadata which makes them searchable, keeping your file collection manageable as it grows. It also enables you group files properly for analyses.\n\nLearn more about [metadata](doc:metadata-on-the-seven-bridges-platform).\n\nWhen you need to change metadata on a file, you should first obtain the file's ID. Then you can either patch the file metadata (adding new and-or changing existing metadata fields) or you can overwrite it (which means any metadata fields you do not explicitly reset will be deleted).\n\nTo edit metadata, use the method `patchMetadata()`. \n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"File updatedFile = user.getFileById(\\\"584d6f2160b2a10069e40d5d\\\");\\nMap<String, String> metaPatch = new HashMap<>();\\nmetaPatch.put(\\\"paired_end\\\", \\\"1\\\");\\nmetaPatch.put(\\\"batch_number\\\", \\\"3\\\");\\nupdatedFile.patchMetadata(metaPatch);\\n// save the changes you made!\\nupdatedFile.save();\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n###Overwrite metadata\n\nTo overwrite metadata, use the method `setMetadata()`.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"Map<String, String> metaOver = new HashMap<>();\\nmetaOver.put(\\\"case_id\\\", \\\"CCLE-HCC1143BL\\\");\\nmetaOver.put(\\\"experimental_strategy\\\", \\\"WGS\\\");\\nmetaOver.put(\\\"investigation\\\", \\\"CCLE-BRCA\\\");\\nmetaOver.put(\\\"paired_end\\\", \\\"2\\\");\\nmetaOver.put(\\\"platform\\\", \\\"Illumina\\\");\\nmetaOver.put(\\\"species\\\", \\\"Homo sapiens\\\");\\nupdatedFile.setMetadata(metaOver);\\n// save the changes you made\\nupdatedFile.save();\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\n###Download a file\n\nEach file also has a URL property, which gives you the URL you can use to download the file. Again, you will need https://dash.readme.io/project/interim/v1.0/docs/java-library-quickstartto know the file ID to do this.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"File toDownload = user.getFileById(\\\"584d6f2160b2a10069e40d5d\\\");\\nString downloadUrl = toDownload.getDownloadInfo().getUrl();\\nInputStream downloadStream = null;\\ntry {\\n  URL url = new URL(downloadUrl);\\n  downloadStream = url.openStream();\\n  log.info(\\\"Downloading file...\\\");\\n  long bytesNum = java.nio.file.Files.copy(downloadStream,   Paths.get(\\\"local_file\\\"), StandardCopyOption.REPLACE_EXISTING);\\n  log.info(\\\"Downloaded {} bytes\\\", bytesNum);\\n} catch (MalformedURLException e) {\\n  log.error(\\\"Malformed exception while creating URL from {} - error message: {}\\\", String.valueOf(downloadUrl), e.getMessage());\\n} catch (IOException e) {\\n  log.error(\\\"Error while downloading file {} \\\", e);\\n} finally {\\n  if (downloadStream != null) {\\n    try {\\n      downloadStream.close();\\n    } catch (IOException skip) {\\n      log.warn(\\\"Error while trying to close download stream - {}\\\", skip);\\n    }\\n  }\\n}\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n##Uploading files\n\nIf you want to use your private data for analysis, you can upload them securely to the Platform.\nIn this section, we will see how to upload files into a project and some actions you can perform on an `upload` object: using an `UploadListener` to listen for related events, polling the number of uploaded bytes while waiting for an upload to complete, blocking further work until upload is completed, cancelling an upload and pausing and restarting an upload.\n \nThe class `AbstractProgressListener` contains callback methods that inform you of the state of your upload. You can implement the methods for the events you want to listen to, like in this example:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"private static class MyProgressListener extends AbstractProgressListener {\\n\\n    private final AtomicInteger cnt;\\n\\n    MyProgressListener(AtomicInteger cnt) {\\n      this.cnt = cnt;\\n    }\\n\\n    :::at:::Override\\n    public void uploadFailed(Exception ex) {\\n      cnt.incrementAndGet();\\n      log.error(\\\"upload failed \\\", ex);\\n    }\\n\\n    @Override\\n    public void uploadFinished() {\\n      cnt.incrementAndGet();\\n      log.info(\\\"upload finished \\\");\\n    }\\n\\n    @Override\\n    public void partUploadFailed(int partNumber, int retryCnt, Exception executionException) {\\n      log.error(\\\"part {} failed, retry {}, exception {}\\\", partNumber, retryCnt, executionException);\\n    }\\n}\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\nThen you can build a synchronized upload request using `uploadBuilder`. \n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"CreateUploadRequestBuilder uploadBuilder = client.getUploadRequestBuilder();\\n\\n    // block on an upload (synchronized upload)\\n    uploadBuilder.setName(\\\"Sync-file\\\")\\n        .setFile(myPrivateFile)\\n        .setOverwrite(true)\\n        .setProject(destinationProject);\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\nBased on this, you can create an `UploadContext` object which will hold information about your upload.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"List<com.sbgenomics.java.file.File> uploadedFiles = new ArrayList<>();\\n\\nUploadContext upload = user.submitUpload(uploadBuilder.build());\\n    try {\\n      com.sbgenomics.java.file.File uploadedFile = upload.getFile(); //blocking call\\n      uploadedFiles.add(uploadedFile);\\n    } catch (RuntimeException e) {\\n      log.error(\\\"Error while waiting for the file to be uploaded - {}\\\", e);\\n    }\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n###Upload a list of files\n\nIf you have a list of files you want to have uploaded (let’s call it `filesToUpload`), you can request upload of those files simultaneously, creating an `UploadContext` and `ProgressListener` for each file.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"static final long KB = 1024L;\\n  static final long MB = 1024 * KB;\\n    \\n// parallel upload without blocking on get, setting up a listener for finished files\\n    AtomicInteger finishedCnt = new AtomicInteger(0);\\n    int numOfFiles = filesToUpload.size();\\n    List<UploadContext> uploads = new ArrayList<>(numOfFiles);\\n    for (int i = 0; i < numOfFiles; i++) {\\n      File testFile = filesToUpload(i);\\n\\n      uploadBuilder.setName(\\\"File-to-upload-no-\\\" + i)\\n          .setFile(testFile)\\n          .setOverwrite(true)\\n          .setProject(destinationProject)\\n// if you want to cut upload in parts\\n          .setPartSize(64 * MB);\\n\\n      UploadContext uploadContext = user.submitUpload(uploadBuilder.build(), new MyProgressListener(finishedCnt));\\n      uploads.add(uploadContext);\\n    }\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\nWhen you submit an upload, the `TransferService` is started if it wasn't previously started by an earlier upload. The `UploadContext` objects allow you to keep track of the progress of your uploads. \n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"    while (finishedCnt.get() < numOfFiles) {\\n      Thread.sleep(1000);\\n      log.info(\\\"---------------------------------------------\\\");\\n      for (UploadContext uploadContext : uploads) {\\n        long transferred = uploadContext.getBytesTransferred();\\n        long size = uploadContext.getUploadSize();\\n        log.info(\\\"Transferred {}% -  {} bytes out of {} for upload {}\\\", (int)((transferred * 100)/size), transferred, size, uploadContext.getUploadName());\\n      }\\n    }\\n\\n    for (UploadContext uploadContext : uploads) {\\n      if (uploadContext.isFinished()) {\\n        uploadedFiles.add(uploadContext.getFile());\\n      }\\n    }\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n###Pause an upload\n\nIf you want to pause an upload, you can do it through `UploadContext.pauseTransfer()`. This will pause the current upload operation and store the information that can be used to resume the upload. The paused `UploadContext` object can later be passed to another `UploadContext` object which will resume upload from the reference point.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"    uploadBuilder\\n        .setName(\\\"File-to-pause-and-resume\\\")\\n        .setFile(testFile)\\n        .setOverwrite(true)\\n        .setProject(destinationProject)\\n        .setPartSize(32 * MB);\\n    UploadContext toPause = user.submitUpload(uploadBuilder.build());\\n\\n    log.info(\\\"toPause state {}\\\", toPause.getState());\\n    Thread.sleep(10_000); // let it work a while\\n    toPause.pauseTransfer();\\n    log.info(\\\"toPause state {}\\\", toPause.getState());\\n    // waiting for the upload to be paused\\n    while (UploadState.PAUSING.equals(toPause.getState())) {\\n      Thread.sleep(5_000);\\n    }\\n    log.info(\\\"toPause state {}\\\", toPause.getState());\\n    UploadContext resumedUpload = user.resumeUpload(toPause, testFile);\\n    log.info(\\\"resumedUpload state {}\\\", resumedUpload.getState());\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\nTrying to obtain the file from a paused upload will throw a `PausedUploadException`. You can use this to resume the paused upload when needed. \n\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n###Abort an upload\n\nOn some occasions your app (or an external factor) might need to abort an upload. If you have instantiated an `UploadContext` in the above described manner, you can obtain the ID of your upload and abort it. Please keep in mind that trying to obtain the file from the `UploadContext` after the abort will throw an exception.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"   String uploadId = uploadContextToBeAborted.getUploadId();\\n    Upload uploadById = user.getUploadById(uploadId);\\n    log.info(\\\"Thread {} is aborting running upload {}\\\", Thread.currentThread().getName(), uploadId);\\n    uploadById.abortUpload();\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n##Finish an upload\n\nOnce you have uploaded all the files you needed, it’s time to close the `TransferService` to make sure you gracefully shutdown daemon threads and release resources.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"   user.shutdownTransferService();\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n##Managing apps\n\nApps on the Platform include both the tools (individual bioinformatics utilities) and the workflows (chains or pipelines of connected tools), either previously existing or user-created. In the section, learn how to get information about publicly available apps, to check which apps are currently in a given project and how to install a new app based on a pre-formatted JSON file.\n\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n###Get information about public apps\n\nUse the following to get the `name` and the `id` of an app. You will need to know the `id` of an app in order to work further with it. \n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"AppList publicApps = user.getPublicApps();\\nlog.info(\\\"There are {} public apps\\\", publicApps.getSize());\\nfor (App publicApp : publicApps) {\\n  log.info(\\\"App id {} name {}\\\", publicApp.getId(), publicApp.getName());\\n}\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n###Copy an app to your project\n\nIf you want to use a certain app inside your project, you can copy it into your project. If you try to copy an app that already exists in the given project, the API will issue an error message, which you can use to take appropriate action and inform the user as necessary.\n\nHere is [the list of API status codes and descriptions](http://docs.sevenbridges.com/reference#api-status-codes).\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"Project destProject = user.getProjectById(String.format(\\\"%s/destination-project\\\", user.getUsername()));\\nApp appToCopy = user.getAppById(\\\"admin/sbg-public-data/fastqc-analysis/2\\\");\\n\\n    // copy action will fail if there is already app with same id in the project\\n    App copiedApp;\\n    try {\\n      copiedApp = appToCopy.copy(destProject);\\n    } catch (ResourceException e) {\\n      log.debug(\\\"Error while trying to copy app - \\\" + e.toString());\\n      if (e.getStatus() == 409) { // CONFLICT, app with same ID already exists in project\\n        copiedApp = user.getAppById(String.format(\\\"%s/fastqc-analysis\\\", destProject.getId()));\\n        log.info(\\\"App already exists in destination project, with id {}\\\", copiedApp.getId());\\n      } else {\\n        log.error(\\\"Error while getting app by id, expected success code or 409 HTTP status, got {}\\\", e.getMessage());\\n        throw e;\\n      }\\n    }\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n###List the apps in your project\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"AppList destProjectApps = destProject.getApps();\\n\\nFinally, if you have a JSON file which describes your app through CWL, you can use it to install the app in a project.\\n\\ntry {\\n  String raw = new String(Files.readAllBytes(Paths.get(\\\"my-app-raw.json\\\")));\\n      destProject.installApp(\\\"my-installed-app\\\", raw);\\n} catch (IOException e) {\\n  log.error(\\\"Error while reading file 'my-app-raw.json', {}\\\", e);\\n}\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n###Add an app with its JSON\n\nLearn more about the [Common Workflow Language(CWL)](http://docs.sevenbridges.com/docs/sdk-overview#section-the-common-workflow-language).\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"try {\\n  String raw = new String(Files.readAllBytes(Paths.get(\\\"my-app-raw.json\\\")));\\n      destProject.installApp(\\\"my-installed-app\\\", raw);\\n} catch (IOException e) {\\n  log.error(\\\"Error while reading file 'my-app-raw.json', {}\\\", e);\\n}\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n##Managing tasks\n\nAn app execution is called a task. Each task is associated with a set of input files and chosen settings for the tool(s) in the app. In this part, we will see how to copy files that satisfy certain criteria, copy a relevant app into the project and build a task. We will see how to poll for task status during execution and how to get other useful information about a task.\n\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n###Create a task\n\nIn order to create a new task, you can create a `TaskBuilder` and set its fields appropriately. Let’s say you want to index a FASTA file with our public app, SAMtools Index FASTA. \n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"// find a fasta file from public repo and copy it to your project\\nFile fastaFile = user.getProjectById(\\\"admin/sbg-public-data\\\")          .getFiles(Files.criteria().withName(\\\"HG19_Broad_variant.fasta\\\"))\\n            .single();\\n\\nProject tasksProject  = user.getProjectById(String.format(\\\"%s/task-test\\\", user.getUsername()));\\n\\nFile input = fastaFile.copy(tasksProject);\\n\\n// copy the app to the project\\nApp sourceApp = \\nuser.getAppById(\\\"admin/sbg-public-data/samtools-index-fasta-1-3\\\");\\n    \\nApp myApp = sourceApp.copy(tasksProject);\\n\\nTaskRequestFactory taskFactory = user.getTaskRequestFactory();\\nCreateTaskRequestBuilder taskBuilder = taskFactory.createTaskBuilder();\\ntaskBuilder.setApp(myApp)\\n  .setDescription(\\\"run from public API v2\\\")\\n  .setName(\\\"API_task_samtools\\\")\\n  .setProject(tasksProject)\\n  .addInput(\\\"input_fasta_file\\\", input)\\n// to run immediately after task creation\\n  .runNow(true);\\n\\nTask task = user.createTask(taskBuilder.build());\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\nWhen the task is executed successfully, its status will change to `COMPLETED`. \n\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n###Check the status of your task\n\nKeep in mind that each check of the status fires up an API request, so this shouldn’t be done in quick succession. \n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"while (!TaskStatus.isFinished(task.getStatus())) {\\n  try {\\n    Thread.sleep(30_000);\\n  } catch (InterruptedException e) {\\n    log.error(\\\"Interrupted from sleep, but task job {} is not finished yet\\\", task.getId());\\n    throw new RuntimeException(e);\\n  }\\n  task.reload();\\n}\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n###Check if a task is completed\n\nIf you want to check which tasks have completed, you can use their status as a search criterion. \nYou can also get other information about each task, e.g. its name, inputs and outputs.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"TaskList tasks = user.getTasks(Tasks.criteria().withStatus(TaskStatus.COMPLETED)); log.info(\\\"List of completed tasks for current user\\\");\\nfor (Task completedTasks : tasks) {\\n   log.info(\\\"  Task name: {}\\\", task.getName());\\n   log.info(\\\"    Inputs: {}\\\", task.getInputs());\\n   log.info(\\\"    Outputs: {}\\\", task.getOutputs());\\n}\\n\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n##Batch tasks\n\nRun identical analyses on different data via [batching](http://docs.sevenbridges.com/docs/perform-batch-analysis), a procedure in which you enter multiple input files and group them by specified metadata criteria.\n\nThis segment on batch tasks follows [this API tutorial](http://docs.sevenbridges.com/docs/api-batch-tutorial), which contains more detailed information. Here we will focus on implementing a batch task using our Java library. We will see how to set up a batch analysis in which we align reads based on their sample metadata. We will see how to copy input and reference files as well as the relevant apps into the project and then create a task and check for errors during creation of the task. We will also show how to check the status of the task while it is running and how to collect the output files once the task is completed.\n\nPreviously, we have discussed creating a project so we will assume that you have a project with a short name “batch-test” in which you want to perform the batch analysis. \n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"Project batchProject = user.getProjectById(String.format(\\\"%s/batch-test\\\", user.getUsername()));\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\nWe'll analyze data that is hosted in the Cancer Cell Line Encyclopedia (CCLE) public project on the CGC. The CCLE public project is specified by the ID of sevenbridges/cancer-cell-line-encyclopedia-ccle-1.  Inside it, we want to find BAM files with an experimental strategy of RNA-Seq. It is possible to filter by metadata fields to retrieve files with certain properties. In this tutorial, however, we already know that we want to find the following three files:\n  * G30630.VM-CUB1.3.bam\n  * G30603.TUHR4TKB.1.bam\n  * G28034.MDA-MB-361.1.bam\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"FileList inputCcleFiles = user.getProjectById(\\\"sevenbridges/cancer-cell-line-encyclopedia-ccle-1\\\")         \\n  .getFiles(Files.criteria()\\n     .withName(\\\"G30630.VM-CUB1.3.bam\\\")\\n     .withName(\\\"G30603.TUHR4TKB.1.bam\\\")\\n     .withName(\\\"G28034.MDA-MB-361.1.bam\\\"));\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\nThe next step is to copy the files into our project.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"List<File> inputFiles = new ArrayList<>(3);\\nfor (File file : inputCcleFiles) {\\n  File copy = file.copy(batchProject);\\n  inputFiles.add(copy);\\n}\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\nMany bioinformatics tools require certain data, such as reference genomes or annotation files, to execute properly.  Seven Bridges maintains a collection of the latest and most frequently used reference genomes and annotation files in the Public Reference Files repository. \n\nFor this analysis, we need to supply the workflow with the following two reference files:\n  * HG19_Broad_variant.fasta\n  * Homo_sapiens.GRCh37.75.gtf\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"FileList inputPublicFiles = user.getPublicFiles(Files.criteria()\\n  .withName(\\\"HG19_Broad_variant.fasta\\\")\\n  .withName(\\\"Homo_sapiens.GRCh37.75.gtf\\\"));\\nfor (File file : inputPublicFiles) {\\n   file.copy(batchProject);\\n}\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\nNext, we need to copy the RNA-seq Alignment STAR app from the list of public apps. Check here how to obtain the ID of the app.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \" App appToCopy = user.getAppById(\\\"admin/sbg-public-data/rna-seq-alignment-star/16\\n\\\");\\n        App appRnaSeqAlign = appToCopy.copy(batchProject);\\n \",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\nAt this point, we need to edit the app so that it can accept the BAM files we copied as input. You can do it by editing its CWL or via the visual interface. Here are the instructions for using Workflow editor on the visual interface. \n\nAfter the app is edited, we can build the task and set appropriate fields.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"TaskRequestFactory taskFactory = user.getTaskRequestFactory();\\n    CreateTaskRequestBuilder taskBuilder = taskFactory.createTaskBuilder();\\n    taskBuilder\\n        .setApp(appRnaSeqAlign)\\n        .setDescription(\\\"run from public API v2 with batch\\\")\\n        .setName(\\\"Batch run RNA seq - api client java\\\")\\n        .setProject(batchProject)\\n// the input port on which you wish to batch\\n        .setBatchInput(\\\"input_file\\\")\\n        .addBatchBy(\\\"CRITERIA\\\", java.util.Collections.singletonList(\\\"metadata.sample_id\\\"))\\n        .addInput(\\\"input_file\\\", inputFiles.toArray(new File[0]))\\n        .addInput(\\\"reference_or_index\\\", batchProject.getFiles(Files.criteria().withName(\\\"HG19_Broad_variant.fasta\\\")).single())\\n        .addInputAsSingletonList(\\\"sjdbGTFfile\\\", batchProject.getFiles(Files.criteria().withName(\\\"Homo_sapiens.GRCh37.75.gtf\\\")).single())\\n    .runNow(true);\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\nSince we set the `runNow` parameter to true, the task will run as soon as we create it. If you leave out this field, the task will be created and stay in DRAFT status until you call `run()` on it. \n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"//creates and runs the task, since runNow is true\\nTask task = user.createTask(taskBuilder.build());\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\nIf task runs into errors during creation, they will be attached to the task object.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \" List<Map<String, Object>> errors = task.getErrors();\\n    if (errors != null && errors.size() > 0) {\\n      log.error(\\\"Error while validating task with id {}\\\", task.getId());\\n      for (Map<String, Object> error : errors) {\\n        log.error(\\\"    {}\\\", error);\\n      }\\n    }\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\nWhile the task is being executed, you can occasionally check its status. You will also be notified by email once your task is completed.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"while (!TaskStatus.isFinished(task.getStatus())) {\\n        log.info(\\\"Sleeping\\\");\\n        try {\\n          Thread.sleep(30_000);\\n        } catch (InterruptedException e) {\\n          Thread.interrupted();\\n          log.error(\\\"Interrupted while waiting for task to finish\\\");\\n          throw new RuntimeException(e);\\n        }\\n        task.reload();\\n        log.info(\\\"Current status {}\\\", task.getStatus().toString());\\n      }\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\nWhen the task is completed, you can obtain the list of files that make up its output. Or you can get information about the task status.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \" if (TaskStatus.COMPLETED.equals(task.getStatus())) {\\n      FileList producedFiles = batchProject.getFiles(Files.criteria().withTaskOrigin(task));\\n      for (File producedFile : producedFiles) {\\n        log.info(\\\"Produced file name {} and id {}\\\", producedFile.getName(), producedFile.getId());\\n      }\\n    } else {\\n      log.warn(\\\"Task is finished, but with status {}\\\", task.getStatus());\\n    }\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\n<div align=\"right\"><a href=\"#top\">top</a></div>\n\n##Managing volumes\n\nVolumes authorize the Platform to access and query objects on a specified cloud storage ([Amazon Web Services](http://docs.sevenbridges.com/docs/aws-cloud-storage-tutorial) or [Google Cloud Storage](http://docs.sevenbridges.com/docs/google-cloud-storage-tutorial)) on your behalf. We will examine setting up a volume on each of the services, listing your volumes, importing and exporting files and polling for an import or export status while waiting for a job to finish.\n\nThe S3 and GCS volumes each have their own builder. Before you can use them, you will need to set up the `AWS_ACCESS_KEY` and the `AWS_SECRET_KEY` for S3 or the `GCS_PRIVATE_KEY` for the GCS. Then an appropriate builder can be used to create a volume. \n\nHere is the S3 example:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"CreateS3VolumeRequestBuilder s3Builder = user.getVolumeRequestFactory().s3Builder();\\n    s3Builder\\n        .setName(\\\"my_new_S3_volume_for_test\\\")\\n        .setAccessMode(AccessMode.RW)\\n        .setAccessKey(AWS_ACCESS_KEY)\\n        .setSecretKey(AWS_SECRET_KEY)\\n        .setBucket(\\\"outer_user_test_bucket\\\")\\n        .setDescription(\\\"S3 Volume for testing imports and exports from and to external (non sbg) S3 bucket\\\")\\n        .setSseAlgorithm(\\\"AES256\\\");\\n    Volume testVolume = user.createVolume(s3Builder.build());\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\nThis is the corresponding GCS example:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"    CreateGcsVolumeRequestBuilder gcsBuilder = user.getVolumeRequestFactory().gcsBuilder();\\n      gcsBuilder\\n        .setName(\\\"my_new_GCS_volume_for_test\\\")\\n        .setAccessMode(AccessMode.RO) //only read mode for GCS\\n        .setBucket(\\\"outer_user_GCS_import\\\")\\n        .setClientEmail(CLIENT_GOOGLE_EMAIL)\\n        .setPrivateKey(\\\"-----BEGIN PRIVATE KEY-----\\\"+ GCS_PRIVATE_KEY + \\\"-----END PRIVATE KEY-----\\\\\\\\n\\\");\\n    Volume gcsVolume = user.createVolume(gcsBuilder.build());\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\nEach file is imported using an import job. Once you have created it, you can refer to it by its ID.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"    ImportJob gcsImportJob = user.startImport(gcsVolume, \\\"files/google_file_key\\\", destProject, \\\"gcs_file\\\", false);\\n    log.info(\\\"GCS volume import id : {}\\\", gcsImportJob.getId());\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\nIf you have a list of import jobs (called imports in this example), you can poll for their status occasionally, until they are all finished.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"    int finishedCnt = 0;\\n    while (finishedCnt < imports.size()) {\\n      try {\\n        Thread.sleep(5000);\\n        Iterator<ImportJob> iterator = imports.iterator();\\n        while (iterator.hasNext()) {\\n          ImportJob next = iterator.next();\\n          next.reload();\\n          if (VolumeJobState.isFinished(next.getState())) {\\n            log.info(\\\"Volume import id : {} name {} done\\\", next.getId(), next.getDestinationName());\\n            finishedCnt++;\\n            iterator.remove();\\n          }\\n        }\\n      } catch (Exception e) {\\n        log.error(\\\" Error while waiting for imports to finish - {}\\\", e.getMessage());\\n        break;\\n      }\\n    }\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\nTo export a file, refer to it by its ID and create an export job.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"File fileToExport = user.getFileById(\\\"584d6f160b2a10069e40d5d\\\");\\n    ExportJob exportJob = user.startExport(fileToExport, testVolume, \\\"exported-file\\\", false);\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\nSimilarly to imports, you can check on the status of your export job occasionally.\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"   try {\\n      while (!VolumeJobState.isFinished(exportJob.getState())) {\\n        Thread.sleep(5000);\\n        exportJob.reload();\\n      }\\n    } catch (Exception e) {\\n      log.error(\\\" Error while waiting for export to finish = {}\\\", e.getMessage());\\n    }\\n\\n    if (VolumeJobState.COMPLETED.equals(exportJob.getState())) {\\n      log.info(\\\"Volume export with id {} is completed successfully\\\", exportJob.getId());\\n    } else {\\n      log.warn(\\\"Volume export with id {} failed\\\", exportJob.getId());\\n    }\\n\",\n      \"language\": \"java\"\n    }\n  ]\n}\n[/block]\n<div align=\"right\"><a href=\"#top\">top</a></div>","excerpt":"","slug":"java-library-quickstart","type":"basic","title":"Java library Quickstart"}

Java library Quickstart


##Overview This guide walks you through everything you need to get started with the Seven Bridges API Java client. <div align="right"><a href="#top">top</a></div> ##Authentication and Configuration To start, you need to authenticate with the API by passing your authentication token and the API endpoint you will be interacting with. You can find your authentication token at https://igor.sbgenomics.com/developers. The API endpoint is https://api.sbgenomics.com/v2. For more information about the API, including details of the available parameters for each API call, you should check the API documentation before using this library at http://docs.sevenbridges.com/docs/the-api. Once you have obtained your authentication token from one of the URLs listed above, you can use the `ClientBuilder` class to initialize the API client: [block:code] { "codes": [ { "code": "ClientBuilder builder = Clients.builder();", "language": "java" } ] } [/block] If you do not pass any other parameters, the builder will automatically attempt to find your API key values in a number of default/conventional locations and then use the discovered values. Locations will be each be checked in the following order: 1. The classpath of the java application which uses this client library. 2. Environment variables `SB_API_ENDPOINT` and `SB_AUTH_TOKEN`. 3. Common configuration file with the defined parameters located at: * Linux, Mac OS X: `$HOME/.sevenbridges/credentials` * MS Windows: `%UserProfile%\.sevenbridges\credentials` This is a special credentials file that may be shared with other Seven Bridges client applications, so you should specify only these two keys in it (if any). If you have more than one profile, you can store the information for each profile in it. For example: [block:code] { "codes": [ { "code": "[default]\nauth_token = 7eab0822exAMPle23d7d14c2eEXampLE\napi_url = https://cgc-api.sbgenomics.com/v2\n\n[my_other_profile]\nauth_token = 1e43fEXampLEa5523dfd14exAMPle3e5\napi_url = https://cgc-api.sbgenomics.com/v2/", "language": "json" } ] } [/block] 4. Library-specific configuration file `sevenbridges.properties` which contains the same keys as above, located at: * Linux, Mac OS X: `$HOME/.sevenbridges/sevenbridges-java/` * MS Windows: `%UserProfile%/.sevenbridges/sevenbridges-java/` If the above-mentioned methods do not satisfy your needs, you can also specify your credentials explicitly when you build a client. [block:code] { "codes": [ { "code": "static Client initClient() {\n ClientBuilder builder = Clients.builder();\n builder.setBaseUrl(\"https://cgc-api.sbgenomics.com/v2\"); \n builder.setAuthenticationScheme(AuthenticationScheme.AUTH_TOKEN);\n builder.setApiKey(ApiKeys.builder()\n .setSecret(\"39bexAMPle2742338EXampLEe3dde8a7\")\n .build());\n return builder.build();\n}", "language": "java" } ] } [/block] [block:callout] { "type": "warning", "body": "Keep your authentication token safe! It encodes all your credentials on the Platform. Generally, we recommend storing the token in a configuration file, which will then be stored in your home folder rather than in the code itself. This prevents the authentication token from being committed to source code repositories." } [/block] After the credentials are loaded, you can use a helper method to initialize the client and obtain the `User` object which holds your user information, such as your name and contact details. [block:code] { "codes": [ { "code": "Client client = initClient();\nUser user = client.getCurrentUser();", "language": "java" } ] } [/block] To obtain your details simply use the appropriate `get()` method. For email address, it would be: [block:code] { "codes": [ { "code": "user.getEmail();", "language": "java" } ] } [/block] For convenience, some code samples contain logging commands. To use logging, invoke a `Logger` instance and pass the current class as a parameter. [block:code] { "codes": [ { "code": "private static final Logger log = LoggerFactory.getLogger(ThisClass.class);", "language": "java" } ] } [/block] <div align="right"><a href="#top">top</a></div> ##Manage projects Projects are the core building blocks of our Platform. All the analyses you design and run take place inside a project. Learn more about [Projects on the CGC](doc:projects-on-the-cgc). Here are some basic methods for dealing with them. ###List all the projects that you are a member of [block:code] { "codes": [ { "code": "ProjectList projects = user.getProjects();", "language": "java" } ] } [/block] You can then iterate through the list of projects and get their information, including names and IDs. A project is referenced through its ID, so you will need to know it in order to work with your project further. [block:code] { "codes": [ { "code": "Iterator<Project> projectIterator = projects.iterator();\nlog.info(\"Project names: \");\nwhile (projectIterator.hasNext()) {\n Project currentProject = projectIterator.next();\n log.info(\" name: {} - id: {}\", \n currentProject.getName(), \n currentProject.getId());\n}", "language": "java" } ] } [/block] The ID for a project consists of your username and the project's name, e.g `my-username/the-first-project`. [block:code] { "codes": [ { "code": "wantedProject = user.getProjectById(String.format(\"%s/my-best-project\", myUsername));", "language": "java" } ] } [/block] Each project also has a name, a description string indicating its use, a type, some tags, a `billing_group` identifier representing the billing group that is attached to the project and the `href`. The property `href` is a URL on the server that uniquely identifies the resource in question. All resources have this attribute. Each of the above attributes can be obtained using the relevant `get` method. <div align="right"><a href="#top">top</a></div> ###Obtain your billing group ID To create a new project, you need to provide its name and the billing group ID. The billing group ID designates which funding resource to charge for the analyses you run in the project you're about to create. Learn more about [billing groups](doc:payments#section-billing-groups). [block:code] { "codes": [ { "code": "BillingGroupList billingGroups = user.getBillingGroups();\nfirstBillingGroup = billingGroups.iterator().next().getId();", "language": "java" } ] } [/block] Now you can create a new project. <div align="right"<a href="#top">top</a></div> ###Create a project Use the billing group ID obtained above to create a that the project will be assigned an ID which consists of your username and the project’s [shortname](doc:the-api#section-project-short-names), which is created from the name you gave it through `setName()` (e.g. `rfranklin/new-test-project`). The human readable name you set can be changed afterwards, but the project ID remains unchanged throughout the life of the project. [block:code] { "codes": [ { "code": "Project newProject = client.instantiate(Project.class);\n newProject.setName(\"New test project\")\n .setDescription(\"This is a project created through V2 API\")\n .setBillingGroupId(user.getBillingGroups().iterator().next().getId());\n log.info(\"Created new project with name '{}' and project id '{}'\", newProject.getName(), newProject.getId());", "language": "java" } ] } [/block] <div align="right"<a href="#top">top</a></div> ##Manage project members Work with project members in your projects. If you are the project admin, you can specify the exact permissions of each project member. ###Add project members and assign permissions To add other users as members of your projects, you need to know their usernames. The `read` permission is assigned by default to each project member and cannot be taken away. Other permissions are modifiable. Learn more about [setting project member permissions](doc:set-permissions). First we instantiate a new project member and then provide the username of the person we want to add and set the necessary permissions. After that we add the user to the desired project. [block:code] { "codes": [ { "code": "\nMember newMember = client.instantiate(Member.class);\nnewMember\n // must be an existing user!\n .setUsername(\"annemarie.jones\")\n .setPermissions(Members.getDefaultPermissions());\ncurrentProject.addMember(newMember);\n", "language": "java" } ] } [/block] <div align="right"><a href="#top">top</a></div> ###List project members Iterate through a list of project members to see who has which permissions. [block:code] { "codes": [ { "code": "MemberList members = currentProject.getMembers();\nIterator<Member> memberIterator = members.iterator();\nSystem.out.println(\"Members of the project \" + currentProject.getId());\nwhile (memberIterator.hasNext()) {\n Member currMember = memberIterator.next();\n log.info(\" Username : {} Permissions : {}\",\n currMember.getUsername(), \n currMember.getPermissions());\n}", "language": "java" } ] } [/block] <div align="right"><a href="#top">top</a></div> ###Modify project member permissions In the following example, let's give our user the right to modify (the `write` permission) and download (the `copy` permission) files from our common project. [block:code] { "codes": [ { "code": "Map<String, Boolean> permissions = newMember.getPermissions();\npermissions.put(\"write\", true);\npermissions.put(\"copy\", true);\nnewMember.setPermissions(permissions);\nnewMember.save();", "language": "java" } ] } [/block] Now, if you want to see the updated list of members and permissions, remember to reload it. [block:code] { "codes": [ { "code": "members.reload();\nfor (Member member : members) {\n log.info(\" Username : {} Permissions : {}\", member.getUsername(), \n member.getPermissions());\n}", "language": "java" } ] } [/block] <div align="right"><a href="#top">top</a></div> ###Remove a project member Use the following to remove a member from your project. [block:code] { "codes": [ { "code": "currentProject.removeMember(newMember.getUsername());", "language": "java" } ] } [/block] <div align="right"<a href="#top">top</a></div> ##Manage files on the Platform Files are an integral part of each analysis. You can either use the files that are already publicly available on the Platform or upload your own. <div align="right"><a href="#top">top</a></div> ###List files in a project You can check what files are currently in your project by iterating through a list of files. [block:code] { "codes": [ { "code": "\nProject myProject = user.getProjectById(String.format(\"%s/source-project\", user.getUsername()));\nlog.info(\"In project {} there are {} files\", myProject.getName(), myProject.getFiles().getSize());\nFileList filesInMyProject = myProject.getFiles();\nfor (File file : filesInMyProject) {\n log.info(\" File {} with name {} and size {}B\", file.getId(), file.getName(), file.getSize());\n}", "language": "java" } ] } [/block] <div align="right"><a href="#top">top</a></div> ###Copy public files into your project If you are starting from an empty project, one way to get going is to copy some of the public files into your project. You can utilize the file tags to find the files you need. The tags are keywords or strings that make it easier to identify and organize files you’ve imported from public datasets. Learn more about [public files](doc:file-repositories) or [tagging your files](doc:tag-your-files). Let’s say you want to copy all the files that are related to human genome version 19. They will be tagged with the “hg19” tag. [block:code] { "codes": [ { "code": "\nProject destinationProject = user.getProjectById(String.format(\"%s/destination-project\", user.getUsername()));\nlog.info(\" There are {} files in destination project before copy\", destinationProject.getFiles().getSize());\nlog.info(\" There are {} files in destination project before copy\", destinationProject.getFiles().getSize());\nFileList publicFiles = user.getPublicFiles(Files.criteria().withTag(\"hg19\"));\nfor (File file : publicFiles) {\n file.copy(destinationProject, file.getName() + \"_copy\");\n}\nlog.info(\" There are {} files in destination project after copy\", destinationProject.getFiles().getSize());", "language": "java" } ] } [/block] <div align="right"><a href="#top">top</a></div> ###Add tags to files Tags can be applied to mark files in any way you find useful. Let’s say you decided you will not use files from a certain project anymore, but do not want to delete them until someone else has checked out your project. You can then use tags to mark the files as ready for deletion. [block:code] { "codes": [ { "code": "FileList destinationFiles = destinationProject.getFiles();\nIterator<File> iterator = destinationFiles.iterator();\nwhile (iterator.hasNext()) {\n File next = iterator.next();\n next.setTags(Collections.singleton(\"for_deletion\"));\n next.save();\n}", "language": "java" } ] } [/block] <div align="right"><a href="#top">top</a></div> ###Edit metadata for a file All files have associated metadata which makes them searchable, keeping your file collection manageable as it grows. It also enables you group files properly for analyses. Learn more about [metadata](doc:metadata-on-the-seven-bridges-platform). When you need to change metadata on a file, you should first obtain the file's ID. Then you can either patch the file metadata (adding new and-or changing existing metadata fields) or you can overwrite it (which means any metadata fields you do not explicitly reset will be deleted). To edit metadata, use the method `patchMetadata()`. [block:code] { "codes": [ { "code": "File updatedFile = user.getFileById(\"584d6f2160b2a10069e40d5d\");\nMap<String, String> metaPatch = new HashMap<>();\nmetaPatch.put(\"paired_end\", \"1\");\nmetaPatch.put(\"batch_number\", \"3\");\nupdatedFile.patchMetadata(metaPatch);\n// save the changes you made!\nupdatedFile.save();", "language": "java" } ] } [/block] <div align="right"><a href="#top">top</a></div> ###Overwrite metadata To overwrite metadata, use the method `setMetadata()`. [block:code] { "codes": [ { "code": "Map<String, String> metaOver = new HashMap<>();\nmetaOver.put(\"case_id\", \"CCLE-HCC1143BL\");\nmetaOver.put(\"experimental_strategy\", \"WGS\");\nmetaOver.put(\"investigation\", \"CCLE-BRCA\");\nmetaOver.put(\"paired_end\", \"2\");\nmetaOver.put(\"platform\", \"Illumina\");\nmetaOver.put(\"species\", \"Homo sapiens\");\nupdatedFile.setMetadata(metaOver);\n// save the changes you made\nupdatedFile.save();", "language": "java" } ] } [/block] ###Download a file Each file also has a URL property, which gives you the URL you can use to download the file. Again, you will need https://dash.readme.io/project/interim/v1.0/docs/java-library-quickstartto know the file ID to do this. [block:code] { "codes": [ { "code": "File toDownload = user.getFileById(\"584d6f2160b2a10069e40d5d\");\nString downloadUrl = toDownload.getDownloadInfo().getUrl();\nInputStream downloadStream = null;\ntry {\n URL url = new URL(downloadUrl);\n downloadStream = url.openStream();\n log.info(\"Downloading file...\");\n long bytesNum = java.nio.file.Files.copy(downloadStream, Paths.get(\"local_file\"), StandardCopyOption.REPLACE_EXISTING);\n log.info(\"Downloaded {} bytes\", bytesNum);\n} catch (MalformedURLException e) {\n log.error(\"Malformed exception while creating URL from {} - error message: {}\", String.valueOf(downloadUrl), e.getMessage());\n} catch (IOException e) {\n log.error(\"Error while downloading file {} \", e);\n} finally {\n if (downloadStream != null) {\n try {\n downloadStream.close();\n } catch (IOException skip) {\n log.warn(\"Error while trying to close download stream - {}\", skip);\n }\n }\n}", "language": "java" } ] } [/block] <div align="right"><a href="#top">top</a></div> ##Uploading files If you want to use your private data for analysis, you can upload them securely to the Platform. In this section, we will see how to upload files into a project and some actions you can perform on an `upload` object: using an `UploadListener` to listen for related events, polling the number of uploaded bytes while waiting for an upload to complete, blocking further work until upload is completed, cancelling an upload and pausing and restarting an upload. The class `AbstractProgressListener` contains callback methods that inform you of the state of your upload. You can implement the methods for the events you want to listen to, like in this example: [block:code] { "codes": [ { "code": "private static class MyProgressListener extends AbstractProgressListener {\n\n private final AtomicInteger cnt;\n\n MyProgressListener(AtomicInteger cnt) {\n this.cnt = cnt;\n }\n\n @Override\n public void uploadFailed(Exception ex) {\n cnt.incrementAndGet();\n log.error(\"upload failed \", ex);\n }\n\n @Override\n public void uploadFinished() {\n cnt.incrementAndGet();\n log.info(\"upload finished \");\n }\n\n @Override\n public void partUploadFailed(int partNumber, int retryCnt, Exception executionException) {\n log.error(\"part {} failed, retry {}, exception {}\", partNumber, retryCnt, executionException);\n }\n}", "language": "java" } ] } [/block] Then you can build a synchronized upload request using `uploadBuilder`. [block:code] { "codes": [ { "code": "CreateUploadRequestBuilder uploadBuilder = client.getUploadRequestBuilder();\n\n // block on an upload (synchronized upload)\n uploadBuilder.setName(\"Sync-file\")\n .setFile(myPrivateFile)\n .setOverwrite(true)\n .setProject(destinationProject);", "language": "java" } ] } [/block] Based on this, you can create an `UploadContext` object which will hold information about your upload. [block:code] { "codes": [ { "code": "List<com.sbgenomics.java.file.File> uploadedFiles = new ArrayList<>();\n\nUploadContext upload = user.submitUpload(uploadBuilder.build());\n try {\n com.sbgenomics.java.file.File uploadedFile = upload.getFile(); //blocking call\n uploadedFiles.add(uploadedFile);\n } catch (RuntimeException e) {\n log.error(\"Error while waiting for the file to be uploaded - {}\", e);\n }", "language": "java" } ] } [/block] <div align="right"><a href="#top">top</a></div> ###Upload a list of files If you have a list of files you want to have uploaded (let’s call it `filesToUpload`), you can request upload of those files simultaneously, creating an `UploadContext` and `ProgressListener` for each file. [block:code] { "codes": [ { "code": "static final long KB = 1024L;\n static final long MB = 1024 * KB;\n \n// parallel upload without blocking on get, setting up a listener for finished files\n AtomicInteger finishedCnt = new AtomicInteger(0);\n int numOfFiles = filesToUpload.size();\n List<UploadContext> uploads = new ArrayList<>(numOfFiles);\n for (int i = 0; i < numOfFiles; i++) {\n File testFile = filesToUpload(i);\n\n uploadBuilder.setName(\"File-to-upload-no-\" + i)\n .setFile(testFile)\n .setOverwrite(true)\n .setProject(destinationProject)\n// if you want to cut upload in parts\n .setPartSize(64 * MB);\n\n UploadContext uploadContext = user.submitUpload(uploadBuilder.build(), new MyProgressListener(finishedCnt));\n uploads.add(uploadContext);\n }", "language": "java" } ] } [/block] When you submit an upload, the `TransferService` is started if it wasn't previously started by an earlier upload. The `UploadContext` objects allow you to keep track of the progress of your uploads. [block:code] { "codes": [ { "code": " while (finishedCnt.get() < numOfFiles) {\n Thread.sleep(1000);\n log.info(\"---------------------------------------------\");\n for (UploadContext uploadContext : uploads) {\n long transferred = uploadContext.getBytesTransferred();\n long size = uploadContext.getUploadSize();\n log.info(\"Transferred {}% - {} bytes out of {} for upload {}\", (int)((transferred * 100)/size), transferred, size, uploadContext.getUploadName());\n }\n }\n\n for (UploadContext uploadContext : uploads) {\n if (uploadContext.isFinished()) {\n uploadedFiles.add(uploadContext.getFile());\n }\n }", "language": "java" } ] } [/block] <div align="right"><a href="#top">top</a></div> ###Pause an upload If you want to pause an upload, you can do it through `UploadContext.pauseTransfer()`. This will pause the current upload operation and store the information that can be used to resume the upload. The paused `UploadContext` object can later be passed to another `UploadContext` object which will resume upload from the reference point. [block:code] { "codes": [ { "code": " uploadBuilder\n .setName(\"File-to-pause-and-resume\")\n .setFile(testFile)\n .setOverwrite(true)\n .setProject(destinationProject)\n .setPartSize(32 * MB);\n UploadContext toPause = user.submitUpload(uploadBuilder.build());\n\n log.info(\"toPause state {}\", toPause.getState());\n Thread.sleep(10_000); // let it work a while\n toPause.pauseTransfer();\n log.info(\"toPause state {}\", toPause.getState());\n // waiting for the upload to be paused\n while (UploadState.PAUSING.equals(toPause.getState())) {\n Thread.sleep(5_000);\n }\n log.info(\"toPause state {}\", toPause.getState());\n UploadContext resumedUpload = user.resumeUpload(toPause, testFile);\n log.info(\"resumedUpload state {}\", resumedUpload.getState());", "language": "java" } ] } [/block] Trying to obtain the file from a paused upload will throw a `PausedUploadException`. You can use this to resume the paused upload when needed. <div align="right"><a href="#top">top</a></div> ###Abort an upload On some occasions your app (or an external factor) might need to abort an upload. If you have instantiated an `UploadContext` in the above described manner, you can obtain the ID of your upload and abort it. Please keep in mind that trying to obtain the file from the `UploadContext` after the abort will throw an exception. [block:code] { "codes": [ { "code": " String uploadId = uploadContextToBeAborted.getUploadId();\n Upload uploadById = user.getUploadById(uploadId);\n log.info(\"Thread {} is aborting running upload {}\", Thread.currentThread().getName(), uploadId);\n uploadById.abortUpload();", "language": "java" } ] } [/block] <div align="right"><a href="#top">top</a></div> ##Finish an upload Once you have uploaded all the files you needed, it’s time to close the `TransferService` to make sure you gracefully shutdown daemon threads and release resources. [block:code] { "codes": [ { "code": " user.shutdownTransferService();", "language": "java" } ] } [/block] <div align="right"><a href="#top">top</a></div> ##Managing apps Apps on the Platform include both the tools (individual bioinformatics utilities) and the workflows (chains or pipelines of connected tools), either previously existing or user-created. In the section, learn how to get information about publicly available apps, to check which apps are currently in a given project and how to install a new app based on a pre-formatted JSON file. <div align="right"><a href="#top">top</a></div> ###Get information about public apps Use the following to get the `name` and the `id` of an app. You will need to know the `id` of an app in order to work further with it. [block:code] { "codes": [ { "code": "AppList publicApps = user.getPublicApps();\nlog.info(\"There are {} public apps\", publicApps.getSize());\nfor (App publicApp : publicApps) {\n log.info(\"App id {} name {}\", publicApp.getId(), publicApp.getName());\n}", "language": "java" } ] } [/block] <div align="right"><a href="#top">top</a></div> ###Copy an app to your project If you want to use a certain app inside your project, you can copy it into your project. If you try to copy an app that already exists in the given project, the API will issue an error message, which you can use to take appropriate action and inform the user as necessary. Here is [the list of API status codes and descriptions](http://docs.sevenbridges.com/reference#api-status-codes). [block:code] { "codes": [ { "code": "Project destProject = user.getProjectById(String.format(\"%s/destination-project\", user.getUsername()));\nApp appToCopy = user.getAppById(\"admin/sbg-public-data/fastqc-analysis/2\");\n\n // copy action will fail if there is already app with same id in the project\n App copiedApp;\n try {\n copiedApp = appToCopy.copy(destProject);\n } catch (ResourceException e) {\n log.debug(\"Error while trying to copy app - \" + e.toString());\n if (e.getStatus() == 409) { // CONFLICT, app with same ID already exists in project\n copiedApp = user.getAppById(String.format(\"%s/fastqc-analysis\", destProject.getId()));\n log.info(\"App already exists in destination project, with id {}\", copiedApp.getId());\n } else {\n log.error(\"Error while getting app by id, expected success code or 409 HTTP status, got {}\", e.getMessage());\n throw e;\n }\n }", "language": "text" } ] } [/block] <div align="right"><a href="#top">top</a></div> ###List the apps in your project [block:code] { "codes": [ { "code": "AppList destProjectApps = destProject.getApps();\n\nFinally, if you have a JSON file which describes your app through CWL, you can use it to install the app in a project.\n\ntry {\n String raw = new String(Files.readAllBytes(Paths.get(\"my-app-raw.json\")));\n destProject.installApp(\"my-installed-app\", raw);\n} catch (IOException e) {\n log.error(\"Error while reading file 'my-app-raw.json', {}\", e);\n}", "language": "java" } ] } [/block] <div align="right"><a href="#top">top</a></div> ###Add an app with its JSON Learn more about the [Common Workflow Language(CWL)](http://docs.sevenbridges.com/docs/sdk-overview#section-the-common-workflow-language). [block:code] { "codes": [ { "code": "try {\n String raw = new String(Files.readAllBytes(Paths.get(\"my-app-raw.json\")));\n destProject.installApp(\"my-installed-app\", raw);\n} catch (IOException e) {\n log.error(\"Error while reading file 'my-app-raw.json', {}\", e);\n}", "language": "java" } ] } [/block] <div align="right"><a href="#top">top</a></div> ##Managing tasks An app execution is called a task. Each task is associated with a set of input files and chosen settings for the tool(s) in the app. In this part, we will see how to copy files that satisfy certain criteria, copy a relevant app into the project and build a task. We will see how to poll for task status during execution and how to get other useful information about a task. <div align="right"><a href="#top">top</a></div> ###Create a task In order to create a new task, you can create a `TaskBuilder` and set its fields appropriately. Let’s say you want to index a FASTA file with our public app, SAMtools Index FASTA. [block:code] { "codes": [ { "code": "// find a fasta file from public repo and copy it to your project\nFile fastaFile = user.getProjectById(\"admin/sbg-public-data\") .getFiles(Files.criteria().withName(\"HG19_Broad_variant.fasta\"))\n .single();\n\nProject tasksProject = user.getProjectById(String.format(\"%s/task-test\", user.getUsername()));\n\nFile input = fastaFile.copy(tasksProject);\n\n// copy the app to the project\nApp sourceApp = \nuser.getAppById(\"admin/sbg-public-data/samtools-index-fasta-1-3\");\n \nApp myApp = sourceApp.copy(tasksProject);\n\nTaskRequestFactory taskFactory = user.getTaskRequestFactory();\nCreateTaskRequestBuilder taskBuilder = taskFactory.createTaskBuilder();\ntaskBuilder.setApp(myApp)\n .setDescription(\"run from public API v2\")\n .setName(\"API_task_samtools\")\n .setProject(tasksProject)\n .addInput(\"input_fasta_file\", input)\n// to run immediately after task creation\n .runNow(true);\n\nTask task = user.createTask(taskBuilder.build());", "language": "java" } ] } [/block] When the task is executed successfully, its status will change to `COMPLETED`. <div align="right"><a href="#top">top</a></div> ###Check the status of your task Keep in mind that each check of the status fires up an API request, so this shouldn’t be done in quick succession. [block:code] { "codes": [ { "code": "while (!TaskStatus.isFinished(task.getStatus())) {\n try {\n Thread.sleep(30_000);\n } catch (InterruptedException e) {\n log.error(\"Interrupted from sleep, but task job {} is not finished yet\", task.getId());\n throw new RuntimeException(e);\n }\n task.reload();\n}", "language": "java" } ] } [/block] <div align="right"><a href="#top">top</a></div> ###Check if a task is completed If you want to check which tasks have completed, you can use their status as a search criterion. You can also get other information about each task, e.g. its name, inputs and outputs. [block:code] { "codes": [ { "code": "TaskList tasks = user.getTasks(Tasks.criteria().withStatus(TaskStatus.COMPLETED)); log.info(\"List of completed tasks for current user\");\nfor (Task completedTasks : tasks) {\n log.info(\" Task name: {}\", task.getName());\n log.info(\" Inputs: {}\", task.getInputs());\n log.info(\" Outputs: {}\", task.getOutputs());\n}\n", "language": "java" } ] } [/block] <div align="right"><a href="#top">top</a></div> ##Batch tasks Run identical analyses on different data via [batching](http://docs.sevenbridges.com/docs/perform-batch-analysis), a procedure in which you enter multiple input files and group them by specified metadata criteria. This segment on batch tasks follows [this API tutorial](http://docs.sevenbridges.com/docs/api-batch-tutorial), which contains more detailed information. Here we will focus on implementing a batch task using our Java library. We will see how to set up a batch analysis in which we align reads based on their sample metadata. We will see how to copy input and reference files as well as the relevant apps into the project and then create a task and check for errors during creation of the task. We will also show how to check the status of the task while it is running and how to collect the output files once the task is completed. Previously, we have discussed creating a project so we will assume that you have a project with a short name “batch-test” in which you want to perform the batch analysis. [block:code] { "codes": [ { "code": "Project batchProject = user.getProjectById(String.format(\"%s/batch-test\", user.getUsername()));", "language": "java" } ] } [/block] We'll analyze data that is hosted in the Cancer Cell Line Encyclopedia (CCLE) public project on the CGC. The CCLE public project is specified by the ID of sevenbridges/cancer-cell-line-encyclopedia-ccle-1. Inside it, we want to find BAM files with an experimental strategy of RNA-Seq. It is possible to filter by metadata fields to retrieve files with certain properties. In this tutorial, however, we already know that we want to find the following three files: * G30630.VM-CUB1.3.bam * G30603.TUHR4TKB.1.bam * G28034.MDA-MB-361.1.bam [block:code] { "codes": [ { "code": "FileList inputCcleFiles = user.getProjectById(\"sevenbridges/cancer-cell-line-encyclopedia-ccle-1\") \n .getFiles(Files.criteria()\n .withName(\"G30630.VM-CUB1.3.bam\")\n .withName(\"G30603.TUHR4TKB.1.bam\")\n .withName(\"G28034.MDA-MB-361.1.bam\"));", "language": "java" } ] } [/block] The next step is to copy the files into our project. [block:code] { "codes": [ { "code": "List<File> inputFiles = new ArrayList<>(3);\nfor (File file : inputCcleFiles) {\n File copy = file.copy(batchProject);\n inputFiles.add(copy);\n}", "language": "java" } ] } [/block] Many bioinformatics tools require certain data, such as reference genomes or annotation files, to execute properly. Seven Bridges maintains a collection of the latest and most frequently used reference genomes and annotation files in the Public Reference Files repository. For this analysis, we need to supply the workflow with the following two reference files: * HG19_Broad_variant.fasta * Homo_sapiens.GRCh37.75.gtf [block:code] { "codes": [ { "code": "FileList inputPublicFiles = user.getPublicFiles(Files.criteria()\n .withName(\"HG19_Broad_variant.fasta\")\n .withName(\"Homo_sapiens.GRCh37.75.gtf\"));\nfor (File file : inputPublicFiles) {\n file.copy(batchProject);\n}", "language": "java" } ] } [/block] Next, we need to copy the RNA-seq Alignment STAR app from the list of public apps. Check here how to obtain the ID of the app. [block:code] { "codes": [ { "code": " App appToCopy = user.getAppById(\"admin/sbg-public-data/rna-seq-alignment-star/16\n\");\n App appRnaSeqAlign = appToCopy.copy(batchProject);\n ", "language": "java" } ] } [/block] At this point, we need to edit the app so that it can accept the BAM files we copied as input. You can do it by editing its CWL or via the visual interface. Here are the instructions for using Workflow editor on the visual interface. After the app is edited, we can build the task and set appropriate fields. [block:code] { "codes": [ { "code": "TaskRequestFactory taskFactory = user.getTaskRequestFactory();\n CreateTaskRequestBuilder taskBuilder = taskFactory.createTaskBuilder();\n taskBuilder\n .setApp(appRnaSeqAlign)\n .setDescription(\"run from public API v2 with batch\")\n .setName(\"Batch run RNA seq - api client java\")\n .setProject(batchProject)\n// the input port on which you wish to batch\n .setBatchInput(\"input_file\")\n .addBatchBy(\"CRITERIA\", java.util.Collections.singletonList(\"metadata.sample_id\"))\n .addInput(\"input_file\", inputFiles.toArray(new File[0]))\n .addInput(\"reference_or_index\", batchProject.getFiles(Files.criteria().withName(\"HG19_Broad_variant.fasta\")).single())\n .addInputAsSingletonList(\"sjdbGTFfile\", batchProject.getFiles(Files.criteria().withName(\"Homo_sapiens.GRCh37.75.gtf\")).single())\n .runNow(true);", "language": "java" } ] } [/block] Since we set the `runNow` parameter to true, the task will run as soon as we create it. If you leave out this field, the task will be created and stay in DRAFT status until you call `run()` on it. [block:code] { "codes": [ { "code": "//creates and runs the task, since runNow is true\nTask task = user.createTask(taskBuilder.build());", "language": "java" } ] } [/block] If task runs into errors during creation, they will be attached to the task object. [block:code] { "codes": [ { "code": " List<Map<String, Object>> errors = task.getErrors();\n if (errors != null && errors.size() > 0) {\n log.error(\"Error while validating task with id {}\", task.getId());\n for (Map<String, Object> error : errors) {\n log.error(\" {}\", error);\n }\n }", "language": "java" } ] } [/block] While the task is being executed, you can occasionally check its status. You will also be notified by email once your task is completed. [block:code] { "codes": [ { "code": "while (!TaskStatus.isFinished(task.getStatus())) {\n log.info(\"Sleeping\");\n try {\n Thread.sleep(30_000);\n } catch (InterruptedException e) {\n Thread.interrupted();\n log.error(\"Interrupted while waiting for task to finish\");\n throw new RuntimeException(e);\n }\n task.reload();\n log.info(\"Current status {}\", task.getStatus().toString());\n }", "language": "java" } ] } [/block] When the task is completed, you can obtain the list of files that make up its output. Or you can get information about the task status. [block:code] { "codes": [ { "code": " if (TaskStatus.COMPLETED.equals(task.getStatus())) {\n FileList producedFiles = batchProject.getFiles(Files.criteria().withTaskOrigin(task));\n for (File producedFile : producedFiles) {\n log.info(\"Produced file name {} and id {}\", producedFile.getName(), producedFile.getId());\n }\n } else {\n log.warn(\"Task is finished, but with status {}\", task.getStatus());\n }", "language": "java" } ] } [/block] <div align="right"><a href="#top">top</a></div> ##Managing volumes Volumes authorize the Platform to access and query objects on a specified cloud storage ([Amazon Web Services](http://docs.sevenbridges.com/docs/aws-cloud-storage-tutorial) or [Google Cloud Storage](http://docs.sevenbridges.com/docs/google-cloud-storage-tutorial)) on your behalf. We will examine setting up a volume on each of the services, listing your volumes, importing and exporting files and polling for an import or export status while waiting for a job to finish. The S3 and GCS volumes each have their own builder. Before you can use them, you will need to set up the `AWS_ACCESS_KEY` and the `AWS_SECRET_KEY` for S3 or the `GCS_PRIVATE_KEY` for the GCS. Then an appropriate builder can be used to create a volume. Here is the S3 example: [block:code] { "codes": [ { "code": "CreateS3VolumeRequestBuilder s3Builder = user.getVolumeRequestFactory().s3Builder();\n s3Builder\n .setName(\"my_new_S3_volume_for_test\")\n .setAccessMode(AccessMode.RW)\n .setAccessKey(AWS_ACCESS_KEY)\n .setSecretKey(AWS_SECRET_KEY)\n .setBucket(\"outer_user_test_bucket\")\n .setDescription(\"S3 Volume for testing imports and exports from and to external (non sbg) S3 bucket\")\n .setSseAlgorithm(\"AES256\");\n Volume testVolume = user.createVolume(s3Builder.build());", "language": "java" } ] } [/block] This is the corresponding GCS example: [block:code] { "codes": [ { "code": " CreateGcsVolumeRequestBuilder gcsBuilder = user.getVolumeRequestFactory().gcsBuilder();\n gcsBuilder\n .setName(\"my_new_GCS_volume_for_test\")\n .setAccessMode(AccessMode.RO) //only read mode for GCS\n .setBucket(\"outer_user_GCS_import\")\n .setClientEmail(CLIENT_GOOGLE_EMAIL)\n .setPrivateKey(\"-----BEGIN PRIVATE KEY-----\"+ GCS_PRIVATE_KEY + \"-----END PRIVATE KEY-----\\\\n\");\n Volume gcsVolume = user.createVolume(gcsBuilder.build());", "language": "java" } ] } [/block] Each file is imported using an import job. Once you have created it, you can refer to it by its ID. [block:code] { "codes": [ { "code": " ImportJob gcsImportJob = user.startImport(gcsVolume, \"files/google_file_key\", destProject, \"gcs_file\", false);\n log.info(\"GCS volume import id : {}\", gcsImportJob.getId());", "language": "java" } ] } [/block] If you have a list of import jobs (called imports in this example), you can poll for their status occasionally, until they are all finished. [block:code] { "codes": [ { "code": " int finishedCnt = 0;\n while (finishedCnt < imports.size()) {\n try {\n Thread.sleep(5000);\n Iterator<ImportJob> iterator = imports.iterator();\n while (iterator.hasNext()) {\n ImportJob next = iterator.next();\n next.reload();\n if (VolumeJobState.isFinished(next.getState())) {\n log.info(\"Volume import id : {} name {} done\", next.getId(), next.getDestinationName());\n finishedCnt++;\n iterator.remove();\n }\n }\n } catch (Exception e) {\n log.error(\" Error while waiting for imports to finish - {}\", e.getMessage());\n break;\n }\n }", "language": "java" } ] } [/block] To export a file, refer to it by its ID and create an export job. [block:code] { "codes": [ { "code": "File fileToExport = user.getFileById(\"584d6f160b2a10069e40d5d\");\n ExportJob exportJob = user.startExport(fileToExport, testVolume, \"exported-file\", false);", "language": "java" } ] } [/block] Similarly to imports, you can check on the status of your export job occasionally. [block:code] { "codes": [ { "code": " try {\n while (!VolumeJobState.isFinished(exportJob.getState())) {\n Thread.sleep(5000);\n exportJob.reload();\n }\n } catch (Exception e) {\n log.error(\" Error while waiting for export to finish = {}\", e.getMessage());\n }\n\n if (VolumeJobState.COMPLETED.equals(exportJob.getState())) {\n log.info(\"Volume export with id {} is completed successfully\", exportJob.getId());\n } else {\n log.warn(\"Volume export with id {} failed\", exportJob.getId());\n }\n", "language": "java" } ] } [/block] <div align="right"><a href="#top">top</a></div>