Start a bulk import job

This call lets you perform a bulk import of files from your volume (either Amazon Web Services or Google Cloud Storage) into your project on the Platform.

You can use this call to either import files to a specific folder or a project but you can also use it to import a folder and its files into another destination folder while preserving folder structure (see below) . One call can contain up to 100 items.

Learn more about using the Volumes API for Amazon S3 and for Google Cloud Storage.

Request

https://cgc-api.sbgenomics.com/v2/bulk/storage/imports/create

Example request

POST v2/bulk/storage/imports/create
Host: cgc-api.sbgenomics.com
X-SBG-Auth-Token: 3210a98c1db9318fa9d9273156740f74

curl -X POST \
  https://cgc-api.sbgenomics.com/v2/bulk/storage/imports/create \
  -H 'Content-Type: application/json' \
  -H 'X-SBG-Auth-Token: 3210a98c1db9318fa9d9273156740f74' \
  -d '{
    "items": [  
             {
            "source": {
                "volume": "rfranklin/my-volume",
                "location": "chimeras.html.gz"
            },
            "destination": {
                "project": "rfranklin/my-project"
            }
         },
         {
            "source": {
                "volume": "rfranklin/my-volume",
                "location": "my-folder/"
            },
            "destination": {
                "project":"rfranklin/my-project"
            },
            "autorename": true,
            "preserve_folder_structure": true
        },
        {
            "source": {
                "volume": "rfranklin/my-volume",
                "location": "my-volume-folder/"
            },
            "destination": {
                "name": "new-folder-name",
                "parent": "5abc23f5e1d23d4c8d7a309a"
            },
            "autorename": true,
            "preserve_folder_structure": true
        }
    ]
}

Header Fields

Name	Description
X-SBG-Auth-Token required	Your CGC authentication token.
Content-Type required	`application/json`

Request body

For every item in the request body, you should enter the list of key-value pairs described in the table below. One API call can contain up to 100 items.

Key	Data type	Description
`source` required	object	The object for specifying the source for importing a file. This is done by specifying `volume` and `location` keys.
`volume` required	string	Volume ID from which to import the file.
`location` required	string	Volume-specific location pointing to the file or folder to import. This location should be recognizable to the underlying cloud service as a valid key or path to the file or folder. Please note that if this volume was configured with a `prefix` parameter when it was created, the prefix will be prepended to `location` before attempting to locate the file on the volume. Furthermore, when specifying the location of a folder the trailing slash is mandatory.
`destination` required	object	This is a dictionary object that describes the Platform destination for the imported file. The destination is specified using either `project` or `parent` keys.
`project` required	string	The project in which to create the alias.
`name`	string	The name of the alias to create. This name should be unique to the project. If the name is already in use in the project, you should use the `overwrite` parameter in this call to force any file with that name to be deleted before the alias is created. If name is omitted, the alias name will default to the last segment of the complete location (including the `prefix`) on the volume. Segments are considered to be separated with forward slashes ('/').
`parent'	string	The ID of the target folder to which the item should be imported. Should not be used together with `project`. If `parent` is used, the import will take place into the specified folder, within the project to which the folder belongs. If `project` is used, the items will be imported to the root of the project's files.
`autorename`	string	Whether to automatically rename the item (by prefixing its name with an underscore and number) if another one with the same name already exists at the destination.
`preserve_folder_structure`	string	Whether to keep the exact source folder structure. The default value is true if the item being imported is a folder. Should not be used if you are importing a file.

Example request body

{
    "items": [  
             {
            "source": {
                "volume": "rfranklin/my-volume",
                "location": "chimeras.html.gz"
            },
            "destination": {
                "project": "rfranklin/my-project"
            }
         },
         {
            "source": {
                "volume": "rfranklin/my-volume",
                "location": "my-folder/"
            },
            "destination": {
                "project":"rfranklin/my-project"
            },
            "autorename": true,
            "preserve_folder_structure": true
        },
        {
            "source": {
                "volume": "rfranklin/my-volume",
                "location": "my-volume-folder/"
            },
            "destination": {
                "name": "new-folder-name",
                "parent": "5abc23f5e1d23d4c8d7a309a"
            },
            "autorename": true,
            "preserve_folder_structure": true
        }
    ]
}

Response

See a list of Seven Bridges Platform-specific response codes that may be contained in the body of the response..

Response body

Key	Data type	Description
`id`	string	ID of this import job.
`state`	string	The state of this import job. Possible values are: `PENDING`: the import is queued; `RUNNING`: the import is running; `COMPLETED`: the import has completed successfully; `FAILED`: the import has failed.
`source`	object	Import source, as passed when this job was started by the call to start an import job.
`destination`	object	Import destination, as passed when this job was start an import job.
`result`	object	File object that was imported.
`error`	object	In case of error in the import job, standard API error is returned here.

Example response body

{
    "items": [
        {
            "resource": {
                "href": "https://cgc-api.sbgenomics.com/v2/storage/imports/jb4t6pFqe11er8w4pJWcgZ4OEugduntP",
                "id": "jb4t6pFqe11er8w4pJWcgZ4OEugduntP",
                "state": "PENDING",
                "overwrite": false,
                "autorename": false,
                "source": {
                    "volume": "rfranklin/my-volume",
                    "location": "chimeras.html.gz"
                },
                "destination": {
                    "project": "rfranklin/my-project",
                    "name": "chimeras.html.gz"
                }
            }
        },
        {
            "resource": {
                "href": "https://cgc-api.sbgenomics.com/v2/storage/imports/33xxpFMsJkVbxEF6SmSarrqhFEBJyYrL",
                "id": "33xxpFMsJkVbxEF6SmSarrqhFEBJyYrL",
                "state": "PENDING",
                "overwrite": false,
                "autorename": true,
                "preserve_folder_structure": true,
                "source": {
                    "volume": "rfranklin/my-volume",
                    "location": "my-volume-folder"
                },
                "destination": {
                    "project": "rfranklin/my-project",
                    "name": "my-folder"
                }
            }
        },
        {
            "resource": {
                "href": "https://cgc-api.sbgenomics.com/v2/storage/imports/Ocy4z7TTits7YsYjc6nZWPisTsJbKS8e",
                "id": "Ocy4z7TTits7YsYjc6nZWPisTsJbKS8e",
                "state": "PENDING",
                "overwrite": false,
                "autorename": true,
                "preserve_folder_structure": true,
                "source": {
                    "volume": "rfranklin/my-volume",
                    "location": "my-volume-folder"
                },
                "destination": {
                    "project": "rfranklin/my-project",
                    "name": "new-folder-name",
                    "parent": "5abc23f5e1d23d4c8d7a309a"
                }
            }
        }
    ]
}

Import a volume folder into a specific folder

To import a volume folder into a specific folder in one of your projects, use the source parameter to specify the volume and the folder you are importing from, as well as the ID of the target folder in your project using the parent parameter. Use the API call for listing files to obtain the folder ID of the desired parent folder.

The folder structure is preserved by setting the preserve_folder_structure parameter to true.

The name parameter can be used to rename the imported folder within the destination folder. The following parameters can be used to determine the behavior in case a folder with the same name already exists (if omitted, the files that exist in the destination folder will be preserved):

autorename - the files that are being imported will be automatically renamed
overwrite - the files that already exist in your project on the Platform will be overwritten

{
    "items": [  
             {
            "source": {
                "volume": "rfranklin/my-volume",
                "location": "my-volume-folder/"
            },
            "destination": {
                "name": "new-folder-name",
                "parent": "5abc23f5e1d23d4c8d7a309a"
            },
            "autorename": true,
            "preserve_folder_structure": true
        }
        }
    ]
}