Import from an FTP or HTTP(S) server

📘

This upload method can also be used with (pre)signed URLs from cloud storage services such as AWS S3 and Google Cloud Storage.

Overview

The FTP/HTTP(S) and cloud storage upload method copies files to the CGC from the following sources:

• Public or private FTP server.
• Public or private HTTP(S) server.
• Cloud storage such as AWS S3 or Google Cloud Storage, using presigned URLs. Unlike Connect Cloud Storage that attaches your cloud storage as a volume to the CGC, this will copy the files to the CGC storage right away, and may cause file transfer costs if the files that are being transferred and the CGC project they are transferred to are in different locations.

This method allows you to import a single file, a directory (FTP), or a list of URLs (HTTP(S) or pre-signed URLs). They are imported in the same job. Additionally, active imports run in the background on the CGC and do not depend on your local network speed. This allows you to multitask as your files are added to your project.

To import files from an FTP or HTTP(S) server:

1. Navigate to the desired project.
2. Go to the Files tab in the project.
3. Click Add Files > FTP/HTTP, as shown below.

3. Type in or paste the URL of the file(s) you want to import. Or, click Browse file to navigate to a text file on your computer that contains these URLs.

📘

Types of uploads

You can upload a single file, a directory (in the case of FTP), or an index page (in the case of HTTP(S)).

You can also upload a list of URLs. They will all be uploaded in the same job.

4. (Optional) Enter tags in the Tag files box. This makes it easier to visually identify and organize files. Separate tags with commas or by hitting the return key. All tags which are already in the selected project are autosuggested in the drop-down menu. Learn more about tagging your files.

👍

Tagging files

You can also tag files as you upload them via the CGC Uploader or via the command line.

6. Resolve naming conflicts - Select the action to be taken if a naming conflict occurs. Available actions are Skip (default option), Auto-rename and Overwrite. Read more about naming conflicts resolution below.

📘

• Allowed characters in file names are all alphanumerics and special characters except slash (/). File names can’t be longer than 255 characters.
• Allowed characters in folder names are a-z, A-Z, 0-9, and special characters (_), (-), and (.). Folder names can’t be longer than 255 characters.

7. Preserve folder structure - If enabled, the imported items will be an exact mirror of the file/folder structure on the server. Otherwise, the items will be "flattened" when imported from the server, i.e. only files will be imported, with no folder structure.
8. Click Import to start the upload.
9. Click Import more files if you have more URLs to import. If you're finished uploading, click Done to close the dialog.

👍

Track your progress

You can track the import progress under the import status icon on the right-hand side of the main menu.

When the upload completes, review your files in the Files tab of the project.

📘

Note that currently tags added to a file can only be viewed via the CGC visual interface. We are working to make these tags available via the API.

Accessing password-protected FTP and HTTP(S) servers

The FTP/HTTP(S) tool can submit usernames and passwords to the server. To do this, you should include the username and password in the links that you pass to the tool, using the standard HTTP syntax: http://username:[email protected]/sample/.

Pasting in a link formatted in this way will attempt to authenticate with the server example.org using the credentials username and password and then try to fetch everything that is hosted under the directory/sample/.

RESOLVING NAMING CONFLICTS

Naming conflicts happen when you try to import an item (file or folder) from an FTP/HTTP server to a project's root folder or subfolder, where an item with the same name already exists. The list below illustrates the possible cases of naming conflicts and the way they are resolved depending on which option is selected in the Resolve naming conflicts dropdown:

1. Importing a file that has the same name as an existing file at the destination path:
   • Skip (default): The file won't be imported. This is the default behavior.
   • Auto-rename: The imported file will be automatically renamed by adding a prefix (underscore and serial number).
   • Overwrite: The existing file will be overwritten by the file imported from the server.
2. Importing a file that has the same name as an existing folder at the destination path:
   • Skip (default): The file won't be imported.
   • Auto-rename: The imported file will be automatically renamed by adding a prefix (underscore and serial number).
   • Overwrite: The file won't be imported.
3. Importing a folder that has the same name as an existing folder at the destination path:
   • The content of the folder that is being imported will be merged with the content of the existing folder, regardless of the option selected in the Resolve naming conflicts dropdown. Any conflicts related to files within the folders will be handled according to file naming conflict resolution rules described above.
4. Importing a folder that has the same name as an existing file at the destination path:
   • The folder and the files it contains won't be imported. This is the default, non-configurable behavior, regardless of the option selected in the Resolve naming conflicts dropdown.

The following table illustrates the entire naming conflict resolution mechanism:

Check import status

To check the progress of your import, click the notification bell in the upper right corner.

Click Open activity center to see more detailed information about each of the imports.

Click on an import in the left pane, and the right pane will show detailed information about that import including:

• FTP/HTTP link - the link the file was imported from

• Status - the status of the import for each of the files; in case of a failed import, the following information will also be available:

  • Error Code - click to see detailed information about the error code
  • Error Message - the description of the error

Troubleshooting

If the Seven Bridges FTP import tool fails to import all the files whose links you provided, please retry by providing shorter lists of FTP links.