How to Use Linode Object Storage

Note

Linode’s Object Storage is available in our Atlanta (USA), Frankfurt (Germany), Newark (USA), and Singapore data centers. For more availability details and billing information, see the Object Storage Overview product documentation.

Caution

Object Storage is similar to a subscription service. Once enabled, you will be billed at the flat rate regardless of whether or not there are active buckets on your account. You must Cancel Object Storage to stop billing for this service.

Linode’s Object Storage is a globally-available, S3-compatible method for storing and accessing data. Object Storage differs from traditional hierarchical data storage (as in a Linode’s disk) and Block Storage Volumes. Under Object Storage, files (also called objects) are stored in flat data structures (referred to as buckets) alongside their own rich metadata.

Additionally, Object Storage does not require the use of a Linode. Instead, Object Storage gives each object a unique URL with which you can access the data. An object can be publicly accessible, or you can set it to be private and only visible to you. This makes Object Storage great for sharing and storing unstructured data like images, documents, archives, streaming media assets, and file backups, and the amount of data you store can range from small collections of files up to massive libraries of information. Lastly, Linode Object Storage has the built-in ability to host a static site.

In this guide you learn about:

• How to get set-up for object storage by creating an Object Storage Key Pair.

• The variety of first-party and third-party tools available to access and use the service.

• How to use Object Storage with the Cloud Manager.

Enabling Object Storage

Object Storage is not enabled for a Linode account by default. All that is required to enable Object Storage is to create a bucket or an Object Storage access key. To cancel Object Storage, see the Cancel Object Storage how-to.

Note

Billing for Object Storage starts when it is enabled on the account, regardless of how it is enabled. For example, if you enable the service by creating an access key, but you have not yet created a bucket, the $5 monthly flat rate (prorated) for Object Storage is charged for the account. Cancelling Object Storage stops billing for it.

Access Keys

The first step towards using Object Storage is to create a pair of keys for the service. This pair is composed of an access key and a secret key:

• The access key allows you to access any objects that you set to have private read permissions.

  Note

  To use the access key when viewing a private object, you first need to generate a signed URL for the object. The signed URL is much like the standard URL for the object, but some extra URL parameters are appended to it, including the access key. Instructions for generating a signed URL can be found for s3cmd and the Linode CLI, outlined in this guide.

• The secret key is used together with the access key to authenticate the various Object Storage tools with the Linode account. You should not share the secret key.

  Note

  Each Object Storage key pair on the Linode account has complete access to all of the buckets on the account.

Generating an Access Key

To use Object Storage with any compatible client or command-line tool, you’ll need to generate an Access Key. This can be done directly in the Cloud Manager.

1. Log into the Linode Cloud Manager.

2. Click the Object Storage link in the sidebar, click the Access Keys tab, and then click the Create an Access Key link.

3. If this is your first bucket and you have not enabled Object Storage previously, a prompt appears asking you to confirm that you’d like to enable Object Storage. Click Enable Object Storage.

   

4. The Create an Access Key menu appears.

5. Enter a label for the Access Key.

• This label is how you reference the Access Key in the Linode Cloud Manager.

• You can also toggle the Limited Access switch on this panel. This allows you to limit certain permissions on a per bucket level for this access key.

  Note

  Regardless of access, all keys can create new buckets and list all buckets. However, after creating a bucket, depending on what you select here, a limited access key may not be able to access those buckets, add items, remove items, and other actions.

1. When you have entered the Label, and optionally selected the desired permissions, click the Submit button.

2. A window appears displaying the Access Key and the Secret Key. Write these down somewhere secure. The access key is visible in the Linode Cloud Manager, but you can not retrieve the secret key again after you close the window!

   You now have the credentials needed to connect to Linode Object Storage.

3. Check the permissions you gave this key by clicking on the more options ellipsis link to the right of the key in the Access Keys list. Then, select View Permissions from the menu.

   An access key with limited permissions displays all the buckets and their selected permissions. An access key that has full permissions displays a statement about unlimited access.

Limiting Access through Access Keys

By default, an Access Key is unrestricted and has full access to all Buckets on an account. When creating an Access Key, you can enable Limited Access and set more granular permissions for each Bucket. These permissions include None, Read, and Read/Write:

• None: Restricts all access to the specified Bucket. This Access Key will still be able to view the Bucket in the list of all Buckets, but will otherwise be unable to access any objects stored within it.

• Read (read_only): Access keys with Read permissions are able to list and retrieve most information about the specified Bucket and objects stored in that Bucket. Technically, read permissions provide access to the following s3 actions (which are used by all s3-compatible clients and tools):

  GetBucketAcl, GetBucketCORS, GetBucketLocation, GetBucketLogging, GetBucketNotification, GetBucketPolicy, GetBucketTagging, GetBucketVersioning, GetBucketWebsite, GetLifecycleConfiguration, GetObjectAcl, GetObject, GetObjectTorrent, GetReplicationConfiguration, GetObjectVersionAcl, GetObjectVersion, GetObjectVersionTorrent, ListBucketMultipartUploads, ListBucket, ListBucketVersions, ListMultipartUploadParts

• Read/Write (read_write): Access keys with Read/Write permissions can list, retrieve, add, delete, and modify most information and objects stored within the specified Bucket. Technically, read/write permissions provide access to all of the same s3 actions as read permissions, as well as the following:

  AbortMultipartUpload, DeleteBucketWebsite, DeleteObject, DeleteObjectVersion, DeleteReplicationConfiguration, PutBucketCORS, PutBucketLogging, PutBucketNotification, PutBucketTagging, PutBucketVersioning, PutBucketWebsite, PutLifecycleConfiguration, PutObject, PutObjectAcl, PutObjectVersionAcl, PutReplicationConfiguration, RestoreObject

A full list of s3 actions is available on Amazon’s S3 API Reference documentation.

Control Access with ACLs and Bucket Policies

Linode Object Storage allows users to share access to objects and buckets with other Object Storage users. There are two mechanisms for setting up sharing: Access Control Lists (ACLs), and bucket policies. These mechanisms perform similar functions: both can be used to restrict and grant access to Object Storage resources. Learn more about ACLs and bucket policies.

Bucket Names

Bucket names, also referred to as labels, need to be unique within the same cluster, including buckets on other users' Linode accounts. This also means if you reserve a bucket name in one cluster, it is not automatically reserved in another. For example, if you have my-bucket.us-east-1.linode.com and want my-bucket.eu-central-1.linode.com you must manually reserve them both. They are separate clusters and not guaranteed. If the label you enter is already in use, then choose a different label. Additionally, bucket labels have the following rules:

• Cannot be formatted as IP addresses.
• Must be between 3 and 63 characters in length.
• Can only contain lower-case characters, numbers, periods, and dashes.
• Must start with a lowercase letter or number.
• Cannot contain underscores (_), end with a dash (-) or period (.), have consecutive periods (.), or use dashes (-) adjacent to periods (.).

Note

A cluster is defined as all buckets hosted by a unique URL; for example: us-east-1.linodeobjects.com, ap-south-1.linodeobjects.com, or eu-central-1.linodeobjects.com.

Object Storage TLS/SSL Certificates

Object storage supports the importing of custom TLS/SSL Certificates in order to encrypt connections with your bucket in transit. Currently, you can Upload, View, or Delete TLS/SSL certificates using the following API Endpoints:

• Upload Object Storage TLS/SSL Cert
• View Object Storage TLS/SSL Cert
• Delete Object Storage TLS/SSL Cert

Like all API endpoints, TLS/SSL certificates can additionally be added using the Linode CLI.

For more information on Object Storage Certificates, including configuration options in the Linode Manager, see our Guide for Custom Object Storage Certificates

Object Storage Tools

There are a number of tools that are available to help manage Linode Object Storage. This guide explains how to install and use the following options:

• The Linode Cloud Manager can be used to create buckets, and upload and delete objects, as well as create access keys for use with the S3 compatible clients.

• The Linode CLI has an Object Storage plugin and can be used to create and remove buckets, add and remove objects, and convert a bucket into a static site from the command line.

• s3cmd is a powerful command line utility that can be used with any S3-compatible object storage service, including Linode’s. s3cmd can be used to create and remove buckets, add and remove objects, convert a bucket into a static site from the command line, plus other functions like syncing entire directories up to a bucket.

• Cyberduck is a graphical utility available for Windows and macOS and is a great option if you prefer a GUI tool.

Cloud Manager

Create a Bucket

The Cloud Manager provides a web interface for creating buckets. To create a bucket:

1. If you have not already, log into the Linode Cloud Manager.

2. Click the Object Storage link in the sidebar, and then click Add a Bucket.

   If you have not created an access key or a bucket before, you are prompted to enable Object Storage.

3. The Create a Bucket menu appears.

4. Add a label for the bucket. See the Bucket Name section for rules on naming the bucket.

5. Choose a region (cluster) for the bucket to reside in. See the Availability section on the Object Storage Overview page for a list of available regions.

   Note

   A cluster is defined as all buckets hosted by a unique URL; for example: us-east-1.linodeobjects.com, ap-south-1.linodeobjects.com, or eu-central-1.linodeobjects.com.

6. Click Submit. You are now ready to upload objects to the bucket.

Upload Objects to a Bucket

Follow these steps to upload objects to a bucket using the Cloud Manager:

1. If you have not already, log into the Linode Cloud Manager.

2. Click the Object Storage link in the sidebar. A list of all the buckets appears. Click the bucket where you want to upload the objects.

3. The bucket’s Objects Listing Page appears. In the example, the my-example-bucket does not yet contain any objects. You can use the Upload Files Pane to drag and drop a file from the local computer to the object storage bucket.

   Note

   You can drag and drop multiple files to the Upload Files Pane at one time.

   Note

   Uploading or renaming objects using non-standard special characters and unusual ASCII/Unicode characters may cause issues.

   You can also click the Browse Files button to bring up the local computer’s file browser and select a file to upload to the bucket.

4. When the upload has completed, the object appears in the Objects Listing Page.

   Note

   Individual object uploads are limited to a size of 5GB each, though larger object uploads can be facilitated with multipart uploads. s3cmd and Cyberduck do this for you automatically if a file exceeds this limit as part of the uploading process.

   Object file names additionally cannot contain the following special characters when using Cloud Manager or the Linode CLI:

   " ' < > & + =

View Bucket Objects

Follow these steps to view your objects from the Cloud Manager:

1. If you have not already, log into the Linode Cloud Manager.

2. Click the Object Storage link in the sidebar. A list of all the buckets appears. Click the bucket whose objects you’d like to view.

3. The bucket’s Objects Listing Page appears, which displays all the objects in the bucket.

4. Click the ellipsis menu corresponding to the object you’d like to view. Then, select Download.

5. The object is downloaded to the local computer.

Delete Objects from a Bucket

Follow these steps to delete objects in a bucket from the Cloud Manager:

1. If you have not already, log into the Linode Cloud Manager.

2. Click the Object Storage link in the sidebar. A list of all the buckets appears. Click the bucket whose objects you’d like to delete.

3. The bucket’s Objects Listing Page appears, which displays all the objects in the bucket.

4. Click the ellipsis menu corresponding to the object you’d like to delete. Then, select Delete.

5. A dialog box appears prompting you to confirm if you’d like to delete the object. Click Delete to proceed. After the object has been deleted, it is no longer be visible on the Objects Listing Page.

Delete a Bucket

Follow these steps to delete an Object Storage bucket from the Cloud Manager:

Note

You can only delete an empty Object Storage bucket. See the Delete Objects from a Bucket how-to for information on deleting a bucket’s objects using the Cloud Manager.

1. If you have not already, log into the Linode Cloud Manager.

2. Click the Object Storage link in the sidebar. A list of all the buckets appears.

3. Click the ellipsis menu corresponding to the bucket you’d like to delete. Then, select Delete.

4. A dialog box appears that prompts you to enter the bucket’s name as a way to confirm that you’d like to delete the bucket. Type the bucket’s name into the text entry field and click Delete.

   

   After the bucket has been deleted, it is no longer be visible on the Buckets Listing Page.

Cancel Object Storage

The following steps outline how to cancel the Object Storage service from the Cloud Manager.

Caution

Cancelling Object Storage deletes all objects and buckets from your account. Consider downloading any important files before continuing.

1. Log in to the Cloud Manager, select Account from the left menu, and navigate to the Settings tab.

2. Find the section labeled Object Storage and click the Cancel Object Storage button.

3. A prompt appears asking you to confirm the cancellation. Click Confirm cancellation to proceed. Any remaining buckets and objects on your account are deleted and you will no longer be billed for Object Storage.

Next Steps

There are S3 bindings available for a number of programming languages, including the popular Boto library for Python, that allow you to interact with Object Storage programmatically.

More Information

You may wish to consult the following resources for additional information on this topic. While these are provided in the hope that they will be useful, please note that we cannot vouch for the accuracy or timeliness of externally hosted materials.

• S3cmd usage and command reference