Working with Amazon S3 Buckets

YatinAugust 12th, 2020Last Updated: August 10th, 2020

0 22 4 minutes read

Welcome readers, in this tutorial, we will make a brief introduction to AWS S3 bucket and analyze the different ways to access S3.

1. Introduction

AWS represents the Amazon Web Services and it provides different services ranging from IaaS (Infrastructure as a Service), PaaS (Platform as a Service), or SaaS (Packaged software as a Service).

It offers flexibility by giving the users flexibility of what to do and what not
Cost-effective solutions
Scalability/Elasticity techniques to automatically scale up/down the application infrastructure when demand increases or decreases respectively
Provides end-to-end security configuration and privacy to its customers
A virtual infrastructure that offers complete privacy and isolation of operations

1.1 AWS Storage Service

AWS Storage Service or simply known as AWS S3 is an online storage facility for the users. It cheap, easy to set up and the user only pays for what they utilize. It offers,

To host static web-content and data or even the dynamic pages
Data storage for analytics
Backup and archival of data
Disaster recovery solutions

1.1.1 S3 Bucket

Amazon S3 Bucket has two primary entities i.e. Object and Bucket, where objects are stored inside the buckets. It provides high availability and durability solutions by replicating the data of one bucket in multiple data centers. Each AWS account offers 100 buckets as free, however, this count can be increased on submitting a request to the support center. Amazon S3 features –

Each object in a bucket is given a unique id
S3 allows a developer to upload/delete or read an object via the REST API
S3 offers two read-after-write and eventual consistency models to ensure that every change command committed to a system should be visible to all the participants
Objects stored in a bucket never leave it’s location unless the user transfer it out
Objects can be made private or public and rights can be granted to specific users

1.1.2 S3 Object

S3 Object(s) are the basic entities stored in the Amazon S3 bucket and is a simple key-value store. An object in S3 consists of the following –

Key – Represents the name assigned to the object
VersionId – Represents the key and version ID uniquely identifies an object. It is a string that S3 generates we add an object to the bucket
Value – The content which we are storing in the bucket and can range up to 5 TB in size
Metadata – Represents the name-value pairs with which we can store information regarding the object
Access Control Information – Through this we control access to the objects stored in Amazon S3

1.1.3 Setting up AWS S3 bucket and IAM user

If someone needs to go through the process of creating an S3 bucket and attach it to an IAM user, please watch this video.

2. Working with S3 buckets

Now to work with AWS S3 there is a different way of doing things and we will see an introduction to each one of them.

2.1 Using Amazon S3 Console

Creating an S3 bucket from a console is one of the easy things to do. Just navigate to the S3 and click the “Create Bucket” button as shown in the below figure.

s3 bucket - Creating an S3 Bucket — Fig. 1: Creating an S3 Bucket

Fill the form (such as Name the bucket (remember the bucket is universally unique) and the region as appropriate) and leave other details are default. After that, the bucket will be created and list in the S3 dashboard as shown in the below figure.

s3 bucket - S3 dashboard listing buckets — Fig. 2: S3 dashboard listing buckets

Once the bucket is successfully created a user can perform operations like upload the data to the bucket (which in turn will be called objects), settings up permission on the object and bucket, deleting object(s) from the bucket and the bucket itself, etc.

2.2 REST API

Creating a bucket using the REST API can be cumbersome because it requires you to write code to authenticate your requests. AWS recommends using the AWS console or the AWS SDK.

2.3 AWS SDK

Creating an S3 bucket or playing around with the S3 operations can be performed with AWS SDK but it’s not easy. When playing around with AWS SDK we need to create a client first and then using the client to send the request to create a bucket. In case, while playing if the region is not specified the AWS creates a bucket in North Virginia (default region). You can read this link to get more information on different regions. But remember as a best practice we should always create a client and bucket in the same region.

While above we saw a different way of doing things or play around in S3. But we should also understand about configuring the bucket policies.

2.4 Permissions in S3

AWS never recommends using the root credentials for creating a bucket
Use the IAM user and grant that user full access to S3. This user will also be helpful to perform the S3 operations using the AWS CLI

Always remember to assign the least privileges to the S3 service in AWS. For instance –

{
  "Version":"2012-10-17",
  "Statement":[
    {
      "Sid":"AddPerm",
      "Effect":"Allow",
      "Principal": "*",
      "Action":["s3:GetObject"],
      "Resource":["arn:aws:s3:::2020MyExampleBucket/*"]
    }
  ]
}

That is all for this tutorial and I hope the article served you whatever you were looking for. Happy Learning and do not forget to share!

3. Summary

In this section, we learned the following:

Introduction to AWS
Introduction to AWS S3 and working with the S3 bucket

Working with Amazon S3 Buckets