Amazon AWS

Getting Started with Python and AWS DynamoDB

Hello readers, in this tutorial, we will perform some basic CRUD operation in AWS DynamoDb using boto3 in Python programming language. But before digging deep, let us make an introduction to AWS DynamoDb.

1. What is AWS DynamoDb?

  • DynamoDb is a fast, scalable, and distributed NoSQL database which is schema-less
  • Offers automatic scaling, availability, and durability
  • Does not require to provision a database. Simply create a table and specify the read and write capacity units. These units automatically scale to meet the scaling needs
  • Offers an expensive serverless mode
  • Heavily used for cases like user profiles, storing session state, shopping cart, and high read/write applications
  • Offers ACID compliance through DynamoDB transactions
  • Data is synchronously replicated across 3 availability zones in a region
  • Provides two read models i.e.
    • Eventual consistent reads are default in nature wherein it maximizes the read throughput. This consistency read might not reflect the results instantly as the consistency across all copies are reached within 1 second
    • Strong consistent reads return the result that reflects all writes that received a successful response before reading (faster consistency)
  • DynamoDB Table’s
    • Consists of the item(s) and its attributes (i.e. key-value pair)
    • Need a mandatory Primary Key which should be unique
    • Other than the Primary Key, the entire table is schema-less where we do not define other attributes or types. Each item can have unique values
    • Max 400KB per item in a table
  • DynamoDB Keys consist of two parts – Mandatory Partition Key and Optional Sort Key
  • DynamoDB Query vs Scan –
    • Query: Search using a partition key attribute and a distinct value to search. Results are sorted by the Primary Key
    • Scan: Reads every item in the table and is expensive compared to the Query. Returns all attributes by default and the items can be filtered using expressions
  • DynamoDB charges for reading, writing, and storing data in your DynamoDB tables, along with any optional features you choose to enable

1.1 Setting up Python

If someone needs to go through the Python installation on Windows, please watch this link. To start with this tutorial, I am hoping that readers at present have the python installed.

2. Getting Started with Python and AWS DynamoDB

I am using JetBrains PyCharm as my preferred IDE. Readers are free to choose the IDE of their choice.

2.1 Application Pre-requisite

To proceed with this tutorial we need an AWS CLI IAM user. If someone needs to go through the process of creating an IAM user and attaching the Administrator Access policy, please watch this video. Readers are free to choose the DynamoDB Full Access policy if they want to allow access to the CLI user for the AWS DynamoDB service only.

2.2 DynamoDB CRUD Operations using Python

Add the AWS CLI user details such as aws_access_key_id, aws_secret_access_key, and region to the AWS credentials file for performing these operations. In our case, we have selected the default region as ap-south-1 wherein we will perform these operations.

2.2.1 Create the DynamoDB Table

To create the DynamoDB table open the DynamoDB service in the AWS console and select the option to Create a table. Fill in the table name and primary key as shown in Fig. 1 and keep all other details as default. Click the Create button.

python dynamodb - Create a table
Fig. 1: Create a table

If everything goes well the table with the name songs would be created successfully.

2.2.2 Put operation in DynamoDB Table

To perform the put operation in the DynamoDB table we will use the following Python script.

create_items.py

01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
import json
 
import boto3
 
# getting the dynamodb resource
dynamodb = boto3.resource('dynamodb')
 
table_name = 'songs'
# resource representing a dynamodb table
table = dynamodb.Table(table_name)
 
 
def create_data():
    total_items = 0
    with open('songs.json') as datafile:
        songs = json.load(datafile)
    for index, song in enumerate(songs):
        # print(song)
 
        item = {
            "song_id": str(index + 1),
            "name": song['name'],
            "artist": song['artist'],
            "publisher": song['publisher'],
            "priceInUsd": song['priceInUsd']
        }
        # print(item)
 
        print('Saving item id = {} to the dB'.format(item['song_id']))
        # put_item(...)
        # Creates a new item or replaces an old item with a new item. If an item that has the same primary key as the
        # new item already exists in the specified table, the new item completely replaces the existing item.
        response = table.put_item(
            Item=item
        )
 
        if response['ResponseMetadata']['HTTPStatusCode'] == 200:
            print('Item saved with status code = {} OK\n'.format(response['ResponseMetadata']['HTTPStatusCode']))
            total_items = total_items + 1
        else:
            print('Item id = {} could be saved in the dB\n'.format(item['song_id']))
 
    print("Total {} item(s) saved to the dB".format(total_items))
 
 
create_data()

If everything goes well the following output will be shown to the user in the IDE console.

Console logs

01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
-- CREATE ITEMS
Saving item id = 1 to the dB
Item saved with status code = 200 OK
 
Saving item id = 2 to the dB
Item saved with status code = 200 OK
 
Saving item id = 3 to the dB
Item saved with status code = 200 OK
 
Saving item id = 4 to the dB
Item saved with status code = 200 OK
 
Saving item id = 5 to the dB
Item saved with status code = 200 OK
 
Total 5 item(s) saved to the dB

2.2.3 Get all Items operation in DynamoDB Table

To perform the get all items operation in the DynamoDB table we will use the following Python script.

get_all_items.py

01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
import boto3
 
# getting the dynamodb resource
dynamodb = boto3.resource('dynamodb')
 
table_name = 'songs'
# resource representing a dynamodb table
table = dynamodb.Table(table_name)
 
 
def get_all_items():
    # scan()
    # Returns one or more items and item attributes by accessing every item in a table or a secondary index.
    # To have dynamodb return fewer items, you can provide a FilterExpression operation.
    response = table.scan()
    items = response['Items']
 
    print("Printing the data from the table.")
 
    for item in items:
        print(item)
        print('\n')
 
 
get_all_items()

If everything goes well the following output will be shown to the user in the IDE console.

Console logs

01
02
03
04
05
06
07
08
09
10
11
-- GET ALL ITEMS
Printing the data from the table.
{'artist': 'Romero Allen', 'song_id': '2', 'priceInUsd': Decimal('392'), 'publisher': 'QUAREX', 'name': 'Atomic Dim'}
 
{'artist': 'Hilda Barnes', 'song_id': '1', 'priceInUsd': Decimal('161'), 'publisher': 'LETPRO', 'name': 'Almond Dutch'}
 
{'artist': 'May Tanner', 'song_id': '5', 'priceInUsd': Decimal('146'), 'publisher': 'CHORIZON', 'name': 'Cadmium Cultured'}
 
{'artist': 'Chang Vance', 'song_id': '4', 'priceInUsd': Decimal('324'), 'publisher': 'DIGIAL', 'name': 'Aureolin Dodger'}
 
{'artist': 'Hall Ramos', 'song_id': '3', 'priceInUsd': Decimal('370'), 'publisher': 'BRAINQUIL', 'name': 'Carmine Drab'}

2.2.4 Get by Id operation in DynamoDB Table

To perform the get by id operation in the DynamoDB table we will use the following Python script.

get_item.py

01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
import boto3
 
# getting the dynamodb resource
dynamodb = boto3.resource('dynamodb')
 
table_name = 'songs'
# resource representing a dynamodb table
table = dynamodb.Table(table_name)
 
songId = '5'
 
 
def get_item():
    print('Fetching item id = {} from the dB\n'.format(songId))
 
    # get_item(...)
    # get a single item from the table.
    response = table.get_item(
        Key={
            'song_id': songId
        }
    )
 
    if 'Item' in response:
        print(response['Item'])
    else:
        print('Item id = {} not found in the dB'.format(songId))
 
 
get_item()

If everything goes well the following output will be shown to the user in the IDE console.

Console logs

1
2
3
4
-- GET ITEM
Fetching item id = 5 from the dB
 
{'artist': 'May Tanner', 'song_id': '5', 'priceInUsd': Decimal('146'), 'publisher': 'CHORIZON', 'name': 'Cadmium Cultured'}

In case the item is not found the database the output message saying Item not found will be shown to the user in the IDE console.

2.2.5 Update by Id operation in DynamoDB Table

To perform the update by id operation in the DynamoDB table we will use the following Python script.

update_item.py

01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
import boto3
 
# getting the dynamodb resource
dynamodb = boto3.resource('dynamodb')
 
table_name = 'songs'
# resource representing a dynamodb table
table = dynamodb.Table(table_name)
 
songId = '5'
 
 
def update_item():
    print('Updating item id = {} from the dB\n'.format(songId))
 
    # get_item(...)
    # get the item from the table.
    response = table.get_item(
        Key={
            'song_id': songId
        }
    )
 
    if 'Item' in response:
        # update_item(...)
        # update an item from the table based on the key.
        response = table.update_item(
            Key={
                'song_id': songId
            },
            UpdateExpression="set artist = :g",
            ExpressionAttributeValues={
                ':g': "Butterfly"
            }
        )
 
        if response['ResponseMetadata']['HTTPStatusCode'] == 200:
            print('Item updated with status code = {} OK'.format(response['ResponseMetadata']['HTTPStatusCode']))
        else:
            print('Some error occurred while updating the item from the dB')
    else:
        print('Item id = {} not found in the dB'.format(songId))
 
 
update_item()

If everything goes well the following output will be shown to the user in the IDE console.

Console logs

1
2
3
4
-- UPDATE ITEM
Updating item id = 5 from the dB
 
Item updated with status code = 200 OK

2.2.6 Delete by Id operation in DynamoDB Table

To perform the delete by id operation in the DynamoDB table we will use the following Python script.

delete_item.py

01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
import boto3
 
# getting the dynamodb resource
dynamodb = boto3.resource('dynamodb')
 
table_name = 'songs'
# resource representing a dynamodb table
table = dynamodb.Table(table_name)
 
songId = '5'
 
 
def delete_item():
    print('Deleting item id = {} from the dB\n'.format(songId))
 
    # get_item(...)
    # get the item from the table.
    response = table.get_item(
        Key={
            'song_id': songId
        }
    )
 
    if 'Item' in response:
        # delete_item(...)
        # delete an item from the table based on the key.
        response = table.delete_item(
            Key={
                'song_id': songId
            }
        )
 
        if response['ResponseMetadata']['HTTPStatusCode'] == 200:
            print('Item deleted with status code = {} OK'.format(response['ResponseMetadata']['HTTPStatusCode']))
        else:
            print('Some error occurred while deleting the item from the dB')
    else:
        print('Item id = {} not found in the dB'.format(songId))
 
 
delete_item()

If everything goes well the following output will be shown to the user in the IDE console.

Console logs

1
2
3
4
-- DELETE ITEM
Deleting item id = 5 from the dB
 
Item deleted with status code = 200 OK

2.2.7 Delete all Items operation in DynamoDB Table

To perform the delete all items operation in the DynamoDB table we will use the following Python script.

delete_all_items.py

01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
import boto3
 
# getting the dynamodb resource
dynamodb = boto3.resource('dynamodb')
 
table_name = 'songs'
# resource representing a dynamodb table
table = dynamodb.Table(table_name)
 
 
def delete_all_items():
    deleted_items = 0
 
    # scan()
    # Returns one or more items and item attributes by accessing every item in a table or a secondary index.
    # To have dynamodb return fewer items, you can provide a FilterExpression operation.
    response = table.scan()
    with table.batch_writer() as batch:
        for song in response['Items']:
            batch.delete_item(
                Key={
                    'song_id': song['song_id'],
                }
            )
            deleted_items = deleted_items + 1
 
    if deleted_items > 0:
        print('{} item(s) are successfully deleted from the dB'.format(deleted_items))
    else:
        print('Either table is empty or some error occurred while deleting the items from the dB')
 
 
delete_all_items()

If everything goes well the following output will be shown to the user in the IDE console.

Console logs

1
2
-- DELETE ALL ITEMS
4 item(s) are successfully deleted from the dB

That is all for this tutorial and I hope the article served you with whatever you were looking for. Happy Learning and do not forget to share!

3. Summary

In this tutorial, we learned:

  • An introduction to AWS DynamoDB
  • Python scripts to perform the basic CRUD operation in the DynamoDB
  • If readers are interested to go through the details boto3 documentation for DynamoDB they can refer the documentation available at this link

You can download the source code of this tutorial from the Downloads section.

4. Download the Project

This was an example of interacting with AWS DynamoDB using Python programming language.

Download
You can download the full source code of this example here: Getting Started with Python and AWS DynamoDB

Yatin

An experience full-stack engineer well versed with Core Java, Spring/Springboot, MVC, Security, AOP, Frontend (Angular & React), and cloud technologies (such as AWS, GCP, Jenkins, Docker, K8).
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

0 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Back to top button