Getting Started with Python and AWS DynamoDB

YatinSeptember 18th, 2020Last Updated: September 17th, 2020

0 95 9 minutes read

Hello readers, in this tutorial, we will perform some basic CRUD operation in AWS DynamoDb using boto3 in Python programming language. But before digging deep, let us make an introduction to AWS DynamoDb.

1. What is AWS DynamoDb?

DynamoDb is a fast, scalable, and distributed NoSQL database which is schema-less
Offers automatic scaling, availability, and durability
Does not require to provision a database. Simply create a table and specify the read and write capacity units. These units automatically scale to meet the scaling needs
Offers an expensive serverless mode
Heavily used for cases like user profiles, storing session state, shopping cart, and high read/write applications
Offers ACID compliance through DynamoDB transactions
Data is synchronously replicated across 3 availability zones in a region
Provides two read models i.e.
- Eventual consistent reads are default in nature wherein it maximizes the read throughput. This consistency read might not reflect the results instantly as the consistency across all copies are reached within 1 second
- Strong consistent reads return the result that reflects all writes that received a successful response before reading (faster consistency)
DynamoDB Table’s
- Consists of the item(s) and its attributes (i.e. key-value pair)
- Need a mandatory Primary Key which should be unique
- Other than the Primary Key, the entire table is schema-less where we do not define other attributes or types. Each item can have unique values
- Max 400KB per item in a table
DynamoDB Keys consist of two parts – Mandatory Partition Key and Optional Sort Key
DynamoDB Query vs Scan –
- Query: Search using a partition key attribute and a distinct value to search. Results are sorted by the Primary Key
- Scan: Reads every item in the table and is expensive compared to the Query. Returns all attributes by default and the items can be filtered using expressions
DynamoDB charges for reading, writing, and storing data in your DynamoDB tables, along with any optional features you choose to enable

1.1 Setting up Python

If someone needs to go through the Python installation on Windows, please watch this link. To start with this tutorial, I am hoping that readers at present have the python installed.

2. Getting Started with Python and AWS DynamoDB

I am using JetBrains PyCharm as my preferred IDE. Readers are free to choose the IDE of their choice.

2.1 Application Pre-requisite

To proceed with this tutorial we need an AWS CLI IAM user. If someone needs to go through the process of creating an IAM user and attaching the Administrator Access policy, please watch this video. Readers are free to choose the DynamoDB Full Access policy if they want to allow access to the CLI user for the AWS DynamoDB service only.

2.2 DynamoDB CRUD Operations using Python

Add the AWS CLI user details such as aws_access_key_id, aws_secret_access_key, and region to the AWS credentials file for performing these operations. In our case, we have selected the default region as ap-south-1 wherein we will perform these operations.

2.2.1 Create the DynamoDB Table

To create the DynamoDB table open the DynamoDB service in the AWS console and select the option to Create a table. Fill in the table name and primary key as shown in Fig. 1 and keep all other details as default. Click the Create button.

python dynamodb - Create a table — Fig. 1: Create a table

If everything goes well the table with the name songs would be created successfully.

2.2.2 Put operation in DynamoDB Table

To perform the put operation in the DynamoDB table we will use the following Python script.

create_items.py

import json
 
import boto3
 
# getting the dynamodb resource
dynamodb = boto3.resource('dynamodb')
 
table_name = 'songs'
# resource representing a dynamodb table
table = dynamodb.Table(table_name)
 
 
def create_data():
    total_items = 0
    with open('songs.json') as datafile:
        songs = json.load(datafile)
    for index, song in enumerate(songs):
        # print(song)
 
        item = {
            "song_id": str(index + 1),
            "name": song['name'],
            "artist": song['artist'],
            "publisher": song['publisher'],
            "priceInUsd": song['priceInUsd']
        }
        # print(item)
 
        print('Saving item id = {} to the dB'.format(item['song_id']))
        # put_item(...)
        # Creates a new item or replaces an old item with a new item. If an item that has the same primary key as the
        # new item already exists in the specified table, the new item completely replaces the existing item.
        response = table.put_item(
            Item=item
        )
 
        if response['ResponseMetadata']['HTTPStatusCode'] == 200:
            print('Item saved with status code = {} OK\n'.format(response['ResponseMetadata']['HTTPStatusCode']))
            total_items = total_items + 1
        else:
            print('Item id = {} could be saved in the dB\n'.format(item['song_id']))
 
    print("Total {} item(s) saved to the dB".format(total_items))
 
 
create_data()

If everything goes well the following output will be shown to the user in the IDE console.

Console logs

-- CREATE ITEMS
Saving item id = 1 to the dB
Item saved with status code = 200 OK
 
Saving item id = 2 to the dB
Item saved with status code = 200 OK
 
Saving item id = 3 to the dB
Item saved with status code = 200 OK
 
Saving item id = 4 to the dB
Item saved with status code = 200 OK
 
Saving item id = 5 to the dB
Item saved with status code = 200 OK
 
Total 5 item(s) saved to the dB

2.2.3 Get all Items operation in DynamoDB Table

To perform the get all items operation in the DynamoDB table we will use the following Python script.

get_all_items.py

import boto3
 
# getting the dynamodb resource
dynamodb = boto3.resource('dynamodb')
 
table_name = 'songs'
# resource representing a dynamodb table
table = dynamodb.Table(table_name)
 
 
def get_all_items():
    # scan()
    # Returns one or more items and item attributes by accessing every item in a table or a secondary index.
    # To have dynamodb return fewer items, you can provide a FilterExpression operation.
    response = table.scan()
    items = response['Items']
 
    print("Printing the data from the table.")
 
    for item in items:
        print(item)
        print('\n')
 
 
get_all_items()

If everything goes well the following output will be shown to the user in the IDE console.

Console logs

-- GET ALL ITEMS 
Printing the data from the table.
{'artist': 'Romero Allen', 'song_id': '2', 'priceInUsd': Decimal('392'), 'publisher': 'QUAREX', 'name': 'Atomic Dim'}
 
{'artist': 'Hilda Barnes', 'song_id': '1', 'priceInUsd': Decimal('161'), 'publisher': 'LETPRO', 'name': 'Almond Dutch'}
 
{'artist': 'May Tanner', 'song_id': '5', 'priceInUsd': Decimal('146'), 'publisher': 'CHORIZON', 'name': 'Cadmium Cultured'}
 
{'artist': 'Chang Vance', 'song_id': '4', 'priceInUsd': Decimal('324'), 'publisher': 'DIGIAL', 'name': 'Aureolin Dodger'}
 
{'artist': 'Hall Ramos', 'song_id': '3', 'priceInUsd': Decimal('370'), 'publisher': 'BRAINQUIL', 'name': 'Carmine Drab'}

2.2.4 Get by Id operation in DynamoDB Table

To perform the get by id operation in the DynamoDB table we will use the following Python script.

get_item.py

import boto3
 
# getting the dynamodb resource
dynamodb = boto3.resource('dynamodb')
 
table_name = 'songs'
# resource representing a dynamodb table
table = dynamodb.Table(table_name)
 
songId = '5'
 
 
def get_item():
    print('Fetching item id = {} from the dB\n'.format(songId))
 
    # get_item(...)
    # get a single item from the table.
    response = table.get_item(
        Key={
            'song_id': songId
        }
    )
 
    if 'Item' in response:
        print(response['Item'])
    else:
        print('Item id = {} not found in the dB'.format(songId))
 
 
get_item()

If everything goes well the following output will be shown to the user in the IDE console.

Console logs

-- GET ITEM
Fetching item id = 5 from the dB
 
{'artist': 'May Tanner', 'song_id': '5', 'priceInUsd': Decimal('146'), 'publisher': 'CHORIZON', 'name': 'Cadmium Cultured'}

In case the item is not found the database the output message saying Item not found will be shown to the user in the IDE console.

2.2.5 Update by Id operation in DynamoDB Table

To perform the update by id operation in the DynamoDB table we will use the following Python script.

update_item.py

import boto3
 
# getting the dynamodb resource
dynamodb = boto3.resource('dynamodb')
 
table_name = 'songs'
# resource representing a dynamodb table
table = dynamodb.Table(table_name)
 
songId = '5'
 
 
def update_item():
    print('Updating item id = {} from the dB\n'.format(songId))
 
    # get_item(...)
    # get the item from the table.
    response = table.get_item(
        Key={
            'song_id': songId
        }
    )
 
    if 'Item' in response:
        # update_item(...)
        # update an item from the table based on the key.
        response = table.update_item(
            Key={
                'song_id': songId
            },
            UpdateExpression="set artist = :g",
            ExpressionAttributeValues={
                ':g': "Butterfly"
            }
        )
 
        if response['ResponseMetadata']['HTTPStatusCode'] == 200:
            print('Item updated with status code = {} OK'.format(response['ResponseMetadata']['HTTPStatusCode']))
        else:
            print('Some error occurred while updating the item from the dB')
    else:
        print('Item id = {} not found in the dB'.format(songId))
 
 
update_item()

If everything goes well the following output will be shown to the user in the IDE console.

Console logs

-- UPDATE ITEM
Updating item id = 5 from the dB
 
Item updated with status code = 200 OK

2.2.6 Delete by Id operation in DynamoDB Table

To perform the delete by id operation in the DynamoDB table we will use the following Python script.

delete_item.py

import boto3
 
# getting the dynamodb resource
dynamodb = boto3.resource('dynamodb')
 
table_name = 'songs'
# resource representing a dynamodb table
table = dynamodb.Table(table_name)
 
songId = '5'
 
 
def delete_item():
    print('Deleting item id = {} from the dB\n'.format(songId))
 
    # get_item(...)
    # get the item from the table.
    response = table.get_item(
        Key={
            'song_id': songId
        }
    )
 
    if 'Item' in response:
        # delete_item(...)
        # delete an item from the table based on the key.
        response = table.delete_item(
            Key={
                'song_id': songId
            }
        )
 
        if response['ResponseMetadata']['HTTPStatusCode'] == 200:
            print('Item deleted with status code = {} OK'.format(response['ResponseMetadata']['HTTPStatusCode']))
        else:
            print('Some error occurred while deleting the item from the dB')
    else:
        print('Item id = {} not found in the dB'.format(songId))
 
 
delete_item()

If everything goes well the following output will be shown to the user in the IDE console.

Console logs

-- DELETE ITEM
Deleting item id = 5 from the dB
 
Item deleted with status code = 200 OK

2.2.7 Delete all Items operation in DynamoDB Table

To perform the delete all items operation in the DynamoDB table we will use the following Python script.

delete_all_items.py

import boto3
 
# getting the dynamodb resource
dynamodb = boto3.resource('dynamodb')
 
table_name = 'songs'
# resource representing a dynamodb table
table = dynamodb.Table(table_name)
 
 
def delete_all_items():
    deleted_items = 0
 
    # scan()
    # Returns one or more items and item attributes by accessing every item in a table or a secondary index.
    # To have dynamodb return fewer items, you can provide a FilterExpression operation.
    response = table.scan()
    with table.batch_writer() as batch:
        for song in response['Items']:
            batch.delete_item(
                Key={
                    'song_id': song['song_id'],
                }
            )
            deleted_items = deleted_items + 1
 
    if deleted_items > 0:
        print('{} item(s) are successfully deleted from the dB'.format(deleted_items))
    else:
        print('Either table is empty or some error occurred while deleting the items from the dB')
 
 
delete_all_items()

If everything goes well the following output will be shown to the user in the IDE console.

Console logs

-- DELETE ALL ITEMS
4 item(s) are successfully deleted from the dB

That is all for this tutorial and I hope the article served you with whatever you were looking for. Happy Learning and do not forget to share!

3. Summary

In this tutorial, we learned:

An introduction to AWS DynamoDB
Python scripts to perform the basic CRUD operation in the DynamoDB
If readers are interested to go through the details boto3 documentation for DynamoDB they can refer the documentation available at this link

You can download the source code of this tutorial from the Downloads section.

4. Download the Project

This was an example of interacting with AWS DynamoDB using Python programming language.

Download
You can download the full source code of this example here: Getting Started with Python and AWS DynamoDB

Getting Started with Python and AWS DynamoDB

1. What is AWS DynamoDb?

1.1 Setting up Python

2. Getting Started with Python and AWS DynamoDB

2.1 Application Pre-requisite

2.2 DynamoDB CRUD Operations using Python

2.2.1 Create the DynamoDB Table

2.2.2 Put operation in DynamoDB Table

2.2.3 Get all Items operation in DynamoDB Table

2.2.4 Get by Id operation in DynamoDB Table

2.2.5 Update by Id operation in DynamoDB Table

2.2.6 Delete by Id operation in DynamoDB Table

2.2.7 Delete all Items operation in DynamoDB Table

3. Summary

4. Download the Project

Thank you!

Yatin

Thank you!

1. What is AWS DynamoDb?

1.1 Setting up Python

2. Getting Started with Python and AWS DynamoDB

2.1 Application Pre-requisite

2.2 DynamoDB CRUD Operations using Python

2.2.1 Create the DynamoDB Table

2.2.2 Put operation in DynamoDB Table

2.2.3 Get all Items operation in DynamoDB Table

2.2.4 Get by Id operation in DynamoDB Table

2.2.5 Update by Id operation in DynamoDB Table

2.2.6 Delete by Id operation in DynamoDB Table

2.2.7 Delete all Items operation in DynamoDB Table

3. Summary

4. Download the Project

Thank you!

Related Articles

Thank you!