Session 6 | API Gateway, Lambda, and DynamoDB
This Session's Setup
Open up the AWS Mangement Console and login to your AWS account using the admin user you created in the setup. Do not log into your root account.
At the top of your Management Console next to your username, you'll see "Ohio" listed as your region. Go ahead and change that to US East (N. Virginia).
Logistics and Expectations
- In this session, we'll dive into the practical side of creating a DynamoDB table and accessing it programatically in addition to how DynamoDB works. We will not be going over the theory or benefits/drawbacks of NoSQL databases. However, as always, we've included links to additional resources you can use to further your learning once you understand the fundamentals.
- Play your part in creating a safe and welcoming learning environment for your classmates.
Agenda and Goals
- Understand the purpose and basic functionality of DynamoDB
- Create and configure a DynamoDB table
- Get all table data from a DynamoDB table using an AWS SDK called from a Lambda function
- Get a single piece of data from a DynamoDB table using an AWS SDK called from a Lambda function
- Post data to a DynamoDB table inside of a Lamdba function
DynamoDB Overview
From AWS:
Amazon DynamoDB is a fully managed NoSQL database service that provides fast and predictable performance with seamless scalability. DynamoDB lets you offload the administrative burdens of operating and scaling a distributed database so that you don't have to worry about hardware provisioning, setup and configuration, replication, software patching, or cluster scaling.
DynamoDB is a good complement to Lambda and API Gateway since it's also serverless. Since DynamoDB is a NoSQL database, there are no relationships between tables, and therefore it is likely not a good choice for an application that is relationship-heavy.
Features of DynamoDB
- Like RDS, DynamoDB is a fully-managed database. This means that you do not need to do any provisioning, software management/patching, security, replication, scaling, etc.
- Because it is serverless, it can be considered "infinitely scalable"
- Extremely fast with low latency
- Automatically replicated across Availability Zones
Structure
With DynamoDB, you do not need to create a database. Instead, you create just a table. You can learn more about data modeling in the additional resources section, but generally you will find that one microservice would access one DynamoDB table. A "row" of data that you would insert into your table of data is called an item.
A DynamoDB table is key-value based and doesn't have a schema. Instead, you can create keys on the fly, (yes, it is very weird) and not all of the items in the database need to have the same keys. For example, these could be two items in the same table:
{
"id": "1",
"name": "Rachel",
"pets": [
{
"name": "Beeba",
"type": "kitten"
}
]
}
{
"id": "2",
"name": "Mike",
"address": "1234 Turing Ln."
}
A table needs to have a partition key specified at creation. This is the only field that a table must have. In the case above, the partition key would be id. If this partition key is unique to every item, then you can consider it your primary key. A partition key should be something where there are many possible values to avoid a "hot partition" issue.
In addition to a primary key, you can specify a sort key that Dynamo uses for ordering the data once it's in the partition. If your partition key is not unique, then the combination of the partition key + the sort key becomes your "composite primary key". The diagram below shows a sort key being used in conjunction with a partition key.
Either the partition key must be unique, OR the combination of the partition key + the sort key must be unique.
Each table that you create has one or more partitions which are 10-GB storage units internal to DynamoDB. When data is inserted into the database, the partition key is hashed (using an internal hashing function). The result of this hashing function determines the partition on which the data is stored. Therefore, the partition key is the determinant of where the data physically lives. Because of this, data that has the same partition key is going to be stored on the same partition.
Below is a Wikipedia diagram showing how a hashing function works:
Each table that you use has its own partitions, as shown in this AWS diagram below:
Together: Tables, partition keys, and partitions
We'll walk through creating a diagram together showing the process of hashing a partition key and storing data in a DynamoDB table.
This next bit of info won't be relevant to what we're doing today, but you'll need to know it for AWS exams: When setting up a table, you must specify your read and write capacity units, which indicate how often you plan to read or write data per second. One read capacity unit (RCU) represents 1 strongly consistent read at 4kb/sec (or 2 eventually consistent reads at 4kb/sec), and 1 write capacity unit (WCU) represents 1 write for 1kb/sec. The number of RCUs and WCUs that you set up will determine to the amount of money you pay. Alternatively, you can set up your DynamoDB table in "on-demand mode" which means that any size workload will be accomodated by automatically scaling up or down. This setup is good for unpredictable workloads.
Data in a DynamoDB table is automatically encrypted at rest.
Additional Resources
Creating a DynamoDB Table
Together: DynamoDB Tables and Items
In this section, we'll create a pets DynamoDB table with an id attribute (this will be both our partition key and our primary key) and a name attribute. Let's head into the Management Console.
Scans vs. Queries
In order to pull out data, you can either do a query or a scan. A query only searches by partition key, but because of this is much more efficient. It only needs to go to the one specific partition where the data lives, grab it, and return it. A scan on the other hand grabs ALL of the data from the table, then filters through it to return the data you're looking for.
Additional Resources
Workshop: Using Lambda to Access DynamoDB Data
Cleanup
- Delete the S3 bucket that is hosting your static files
- Delete all of your Lambda functions.
- Delete your API gateway.
- Delete your DynamoDB table.
- Delete any IAM roles related to your Lambda functions, API Gateway, or DynamoDB table.
Wrap Up
You finished the 12-hour AWS Fundamentals class! In true Turing form, we'll end with a feedback survey.