Read Write Request Units for DynamoDB Table and Index — LSI and GSI

DynamoDB is a simple, scalable, serverless key-value store based database. Being serverless, it’s used more often with lambdas and is billed per usage in terms of request units — Read Request Units (RRU) and Write Request Units (WRU). Refer here for more details

• For each 4KB, it consumes 0.5 RRU for eventual read, 1 RRU for consistent read, 2 RRU for transactional read.
• For each 1KB, it consumes 1 WRU for standard writes and 2 WRU for transactional writes.

But if I have index, how does these units are calculated? If I do not project all attributes, do I still billed? LSI uses same partition as base table. So do I still billed for LSI?

Lets understand how it is billed for read and writes for different types of indices with different projections.

I created a table with 2 LSI and 2 GSI as with different projections as mentioned below.

DB Design with Local and Global Secondary Index

Create

In this, we use only standard writes which takes 1 WRU for each 1KB

I inserted 15k records simply running the below in for loop. This item sized around 87 bytes. In real, it would be more than this.

const create = (i) => new PutCommand({
  TableName,
  Item: {
    tenantId: 'tenant-a',
    userId: 'ta-user-' + i,
    appId: 'ta-appln-' + i,
    subjecId: 'sub' + i,
    email: i + '@gmail.com',
  },
  ReturnConsumedCapacity: 'INDEXES'
});

Every write takes 5 WRU

• 1 for base table
• 2 for LSI (1 for each)
• 2 for GSI (1 for each)

Update

new UpdateCommand({
  TableName,
  Key: {
    tenantId: 'tenant-a',
    userId: 'ta-user-1',
  },
  UpdateExpression: "SET username = :username",
  ExpressionAttributeValues: {
    ":username": "user-one",
  },
  ReturnValues: "ALL_NEW",
  ReturnConsumedCapacity: 'INDEXES'
})

In this, a new attribute is added, so it would impact only those indices that project all attributes. So this takes, 3 WRU

• 1 for base table
• 1 for “app-index” (LSI)
• 1 for “email-gsi-index” (GSI)

Update with same values

I reran the same update query, there were zero writes made to any index as there are no changes, but it takes 1 WRU for base table.

Query

Numbers to remember

• Each Scan/Query operation is limited to 1MB
• Eventual Read, takes 0.5 RRU for each 4KB, (also can say 8KB = 1RRU). So each Scan/Query operation can have 128 RRU (1024/8).
• Maximum Item size in DynamoDB is 400KB
• Each Scan/Query can return (1MB / avg item size) items. In my case, the average item size is very small 87 bytes. So I can get around 12k items (1MB / 87 bytes) in single Scan/Query

Note: consistent read, takes 1 RRU for each 4KB. So consistent read can have in 256 RRU (1024/4)

Let’s try

new QueryCommand({
  TableName,
  KeyConditionExpression: "tenantId = :tenantId",
  ExpressionAttributeValues: {
    ":tenantId": "tenant-a"
  },
  ReturnConsumedCapacity: 'INDEXES'
});

{
  "ConsumedCapacity": {
    "CapacityUnits": 128.5,
    "Table": {
      "CapacityUnits": 128.5
    },
    "TableName": "DDB_Analysis"
  },
  "Items": [ ... ],
  "Count": 11997,
  "LastEvaluatedKey": {
    "userId": "ta-user-7295",
    "tenantId": "tenant-a"
  },
  "ScannedCount": 11997
}

As expected, single query results in 11997 records consuming 128.5 RRU.

Pagination: Continuing the query, returns the remaining 3003 items totaling 15k.

new QueryCommand({
  TableName,
  KeyConditionExpression: "tenantId = :tenantId",
  ExpressionAttributeValues: {
    ":tenantId": "tenant-a"
  },
  ExclusiveStartKey: {
    "userId": "ta-user-7295",
    "tenantId": "tenant-a"
  },
  ReturnConsumedCapacity: 'INDEXES'
});

{
  "ConsumedCapacity": {
    "CapacityUnits": 31.5,
    "Table": {
      "CapacityUnits": 31.5
    },
    "TableName": "DDB_Analysis"
  },
  "Items": [ ... ],
  "Count": 3003,
  "ScannedCount": 3003
}

Query with Index

You can choose to project “ALL”, “KEYS_ONLY”, “INCLUDE” specific attributes. So if ALL is not chosen, then the item size should be smaller in index than in base table and should return more number of items.

new QueryCommand({
  TableName,
  IndexName: "email-index",
  KeyConditionExpression: "tenantId = :tenantId",
  ExpressionAttributeValues: {
    ":tenantId": "tenant-a"
  },
  ReturnConsumedCapacity: 'INDEXES'
});

• email-index → 14062 items and uses 92 RRU
• app-index → 10521 items and uses 113 RRU
• base table → 11997 items and uses 128.5 RRU

Scan with Index

const scan = () => new ScanCommand({
  TableName,
  IndexName: "email-gsi-index",
  ReturnConsumedCapacity: 'INDEXES'
});

• email-gsi-index → all 15000 items with 99.5 RRU
• email-index → 14062 items and uses 92.5 RRU
• app-index → 10521 items and uses 113.5 RRU
• base table → 11997 items and uses 129 RRU

Since I have only one value for partition key, Scan and Query all operation are same in this case. The resultant count are exactly same, but scan operation incurs 0.5 RRU extra.

Conclusion

Having N indices (both LSI and GSI) would incur N times the write cost and storage cost. The cost just simply multiplies.

Updating an attribute(s) will incur cost for the index that projects the attribute and will not incur cost for the index that doesn’t project that attribute. If you project ALL in the index, then it would incur N times the cost, every time you update.

Scanning / querying by index with less projected attributes, will results in more number of items.

Create Index only for required cases. Choose the keys and project only the attributes that are required.

Happy Indexing!