TCP #62: One Architecture Pattern to Solve Your AWS Scaling Problems
Master CQRS on AWS: From Theory to Production-Ready Implementation
You can also read my newsletters from the Substack mobile app and be notified when a new issue is available.
Become a Founding Member
As a founding member, you will receive:
Everything included in paid subscriber benefits + exclusive toolkits and templates.
High-quality content from my 11+ years of industry experience, where I solve specific business problems in the real world using AWS Cloud. Learn from my actionable insights, strategies, and decision-making process.
Quarterly report on emerging trends, AWS updates, and cloud innovations with strategic insights.
Public recognition in the newsletter under the “Founding Member Spotlight” section.
Early access to deep dives, case studies, and special reports before they’re released to paid subscribers.
The Command Query Responsibility Segregation (CQRS) pattern represents a powerful architectural approach that separates your application's read and write operations.
This separation allows each side to be optimized independently, addressing different performance requirements, scaling needs, and consistency models.
AWS serverless architecture provides an ideal foundation for implementing CQRS due to its inherent scalability, pay-per-use pricing model, and easy service integration.
By leveraging Lambda for compute, DynamoDB for storage, and EventBridge for event routing, we can create a flexible, maintainable CQRS implementation without managing infrastructure.
In today’s newsletter issue, I will walk you through a complete implementation strategy, from understanding core concepts to deploying a production-ready solution.
Understanding CQRS Fundamentals
Command vs. Query Separation
In traditional architectures, the same data model handles both read and write operations. CQRS takes a different approach:
Commands are operations that change state (create, update, delete). They are optimized for write performance and data integrity.
Queries: Operations that read state without modification. These are optimized for read performance and tailored to specific use cases.
By separating these concerns, we can design each side to excel at its specific task.
Benefits of CQRS
Independent Scaling: Scale read and write workloads separately based on their unique demands
Performance Optimization: Tailor data models for specific read patterns without compromising write operations
Simplified Models: Query models can be denormalized for specific use cases, avoiding complex joins
Enhanced Security: Apply different permission sets to command and query operations
Common Challenges and Misconceptions
Eventual Consistency: CQRS often introduces eventual consistency, which requires careful consideration in UX design
Complexity: Implementing CQRS adds complexity that may not be justified for simple applications
Synchronization: Maintaining multiple data representations requires reliable event propagation
AWS Services Overview
Lambda for Serverless Compute
AWS Lambda provides the perfect compute platform for CQRS implementations:
Stateless execution model aligns with command and query handlers
Automatic scaling based on workload
Different runtime configurations for command vs. query functions
Pay only for actual compute time used
DynamoDB for Data Persistence
DynamoDB offers several benefits for CQRS:
Separate tables for write and read models
Single-digit millisecond performance at any scale
Fine-grained access control
Different throughput allocation for command vs. query tables
Flexible schema for varied read models
EventBridge for Event Routing
EventBridge serves as the nervous system connecting commands to their effects:
Decouples command processing from read model updates
Enables event filtering and routing
Provides reliable delivery with retry capabilities
Supports multiple subscribers for the same events
Enables event archiving for replay and debugging
Integration Points
These services connect through well-defined integration points:
Lambda functions process commands and update the command store
Events are published to EventBridge upon state changes
EventBridge routes events to the appropriate Lambda functions
Projection Lambdas update read models in DynamoDB
Query Lambdas access read models and return results
High-Level Architecture
Command Flow Walkthrough
Client submits a command via the API Gateway
Command Handler Lambda validates the command
If valid, the command is processed, and state changes are saved to the Command Store
An event is published to EventBridge reflecting the state change
Success/failure response is returned to the client
Query Flow Walkthrough
Client requests data via API Gateway
Query Lambda receives the request
Lambda queries the appropriate read model(s) in DynamoDB
Results are transformed to the required format and returned to the client
Event Propagation Process
Command Handler publishes domain events to EventBridge
EventBridge routes events to subscribed Lambda functions
Projection Lambdas receive events and update read models accordingly
Multiple projections can be updated from the same event
Implementation Guide
Setting up DynamoDB Tables
Command Store
{
"TableName": "CommandStore",
"KeySchema": [
{ "AttributeName": "aggregateId", "KeyType": "HASH" },
{ "AttributeName": "version", "KeyType": "RANGE" }
],
"AttributeDefinitions": [
{ "AttributeName": "aggregateId", "AttributeType": "S" },
{ "AttributeName": "version", "AttributeType": "N" }
],
"ProvisionedThroughput": {
"ReadCapacityUnits": 5,
"WriteCapacityUnits": 20
}
}Read Model (Example: Product Catalog)
{
"TableName": "ProductCatalog",
"KeySchema": [
{ "AttributeName": "productId", "KeyType": "HASH" }
],
"AttributeDefinitions": [
{ "AttributeName": "productId", "AttributeType": "S" },
{ "AttributeName": "category", "AttributeType": "S" }
],
"GlobalSecondaryIndexes": [
{
"IndexName": "CategoryIndex",
"KeySchema": [
{ "AttributeName": "category", "KeyType": "HASH" }
],
"Projection": {
"ProjectionType": "ALL"
},
"ProvisionedThroughput": {
"ReadCapacityUnits": 20,
"WriteCapacityUnits": 5
}
}
],
"ProvisionedThroughput": {
"ReadCapacityUnits": 50,
"WriteCapacityUnits": 10
}
}Creating Lambda Functions for Command Handling
Command Handler Structure
exports.handler = async (event) => {
try {
// Parse and validate command
const command = JSON.parse(event.body);
validateCommand(command);
// Retrieve current state
const currentState = await getAggregateState(command.aggregateId);
// Apply business rules and generate events
const events = processCommand(command, currentState);
// Store events and update state
await storeEventsAndUpdateState(command.aggregateId, events, currentState.version);
// Publish events to EventBridge
await publishEvents(events);
return {
statusCode: 200,
body: JSON.stringify({ success: true, events: events })
};
} catch (error) {
return {
statusCode: error.statusCode || 500,
body: JSON.stringify({ error: error.message })
};
}
};Optimistic Concurrency Control
async function storeEventsAndUpdateState(aggregateId, events, expectedVersion) {
const params = {
TransactItems: events.map((event, index) => ({
Put: {
TableName: 'CommandStore',
Item: {
aggregateId: aggregateId,
version: expectedVersion + index + 1,
eventType: event.type,
data: event.data,
timestamp: new Date().toISOString()
},
ConditionExpression: 'attribute_not_exists(version)'
}
}))
};
try {
await dynamoDB.transactWrite(params).promise();
} catch (error) {
if (error.code === 'TransactionCanceledException') {
throw new Error('Concurrency conflict detected. Please retry with latest version.');
}
throw error;
}
}Implementing Read-Model Projectors
Event Subscriber Lambda
exports.handler = async (event) => {
const results = [];
for (const record of event.Records) {
const domainEvent = JSON.parse(record.body);
// Route to appropriate projector based on event type
switch (domainEvent.type) {
case 'ProductCreated':
results.push(await handleProductCreated(domainEvent));
break;
case 'ProductUpdated':
results.push(await handleProductUpdated(domainEvent));
break;
case 'ProductDeleted':
results.push(await handleProductDeleted(domainEvent));
break;
default:
console.log(`Unknown event type: ${domainEvent.type}`);
}
}
return { results };
};
async function handleProductCreated(event) {
const product = {
productId: event.data.productId,
name: event.data.name,
description: event.data.description,
price: event.data.price,
category: event.data.category,
lastUpdated: new Date().toISOString()
};
await dynamoDB.put({
TableName: 'ProductCatalog',
Item: product
}).promise();
return { success: true, productId: product.productId };
}Configuring EventBridge for Event Routing
Event Bus Configuration
{
"Name": "CQRSDomainEventBus",
"Description": "Event bus for domain events in CQRS architecture"
}Event Rules
{
"Name": "ProductEventsRule",
"EventPattern": {
"source": ["com.mycompany.product"],
"detail-type": ["ProductCreated", "ProductUpdated", "ProductDeleted"]
},
"Targets": [
{
"Id": "ProductProjectorTarget",
"Arn": "arn:aws:lambda:region:account-id:function:product-projector"
}
]
}Connecting the Components
Infrastructure as Code Example (AWS CDK)
import * as cdk from 'aws-cdk-lib';
import * as dynamodb from 'aws-cdk-lib/aws-dynamodb';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import * as events from 'aws-cdk-lib/aws-events';
import * as targets from 'aws-cdk-lib/aws-events-targets';
import * as apigateway from 'aws-cdk-lib/aws-apigateway';
export class CqrsInfrastructureStack extends cdk.Stack {
constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);
// Command Store Table
const commandStore = new dynamodb.Table(this, 'CommandStore', {
partitionKey: { name: 'aggregateId', type: dynamodb.AttributeType.STRING },
sortKey: { name: 'version', type: dynamodb.AttributeType.NUMBER },
billingMode: dynamodb.BillingMode.PAY_PER_REQUEST
});
// Read Model Table
const productCatalog = new dynamodb.Table(this, 'ProductCatalog', {
partitionKey: { name: 'productId', type: dynamodb.AttributeType.STRING },
billingMode: dynamodb.BillingMode.PAY_PER_REQUEST
});
productCatalog.addGlobalSecondaryIndex({
indexName: 'CategoryIndex',
partitionKey: { name: 'category', type: dynamodb.AttributeType.STRING },
projectionType: dynamodb.ProjectionType.ALL
});
// Event Bus
const eventBus = new events.EventBus(this, 'CQRSDomainEventBus', {
eventBusName: 'CQRSDomainEventBus'
});
// Command Handler Lambda
const commandHandler = new lambda.Function(this, 'CommandHandler', {
runtime: lambda.Runtime.NODEJS_18_X,
handler: 'index.handler',
code: lambda.Code.fromAsset('lambda/command-handler'),
environment: {
COMMAND_STORE_TABLE: commandStore.tableName,
EVENT_BUS_NAME: eventBus.eventBusName
}
});
commandStore.grantReadWriteData(commandHandler);
eventBus.grantPutEventsTo(commandHandler);
// Projector Lambda
const productProjector = new lambda.Function(this, 'ProductProjector', {
runtime: lambda.Runtime.NODEJS_18_X,
handler: 'index.handler',
code: lambda.Code.fromAsset('lambda/product-projector'),
environment: {
PRODUCT_CATALOG_TABLE: productCatalog.tableName
}
});
productCatalog.grantReadWriteData(productProjector);
// Event Rule
const productEventsRule = new events.Rule(this, 'ProductEventsRule', {
eventBus: eventBus,
eventPattern: {
source: ['com.mycompany.product'],
detailType: ['ProductCreated', 'ProductUpdated', 'ProductDeleted']
},
targets: [new targets.LambdaFunction(productProjector)]
});
// Query Lambda
const queryHandler = new lambda.Function(this, 'QueryHandler', {
runtime: lambda.Runtime.NODEJS_18_X,
handler: 'index.handler',
code: lambda.Code.fromAsset('lambda/query-handler'),
environment: {
PRODUCT_CATALOG_TABLE: productCatalog.tableName
}
});
productCatalog.grantReadData(queryHandler);
// API Gateway
const api = new apigateway.RestApi(this, 'CQRSAPI', {
restApiName: 'CQRS Service'
});
const commandsResource = api.root.addResource('commands');
commandsResource.addMethod('POST', new apigateway.LambdaIntegration(commandHandler));
const queriesResource = api.root.addResource('queries');
queriesResource.addResource('products').addMethod('GET', new apigateway.LambdaIntegration(queryHandler));
}
}Best Practices
Handling Eventual Consistency
Provide Feedback to Users:
Acknowledge command receipt immediately
Set expectations for when changes will be visible
Optimistic UI Updates:
Update UI immediately after command submission
Refresh data after a short delay
Versioning and Timestamps:
Include version numbers or timestamps in responses
Allow clients to poll until versions match
Command Tracing:
Assign unique identifiers to commands
Enable status checking for long-running operations
Optimizing DynamoDB for Read vs. Write Operations
Write Optimization (Command Store):
Use a simple key structure for efficient writes
Consider Time-To-Live (TTL) for command expiry
Use transactions for atomic operations
Consider write sharding for high-throughput workloads
Read Optimization (Read Models):
Create purpose-specific tables with denormalized data
Design Global Secondary Indexes (GSIs) for access patterns
Use sparse indexes for selective data subsets
Consider caching for frequently accessed data
Error Handling and Retry Mechanisms
Command Errors:
Validate commands early to catch errors
Use consistent error formats with appropriate HTTP status codes
Implement idempotency to support safe retries
Event Processing Errors:
Configure Dead Letter Queues (DLQs) for failed event processing
Implement event versioning for backward compatibility
Create monitoring alerts for projection failures
Recovery Strategies:
Implement event replay capability
Create administration tools for manual intervention
Consider using Step Functions for complex command workflows
Final Thoughts
Implementing CQRS using AWS Lambda, DynamoDB, and EventBridge provides a robust, scalable, and cost-effective architecture for modern applications.
The separation of read and write concerns allows each side to be optimized independently, providing significant performance benefits and flexibility.
Key takeaways:
Start with a clear understanding of your domain and data access patterns
Design events as the source of truth in your system
Leverage AWS-managed services to reduce operational overhead
Implement proper monitoring and error handling
Address eventual consistency in your application design
SPONSOR US
The Cloud Playbook is now offering sponsorship slots in each issue. If you want to feature your product or service in my newsletter, explore my sponsor page
That’s it for today!
Did you enjoy this newsletter issue?
Share with your friends, colleagues, and your favorite social media platform.
Until next week — Amrut
Get in touch
You can find me on LinkedIn or X.
If you wish to request a topic you would like to read, you can contact me directly via LinkedIn or X.