Filtering DynamoDB event for AWS Lambda functions
AWS Lambda now provides content filtering options DynamoDB as event sources. With this, you can filter the event to only include the data you want. This helps reduce traffic to customers’ Lambda functions, simplifies code, and reduces overall cost.
But AWS CDK is not yet able to support this. You can track the progress of this feature in the AWS CDK Lambda event filtering
How to filter DynamoDB event for AWS Lambda functions in CDK
Let’s look at an example of how to filter DynamoDB event for AWS Lambda functions in CDK.
// 1. Create a DynamoDB table const dynamoTable = new dynamodb.Table(this, 'DynamoTable', partitionKey: name: 'id', type: dynamodb.AttributeType.STRING , billingMode: dynamodb.BillingMode.PAY_PER_REQUEST, removalPolicy: RemovalPolicy.DESTROY, stream: dynamodb.StreamViewType.NEW_AND_OLD_IMAGES, ) // 2. Create a Lambda function that will be triggered by the DynamoDB stream // you can see how to create lambda functions in typescript in the following link: // https://www.codewithyou.com/blog/writing-typescript-lambda-in-aws-cdk const fn = new NodejsFunction(this, 'Handler', entry: './lambda/index.ts', handler: 'main', runtime: lambda.Runtime.NODEJS_14_X, architecture: lambda.Architecture.ARM_64, bundling: externalModules: ['aws_sdk'], minify: true, , ) // 3. currently, aws cdk is not yet support filtering event sources // https://github.com/aws/aws-cdk/issues/17874 // So, we need create a source mapping to map the DynamoDB table to the Lambda function const sourceMapping = new lambda.EventSourceMapping(this, 'SourceMapping', startingPosition: lambda.StartingPosition.TRIM_HORIZON, batchSize: 1, // only process one record at a time target: fn, // the lambda function to trigger eventSourceArn: dynamoTable.tableStreamArn, // the DynamoDB stream to read from bisectBatchOnError: true, // if the Lambda function fails, split the batch into two retryAttempts: 3, // retry 3 times if the Lambda function fails ) // grant the function stream access to the DynamoDB table dynamoTable.grantStreamRead(fn) // filter the event source mapping to only process records with the specified event type (in this case, "INSERT") const cfnSourceMapping = sourceMapping.node.defaultChild as lambda.CfnEventSourceMapping // get the underlying CloudFormation resource cfnSourceMapping.addPropertyOverride('FilterCriteria', Filters: [ Pattern: JSON.stringify( eventName: ['INSERT'], // only process INSERT events ), , ], )
Let’s go through the code above.
- We created a DynamoDB table. This table will be the source of the event.
- We created a Lambda function written in TypeScript. You can see how to create lambda functions in TypeScript in the following link:
- We created a source mapping to map the DynamoDB table to the Lambda function. This source mapping will be the event source for the Lambda function.
- We filtered the source mapping to only process records with the specified event type (in this case, “INSERT”).
cfnSourceMapping.addPropertyOverride('FilterCriteria', Filters: [ Pattern: JSON.stringify( eventName: ['INSERT'], // only process INSERT events ), , ], )
This code snippet will add a filter to the source mapping. The filter will only process records with the event type “INSERT”. The filter will be added to the source mapping as a property override.
The code for this article is available on GitHub
Let deploy the whole stack to AWS.
After the stack is deployed, you can see the Lambda function is triggered by the DynamoDB stream by add a item to the DynamoDB table.
Don’t forget to delete the stack. You can do this by running the following command:
npx cdk destroy
Thanks for reading this article. If you have any questions, please leave a comment. I will try to answer your questions as soon as possible.