The Analytics Team has complained that even though the application is running, there is no data being sent to their data lake’s staging area. Let’s take another look at the Alien Attack architecture diagram:
According to this diagram, the Kinesis Data Stream sends the data through a Kinesis Data Firehose to a bucket in S3. The Analytics Team has appended the following information: “We have already created the S3 bucket to store this data in our AWS account. It has the suffix ‘raw’”.
Action Item: before we move on, try to navigate to your S3 console and find the bucket that the Analytics Team is referring to. Click the checkbox to the left of the bucket, select Copy Bucket ARN and paste the ARN to a separate text editor file.
The team said that this fix should be pretty simple: “All you need to do is connect the Kinesis Data Firehose to our existing Kinesis Data Stream. If the Kinesis Data Firehose doesn’t exist, create one! Grant us access to the environment and we can help, or call us if you need”.
Let’s get started.
What are we fixing? Fix or create a Kinesis Data Firehose so that it is properly sending data from our Kinesis Data Stream to the Analytics Team’s S3 bucket. We are currently missing a mechanism to do this within our AWS architecture.
Do you see any Kinesis Firehose Delivery Streams for your Alien Attack environment?
It looks like our application doesn’t have any Kinesis Firehose Delivery Streams built, so let’s create one:
cdk diff -c envname=$envname -c sessionparameter=true -c kinesisintegration=true -c firehose=truein the terminal window. This will show you what will be changed in your environment.
cdk deploy -c envname=$envname -c sessionparameter=true -c kinesisintegration=true -c firehose=trueto deploy the changes.