CloudGuild · Blog · Cheat sheets · Lessons · Certifications
Amazon Kinesis — Real-time data streaming and analytics
Amazon Kinesis enables you to collect, process, and analyze real-time, streaming data.
What it is
- A platform for real-time data streaming and analytics.
- Comprises Kinesis Data Streams, Kinesis Data Firehose, Kinesis Data Analytics, and Kinesis Video Streams.
When I reach for it
- When you need to process and analyze streaming data in real-time, such as log data, telemetry, or social media feeds.
- When you have large volumes of data generated continuously and need to act on it immediately.
Key architectural decisions
- Data Ingestion: Choose between Kinesis Data Streams (for custom applications) or Kinesis Data Firehose (for easy delivery to AWS services).
- Data Processing: Use Kinesis Data Analytics for SQL-like queries on streaming data or integrate with AWS Lambda for custom processing.
- Data Retention: Decide how long to retain data in streams (default is 24 hours, max is 7 days).
Gotchas & exam traps
- Kinesis Data Streams has a limit of 1 MB per record and a maximum of 1,000 records per second per shard.
- Ensure you understand the difference between Kinesis Data Streams and Firehose, especially for exam questions.
- Be cautious with data retention settings; exceeding limits can incur costs.
The architect view
- Kinesis is essential for real-time analytics and can be integrated seamlessly with other AWS services like Lambda, S3, and Redshift.
- Consider using Kinesis for event-driven architectures and microservices that require low-latency processing.