Core Concept: A Durable, In-Memory Database
The key differentiator for MemoryDB is durability. Unlike a traditional cache where data can be lost on a restart or failure, MemoryDB is designed for zero data loss.
- In-Memory Speed: All data is stored in memory, enabling microsecond read latency and single-digit millisecond write latency.
- Database Durability: All data and transactions are redundantly stored on disk in a distributed, Multi-AZ transactional log. This log ensures data is preserved across node failures, reboots, and failovers.
MemoryDB vs. ElastiCache for Redis
Choosing between MemoryDB and ElastiCache for Redis depends entirely on your need for data durability.
| Feature | Amazon MemoryDB for Redis | Amazon ElastiCache for Redis |
|---|---|---|
| Primary Use Case | Primary Database. Use when you need an ultra-fast, durable database. | In-Memory Cache or Datastore. Use for accelerating existing databases, not as the primary source of truth. |
| Data Durability | Durable. Data is safely stored in a Multi-AZ transactional log. No data loss on node failure. | Not Durable (by default). Data is purely in-memory. Data loss can occur during node failures or restarts. |
| Consistency | Strongly Consistent writes on the primary node. | Not Strongly Consistent. Writes are asynchronous to replicas. |
| Performance | Microsecond read latency; single-digit millisecond write latency. | Microsecond read and write latency (writes are faster as they don't wait for a transaction log). |
| Failover | Fast, automatic failover to a replica with zero data loss. | Automatic failover is supported, but some data loss can occur (the "replica lag"). |
| Target Application | Microservices requiring a high-performance, durable, stateful database. | Caching layers to reduce latency and load on a primary database like RDS or DynamoDB. |
Rule of Thumb: If you need the speed of Redis but cannot afford to lose the data (i.e., you need a primary database), choose MemoryDB. If you are simply caching data that exists elsewhere and can be refetched if lost, choose ElastiCache.
Architecture & Components
MemoryDB uses a hierarchical architecture for scalability and high availability.
- Cluster: A collection of one or more shards that collectively hold the entire dataset.
- Shards: Each cluster is composed of one or more shards. A shard is a partition of the data, allowing the database to scale horizontally.
- Nodes: Each shard contains a single primary node and up to five replica nodes.
- Primary Node: Serves both read and write requests for that shard. It is responsible for writing to the transaction log.
- Replica Nodes: Serve only read requests, increasing the read throughput for the shard. They replicate data from the primary.
- Multi-AZ Transaction Log: This is the key to durability. Every write to a primary node is synchronously recorded in a distributed log that spans multiple Availability Zones before a success acknowledgement is sent to the client.
Key Features
- Redis Compatibility: Fully compatible with the open-source Redis API. You can use the same Redis data structures, clients, and tools.
- High Availability: In the event of a primary node failure, MemoryDB automatically promotes a replica to primary and replays the transaction log to ensure no data is lost.
- Scalability:
- Write Scaling (Horizontal): Increase the number of shards in your cluster to scale write throughput.
- Read Scaling (Horizontal): Add more replica nodes (up to 5 per shard) to scale read throughput.
- Vertical Scaling: Change the node instance type to scale memory and compute resources up or down.
- Data Tiering: Supported on
r6gdnode types. Data tiering automatically moves less frequently used data from memory to lower-cost, local SSD storage, helping you scale capacity cost-effectively.
Security
- Access Control Lists (ACLs): The primary mechanism for authentication and authorization. You can create users, assign them passwords, and grant them specific permissions (e.g., read-only, write-only) on a per-keyspace basis.
- Encryption: Supports both encryption in-transit (TLS) and encryption at-rest using AWS KMS keys.
- VPC Integration: MemoryDB clusters are deployed within an Amazon VPC, allowing you to control network access via security groups and network ACLs.
Data Persistence & Backup
- Transactional Log (Durability): The always-on, Multi-AZ log provides continuous data persistence and is used for automatic recovery and failover.
- Snapshots (Backup & Restore): You can create point-in-time snapshots of your cluster, which are stored in Amazon S3. These are useful for creating new clusters, archiving data, or for point-in-time recovery. Snapshots can be configured to be taken automatically or initiated manually.
Sources