Amazon EBS Multi-Attach

Primary Use Case: High-Availability Clustered Applications

The sole purpose of EBS Multi-Attach is to provide shared block storage for clustered applications that require concurrent access to data and are capable of managing storage consistency.

• Cluster-Aware Applications: It is designed for applications that have built-in mechanisms to manage concurrent write operations from multiple servers.

• Cluster File Systems: To use Multi-Attach as a shared file system, you must use a cluster-aware file system like GFS2 or OCFS2. Using a standard file system like ext4, XFS, or NTFS will result in data corruption and data loss.

• Examples:

  • Building high-availability applications that manage data consistency at the application layer.

  • Clustered databases that are certified for shared block storage deployments.

  • Simplifying management for applications that require each node to have access to the same underlying storage device.

Key Requirements & Configuration

To use EBS Multi-Attach, your environment must meet several specific requirements:

• Volume Type: Only Provisioned IOPS SSD (io1 and io2) volumes are supported. It is not available for General Purpose (gp2/gp3), Throughput Optimized (st1), or Cold HDD (sc1) volumes.

• Instances: All instances must be built on the AWS Nitro System.

• Availability Zone: The EBS volume and all EC2 instances it is attached to must reside in the same Availability Zone.

• Enablement: The Multi-Attach feature must be explicitly enabled when the volume is created. It cannot be enabled or disabled on an existing volume.

• Maximum Attachments: A single Multi-Attach enabled volume can be attached to a maximum of 16 Linux-based EC2 instances.

Critical Considerations & Limitations

Misunderstanding the limitations of Multi-Attach can lead to severe data integrity issues.

• !! CRITICAL WARNING on Data Consistency !!

  • EBS Multi-Attach does not provide I/O fencing. It does not manage write ordering or prevent race conditions between instances.

  • The attached instances, through a cluster-aware file system or application-level logic, are entirely responsible for maintaining data consistency.

  • Attaching a Multi-Attach volume to multiple instances and mounting it with a standard, non-cluster-aware file system will corrupt your data.

• No Windows Support: Multi-Attach is only supported for Linux workloads.

• No Boot Volumes: A Multi-Attach enabled volume cannot be used as an EC2 instance's boot volume.

• Immutable Configuration: After a Multi-Attach enabled volume is created, you cannot change its volume type, size, or Provisioned IOPS setting. You also cannot disable the Multi-Attach feature.

• Single Point of Failure: While Multi-Attach can improve application availability, the EBS volume itself can still be a single point of failure. An issue with the underlying EBS infrastructure for that volume will impact all attached instances.

• Snapshots: Taking a snapshot of a Multi-Attach volume is possible, but for a consistent snapshot, it is strongly recommended to either detach the volume from all instances or pause all I/O from all instances before initiating the snapshot.

How to Enable Multi-Attach

You enable the feature during the creation of a new Provisioned IOPS (io1 or io2) volume via the AWS Management Console, AWS CLI, or SDKs. In the console, it is a simple checkbox in the "Create volume" screen.