Running RabbitMQ on Amazon EC2

RabbitMQ

Features
Get Started
Support
Community
Docs
Blog

Overview

This guide assumes familiarity with the general clustering guide as well the guide on cluster peer discovery.

Using RabbitMQ on EC2 is quite similar running it on other platforms. However, there are certain minor aspects to EC2 that need to be accounted for. They primarily have to do with hostnames and their resolution.

This guide demonstrates manual (CLI-driven) RabbitMQ clustering. Peer discovery plugin for AWS (RabbitMQ 3.7.0 or later) is an option more suitable for automation.
AMIs

Pivotal currently does not provide AMI images with RabbitMQ pre-provisioned. RabbitMQ works well on up-to-date Ubuntu, Debian and CentOS AMIs as long as a compatible version of Erlang/OTP is installed.
Choosing an Instance Type

RabbitMQ will work on every instance type, but there are a few considerations worth bearing in mind:

Use 64-bit instances.
Depending on the workload and settings, RabbitMQ can require substantial amounts of memory. Make sure that your host does have an appropriate amount of RAM and always have at least a few gigabytes of swap space enabled. Workloads can be simulated using PerfTest. A separate guide on reasoning about node memory usage is available.
RabbitMQ generally will take advantage of all the CPU cores in the system provided the workload uses multiple queues. Other factors should be taken into account (e.g. disk and network I/O throughput).
Monitoring RabbitMQ nodes as well as applications that use it on real or simulated workloads will help assess how suitable a particular instance type is.

Operaing Systems

Although RabbitMQ is tested with most major Linux distributions, Ubuntu support for Amazon EC2 seems to be strongest, so that’s the distribution this guide will use.

See Ubuntu Cloud Images and Ubuntu EC2 Starters Guide.
Installation

Please consult the installation guides for

Debian and Ubuntu
RHEL, CentOS and Fedora

A wide variety of deployment tools can be used to automate RabbitMQ deployment.

Community Docker image (on GitHub)
Chef cookbook
Puppet module

are some popular options.
Durable Storage on EBS Volumes

On Linux, RabbitMQ will use the following directories for its node data directory:

/var/lib/rabbitmq/ to store persistent data like the messages or queues
/var/log/rabbitmq/ to store logs

See the File and Directory Locations for details.

Those directories can be symlinks to a dedicated storage volume. The node must be stopped before symlinking is performed:

sudo service rabbitmq-server stop

We recommend performing symlinking and other storage preparation steps before installing RabbitMQ when possible.

Note that EBS volumes have an IOPS limits, which can impact thoughput and RabbitMQ operations. If an EBS volume hits the limit, disk writes will worsen. It is also possible that the RabbitMQ message store compaction (garbage collection of on-disk data) can fall behind disk writes, which means the disk will be filled up quicker than disk space can be reclaimed after messages were consumed and acknowledged. This will eventually lead to resource alarms and publisher throttling. Please make sure the limit is high and set up I/O operation rate monitoring.
Further Reading

Several other guides cover topics highly relevant for running RabbitMQ clusters in public clouds:

Custering Fundamentals
Peer Discovery
Configuration
Monitoring
Production Checklist
File and Directory Locations

Getting Help and Providing Feedback

If you have questions about the contents of this guide or any other topic related to RabbitMQ, don’t hesitate to ask them on the RabbitMQ mailing list.
Help Us Improve the Docs ❤️

If you’d like to contribute an improvement to the site, its source is available on GitHub. Simply fork the repository and submit a pull request. Thank you!
In This Section

Install: Windows
Install: Debian and Ubuntu
Install: RPM-based Linux
Install: Homebrew
Install: Windows (manual)
Install: Generic binary build
Install: Solaris
Install: EC2
Upgrade
Blue-green deployment-based upgrade
Supported Platforms
Changelog
Erlang Versions
Signed Packages
Java Client Downloads
.NET Client Downloads
Erlang Client Downloads
Community Plugins
Snapshots

RabbitMQ

Features
Get Started
Support
Community
Docs
Blog

Copyright © 2007-2020 VMware, Inc. or its affiliates. All rights reserved. Terms of Use, Privacy and Trademark Guidelines