RabbitMQ for Beginners

RabbitMQ is a powerful Message Queue implementing AMQP (wire-level protocol and semantic framework for high performance enterprise messaging). This is great for those of you who like buzzwords, but here’s the simple explanation for the rest of us:

RabbitMQ is a software service which you typically run on many servers, either in a local Lan, and/or across the Internet. The services all connect with each other in a very fault tolerant(1) manner, and allow you to transmit messages between those servers very quickly. A producer of messages inserts messages into one or more queues. A consumer of messages subscribes(2) to one or more queues. A software can be a producer, or consumer, or both a consumer and a producer of messages. This can be used to facilitate communications between many servers, on many topics, with any possible combination of routing and subscriptions, and access control. For really non-techie managers: Imagine it like a series of tubes in a house, with water containers on the roof being the producers, and taps all around the house being the consumers 

(1) By “Fault Tolerant” I mean nodes can fail, then maybe come back up, but the rest of the nodes that are still up will keep running and transmitting messages with each other. In addition, when the fallen nodes come back up, any messages they stored on disk will be reloaded and retransmitted. Nothing is lost.

(2) The manner in which consumers subscribe to queues and exchanges is quite complex at first look, but fear not, some reading and then experimenting will fix it for you. Hopefully thanks to this guide, you will have a working, solid set of servers to start experimenting with.

Update: Found this awesome Slide Show about Interop with RabbitMQ. It explains the advantages of using AMQP and RabbitMQ, and I strongly recommend you check it out.

Assumptions for this guide

The first assumption is that you have 3 servers, two are on a local lan, while the third is in another data center, and not only is it on another completely different IP network, it even has another domain name!

Here are the server names we’ll use for this guide:

ren.AcmeNetwork.com on 192.168.1.101
stimpy.AcmeNetwork.com on 192.168.1.102
dynamite.WildCoyote.com on 10.0.0.150

The second assumption is that you are running under Ubuntu 9.10 Server, and your boxes are connected to the internet so that they can fetch the rabbitmq-server packages (and related packages).

Step 1 – Setting up RabbitMQ

Fortunately, RabbitMQ have released packages for most popular operating systems. The OS of choice for me is Ubuntu 9.10 Server, however I’m pretty sure you can adapt this easily to any other operating system. Ubuntu’s repository currently contains an older version of RabbitMQ 1.6.0, but we want the latest and greatest. So here’s how we do it:

root@ren:/root# wget http://www.rabbitmq.com/releases/rabbitmq-server/v1.7.2/rabbitmq-server_1.7.2-1_all.deb
root@ren:/root# dpkg -i rabbitmq-server_1.7.2-1_all.deb

Well, that was easy, wasn’t it?

If Ubuntu or Debian complain that some dependencies were missing, just issue an apt-get -f install to correct the issue. The dependencies will be satisfied, and the RabbitMQ installation will complete successfully.

You will notice Ubuntu will install quite a few erlang related packages (RabbitMQ is written in Erlang, which is a great thing). When apt-get is done,  your RabbitMQ server will already be installed and running, and this is where people usually start scratching their heads and start to google their way around. The guides on the RabbitMQ site, while technically correct, lack in the “Teach by example” area.

And before you continue! Some best practices and preparations:

1) First, Let’s change the default guest password (“guest”) to something else (the part in bold is what you type):

root@ren:/root# rabbitmqctl change_password guest 123456
Changing password for user "guest" ...
...done.

Do this on all 3 nodes. Great, now Mr. H4ck3r can’t connect to our RabbitMQ Servers with the default password.

2) The next thing to do is add the hostname’s short names in your  /etc/hosts file. Why? Because RabbitMQ does not like FQDN (Attempting to do this will result in the following error:  ** System NOT running to use fully qualified hostnames **). Just this single issue has cost me at least 10 minutes of research.

So assuming the host names above, with the IP addresses I specified, here is what our section looks like in  /etc/hosts:

# Start RabbitMQ shortstop names definition
192.168.1.101  ren       ren.acmenetwork.com
192.168.1.102  stimpy    stimpy.acmenetwork.com
10.0.0.150     dynamite  dynamite.wildcoyote.com
# End RabbitMQ shortstop names definition

Again, put the above section in  all of your hosts, at the end of your  /etc/hosts file,  before you continue.

3) When RabbitMQ was first started, it generated a “ cookie” automatically into a file, here:  /var/lib/rabbitmq/.erlang.cookie. This cookie is different on all of our 3 nodes, but for our cluster to be able to talk to each other this cookie has to be the same. We will simply take the cookie from the first node (ren), and copy it to the other two nodes (stimpy & dynamite). Here’s how to do this. First le’ts find out what the cookie is on the first server:

root@ren:/root# cat /var/lib/rabbitmq/.erlang.cookie
EOKOWXQREETZSHFNTPEY

Ok so now we know Ren’s cookie. Before we change the cookie on Stimpy and Dynamite, we must stop rabbitmq-server – Why? Because the cookie is used to communicate the shutdown request to rabbitmq-server. If we change this cookie before we shut it down, we will no longer be able to shut it down! 

Let’s shut rabbitmq-server down on Stimpy, replace the cookie, then restart:

root@stimpy:/root# service rabbitmq-server stop
Stopping rabbitmq-server: rabbitmq-server.

root@stimpy:/root# echo -n EOKOWXQREETZSHFNTPEY > /var/lib/rabbitmq/.erlang.cookie
root@stimpy:/root# service rabbitmq-server start

Let’s do the same on Dynamite as well:

root@stimpy:/root# service rabbitmq-server stop
Stopping rabbitmq-server: rabbitmq-server.

root@stimpy:/root# echo -n EOKOWXQREETZSHFNTPEY > /var/lib/rabbitmq/.erlang.cookie
root@stimpy:/root# service rabbitmq-server start

Awesome! now our servers all share the same cookie, and we can continue with the setup.

Step 2 – Creating the cluster

Let the magic begin – There are two ways to create the cluster. The first way is to define the nodes in a configuration file. The second way is to use the command line utility rabbitmqctl to create the cluster. We will use the second method, because it uses some commands which I think are important for the understanding of the way RabbitMQ works behind the scenes.

The process is simple, and involves the following steps:

1. Stop the MQ Application, without stopping the rabbitmq management process
2. “Reset” the state of the management process
3. Ask the management process to discover the new node(s)
4. Restart the MQ application

And this is how it’s done in the command line:

root@stimpy:/root# rabbitmqctl stop_app
Stopping node rabbit@stimpy ...
...done.

root@stimpy:/root# rabbitmqctl reset

 
 

  
  Resetting node rabbit@stimpy ...
 
 

 
 

  
  ...done.
 
 

root@stimpy:/root# rabbitmqctl cluster rabbit@ren
Clustering node rabbit@stimpy with [rabbit@ren] ...
...done.

root@stimpy:/root# rabbitmqctl start_app
Starting node rabbit@stimpy ...
...done.

Do the same with  Dynamite. You know you want to:

root@dynamite:/root# rabbitmqctl stop_app
Stopping node rabbit@dynamite ...
...done.

root@dynamite:/root# rabbitmqctl reset
Resetting node rabbit@dynamite ...


  
  

   
   ...done.
  
  

root@dynamite:/root# rabbitmqctl cluster rabbit@ren
Clustering node rabbit@dynamite with [rabbit@ren] ...
...done.

root@dynamite:/root# rabbitmqctl start_app
Starting node rabbit@dynamite ...
...done.

Step 3 – Enabling some Plugins

This section is important if you intend to use some of the plugins supplied with rabbitmq, such as the json and mochiweb plugins.

In this area, the rabbitmq documentation was again lacking. It explains how to compile the plugins, and it even compiles without a hitch, however they don’t tell you where the plugin directory is located. You would expect the  README.Debian file to contain this information but  it doesn’t. I had to dig into a script called  rabbitmq-activate-plugins to find out where it expects the plugins directory to be.

The steps in general are:

1. Install some additional software which the compiler will need later (such as Mercurial, Zip, Unzip and Subversion)
2. Obtain the plugin sources via Mercurial
3. Get them compiled (the scripts in here use zip and subversion for example)
4. Once compiled, create symbolic links into the plugins directory
5. Ask rabbitmq to activate all the plugins it finds in the plugins directory
6. restart rabbitmq

And here’s the actual walkthrough. First let’s install the dependencies:

root@ren:/root# apt-get install mercurial zip unzip subversion erlang-dev erlang-src erlang-tools erlang-nox

Next we’ll go into the rabbitmq lib directory, and get the plugin sources from the Mercurial repository:

root@ren:/root# cd /usr/lib/rabbitmq/lib/

root@ren:/usr/lib/rabbitmq/lib# hg clone http://hg.rabbitmq.com/rabbitmq-public-umbrella
destination directory: rabbitmq-public-umbrella
requesting all changes
adding changesets
...
...
12 files updated, 0 files merged, 0 files removed, 0 files unresolved

root@ren:/usr/lib/rabbitmq/lib# cd rabbitmq-public-umbrella/
root@ren:/usr/lib/rabbitmq/lib/rabbitmq-public-umbrella# make co
root@ren:/usr/lib/rabbitmq/lib/rabbitmq-public-umbrella# make

After a pretty long output, you should have all the plugins installed. Note that this builds the plugins in development mode. This means that you can change the code, run make again, and then simply restart rabbitmq for changes to take effect immediately.

Now to the actual installation of the plugins into rabbitmq:

cd ../rabbitmq_server-1.7.2
# Yes, the plugins directory isn't even created for us initially. We'll just
# make it now:
mkdir plugins
cd plugins
ln -s ../../rabbitmq-public-umbrella/erlang-rfc4627
ln -s ../../rabbitmq-public-umbrella/rabbitmq-jsonrpc
ln -s ../../rabbitmq-public-umbrella/rabbitmq-jsonrpc-channel
ln -s ../../rabbitmq-public-umbrella/rabbitmq-mochiweb
ln -s ../../rabbitmq-public-umbrella/rabbitmq-mochiweb/deps/mochiweb

Now we should tell rabbitmq to activate those plugins:

rabbitmq-activate-plugins

Now restart rabbitmq-server and you’re done:

service rabbitmq-server restart

Congratulations!

Let’s go over what we’ve done quickly:

1. Downloaded and installed RabbitMQ Server 1.7.2
2. Made it a bit more secure, and also configured short hostnames for all of our cluster nodes
3. Connected all of our rabbits together into a cluster
4. Downloaded & Compiled the provided umbrella plugins
5. Activated some plugins

If everything went smooth, you should be able to start playing with queues, exchanges, etc. I hope you are excited as I was, when you send your first messages onto the queue and start receiving them on the other side of your network.

I would love to hear your feedback and corrections on this guide.

 

原文地址： http://www.skaag.net/2010/03/12/rabbitmq-for-beginners/