ZeroMQ Guide Chapter 0 - 1 Notes

The original

Preface

Service: sockets that carry atomic messages in concurrency framworks.

• Multiple transport occasions.
• Multiple patterns among nodes.
• Asynchronous I/O.
• Low latency, low cost, low management. (“Zero”)

Basics

Request-Reoly Pattern

Server

CONTEXT; SOCKET(REP);
WHILE(1) {
    RECV;    
    ...
    SEND;
}

Client

CONTEXT; SOCKET(REQ);
WHILE(1) {
    SEND;    
    ...
    RECV;
}

Note: the client can corrupt (eg. return -1) when the server is down.

Strings with Suffix 0 in C/C++/… Languages

ZeroMQ strings are length-specified and are sent on the wire WITHOUT a trailing null.

eg.

char buf[6] = "hello";
SEND(buf, 5);

Publisher-Subscriber Pattern

Publisher

CONTEXT; SOCKET(PUB);
BIND;
WHILE(1) {
    SEND;    
    ...
}
CLOSE;

Subscriber

CONTEXT; SOCKET(SUB);
CONNECT;
SETSOCKOPT(zipcode);
WHILE(1) {
    RECV;
    ...
}
CLOSE;

Note:

• Asynchronous pattern.
• If zipcode is left empty, no message will be received.
• It is forbidden to SEND to a specific SUB, or call RECV in a PUB.
• SUBs won’t corrupt when PUBs restart. They just re-connect.
• When a SUB connects to multiple PUBs, messages will be fair-queued so that no PUB can drown out others.

Slow Joiner Symptom: A Synchronous Problem

When a PUB starts, it takes the SUBs a while (relatively) to build the connection. Before they finish, the PUB may have started to send some messages, which is impossible for the SUBs to receive.

Divide and Conquer (Parallel Pipeline)

	VENTILATOR
↙	$\downarrow$	↘
WORKER	WORKER	$\cdots$	$\cdots$
↘	$\downarrow$	↙
	SINK

• A VENTILATOR that produces tasks that can be done in parallel
• A set of WORKERs that process tasks
• A SINK that collects results back from the worker processes

Ventilator

CONTEXT;
sender=SOCKET(PUSH);
sender.BIND(addr1);
sink=SOCKET(PUSH);
sink.CONNECT(addr2);
sink.SEND;  // signal the start to SINK
for (task: tasks) {
    sender.SEND;
    ...
}

Worker

CONTEXT;
receiver = SOCKET(PULL);
receiver.CONNECT(addr1);
sender = SOCKET(PUSH);
sender.CONNECT(addr2);
while (1) {
    receiver.RECV;
    ...
    sender.SEND;
}

Sink

CONTEXT;
receiver=SOCKET(PULL);
receiver.BIND(addr2);
receiver.RECV;  // receive the start signal
for (i : task_num) {
    receiver.RECV;
    ...
}

Note:

• Stable parts: VENTILATOR & SINK; Dynamic parts: WORKERs (add and remove at any time).
• Synchronous need: the VENTILATOR must wait until all WORKERs are connected to send the tasks. Otherwise, one WORKER may receive all tasks. (Slow Joiner Syndrome) Are the former 2 points contradictory? Any misunderstanding?
• Load balancing supported in task distribution.
• Fair queuing supported in task collection.

Context

A context is the container of all sockets in a single process, so create it at the beginning of the process and destroy at the end.

fork(): create it at the beginning of the child process.

Close all sockets (unless they are set LINGER=0) and then destroy the context.

Multithreading case:

1. Do NOT share a socket in threads.
2. Shut down each socket that has ongoing requests (eg. set LINGER=1sec), and close it.
3. Destroy the context. All blocking receives, polls, and sends in attached threads will return with an error.
4. Catch the errors, set linger on, close sockets in that thread, and exit.

Advantageous features of ZeroMQ

• Asynchronous I/O in background threads. Lock-free for concurrent cases.
• Dynamic components. Automatically reconnection support.
• Message queuing.
• High water mark: when the queue is full, ZMQ will block the sender or throw away messages depending on its pattern.
• Encapsualed arbitrary transports: TCP, multicast, in-progress, inter-progress. No need to change the code.
• No format on messages.
• Handles network errors, slow readers and other case intelligently.
• Sockets are doorways to external communications, rather than simple “one socket one connection” mode in traditional networking.