Avro and Qpid are two Apache projects designed to help with message passing in and between applications. Here we explore using them together to create a flexible and powerful messaging system for distributed applications.

Avro is a data serialization project developed within the Apache Hadoop project. It allows you to serialize data into a compact binary format which can be shared across languages. Data types are defined in JSON schemas, giving a common definition between languages. Avro can also generate code from the JSON schemas for staticly typed languages, however this is not required and you can deserialize using just a schema.

Qpid is a message queueing project which fully implements AMQP (Advanced Message Queueing Protocol). From the Qpid website:
"AMQP is the first open standard for Enterprise Messaging. It is designed to support messaging for just about any distributed or business application. Routing can be configured flexibly, easily supporting common messaging paradigms like point-to-point, fanout, publish-subscribe, and request-response.

Apache Qpid implements the latest AMQP specification, providing transaction management, queuing, distribution, security, management, clustering, federation and heterogeneous multi-platform support and a lot more. And Apache Qpid is extremely fast." Qpid uses a broker/client architecture. You launch a broker service which is bound to an ip address and port. Once a client connects and authenticates with the broker, it can send and receive messages with other clients.

Both projects have overlapping support for many popular languages.
The Qpid broker is implemented in Java and C++.
Avro and the Qpid client are implemented in:

• Java
• C++
• C#
• Python
• Ruby

This example was done using the Java and C++ implementations on Ubuntu.

We will use Avro to serialize data to binary, and then pack it into a Qpid message for transporting.
Avro does include the ability to pass messages over HTTP and other protocols, and Qpid has features for message serialization and language independent data representation. However, these are not the core concerns of each project.
While they are each adequate as complete messaging solutions alone, their strengths compliment each other very well when used together.

The rest of this entry will demonstrate using Avro and Qpid for distributed video processing and display using C++ and Java.
This project requires the following:

• Avro
• Qpid
• OpenCV

Overview:
Our goal will be creating a system that captures images from a camera, does some processing on the images, and draws the results.
To do this, we will be making three distributed components:

1. Camera Capture using OpenCV (C++)
2. Image Processing, we will detect faces using the OpenCV haar face detection (C++)
3. Video Display with results from the Image Processing (Java & C++)

The Camera Capture component will pull images from the camera and pass them to both the Image Processing and Video Display components.
The Image Processor will recieve images, perform haar face detection, and send the face cooridinates to the Video Display.
The Video Display will receive an image from the Camera Capture, and wait for results from the Image Processor. It will then draw the image and results.