Cpp OTF User Guide

Some applications, such as network sniffers, need to process messages dynamically and thus have to use the Intermediate Representation to decode the messages on-the-fly (OTF). An example of using the OTF API can be found here. Doxygen documentation for the C++ OTF API can be found in the header files here. You will need to have doxygen installed to build the doc.

The C++ OTF decoder follows the design principles of the generated codec stubs.

Note: Due to the dynamic nature of OTF decoding, the stubs generated by the SBE compiler will yield greater relative performance.

Introduction

The C++ OTF decoder for SBE is concerned with being able to take a piece of data and a schema and being able to decode it, well, on-the-fly. The decoder uses a reactive, Rx, style API to accomplish this. Applications subclass callback interfaces, such as OnNext, OnError, and OnComplete to be notified of decoding actions. Listener is a decoding engine object. It is configured with the data to decode as well as the schema. The schema is represented in a "compiled" form called Intermediate Representation, or IR.

Listener - Encapsulation of a Decoding Engine

Instances of the Listener class may be reused for different buffers as well as different Ir. The methods use a fluent style for composition.

The basic usage pattern is:

• Instantiate a Listener. This can be on the stack or via new, etc.
• Set the Ir to use for the decoding that describes the data format
• Pass in a pointer to the start of the data along with the length of the data in bytes
• Subscribe callbacks for Field, Group, Error, and OnCompleted events. When called, Listener::subscribe initiates the decoding of the message. Thus all callbacks come from the calling thread.

This is demonstrated in the example below.

 Listener listener();                   // instantiate a decoder
 Ir ir(irBuffer, irLen);                // create an Ir object for the format based on Ir in buffer
 listener.resetForDecode(buffer, len)   // get ready to decode data located at buffer for len bytes
         .ir(ir)                        // pass in the Ir to use
         .subscribe(...);               // subscribe callbacks and initiate decoding

A more advanced usage pattern is when a header is used to dispatch to a set of different formats for the data. This is accomplished using the Listener::dispatchMessageByHeader method. An example is below.

 Listener listener();
 Ir headerIr(headerIrBuffer, headerIrLen);                   // the Ir for the header
 listener.resetForDecode(buffer, len)                        // get ready to decode data at buffer for len bytes
         .dispatchByMessageHeader(headerIr,                  // the Ir of the header
                                  irCallback)                // the callback called for dispatch choices
         .subscribe(...);

Decoding multiple messages in a single buffer is straight forward. Simply bump the pointer to the data by the offset of the Listener after it is done and reuse the decoder. The Listener keeps track of its current offset within the buffer and this offset can be retrieved via Listener::bufferOffset.

 Listener listener();
 Ir headerIr(headerIrBuffer, headerIrLen);
 listener.resetForDecode(buffer, len)
         .dispatchByMessageHeader(headerIr, irCallback)
         .subscribe(...);                                   // go ahead and decode single message header plus message and return

 listener.resetForDecode(buffer + listener.bufferOffset(), len - listener.bufferOffset())
         .dispatchByMessageHeader(headerIr, irCallback)
         .subscribe(...);                                   // go ahead and decode single message header plus message and return     

Fields

During decoding a Listener will call OnNext::onNext(const Field &) and pass encountered fields to the application. These fields may be of varying types, including composites (or structs), enumerations, bit sets, or variable length data. All of these types may be accessed via the Field class.

Groups

Groups are markers in the event sequence of calls to OnNext::onNext. Groups contain fields. When a group starts, OnNext::onNext(const Group &) is called with a Group::Event type of Group::START, the name of the group, the iteration number (starting at 0), and the expected number of iterations. After that, a set of calls to OnNext(const Field &) should occur. A group is ended by a call to OnNext::onNext(const Group &) with a Group::Event type of Group::END. Nested repeating groups are handled as one would expect with Group::START and Group::END within an existing Group sequence.

Error Handling

Message Completion

Ir Collections