Skip to content

Building an API Reference Implementation

Developing an OLIVE API Reference Implementation

If SRI’s Java client library does not meet your needs or you need to create an implementation in a language other than Java, then the following information is helpful for creating a new reference implementation of the OLIVE Enterprise API.

Things to know before you start

Before you start, it is a good idea to first develop an understanding of the current OLIVE Java Client API. Reading the API Primer page and reviewing the provided Java Client API source files are all good places to start. As an alternative,

To enable users to quickly put together client codes, the OLIVE Java API hides many of the low level implementation details such as assemby of request messages, submission of requests over a port, etc. This section describes some of these implementation details you need to consider when creating a brand new OLIVE API.

1. Communicating with the OLIVE Server

A messaging system called ZeroMQ, or ZMQ for short, is used as the backbone for communicating with the OLIVE Server.

In order to communicate with a running server, you must initialize ZeroMQ (ZMQ) sockets in your client code. There are two sockets:

  • request socket: This is the socket over which you send requests and receive replies, all of which are serialized protobuf messages. You must conect to this socket using the ZMQ.DEALER configuration.
  • status socket: This socket provides a simple heartbeat broadcast from the server. You may optionally monitor this socket to determine the up/down status of the server. Connect to this socket using the ZMQ.PUB configuration.

The code for initializing a ZMQ context and creating/connecting the necessary sockets can differ by language. Below is a ZMQ initialization example in Java.

ZMQ.Context context = ZMQ.context();

ZMQ.Socket request_socket = context.socket(ZMQ.DEALER);
ZMQ.Socket status_socket  = context.socket(ZMQ.SUB);

request_socket.connect("tcp://myserver:6678");
status_socket.connect("tcp://myserver:6679");
status_socket.subscribe(""); // Don’t forget this

After connecting the sockets, you can begin sending requests and receiving responses over the request port. The specifics of how this operation is performed are described in the next section.

2. Serialization of messages to and from the OLIVE Server

Messages are exchanged between the client and server in serialized form, over the request port. Serialization is provided by the Google Protobufs library. You should familiarize yourself with protobufs before beginning your integration. In order to utilize protobufs, you must first take the scenic.proto message definition file (provided in the example code package or available upon request) and use protoc (or protobuf.net) to automatically generate classes that represent the OLIVE API messages. For more information see the Google Protocol Buffers Documentation.

For example for the OLIVE Java Client API, protoc is used to produce the Java file Scenic.java, located in src/main/java/com/sri/scenic/api within the example code package. This file contains the class Scenic which is used in the Java API for all message related classes and definitions.

Once you have compiled the OLIVE messages into your code base you can begin your integration. As you may have seen in the OLIVE Java Client API code, (Scenic.java), every OLIVE message is an instance of a class named Envelope. As its name implies, Envelope acts as a container for enclosing messages. Messages are instances of a class named ScenicMessage. An Envelope can contain multiple instances of ScenicMessage, allowing you to batch your communications to the server.

Envelope and ScenicMessage are special because they are used for every communication across the request_port and are basically just wrappers. They’re analogous to the envelope and paper when writing someone a letter. The remaining OLIVE messages comprise the actual API requests and responses. The remaining messages each have an entry in the MessageType enum, allowing you to request certain types when retrieving data from database as well as dynamically deserializing data returned by the server.

Message Building Example

To get started, your integration will probably need to retrieve some information from the server. For example, you may wish to know the list of available plugins. An example pseudocode excerpt accomplishing this is shown below.

PluginDirectoryRequest.Builder req = PluginDirectoryRequest.newBuilder()

String id = getUUIDString()
ScenicMessage msg = ScenicMessage.newBuilder()
    .setMessageType(MessageType.PLUGIN_DIRECTORY_REQUEST)
    .addMessageData(req.build().toByteString())
    .setMessageId(id)
      .build();

Envelope env = Envelope.newBuilder()
    .setSenderId(“third-party-integration”)
    .addMessage(msg).build();

// Now send the message to the server
request_socket.send(env.toByteArray());

Envelope resp = Envelope.parseFrom(request_socket.recv());

for (ScenicMessage sm : resp.getMessageList()) {
    // For purposes of this example, we assume the above message was 
    // the first and only sent so we can assume things about the response
    // message, namely that it corresponds to  our request. 
    assert(id == msg.getMessageId())’
    assert(msg.getMessageType() == MessageType.PLUGIN_DIRECTORY_RESULT);
    if (sm.hasError()) {
        System.out.println(“Dang: “ + sm.getError());
        continue;
    }

    PluginDirectoryResult rep = PluginDirectoryResult.parseFrom(sm.getMessageData(0));

    for(Plugin p : rep.getPluginsList()){
  System.out.println(p.getId() + ": " + p.getDesc() );
    }

}

Please note the following:

  • We could have put other requests in the Envelope. Their responses may or may not have come back in the same envelope, but they would have come back in order.
  • We are guaranteed that our messages are received in order by the server and responses sent in order. However, for messages such as scoring requests and class modification requests (enrollment), which are highly asynchronous, there is no guarantee about the order in which they will finish.
  • The sure-fire way to ensure that you process a message from the server correctly is to base your actions on the message id (which you originally assigned in your request).
  • To properly deserialize the data contained within a ScenicMessage, you must check or otherwise be sure of the MessageType.
  • Some OLIVE plugins need to be preloaded by the server in order to fulfill a request. In such cases either the API or the client program must first send a Load Plugin Domain Request to have the targeted plugin preloaded.

An Analysis (Scoring) Message Example

Now let’s assume we wish to perform language identification on an audio file. We can create a LID like request as follows:

// Variable init
String plugin = “lid-embed-v2”
String domain = “multi-v1”
String audioFilePath = “/home/user/audio/file1.wav”

// Build audio object
Scenic.Audio.Builder audio = Scenic.Audio.newBuilder().setPath(audioFilePath.toAbsolutePath().toString;

// Create LID request
// If specifically processing stereo audio files and wish to score 
// both channels, please use the FrameScorerStereoRequest or 
// GlobalScorerStereoRequest messages, that will be responded to with 
// an FrameScoreStereo or GlobalScoreStereo message, containing score results
// for both channels of the submitted audio.
// If submitting a stereo audio file using the standard xScorerRequest functions,
// and you don’t desire to score both channels independently, there are two 
// options:
//   - Specify the channel you wish to be scored -> you will receive results
//             for that channel only.
//   - Do not specify a channel -> you will receive a single set of results
//             corresponding to the merged mono representation of the stereo file. 

Scenic.GlobalScorerRequest.Builder req = Scenic.GlobalScorerRequest.newBuilder()
  .setAudio(audio)
  .setPlugin(plugin)
  .setDomain(domain);

Note that this example relies on code written in our Java Client API, but the general steps to perform the task are the same.

Now we wrap the request in a ScenicMessage and Envelope like we did in the last example and send it across the request socket. Analysis requests as well as Enrollment requests take significant time to process on the server. It’s likely you’ll want your integration to be doing other things while it is waiting for the response, such as issuing further analysis requests. This is fully supported. However, you don’t know when or in what order the responses to your analyze request will emerge from the server. Therefore, it’s advantageous to track the message ids that you’ve issued in a map of the form message_id -> request message, so that you know the request to which a newly received response pertains.

Let’s assume we’ve received a GlobalScorerResult message and have deserialized it into a variable named res. We could process the result as follows:

// Currently OLIVE (SCENIC) will only send back one score reply per score
// request. Future releases may be able to send back multiple.
// Because of this, we must iterate though all of the scores.

List<Scenic.GlobalScore> scores = res.getScoreList();

for(Scenic.GlobalScore gs : scores){
        system.out.println("LID Score: class " +  gs.getClassId() “ = “ +  gs.getScore());
}

For more details regarding the specifications and breadth of the possible requests, their replies, and the structure of each of these data objects, including how results are represented, please refer to the OLIVE API Message Reference documentation.  

Request a new OLIVE API Reference Implementation

If there is a need for an OLIVE API in another language, SRI International would be an ideal candidate to undertake the task because of its rich experience having already done it in Java. However, it will still be a complex software engineering undertaking which will definitely take up a significant amount of project resources. For more information about what this would entail, please reach out to olive-support@sri.com.