Skip to content

OLIVE Server

Overview

In the OLIVE architecture, if the individual plugins are the muscles of the system, the Server serves as the brains. It provides coordination and tasking, and is responsible for properly receiving and interpreting messages from client applications, kicking off the appropriate plugin jobs that these messages may request, as well as routing the proper response or results from these jobs back to the requesting client. The OLIVE Enterprise API is client/server based. Therefore, you must run the OLIVE server and manage its lifecycle as part of your integration effort. The server is included as part of the primary system installation.

The OLIVE server communicates with clients over two ports, 5588 and 5589 by default (configurable), using ZeroMQ and Google Protocol Buffers. It relies on other components to perform its duties; namely an assembly of third party and other libraries that are delivered with OLIVE as the OLIVE Runtime, two SRI-built libraries; Idento and dnn, that empower the final puzzle piece, the plugins themselves, to complete their assigned tasks. OLIVE is usually delivered with a simple startup script for ease of use and to facilitate a rapid deployment, so the actual details of the relationship between these pieces does not need to be considered too closely if sticking, but the next section outlines how to properly establish the environmental setup that the Server needs to properly function, and how to manually start and configure the server.

Installation, Environment Setup and Startup

This section outlines the steps needed to prepare the OLIVE environment for proper operation, describes the contents of the OLIVE runtime library, and covers starting the actual server. The final section of this page also outlines some common pitfalls encountered with running the OLIVE server, and offers troubleshooting advice.

Installation

If you haven't already installed the OLIVE software package, please jump over and refer to our OLIVE Installation Guide for information about the layout of the software and a quick-start setup guide before continuing on.

Environment Variables

As discussed in the Installation Guide, we recommend using the provided olive_env.sh scripts to do the heavy lifting and set up the OLIVE environment, but it may be helpful to know what some of the important environment variables being set are referring to. Below is a selection of some of the major variables, with a brief description of their function.

OLIVE

The OLIVE environment variable points to the actuall OLIVE installation location - If the software is installed as shown in the install page, this is typically something like $HOME/olive5.0.0/. From this, the OLIVE software is able to properly add the relevant bin/ and lib directories to the $PATH and $LD_LIBRARY_PATH environment variables, respectively, and find/set other important variables.

OLIVE_RUNTIME

OLIVE_RUNTIME points to the location of the OLIVE Runtime library outlined below. It's very important that this is set, as OLIVE will not function properly without access to the libraries and dependencies in the runtime. In the two examples in the install page, the OLIVE_RUNTIME variable would be set to:

    $HOME/olive5.0.0/runtime-5.0.0-centos-7.3.1611-x86_64/

OLIVE_APP_DATA

This variable is very important if running the OLIVE Server, but not necessary when using the OLIVE command line tools. OLIVE_APP_DATA tells the Server where to find the OLIVE plugins, and also determines where it will store things like class enrollments (i.e. speakers for SID plugins, keyword examples for QBE plugins), as well as server logs and other files that must be written out by the system. OLIVE_APP_DATA must contain a directory called plugins that contains valid OLIVE plugins. In the two examples in the install page, the OLIVE_APP_DATA variable would be set to:

    $HOME/oliveAppData/

Starting the Server

Once the OLIVE environment has been properly established, launching the server is as simple as entering this into the appropriate terminal:

    oliveserver

Additional runtime/configuration options are available for advanced operation, and their details can be found in the oliveserver usage statement:

usage: oliveserver [-h] [--version] [--verbose] [--interface INTERFACE] 
                  [--port REQUEST_PORT] [--upload_port UPLOAD_PORT] 
                  [--enable_oob_upload] [--workers WORKERS] [--work_dir WORK_DIR]
                  [--enroll_dir ENROLL_DIR] [--debug] [--monitor] [--quiet] 
                  [--timeout TIMEOUT] [--options OPTIONS_PATH]

optional arguments:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
  --verbose             increase server logging output
  --interface INTERFACE
                        server binds to this address; default * (all)
  --port REQUEST_PORT   the first of three sequentially numbered ports 
                        used by the server. The request port is the first 
                        port in this range, followed by the status port, and then an
                        interal port only used by the server; default value is 5588.
  --upload_port UPLOAD_PORT
                        port for out of band uploading of files
  --enable_oob_upload   enable out of band file uploads
  --workers WORKERS, -j WORKERS
                        specify number of parallel WORKERS to run; default is the 
                        number of local processors
  --work_dir WORK_DIR   path to work dir to create. default value of environment 
                        variable $OLIVE_APP_DATA
  --enroll_dir ENROLL_DIR
                        path for storage of enrollment data. default value of 
                        environment variable $OLIVE_APP_DATA
  --debug               debug mode prevents deletion of logs and intermediate 
                        files on success
  --monitor             special debug option to monitor memory usage. Must be 
                        used with the debug flag
  --quiet, -q           suppress sending log information to the console
  --timeout TIMEOUT     timeout, in seconds, for all jobs regardless of the 
                        audio duration. otherwise the job will timeout based 
                        on the duration of audio to process and the domain's
                        timeout_weight
  --options OPTIONS_PATH
                        Optional file containing name/value pairs. The option 
                        file must have more or more configuration sections. 
                        Currently only the server 'server' section is supported

An example for setting up the OLIVE runtime environment using the first directory setup shown on the install page, and launching the OLIVE server:

    $ cd $HOME/olive5.0.0/runtime-5.0.0-centos-7.3.1611-x86_64
    $ source olive_env.sh
    $ cd $HOME/olive5.0.0/olive-5.0.0-centos-7.3.1611-x86_64
    $ source olive_env.sh
    $ export OLIVE_APP_DATA=$HOME/oliveAppData/

    $ oliveserver --port 6678 --work_dir /home/sysadmin/OLIVE

In this example, the ports that the server will listen on and broadcast its status heartbeat messages on have been update, in addition to setting a custom WORK directory. For more information on the available server options, please refer to the OLIVE Server Options section below.

Note that upon launching the server, if the operation was successful, the user will be greeted with a display of the plugins and domains that the server has access to, as well as a “Server ready” message. If a list of plugins does not appear, but the “Server ready” message does, the OLIVE_APP_DATA environment variable should be checked to ensure that it contains a valid plugins directory. An example output of a successful server startup:

    [oliveuser@localhost olive5.7.0]$ oliveserver

TASK    PLUGIN                               VERSION    DOMAINS
------  -----------------------------------  ---------  -------------------------------------------
ASR     asr-end2end-v1.3.1                   1.3.1      english-augmented-v1 (gpu0)
                                                        english-spanish-v1 (cpu)
                                                        farsi-v1 (cpu)
                                                        french-v1 (cpu)
                                                        iraqiArabic-v1 (cpu)
                                                        japanese-augmented-v1 (cpu)
                                                        khmer-augmented-v2 (gpu0)
                                                        korean-augmented-v1 (cpu)
                                                        levantineArabic-v1 (cpu)
                                                        mandarin-augmented-v2 (gpu0)
                                                        pashto-v1 (cpu)
                                                        russian-augmented-v1 (cpu)
                                                        spanish-augmented-v1 (cpu)
                                                        ukrainian-augmented-v1 (cpu)
LID     lid-hdplda-v2.0.0                    2.0.0      multi-araDialects-v1 (cpu)
                                                        multi-v1 (cpu)
SAD     sad-dnn-v8.2.0                       8.2.0      fast-multi-v1 (cpu)
                                                        multi-v1 (cpu)
                                                        vtd-v1 (cpu)
SDD     sdd-diarizeEmbedSmolive-v1.0.4       1.0.4      telClosetalk-smart-v1
SHL     shl-sbcEmbed-v1.0.2                  1.0.2      micFarfield-v1
                                                        telClosetalk-v1
SID     sid-dplda-v3.1.0                     3.1.0      multi-v1 (cpu)
TMT     tmt-ctranslate-v1.3.0                1.3.0      english-mandarin-v2 (cpu)
                                                        english-russian-v2 (cpu)
                                                        english-spanish-v2 (cpu)
                                                        english-ukrainian-v2 (cpu)
                                                        iraqiArabic-english-v1 (cpu)
                                                        mandarin-english-text-v2 (cpu)
                                                        mandarin-english-v2 (cpu)
                                                        russian-english-v2 (cpu)
                                                        spanish-english-v2 (cpu)
                                                        ukrainian-english-v2 (cpu)

--------- Olive Server (v5.7.0) ready, Thu Jun 13 20:54:38 2024 ---------

The olive_env.sh file is included with OLIVE deliveries and does the bulk of the environment setup required for running the server. Important variables that are handled with this script and that are required for running the OLIVE Server include:

  • OLIVE_RUNTIME
    • Must be set to the path of the OLIVE runtime directory. Allows OLIVE to find the libraries it requires.
    • Ex: /home/user1/olive5.0.0-installation/runtime-5.0.0-centos-7.3.1611-x86_64/
  • OLIVE
    • Contains the path to the OLIVE installation directory. Allows OLIVE to find the binaries and utilities it relies on to function.
    • Ex: /home/user1/olive5.0.0-installation/olive-5.0.0-centos-7.3.1611-x86_64/ or /home/user1/olive5.0.0-installation/runtime-5.0.0-centos-7.3.1611-x86_64/olive-5.0.0-centos-7.3.1611-x86_64/
  • OLIVE_APP_DATA
    • This is the path used by the server to store class enrollments and log files. This is also where the plugins that the server will have access to must be stored, in a directory simply named plugins. If OLIVE_APP_DATA is set to a directory that does not contain a plugins folder, the server will launch with no plugins.
    • Ex: /home/user1/oliveAppData/
      • Where the contents of oliveAppData might be:
        • plugins/
          • sad-dnn-v4a
          • sid-embed-v2
          • lid-embed-v2
        • server/ (generated by the server)
          • enrollments/
          • processing/

If your server is protected by a firewall you must ensure that any ports you attempt to utilize for the server are able to send and receive network traffic. The path you provide to --work_dir will ideally point to a large and performant hard drive. The location will house the enrollments, log files and temporary workspaces utilized by the server.

Server Options

interface

Use this option to specify a single interface for the server to use for communication, otherwise the ports used by the server (5588, 5589, 5590 by default ) are available to all interfaces on the host. For example, --interface 192.168.10.20 would restrict the server to using only that interface, and only available to clients on that interface

port

Use this option to change the set of ports used by the server for external and internal communication. For example --port 6000 instructs the server to use ports 6000 (requests), 6001 (heartbeats), 60002 (internal communication)

upload_port

Specify the port used for experimental "out of band" upload feature. For advanced users only.

enable_oob_upload

Enable "out of band" file uploads. This is an experimental feature for very specific scenarios that should not be used without proper guidance.

workers

Use this option to specify the number of current jobs the server can process, this value should be set between 1 and the number of processors on the host

work_dir

Use this option to specify an alternate location of the directory where the server stores log files, enrollments, and temporary data. Setting this does NOT change the location of the plugins, which should still be located at $OLIVE_APP_DATA/plugins

enroll_dir

Use this option to specify an alternate location of the enrollments directory, when specified the work directory and plugins location is not changed.

debug

Use this option to preserver all logs files and provide extra output from OLIVE and plugins in some circumstances

monitor

Used in conjunction with the debug flag, enable a special memory-usage monitoring mode for internal troubleshooting.

verbose

Use this option with the --debug option to produce to maximize debug output

quiet

Use this option to start the server without non-error output messages

timeout

Use this option adjust the timeout (in seconds) used for jobs, unless instructed by OLIVE technical support this value should not specified

options

Used to pass special options to the server, this option should only be used with guidance from OLIVE technical support

Remote Access

The OLIVE server uses three sequential ports for communication; by default these are ports are 5588, 5589, and 5590. An alternate set of ports can be specified using the --port argument, which instructs the server to use 3 ports starting at the given value, so if --port 6000 is specified then ports 6000 (request port), 6001 (status port), and 6002 (internal port) are used.

Clients send and receive responses on the request port (5588 by default), while the server publishes health and status information on the status port (5589 by default). The internal IPC port (5590 by default) is only used for internal server communication.

You can restrict these ports to a specific interface by using the --interface option, so that the server is only available on that network (by default the server should be available on all interfaces, which may not be desirable on a multi-homed server). For example, specifying an interface of --interface 192.168.10.99 would restrict the server to communication only on that port. You could also use the interface option to restrict the server to local processing, not networked, by setting interface as: --interface 127.0.0.1 (on some OSs you may be able to specify --interface localhost instead of the loopback address, 127.0.0.1). If using the localhost interface, the server is not accessible to remote clients.

Important Information

Audio Formats

The OLIVE server can handle a wide range of input audio file formats, but does have some limitations. For more information on the details of these limitations, please refer to the OLIVE Audio Formats information page.

Common Pitfalls

One common mistake that's encountered is when informing the OLIVE Server where to find plugins. There is a directory within the OLIVE distribution, olive/plugins that is often mistaken for the directory where OLIVE plugins should oro do reside. This directory has a different purpose, however, and should not be used as such. Guidelines are provided above for general recommendations for where to place your OLIVE plugins.