Skip to content

Integrating with the OLIVE Python API (olivepy)

olivepy is an OLIVE Python API that's built on top of the OLIVE Enterprise API.

New Python integrators are advised to read through the Enterprise API Primer for an overview of key concepts that also apply to olivepy.

Installation

olivepy is distributed as a Python wheel package for easy installation. To install, navigate into the api folder distributed with your OLIVE delivery, and install it using your native pip3:

    $ pip3 install olivepy-*-py3-none-any.whl

A source distribution is also available as olivepy-*.tar.gz, but it should only be needed for unique operating environments and most clients should use the wheel instead.

Dependencies

olivepy direct dependencies include:

  • protobuf
  • soundfile
  • numpy
  • requests
  • zmq

These will be downloaded by pip when the olivepy package is installed.

Integration

Quickstart

Here's a complete example for those in a hurry:

from olivepy.api.olive_async_client import AsyncOliveClient
from olivepy.messaging.msgutil import package_audio, serialize_audio, InputTransferType
from olivepy.messaging.olive_pb2 import Audio

# connect
client = AsyncOliveClient("example client")
client.connect()
print("Client connected...")

print("\n-- Connection Information --")
print(f"Client: {client.client_id}")
print(f"Server: {client.server_address}")
print(f"Port: {client.server_status_port}")

# find a Language ID (LID) plugin
plugin_request = client.request_plugins()
plugin_response = plugin_request.get_response()

for plugin in plugin_response.plugins:
    if plugin.task == "LID":
        plugin_id = plugin.id
        plugin_domain = plugin.domain[0].id
        break

# run Language ID (LID) on serialized audio file
print("\n-- Language ID Results --")
packaged_audio = package_audio(Audio(), serialize_audio("Edmund_Yeo_voice_ch.wav"), mode=InputTransferType.SERIALIZED)
analyze_request = client.analyze_global(plugin_id, plugin_domain, packaged_audio, None)
analyze_response = analyze_request.get_response()

top_scoring = analyze_response.score[0]
for score in analyze_response.score:
    print(f"{score.class_id}={score.score}")
    if score.score > top_scoring.score:
        top_scoring = score

print(f"\nTop scoring: {top_scoring.class_id} ({top_scoring.score})")

# disconnect
client.disconnect()
print("\nClient disconnected!")

Establish Server Connection

Before making any task request, a client must establish a connection with the server. By default, the OLIVE server listens on ports 5588 (request port) and 5589 (status port) for client connection and status requests. These ports are configurable, but if the server has not been instructed to change its listening ports, the code below should establish a connection. 

from olivepy.api.olive_async_client import AsyncOliveClient

client = AsyncOliveClient("example client")
client.connect()

...

client.disconnect()

The request port (5588 by default) is used for call and response messages. Each request message sent to this port is guaranteed a response from the server (this is why the messages in the API Message Reference are often suffixed with 'Request' and 'Result'). There is no need to poll the server for information on a submitted request, as the result/response for the a request is returned to the client as soon as it is available.

The status port (5589 by default) is used by the Server to publish health and status messages (Heartbeat) to client(s). Clients can not send requests on this port.

Request Available Plugins

In order to submit most server requests, the client must specify the plugin and domain to handle the request. To obtain the handle of a targeted plugin, the client first requests a list of all currently available valid plugins from the server. From the returned plugins list, the client looks up the specific plugin handle by the plugin's unique name (id) and its associated trait for the task to be performed. This handle can then be used in a future request message sent to the server.

from olivepy.api.olive_async_client import AsyncOliveClient

client = AsyncOliveClient("example client")
client.connect()

plugin_request = client.request_plugins()
plugin_response = plugin_request.get_response()
for plugin in plugin_response.plugins:
    print(f'Plugin {plugin.id} ({plugin.task},{plugin.group}) {plugin.version} has {len(plugin.domain)} domain(s):')
    for domain in plugin.domain:
        print(f'\tDomain: {domain.id}, Description: {domain.desc}')

client.disconnect()

The targeted plugin's id and domain can then be used with subsequent requests.

Note

Each plugin has additional useful information that can be retrieved as needed:

  • task: The type of task (e.g. LID, SID, SAD, KWS, AED, etc.)
  • trait: See Traits
  • group: Allows additional grouping of plugins such as Keyword, Speaker, Language, etc

Audio Submission Guidelines

One of the core client activities is submitting Audio with a request. olivepy provides three ways for a client to package audio data to send to the OLIVE server:

  • file path
  • buffer of raw audio sample data
  • serialized file buffer object

The enum InputTransferType is used to specify they type of audio transfer to use.

When the client and the OLIVE server share the same file system, the easiest way for the client to send audio data to the server is by specifying the audio's file path on disk along with InputTransferType.PATH:

packaged_audio = package_audio(Audio(), "/home/olive/samples/Edmund_Yeo_voice_ch.wav", mode=InputTransferType.PATH)

When the client and the server don't share the same file system, as in the case of a client making a remote connection to the OLIVE server running in docker (Martini), it is necessary to send the client's local audio files using InputTransferType.SERIALIZED or InputTransferType.DECODED:

packaged_audio = package_audio(Audio(), serialize_audio("Edmund_Yeo_voice_ch.wav"), mode=InputTransferType.SERIALIZED)

Serialized Buffer Note

Sending audio data as a serialized file buffer ensures that all audio header information is provided intact to the server. This allows the server to properly decode and process the audio once its received, since it can directly access the bit depth, encoding type, sample rate and other necessary information from the header itself. The tradeoff with serialized files is that there may be additional overhead needed to process the audio into a consumable form. If the client and server reside on the same hardware and file system, it is advisable to simply pass filepaths when possible. This saves the memory overhead burden of both the client and server loading audio into memory.

When using 16-bit PCM .wav files, or if first processing audio samples to be in one of the formats supported by the OLIVE server, it's also possible to pass a non-serialized buffer of raw samples along with InputTransferType.DECODED:

packaged_audio = package_audio(Audio(), audio_samples, mode=InputTransferType.DECODED, num_channels=1, sample_rate=8000, num_samples=1080)

Buffer of Raw Samples Caution

When submitting audio to the server as a buffer of raw samples, it is important to include information characterizing the audio, such as the bit depth, audio encoding, sample rate, and number of channels, to ensure the server knows how to properly treat the buffer it receives.

Synchronous vs. Asynchronous Message Submission

olivepy allows the client to choose between processing a task request synchronously or asynchronously. Processing a task request synchronously means the client will block and wait for the task result to return before proceeding to other task requests. On the other hand, asynchronous processing means the client will not wait for the result to come back before moving on, allowing several jobs to be submitted in parallel.

The olivepy.api.olive_async_client.AsyncOliveClient class supports both synchronous and synchronous submissions. Most requests sent to the server accept a typing.Callable that will be executed when the asynchronous request has been processed. When a synchronous request is desired (which is the case in most of the examples in this reference guide for conveience), then None needs to be explicitly passed in place of the argument reserved for the typing.Callable.

Legacy OliveClient Note

There is also a legacy olivepy.api.oliveclient.OliveClient that only supports synchronous submissions, but it will be deprecated in the future and all integrators are encouraged to switch over to the AsyncOliveClient.

Frame Score Request

The snippet below submits a Frame Score Request to the connected server.

analyze_request = client.analyze_frames(plugin_id, plugin_domain, packaged_audio, None)
analyze_response = analyze_request.get_response()

for i, score in enumerate(analyze_response.result[0].score):
    print(f"frame[{i}]={score}")

Note

Only a plugin that support the FrameScorer trait can handle this request.

See the Frame Scorer Messages section of the Enterprise API Message Reference for details about the FrameScorerRequest and FrameScorerResult objects.

Global Score Request

The snippet below submits a Global Score Request to the connected server.

analyze_request = client.analyze_global(plugin_id, plugin_domain, packaged_audio, None)
analyze_response = analyze_request.get_response()

for score in analyze_response.score:
    print(f"{score.class_id}={score.score}")

Note

The code required to submit a GlobalScorerRequest message doesn't care what type of plugin is going to be doing the scoring, as long as the plugin implements the GlobalScorer Trait. This means that the exact same code can be used for submitting audio to global scoring SID plugins, LID plugins, or any other global scoring plugin.

See the Global Scorer Messages section of the Enterprise API Message Reference for details about the GlobalScorerRequest and GlobalScorerResult objects.

Region Score Request

The snippet below submits a Region Score Request to the connected server.

analyze_request = client.analyze_regions(plugin_id, plugin_domain, packaged_audio, None)
analyze_response = analyze_request.get_response()

for region in analyze_response.region:
    print(f"{region.start_t:.2f}-{region.end_t:.2f}secs {region.class_id}")

Note

Only a plugin that support the RegionScorer trait can handle this request.

See the Region Scorer Messages section of the Enterprise API Message Reference for details about the RegionScorerRequest and RegionScorerResult objects.

Enrollments

See the appropriate section of the OLIVE Plugin Overview documentation for a description of Enrollments.

Enroll

The snippet below enrolls a sample audio file for speaker EDMUND_YEO:

# submit enrollment (i.e. Class Modification Request) for Speaker ID plugin using a serialized audio file
class_modification_request = client.enroll(plugin_id, plugin_domain, "EDMUND_YEO", "Edmund_Yeo_voice_ch.wav", None, mode=InputTransferType.SERIALIZED)
class_modification_response = class_modification_request.get_response()

for addition_result in class_modification_response.addition_result:
    status_label = "succeeded" if addition_result.successful else "failed"
    additional_details = f"Additional details for request: {addition_result.message}" if addition_result.message else ""
    print(f"Update to {plugin_id}/{plugin_domain} {status_label}! {additional_details}")

Not all plugins support enrollments. Refer to the individual plugin documentation to confirm if it supports class enrollment before proceeding.

Unenroll

The snippet below unenrolls speaker EDMUND_YEO:

# submit unenrollment (i.e. Class Modification Request) for Speaker ID plugin for speaker
class_modification_request = client.unenroll(plugin_id, plugin_domain, "EDMUND_YEO", None)
if (class_modification_request.is_successful()):
    print("Unenrollment was successful!")
else:
    print(f"Unenrollment failed with error: {class_modification_request.get_error()}")

Workflow Integration

In addition to the basic API integration mechanisms described above, olivepy also supports OLIVE Workflows.

See Workflows for a primer on OLIVE Workflows and then continue reading below for olivepy specific details.

Quickstart

Here's a complete workflow example for those in a hurry:

import os

from olivepy.api.olive_async_client import AsyncOliveClient
from olivepy.api.workflow import OliveWorkflowDefinition
from olivepy.messaging.msgutil import InputTransferType

# connect
client = AsyncOliveClient("example client")
client.connect()

# load the workflow_definition into the AsyncOliveClient to get a workflow helper object
workflow_definition = OliveWorkflowDefinition("sad_lid_sid.workflow.json")
sad_lid_sid_workflow = workflow_definition.create_workflow(client)

# package audio as a serialized buffer
serialized_audio = sad_lid_sid_workflow.package_audio("Edmund_Yeo_voice_ch.wav", InputTransferType.SERIALIZED, label=os.path.basename('Edmund_Yeo_voice_ch.wav'))

# submit workflow request on serialized audio file
response = sad_lid_sid_workflow.analyze([serialized_audio])

print(response.to_json(indent=2))

# disconnect
client.disconnect()

Initializing a Workflow

As described in Workflows, workflow logic is encapsulated in a Workflow Definition file distributed as either binary (i.e. *.workflow - deprecated) or JSON (i.e. *.workflow.json). Workflows are preconfigured to perform tasks such as Speech Activity Detection (SAD), Language Identification (LID), Speaker Identification (SID), etc. with a single call to the OLIVE server. These Workflow Definition files must be initialized (aka 'created') with the olivepy client before the workflow can be used.

The snippet below initializes a workflow with the client:

workflow_definition = OliveWorkflowDefinition("sad_lid_sid.workflow.json")
sad_lid_sid_workflow = workflow_definition.create_workflow(client)

A workflow 'helper' object (OliveWorkflow) is returned and is used to submit audio files directly to that workflow for analysis, enrollment, or unenrollment. In the snippet above, sad_lid_sid_workflow is the 'helper'.

Packaging Audio

The same Audio Submission Guidelines discussed earlier in this guide apply to workflows and each of the following are supported for workflows:

  • InputTransferType.PATH
  • InputTransferType.SERIALIZED
  • InputTransferType.DECODED

The difference with workflows is that the workflow 'helper' is used to package audios rather than olivepy client directly; the audio is wrapped in a WorkflowDataRequest that is submitted to OLIVE for processing:

# package audio as a serialized buffer using the workflow 'helper'
packaged_audio = workflow.package_audio("Edmund_Yeo_voice_ch.wav", InputTransferType.SERIALIZED, label=os.path.basename('Edmund_Yeo_voice_ch.wav'))

Multi-channel Audio

The default workflow behavior is to merge multi-channel audio into a single channel, which is known as MONO mode. To perform analysis on each channel individually instead of a merged channel, the Workflow Definition must be authored with a mode of SPLIT. When using the split mode, each channel in a multi-channel audio input is "split" into a job.

Here is a mode within a workflow definition file that merges multi-channel audio into a single channel audio input:

      "data_properties": {
        "min_number_inputs": 1,
        "max_number_inputs": 1,
        "type": "AUDIO",
        "preprocessing_required": true,
        "resample_rate": 8000,
        "mode": "MONO"
      },

... and one that handles each channel individually:

      "data_properties": {
        "min_number_inputs": 1,
        "max_number_inputs": 1,
        "type": "AUDIO",
        "preprocessing_required": true,
        "resample_rate": 8000,
        "mode": "SPLIT"
      },

Audio Annotations

The audio submitted for analysis (or enrollment) can be annotated with start/end regions when packaging audio using the package_audio() function. The snippet below specifies two regions within a file:

    # Provide annotations for two regions: 0.3 to 1.7 seconds, and 2.4 to 3.3 seconds in audio
    regions = [(0.3, 1.7), (2.4, 3.3)]
    packaged_audio = workflow.package_audio('Edmund_Yeo_voice_ch.wav', InputTransferType.AUDIO_SERIALIZED, annotations=regions)

Submitting Audio

Submitting audio for analysis, enrollment, and unenrollment using workflows is supported.

Analysis

The snippet below packages and submits an audio file to the workflow 'helper' for analysis:

# package audio as a serialized buffer
serialized_audio = workflow.package_audio("Edmund_Yeo_voice_ch.wav", InputTransferType.SERIALIZED, label=os.path.basename('Edmund_Yeo_voice_ch.wav'))

# submit workflow request on serialized audio file
response = workflow.analyze([serialized_audio])
Batch request

The analyze function accepts a list of list of audio files to so multiple files can be analyzed as a complete 'batch' request:

# package audio files for analysis
serialized_audio_1 = sad_lid_sid_workflow.package_audio("Edmund_Yeo_voice_ch.wav", InputTransferType.SERIALIZED, label=os.path.basename('Edmund_Yeo_voice_ch.wav'))
serialized_audio_2 = sad_lid_sid_workflow.package_audio("Edmund_Yeo_voice_en.wav", InputTransferType.SERIALIZED, label=os.path.basename('Edmund_Yeo_voice_en.wav'))

# submit workflow request on multiple serialized audio files
response = sad_lid_sid_workflow.analyze([serialized_audio_1, serialized_audio_2])

Enrollments

Enroll

Some workflows support enrollment for one or more jobs. To list the jobs in a workflow that support enrollment, use the get_enrollment_job_names() function:

print(f"Enrollment Jobs: {sid_enrollments_workflow.get_enrollment_job_names()}")
# Enrollment jobs '['SID Enrollment']'

To enroll a speaker via this workflow, use the workflow 'helper's enroll function:

# find a Speaker Identification (SID) enrollment job name
for enrollment_job_name in sid_enrollments_workflow.get_enrollment_job_names():
    if enrollment_job_name.startswith("SID"):
        sid_enrollment_job_name = enrollment_job_name
        break

# submit enrollment (i.e. Class Modification Request) for Speaker ID plugin using a serialized audio file
speaker_label = "EDMUND_YEO"
packaged_audio = sid_enrollments_workflow.package_audio("Edmund_Yeo_voice_ch.wav", InputTransferType.SERIALIZED, label=os.path.basename("Edmund_Yeo_voice_ch.wav"))

class_modification_response = sid_enrollments_workflow.enroll([packaged_audio], speaker_label, [sid_enrollment_job_name])

Since there was only one enrollment job in the workflow definition, the enrollment request could have been made without specifying the job name. However, it is a best practice to explicitly request the enrollment job by name.

Note also that not all workflows support enrollment. Please check that the workflow being used supports this before submitting an enrollment request.

To confirm the new speaker name was added:

sid_class_ids = sid_enrollments_workflow.get_analysis_class_ids()
print(sid_class_ids.to_json(indent=1))

Which produces the following output:

 {
 "job_class": [
  {
   "job_name": "SID analysis",
   "task": [
    {
     "task_name": "SID",
     "class_id": [
      "EDMUND_YEO"
     ]
    }
   ]
  }
 ]
}

'Class ID' is a general term used to describe 'labels' that apply to the specific plugin. Here, class_id represents all the speaker labels that are enrolled.

Unenroll

Workflow definitions that support enrollment often also support unenrollment. Similar to enrollment, use the get_unenrollment_job_names() to get a list of jobs that support unenrollment, send unenrollment requests to unenroll, and use get_analysis_class_ids() to list. Here is an example snippet of all three:

# find Speaker Identification (SID) unenrollment job name
for unenrollment_job_name in sid_enrollments_workflow.get_unenrollment_job_names():
    if unenrollment_job_name.startswith("SID"):
        sid_unenrollment_job_name = unenrollment_job_name
        break

# submit unenrollment (i.e. Class Modification Request) for Speaker ID plugin
speaker_label = "EDMUND_YEO"
class_modification_response = sid_enrollments_workflow.unenroll(speaker_label, [sid_unenrollment_job_name])

sid_class_ids = sid_enrollments_workflow.get_analysis_class_ids()
print(sid_class_ids.to_json(indent=1))

Which produces the following output:

{
 "job_class": [
  {
   "job_name": "SID analysis",
   "task": [
    {
     "task_name": "SID"
    }
   ]
  }
 ]
}

There is no class_id attribute because the speaker was successfully unenrolled and there aren't any others enrolled in the system.

Parsing Workflow Responses

A successful workflow request produces a response that includes information about the results. Information is grouped into one or more 'jobs', where a job is includes the name, tasks that were performed as part of the job, the results of each task, and potentially information about the input audio.

For example the following code snippet:

# package audio files for analysis
serialized_audio_1 = sad_lid_sid_workflow.package_audio("Edmund_Yeo_voice_ch.wav", InputTransferType.SERIALIZED, label=os.path.basename('Edmund_Yeo_voice_ch.wav'))
serialized_audio_2 = sad_lid_sid_workflow.package_audio("Edmund_Yeo_voice_en.wav", InputTransferType.SERIALIZED, label=os.path.basename('Edmund_Yeo_voice_en.wav'))

# submit workflow request on multiple serialized audio files
response = sad_lid_sid_workflow.analyze([serialized_audio_1, serialized_audio_2])

print(response.to_json(indent=2))

would produce the following output:

Example Workflow Output (click to expand)
[
  {
    "job_name": "SAD, LID, SID analysis",
    "data": [
      {
        "data_id": "Edmund_Yeo_voice_ch.wav",
        "msg_type": "PREPROCESSED_AUDIO_RESULT",
        "mode": "MONO",
        "merged": false,
        "sample_rate": 8000,
        "duration_seconds": 29.2825,
        "number_channels": 1,
        "label": "Edmund_Yeo_voice_ch.wav",
        "id": "cbc95af3f693de48654a72ec288adb8ad182a8f86993a3d0d42e3e2a5b4d5548"
      }
    ],
    "tasks": {
      "SAD": [
        {
          "task_trait": "REGION_SCORER",
          "task_type": "SAD",
          "message_type": "REGION_SCORER_RESULT",
          "plugin": "sad-dnn-v8.0.0",
          "domain": "multi-v1",
          "analysis": {
            "region": [
              {
                "start_t": 0.0,
                "end_t": 28.95,
                "class_id": "speech",
                "score": 0.0
              }
            ]
          }
        }
      ],
      "LID": [
        {
          "task_trait": "GLOBAL_SCORER",
          "task_type": "LID",
          "message_type": "GLOBAL_SCORER_RESULT",
          "plugin": "lid-embedplda-v4.0.0",
          "domain": "multi-v1",
          "analysis": {
            "score": [
              {
                "class_id": "Mandarin",
                "score": 6.0677257
              },
              {
                "class_id": "Korean",
                "score": -1.3737706
              },
              {
                "class_id": "English",
                "score": -4.7524
              },
              {
                "class_id": "Vietnamese",
                "score": -4.7861123
              },
              {
                "class_id": "Japanese",
                "score": -6.1542406
              },
              {
                "class_id": "Iraqi Arabic",
                "score": -8.356203
              },
              {
                "class_id": "Levantine Arabic",
                "score": -8.655978
              },
              {
                "class_id": "Tagalog",
                "score": -8.790578
              },
              {
                "class_id": "French",
                "score": -9.969812
              },
              {
                "class_id": "Modern Standard Arabic",
                "score": -10.236586
              },
              {
                "class_id": "Iranian Persian",
                "score": -11.295098
              },
              {
                "class_id": "Amharic",
                "score": -12.617832
              },
              {
                "class_id": "Spanish",
                "score": -14.459879
              },
              {
                "class_id": "Portuguese",
                "score": -14.844175
              },
              {
                "class_id": "Russian",
                "score": -15.082703
              },
              {
                "class_id": "Pashto",
                "score": -15.151446
              }
            ]
          }
        }
      ],
      "SID": [
        {
          "task_trait": "GLOBAL_SCORER",
          "task_type": "SID",
          "message_type": "GLOBAL_SCORER_RESULT",
          "plugin": "sid-dplda-v3.0.0",
          "domain": "multi-v1",
          "analysis": {
            "score": [
              {
                "class_id": "EDMUND_YEO",
                "score": 10.516918
              }
            ]
          }
        }
      ]
    }
  },
  {
    "job_name": "SAD, LID, SID analysis",
    "data": [
      {
        "data_id": "Edmund_Yeo_voice_en.wav",
        "msg_type": "PREPROCESSED_AUDIO_RESULT",
        "mode": "MONO",
        "merged": false,
        "sample_rate": 8000,
        "duration_seconds": 37.0825,
        "number_channels": 1,
        "label": "Edmund_Yeo_voice_en.wav",
        "id": "65a53db9b3ac1d1571082512cba37665634712738fed530ca00b3d5922e0d129"
      }
    ],
    "tasks": {
      "SAD": [
        {
          "task_trait": "REGION_SCORER",
          "task_type": "SAD",
          "message_type": "REGION_SCORER_RESULT",
          "plugin": "sad-dnn-v8.0.0",
          "domain": "multi-v1",
          "analysis": {
            "region": [
              {
                "start_t": 0.0,
                "end_t": 36.78,
                "class_id": "speech",
                "score": 0.0
              }
            ]
          }
        }
      ],
      "LID": [
        {
          "task_trait": "GLOBAL_SCORER",
          "task_type": "LID",
          "message_type": "GLOBAL_SCORER_RESULT",
          "plugin": "lid-embedplda-v4.0.0",
          "domain": "multi-v1",
          "analysis": {
            "score": [
              {
                "class_id": "English",
                "score": 5.056248
              },
              {
                "class_id": "Levantine Arabic",
                "score": -1.0983323
              },
              {
                "class_id": "Tagalog",
                "score": -1.4544584
              },
              {
                "class_id": "Iraqi Arabic",
                "score": -1.7203265
              },
              {
                "class_id": "Vietnamese",
                "score": -1.7773396
              },
              {
                "class_id": "French",
                "score": -2.4661431
              },
              {
                "class_id": "Korean",
                "score": -3.766336
              },
              {
                "class_id": "Mandarin",
                "score": -3.8002808
              },
              {
                "class_id": "Japanese",
                "score": -4.7714725
              },
              {
                "class_id": "Modern Standard Arabic",
                "score": -5.692904
              },
              {
                "class_id": "Spanish",
                "score": -6.5970984
              },
              {
                "class_id": "Iranian Persian",
                "score": -6.6558266
              },
              {
                "class_id": "Portuguese",
                "score": -8.045044
              },
              {
                "class_id": "Amharic",
                "score": -8.093353
              },
              {
                "class_id": "Pashto",
                "score": -9.445313
              },
              {
                "class_id": "Russian",
                "score": -11.597008
              }
            ]
          }
        }
      ],
      "SID": [
        {
          "task_trait": "GLOBAL_SCORER",
          "task_type": "SID",
          "message_type": "GLOBAL_SCORER_RESULT",
          "plugin": "sid-dplda-v3.0.0",
          "domain": "multi-v1",
          "analysis": {
            "score": [
              {
                "class_id": "EDMUND_YEO",
                "score": 7.9177084
              }
            ]
          }
        }
      ]
    }
  }
]

In the above output, note that the root array contains two objects because two audio files were submitted to the workflow.

Each object contains information about job including the job_name that identifies which job was run, the data attribute that contains information about the input audio, and the tasks attribute has holds results of each task performed.

Each task object with tasks has a message_type attribute identifies the type of output produced by the task which; it can be used to determine which fields will be available in the analysis attribute. See the Enterprise API Message Reference for a reference on what data will be available for each message type (ex. GlobalScorerResult and RegionScorerResult)

The results can be converted to a Python dict and iterated through programmatically:

# convert to Python dict and iterate
for result in json.loads(response.to_json()):
    label = result['data'][0]['label']
    for task_type, task_result in result['tasks'].items():
        plugin = task_result[0]['plugin']
        domain = task_result[0]['domain']
        analysis = task_result[0]['analysis']

        print(f"{task_type} on {label} using {plugin}/{domain}: {analysis}")

Which produces the following output:

SAD on Edmund_Yeo_voice_ch.wav using sad-dnn-v8.0.0/multi-v1: {'region': [{'start_t': 0.0, 'end_t': 28.95, 'class_id': 'speech', 'score': 0.0}]}
LID on Edmund_Yeo_voice_ch.wav using lid-embedplda-v4.0.0/multi-v1: {'score': [{'class_id': 'Mandarin', 'score': 6.0677257}, {'class_id': 'Korean', 'score': -1.3737706}, {'class_id': 'English', 'score': -4.7524}, {'class_id': 'Vietnamese', 'score': -4.7861123}, {'class_id': 'Japanese', 'score': -6.1542406}, {'class_id': 'Iraqi Arabic', 'score': -8.356203}, {'class_id': 'Levantine Arabic', 'score': -8.655978}, {'class_id': 'Tagalog', 'score': -8.790578}, {'class_id': 'French', 'score': -9.969812}, {'class_id': 'Modern Standard Arabic', 'score': -10.236586}, {'class_id': 'Iranian Persian', 'score': -11.295098}, {'class_id': 'Amharic', 'score': -12.617832}, {'class_id': 'Spanish', 'score': -14.459879}, {'class_id': 'Portuguese', 'score': -14.844175}, {'class_id': 'Russian', 'score': -15.082703}, {'class_id': 'Pashto', 'score': -15.151446}]}
SID on Edmund_Yeo_voice_ch.wav using sid-dplda-v3.0.0/multi-v1: {'score': [{'class_id': 'EDMUND_YEO', 'score': 10.516918}]}
SAD on Edmund_Yeo_voice_en.wav using sad-dnn-v8.0.0/multi-v1: {'region': [{'start_t': 0.0, 'end_t': 36.78, 'class_id': 'speech', 'score': 0.0}]}
LID on Edmund_Yeo_voice_en.wav using lid-embedplda-v4.0.0/multi-v1: {'score': [{'class_id': 'English', 'score': 5.056248}, {'class_id': 'Levantine Arabic', 'score': -1.0983323}, {'class_id': 'Tagalog', 'score': -1.4544584}, {'class_id': 'Iraqi Arabic', 'score': -1.7203265}, {'class_id': 'Vietnamese', 'score': -1.7773396}, {'class_id': 'French', 'score': -2.4661431}, {'class_id': 'Korean', 'score': -3.766336}, {'class_id': 'Mandarin', 'score': -3.8002808}, {'class_id': 'Japanese', 'score': -4.7714725}, {'class_id': 'Modern Standard Arabic', 'score': -5.692904}, {'class_id': 'Spanish', 'score': -6.5970984}, {'class_id': 'Iranian Persian', 'score': -6.6558266}, {'class_id': 'Portuguese', 'score': -8.045044}, {'class_id': 'Amharic', 'score': -8.093353}, {'class_id': 'Pashto', 'score': -9.445313}, {'class_id': 'Russian', 'score': -11.597008}]}
SID on Edmund_Yeo_voice_en.wav using sid-dplda-v3.0.0/multi-v1: {'score': [{'class_id': 'EDMUND_YEO', 'score': 7.9177084}]}