Skip to content

lid-embedplda-v4 (Language Identification)

Version Changelog

Plugin Version Change
v4.0.0 Initial plugin release builds off of v3.0.1 and adds GPU support, released with OLIVE 5.5.0
v4.1.0 Bug fix for increased memory usage for larger input audio files

Description

LID plugins analyze an audio segment to produce a detection score for each of the enabled language or dialect classes for the domain in use. A plugin domain could consist of 50 or more languages and dialects in a single plugin, or as few as one for use cases where the customer is only focused on a single target class. Some plugin domains are solely focused on dialect or sub-language recognition, such as languages of China. Several LID plugins allow users to add new classes or augment existing classes with more data for the class to improve accuracy.

Language recognition plugin for clean telephone or microphone data, based on a language embeddings DNN fed with acoustic DNN bottleneck features, and language classification using a PLDA backend and multi-class calibration. This plug-in has been configured to allow enrollment for the augmentation of existing classes and the addition of new classes. The backend and calibration modules are rapidly retrained with the addition of enrollment data.

Domains

  • multi-v1
    • Generic domain for most close talking conditions with signal-to-noise ratio above 10 dB. Currently set up with 16 languages configured (optionally configurable to up to 54 languages). See below for the currently-configured and available languages. See the configuring languages section for instructions on reconfiguring the available languages if necessary.

Inputs

Audio file or buffer and an optional identifier.

Outputs

Generally, a list of scores for all classes in the domain, for the entire segment. As with SAD and SID, scores are generally log-likelihood ratios where a score of greater than “0” is considered a detection. Plugins may be altered to return only detections, rather than a list of classes and scores, but this is generally done on the client side for sake of flexibility.

An example output excerpt:

        input-audio.wav Amharic -6.73527861
        input-audio.wav Arabic -3.31796265
        input-audio.wav English 8.22701168
        input-audio.wav French -2.98071671
        input-audio.wav Iranian Persian -5.55558729
        input-audio.wav Japanese -6.01283073
        input-audio.wav Korean -5.64162636
        input-audio.wav Mandarin -4.81163836
        input-audio.wav Portuguese -1.93523705
        input-audio.wav Russian -5.60199690
        input-audio.wav Spanish -3.70800495
        input-audio.wav Tagalog -4.86510944
        input-audio.wav Vietnamese -5.10995102

Functionality (Traits)

The functions of this plugin are defined by its Traits and implemented API messages. A list of these Traits is below, along with the corresponding API messages for each. Click the message name below to go to additional implementation details below.

Compatibility

OLIVE 5.2+

Limitations

Known or potential limitations of the plugin are outlined below.

All current LID plugins assume that an audio segment contains only a single language and may be scored as a unit. If a segment contains multiple languages the entire segment will still be scored as a unit. In many cases, a minimum duration of speech of 2 seconds is required in order to output scores. This value can optionally be overwritten, but scores provided for such short segments will be volatile.

Minimum Speech Duration

The system will only attempt to perform language identification if the submitted audio segment contains more than 2 seconds of detected speech.

Languages of Low Confidence

Many of the language models that are included and hidden within the domain's data model, disabled by default, do not contain enough data within the model for reliable detection of this language, and are included solely to help with score calibration, and differentiating other languages. If in doubt regarding whether an enrolled language should be used for detection or not, please reach out to SRI for clarification.

Comments

GPU Support

Please refer to the OLIVE GPU Installation and Support documentation page for instructions on how to enable and configure GPU capability in supported plugins. By default this plugin will run on CPU only.

Language/Dialect Detection Granularity

LID plugins attempt to distinguish dialects (ie., Tunisian Arabic and Levantine Arabic) or a base language class (such as Arabic). These can be mapped back to the base language if desired. This requires one change to be enabled.

  • A mapping file 'dialect_language.map' must exist within the domain of the plugin for which mapping is to be performed (eg. domains/multi-v1/dialect_language.map). This file is a tab-delimited, two-column file that lists each mapping for the dialect to the languages as "\t". Example lines include:
    Levantine Arabic    Arabic
Tunisian Arabic     Arabic

In the example above, the output labels of the dialects will be mapped to the same base language 'Arabic'. Note the exception in which mapping is not performed is for user-enrolled languages where it is assumed the user has provided the dialect or language label based on their requirements.

Note that we recommend users request these mapping files from SRI, or request the mapping to be performed before delivery of the plugin so that SRI can test and validate the final mapping before delivery.

Note also that the system will not allow you to create an enrollment with the same class name that you have languages mapped to. This is to avoid confusing situations where the system isn't sure if it should be considering the original pre-mapped models, or the newly enrolled user model. You must provide a unique name for any new language enrollments, that does not conflict with the dialect_langage.map. If you have already enrolled a conflicting model, and then add a mapping to this same name, the plugin will provide a warning message and intentionally fail to load.

Enrollments

Some recent LID plugins allows class modifications. A class modification is essentially an enrollment capability similar to SID. A new enrollment is created with the first class modification request (sending the system audio with a language label, generally 30 seconds or more per cut). A new language class will provide volatile scores unless sufficient cuts have been provided (approximately 10). In general, 30 minutes from around 30 samples is the minimum amount of data required to produce a reasonable language model. This enrollment can be augmented with subsequent class modification requests by adding more audio from the same language to an existing class, again, like SID or SDD. In addition to user enrolled languages, most LID plugins are supplied with several pre-enrolled languages. Users can replace these existing languages using their own data by enrolling audio with the same label as an existing language.

Note that LID enrollments are not independent. This means that the more languages the user enrolls from their domain, the better a given language they enroll will be.

Configuring Languages

Most LID plugins have the ability to re-configure the languages available in a domain. Configuring languages in the domain can be done by entering the domain directory of interest within the plugin folder and editing domain_config.txt. This file lists the pre-enrolled languages available in the plugin. Disabled languages are indicated by a # at the start of the line. To enable a language, remove the #. To disable a language, add a # at the start of the line.

Note that you cannot add languages to this list that are not supported by underlying models. If nonexistent language are added to this file, the plugin will intentionally fail.

Default Enabled Languages

The following languages are identified as high-confidence languages, supported by a sufficient amount of training data to make them reliable language detectors. As such, they are enabled by default in the plugin as-delivered, and serve as a general purpose base language set.

Language Name (cont'd)
Amharic Modern Standard Arabic
English Mandarin
French Pashto
Iranian Persian Portuguese
Japanese Russian
Korean Spanish
Levantine Arabic Tagalog
Iraqi Arabic Vietnamese

Supported Languages

The full list of languages that exist as an enrolled class within this plugin as delivered are provided in the chart below. Note that as mentioned previously, not all of these languages were enrolled with enough data to serve as reliable detectors, but remain in the domain for the benefits to differentiating other languages, and for score calibration. If in doubt regarding whether an enrolled language should be used for detection or not, please reach out to SRI for clarification.

Language Name
Albanian German Modern Standard Arabic Somali Wu
Azerbaijani Greek Macedonian Spanish
Belarusian Haitian Mandarin Swahili
Bengali Hausa Min Nan Tagalog
Bosnian Hindi Ndebele Tamil
Bulgarian Indonesian Oromo Thai
Burmese Iranian Persian Panjabi Tibetan
Cantonese Italian Pashto Tigrinya
Cebuano Japanese Portuguese Tunisian Arabic
Croatian Khmer Romanian Turkish
English Korean Russian Ukranian
French Levantine Arabic Serbian Uzbek
Georgian Iraqi Arabic Shona Vietnamese

Global Options

This plugin offers several basic user-configurable parameters which can be edited directly in plugin_config.py or passed via the API. Note that if changed in the plugin_config.py file, a server running the plugin will need to be restarted in order to use the new parameters, while parameters passed via the API are dynamically updated and do not require a restart of the server.

The options available and their default values are described below:

Option Name Description Default Expected Range
min_speech The minimum amount of detected speech in order to process a file. A higher value will prevent shorter files from being processed with the benefit of more reliable outputs. 2.0 0.5 - 4.0
sad_threshold The threshold used to determine speech for processing. A higher value results in less speech detected, while removing more noise. 1.0 -3.0 - 6.0