Skip to content

ner-llm-commercial (Named Entity Recognition)

Version Changelog

Plugin Version Change
v1.0.0 Initial plugin release with OLIVE 6.1.0.

Description

Named Entity Recognition plugins extract key information in terms of categories - in this case 'people,' 'organizations,' and 'locations' (places) from unstructured text inputs. They transform raw text inputs into structured data, thereby enabling easier subsequent filtering or processing.

This plugin performs this task using a large language model (LLM) external to the plugin, either run in OLIVE or hosted separately.

Domains

  • multi-v1
    • Uses an LLM to extract named entities of the categories people, organizations, and locations (places). The accuracy as well as language support varies between different LLMs.

Inputs

A text string or text-populated file to process.

An example input string, in English:

    Jim bought 300 shares of Acme Corp. in 2006.

Alternatively, output from an ASR plugin can be passed directly to the plugin via the optional parameter 'asr_regions' in a workflow.

Outputs

The output format for Named Entity Recognition plugins is JSON-formatted text, with a mapping between named entity category labels "People", "Organizations", and "Locations" and lists of extracted named entities.

An example output string with extracted named entities:

    {"People": ["Jim"], "Organizations": ["Acme Corp."], "Locations": []}

Alternatively, if the input is directly provided as output from an ASR plugin via the optional parameter 'asr_regions' in a workflow, the plugin will return time regions marking the start and end times of each extracted named entity.

Functionality (Traits)

The functions of this plugin are defined by its Traits and implemented API messages. A list of these Traits is below, along with the corresponding API messages for each. Click the message name below to be brought to additional implementation details below.

  • CompositeScorer – Plugin accepts and analyzes one or multiple inputs, and outputs a combination of known Scorer result types as output. In the case of the Named Entity Recognition plugin, text strings are provided as input, with the expectation that the output will be a text string encoding named entity categories as JSON object with a mapping between named entity category labels "People", "Organizations", and "Locations" and lists of strings with the extracted named entities. If a transcript from an ASR plugin is provided directly via the optional parameter asr_regions in a workflow, the plugin will extract the named entities from the transcript and will return time regions marking the start and end times of each extracted named entity as regions.

Compatibility

OLIVE 6.1+

Limitations

This plugin is based on an LLM. Its performance critically depends therefore on the LLM's performance on the task, in particular for low-resource languages. There is often a correlation between the number of parameters of an LLM and its performance on complex tasks. Hence, larger LLMs tend to perform better. However, they also tend to use more resources and have lower processing speed given the same hardware.

We tested this plugin using Google's Gemma-3-4B-it-qat-q4_0-gguf and Gemma-3-12B-it-qat-q4_0-gguf. Performance with other LLMs may vary.

Comments

Large Language Model (LLM) Required

This plugin relies on a Large Language Model (LLM) for the heavy lifting of its task. This plugin can only be used with an appropriately configured OLIVE server that has been started with the LLM server active. See the LLM Configuration Documentation for more information, and refer to the Martini documentation to make sure the appropriate startup procedure is followed.

GPU Support

Please refer to the OLIVE GPU Installation and Support documentation page for instructions on how to enable and configure GPU capability in supported plugins. This plugin will run on CPU only; however, the LLM used in OLIVE may use GPU by default.

Text Transformation Options

The following region scoring options are available to this plugin, adjustable in the plugin's configuration file; plugin_config.py.

Option Name Description Default Expected Range
llm_base_url LLM base URL of an OpenAI API compatible LLM server (such as llama-server, vLLM, or hosted LLMs such as OpenAI). http://127.0.0.1:5007/v1
model Name of the LLM model to use. gemma3-4b
api_key API key/token to use for the LLM server, if required. token
llm_ctx_window LLM context window length, i.e., the number of tokens in the context window of the LLM. By default, the plugin queries the LLM server for the max. LLM context window. This setting is used as fallback if the query fails. 8192

If you find this plugin to not perform adequately for your data conditions, or have a specific use case, please get in touch with SRI to discuss how the plugin can be tuned for optimal performance on your data.