tpr-llm-commercial (Text Punctuation/Capitalization Restoration)
Version Changelog
| Plugin Version | Change |
|---|---|
| v1.0.0 | Initial plugin release with OLIVE 6.1.0. |
Description
Text Punctuation/Capitalization Restoration plugins add missing punctuation and capitalization to input text. This can increase the readability of automatic speech recognition transcripts and help downstream natural language processing tasks, such as machine translation.
The goal of this plugin is to ingest text without punctuation and output the same text with added punctuation and restored capitalization. This plugin performs this task using a large language model (LLM) external to the plugin, either run in OLIVE or hosted separately.
Domains
multi-v1- Uses an LLM to predict missing punctuation/capitalization. The accuracy as well as language support varies between different LLMs.
Inputs
A text string or text-populated file to process.
An example input string, in English:
i want you to go to town on saturday and buy a lot of little clothes all right if you won't give it to me i'll take it
Alternatively, output from an ASR plugin can be passed directly to the plugin via the optional parameter 'asr_regions' in a workflow.
Outputs
The output format for Text Punctuation/Capitalization Restoration plugins is simply text. The output text is the same text with added punctuation and restored capitalization.
An example output string with added punctuation/capitalization:
I want you to go to town on Saturday and buy a lot of little clothes. All right, if you won't give it to me, I'll take it.
Functionality (Traits)
The functions of this plugin are defined by its Traits and implemented API messages. A list of these Traits is below, along with the corresponding API messages for each. Click the message name below to be brought to additional implementation details below.
- TextTransformer – Plugin accepts and analyzes text string inputs, and outputs a new text string as output. In the case of Text Punctuation/Capitalization Restoration plugins, a text string is provided as input, with the expectation that the output will be a text string with added punctuation/capitalization.
Compatibility
OLIVE 6.1+
Limitations
This plugin is based on an LLM. Its performance critically depends therefore on the LLM's performance on the task, in particular for low-resource languages. There is often a correlation between the number of parameters of an LLM and its performance on complex tasks. Hence, larger LLMs tend to perform better. However, they also tend to use more resources and have lower processing speed given the same hardware.
We tested this plugin using Google's Gemma-3-4B-it-qat-q4_0-gguf and Gemma-3-12B-it-qat-q4_0-gguf. Performance with other LLMs may vary.
Comments
Large Language Model (LLM) Required
This plugin relies on a Large Language Model (LLM) for the heavy lifting of its task. This plugin can only be used with an appropriately configured OLIVE server that has been started with the LLM server active. See the LLM Configuration Documentation for more information, and refer to the Martini documentation to make sure the appropriate startup procedure is followed.
GPU Support
Please refer to the OLIVE GPU Installation and Support documentation page for instructions on how to enable and configure GPU capability in supported plugins. This plugin will run on CPU only; however, the LLM used in OLIVE may use GPU by default.
Text Transformation Options
The following region scoring options are available to this plugin, adjustable in the plugin's configuration file; plugin_config.py.
| Option Name | Description | Default | Expected Range |
|---|---|---|---|
| llm_base_url | LLM base URL of an OpenAI API compatible LLM server (such as llama-server, vLLM, or hosted LLMs such as OpenAI). | http://127.0.0.1:5007/v1 | |
| model | Name of the LLM model to use. | gemma3-4b | |
| api_key | API key/token to use for the LLM server, if required. | token | |
| asr_threshold | Threshold for using input words in optional 'asr_regions' input. (Only relevant for ingesting ASR region outputs directly from an ASR plugin in a workflow.) | 0.0 | |
| llm_ctx_window | LLM context window length, i.e., the number of tokens in the context window of the LLM. By default, the plugin queries the LLM server for the max. LLM context window. This setting is used as fallback if the query fails. | 8192 |
If you find this plugin to not perform adequately for your data conditions, or have a specific use case, please get in touch with SRI to discuss how the plugin can be tuned for optimal performance on your data.