tmt-llm-commercial-v1 (Text Machine Translation)
Version Changelog
| Plugin Version | Change |
|---|---|
| v1.0.0 | Initial plugin release with OLIVE 6.1.0. Requires LLM. Autodetects input language. Hard-coded prompt to translate to English (override with options) |
Description
Text Machine Translation plugins perform translation of text from one language to another, typically from one language, specified by the domain, to English. This plugin is the first to break the assumption that each MT domain is language specific, and instead, this plugin will auto-detect the input language and translate the output to English, by default. This is overridable at runtime by setting options (see below) if desired.
This plugin is essentially encapsulating a Large Language Model (LLM) prompt, rather than the model itself. It depends on the OLIVE server hosting an LLM server with an active LLM and will query the global model that the OLIVE LLM server is hosting at the time. The plugin will not function without this LLM available; please review the LLM configuration and LLM startup documentation to be sure this is available.
This plugin is a departure from previous plugins where all TMT domains were language-dependent and meant to work with a single, specific language pair (translating from one target language to another). This plugin will attempt to automatically detect the input language or languages, and translate to English.
If the input language is known, the user can override the automatic language detection by passing in an option.
Inputs
For scoring, a text string or text-populated file is required. There is no verification performed by OLIVE or by TMT plugins that the text passed as input is actually in a language that the plugin is capable of recognizing. The burden lies on the user to manually or automatically screen this audio before attempting to recognize. Note that output may fail or be very confusing if the input language does not match the domain's capabilities.
An example input string, in Spanish:
manual de fotografía para aprender todo lo esencial sobre fujifilm cámaras
Note that some punctuation and special characters will be stripped from the input during preprocessing.
Input Size
Caution about allowed length of input
This plugin is relying on an LLM to perform the actual translation. The server hosting the LLM has a 'context size' setting taht determines how many tokens it is able to consider at once when giving answer; and this setting is set globally for the LLM server. If the input to this plugin is too long, it may overrun this context size, causing incomplete or unexpected translations to come back. For best results, break the input into smaller pieces, or experiment with the context size setting for your LLM server deployment to learn where the limits lie. OLIVE does configure the context size to a default value for each of the available LLM models, to give a reasonable trade-off between performance and memory usage; but if a user has the GPU VRAM available, the value can be increased for more processing headroom. For more information on the Context Size and LLM Configuration, view the linked documentation page.
Outputs
The output format for TMT plugins is simply text. Words that the system does not recognize or can't translate may be tagged as unknown.
An example output string translation of the above example, as provided by the default multi-v1 domain and auto-detecting the language:
A photography manual to learn everything essential about Fujifilm cameras.
Note that unlike previous TMT plugins, this one is capable of automatically detecting sentence boundaries and inserting capitalization and punctuation for more human readable output.
Output Caveats
Unlike traditional TMT plugins, where the model performing the translation is all self contained within the plugin and domain, the model performing translation when using this plugin is a Large Language Model external to the plugin. This means that results and performance (both accuracy and speed/resource-wise) can and will vary depending on which LLM is being used. For more information on currently available LLM distributions, including their rough GPU memory requirements, please refer to the linked documentation.
In general, larger models will produce more accurate results, but at the cost of large GPU memory requirements, and often at a reduced speed.
Domains (Supported Languages)
multi-v1- Auto-detects the input language, and attempts to translate it into English text.
Functionality (Traits)
The functions of this plugin are defined by its Traits and implemented API messages. A list of these Traits is below, along with the corresponding API messages for each. Click the message name below to be brought to additional implementation details below.
- TextTransformer – Plugin accepts and analyzes text string inputs, and outputs a new text string as output. In the case of TMT plugins, a text string in the source language should be provided as input, with the expectation that the output will be a text string translated to the desired destination language.
Compatibility
OLIVE 6.1+
Limitations
This plugin is based on an LLM. Its performance critically depends therefore on the LLM's performance on the task, in particular for low-resource languages. There is often a correlation between the number of parameters of an LLM and its performance on complex tasks. Hence, larger LLMs tend to perform better. However, they also tend to use more resources and have lower processing speed given the same hardware.
We tested this plugin using Google's Gemma-3-4B-it-qat-q4_0-gguf and Gemma-3-12B-it-qat-q4_0-gguf. Performance with other LLMs may vary.
Context Length Disclaimer
If the LLM context length is reached, incomplete or unexpected results may occur. See the LLM Configuration page for more details
Spelling Errors
Note that there is no spell-checking or other types of spelling related pre-processing that occurs on the input data. Therefore any spelling mistakes in the input cause the system to output unexpected results, or even to auto-detect the wrong input language, causing odd or unexpected translations.
Comments
Global Options
| Option Name | Description | Default | Expected Range |
|---|---|---|---|
| tgt_lang_code | The 3-letter ISO-639-3 code representing the language to translate text to. | eng (English) | The exact set of supported language codes is up to the individual LLM being used. See the table below for the codes this plugin is expecting and will allow. Note that if not using the standard LLM models typically bundled with OLIVE, the presence of this code in the 'allowed' list below doesn't guarantee that it is supported by the LLM being used. |
Supported tgt_lang_code Table
| Language Code | Language Name |
|---|---|
| eng | English |
| spa | Spanish |
| fra | French |
| deu | German |
| ita | Italian |
| por | Portuguese |
| rus | Russian |
| cmn | Chinese |
| zho | Chinese |
| jpn | Japanese |
| jap | Japanese |
| kor | Korean |
| ara | Arabic |
| hin | Hindi |
| ben | Bengali |
| ind | Indonesian |
| tur | Turkish |
| vie | Vietnamese |
| nld | Dutch |
| swe | Swedish |
| pol | Polish |
| ukr | Ukrainian |
| hun | Hungarian |
| gre | Greek |
| cze | Czech |
| dan | Danish |
| tha | Thai |
| fin | Finnish |
| nor | Norwegian |
| heb | Hebrew |
| fas | Persian |
| urd | Urdu |
| tam | Tamil |
| tel | Telugu |
| mar | Marathi |
| guj | Gujarati |
| mal | Malayalam |
| kan | Kannada |
| pan | Punjabi |
| amh | Amharic |
| som | Somali |
| swa | Swahili |