enh-mamba-v1 (Enhancement - SEMamba)
Version Changelog
| Plugin Version | Change |
|---|---|
| v1.0.0 | Initial plugin release with OLIVE 6.2.0 |
Description
The enh-mamba plugin is a very powerful Speech Enhancement plugin that relies on the SE Mamba model architecture. It far exceeds the capabilities of previous enhancement plugins.
Domains
There is currently one domain included, that packages the default NVIDIA RE-USE Multilingual Universal Speech Enhancement model. In the future, there may be additional domains included that have been fine-tuned for specific purposes or data conditions, or to address encountered issues.
The included domains:
- default-v1
- The original NVIDIA RE-USE Multilingual Universal Speech Enhancement model. Very robust model capable of handling a wide range of interference and degradation, and several input sample rates.
Inputs
For redaction-voice transformation, an audio file is the expected and required input. At this time, the plugin does not accept optional regions to subselect portions to enhance; the entire file will be considered.
Outputs
The output of red-transform-v1 is an audio file that has been transformed according the REUSE model to attempt to enhance the quality and listenability of the model. The output will likely be dramatically improved to a human listener in most cases. The performance of machine systems may also improve, but the existence and amount of improvement will depend on the actual degradations and general audio conditions, and can also vary greatly depending on how each model was trained. For models that were trained exposed to a wide range of degradations as part of the model creation process, the improvements may not be as dramatic, since they are able to better handle the poorer audio quality.
Functionality (Traits)
The functions of this plugin are defined by its Traits and implemented API messages. A list of these Traits is below, along with the corresponding API messages for each. Click the message name below to go to additional implementation details below.
- AUDIO_CONVERTER – Take an audio file or buffer, potentially perform some modification(s) on it, and return audio to the requestor, this can be the same audio untouched, a modified version of this audio, or completely different audio, depending on the purpose of the plugin.
Compatibility
OLIVE 6.2+
Limitations
GPU Required
This plugin was designed and developed to run optimally on GPU hardware. It is currently not capable of running on CPU only.
Please refer to the OLIVE GPU Installation and Support documentation page for instructions on how to enable and configure GPU capability in supported plugins. By default this plugin will run on CPU only.