Stigma Studios: 所有产品五折优惠

Speech To Life

Offline Speech Recognition for Windows and Android platforms including Quest 1 / 2

  • 支持的平台
  • 支持的引擎版本
    4.26 - 4.27, 5.0
  • 下载类型
    引擎插件
    此产品包含一款代码插件,含有预编译的二进制文件以及与虚幻引擎集成的所有源代码,能够安装到您选择的引擎版本中,并根据每个项目的需求启动。

Speech To Life is a voice recognition plugin for Unreal Engine which gives you performant offline voice recognition for Windows and Android platforms including Quest 1 / 2. It includes a sentence fuzzy matching library to allow conversation selection like gameplay and easily supports voice command control found in games where your voice causes actions/events to occur in the game world. This can be a very powerful mechanic in VR games where it adds an extra layer of 'presence'.


Some examples of usage:

  • Choose dialog choices with NPCs using your voice
  • Have NPCs react to things you say, give commands
  • Ghost hunting game questions to the ghost to get responses / make the ghost do things.
  • Add an extra layer of controls for VR/AR applications


Considerations:

  • Speech recognition is completely offline and doesn't require an internet connection.
  • A speech model is around 40-80mb in size on average and needs to be completely loaded into memory to be used. You should include this in your memory budget when using this plugin.
  • This plugin only supports Windows and Android platforms. Linux, MacOS, and iOS will come at a future time.
  • The plugin uses Vosk for voice recognition, but if there is interest / if I feel the need I will add new recognition systems. Vosk is great because it is offline, works on all platforms, and is fast.


Speech To Life can support over 20 languages using the exceptional voice recognition supplied by Vosk.


This plugin contains a BONUS Fuzzy String Matching library exposed to blueprint for using a number of fuzzy string matching methods to match a voice recognition result sentence (or the words of) to a number of strings you are checking for. You can use this library for purposes other than voice recognition matching as well.


The plugin can be simply implemented into your own project and contains a fully working example found here.

NOTE: The example project is for Engine 4.27 and greater. Let me know on the Solar Storm Interactive Discord if you need a version for 4.26 and we can arrange something.

Engine 5.0 Example Project

Engine 5.0 Quest Specific Example Project

Engine 4.27 Example Project


Demos can be found here:

Documentation can be found here

Join the Solar Storm Interactive Discord here to get support!


This plugin would not be possible without the amazing work done on the Kaldi and Vosk projects. This plugin also contains the RapidFuzz C++ library for matching recognizer output sentence strings to strings pre-defined by the designer to be checked against.


Vosk : https://alphacephei.com/vosk/

RapidFuzz : https://github.com/maxbachmann/RapidFuzz

技术细节

Features:

  • Offline, no internet required, voice recognition
  • Runs on Mobile Android Devices
  • Supports 20+ languages
  • Performant threaded implementation
  • Fuzzy matching library included for large sentence matching

Code Modules:

  • SpeechToUnreal (Runtime) Contains a game instance subsystem for globally controlling the current locale (language) used, a component to feed audio for voice recognition and get results, a function library for matching results in different ways, an extended audio device capture source for feeding microphone audio into the recognizer component, a bonus audio generator synth component for taking the microphone (or any audio generator source) and mixing it back into the games audio or to feed through mixers for creating walkie-talkie voice FX for example.
  • SpeechToUnrealRecognizers (Runtime) Recognizer implementation, currently includes Vosk but might contain more in the future if requested.
  • SpeechToUnrealEditor (Editor) For editor settings extension.
  • RapidFuzz (Runtime) A fuzzy string matching library

Number of C++ Classes: 10

Supported Development Platforms: Windows 64-bit

Supported Target Build Platforms: Windows 64-bit, Android, Oculus, Steam VR

Nativizable: Yes

Documentation: Here

Example Project: Here