TTS (NeuTTS)

On-device neural text-to-speech with batch and push-based streaming. Two public pipelines:

• NeuTTSMultilingual — Qwen3-based, 9 languages.

• NeuTTS (nano) — phoneme-based, English only, faster.

Flutter consumers go through the singleton start_model + infer / infer_stream (JSON) path — there is no direct TTS pipeline constructor on Dart. Both surfaces share the same on-disk cache and response shape.

Main features

• Two pipelines: NeuTTSMultilingual (Qwen3-based, 9 languages) and the faster English-only NeuTTS nano.

• Batch synthesis plus push-based streaming — pipe LLM tokens straight into TTS as they arrive.

• Custom voices: ship your own voice.json as app assets, selectable exactly like the built-ins.

• 24 kHz mono PCM output in [-1.0, 1.0], with optional resampling via output_sample_rate.

• Deterministic synthesis with the optional seed input.

Supported models

Model	HF Repo	Notes
NeuTTSMultilingual	TheStageAI/neutts-multilingual	Qwen3-based, 9 languages; phonemizes internally
NeuTTS (nano)	TheStageAI/neutts	Phoneme-based, English only, faster

The multilingual model supports:

english, french, german, spanish, portuguese, japanese, korean, chinese, urdu

The nano variant is English-only.

API surface

Singleton (TheStageAI)

TheStageAI is a Kotlin object (singleton); its methods are suspend.

TheStageAI.start_model(
    model_name = "tts",
    engines_path = "TheStageAI/neutts-multilingual",
    config = mapOf("voice_id" to "paul", "language" to "english")
)

val json = TheStageAI.infer(
    model_name = "tts",
    input_json = mapOf(
        "text" to "Hello, world!",
        "seed" to 42L                  // optional
    )
)
val audio = json[0]["audio"]

JSON response keys: audio, sample_rate (Int), duration (Double), tokens_per_second (Double), rtf (Double), plus per-stage timings.

JSON streaming yields typed InferenceStreamChunk values; PCM samples live on chunk.audio:

TheStageAI.infer_stream(
    model_name = "tts",
    input_json = mapOf("text" to "A long paragraph of text to speak.")
).collect { chunk ->
    val audio = chunk.audio
    if (audio != null && audio.isNotEmpty()) {
        play(audio, chunk.sample_rate ?: 24000)
    }
    if (chunk.is_final) return@collect
}

A push-based streamer is also reachable via the singleton:

val streamer = TheStageAI.open_tts_streamer(model_name = "tts")
// same `streamer.send(...)` / `streamer.finish()` shape as below

The Flutter TheStageFlutterSDK.infer / infer_stream calls hit this exact JSON path, so the response keys above apply unchanged on Dart. PCM audio crosses the platform channel as Float32List; do not promote to Float64List.

Direct constructors

val tts = NeuTTSMultilingual(
    engines_path = dir,               // local (prefetched) dir
    voice_id = "paul",                // voice subfolder under voices/
    language = "english"              // optional language override
)

val nano = NeuTTS(
    engines_path = nanoDir,
    voice_id = "dave"
)

Per-component device placement (LLM on NPU, NeuCodec pre/post on CPU) is set in the bundle’s metadata.json; when downloading through the singleton, pass overrides via start_model(device = ..., devices = ...). TheStageAI.initialize(...) must have succeeded before either call.

Quick start

Kotlin — via the singleton (recommended; auto-downloads the bundle):

// build.gradle.kts: implementation(files("libs/TheStageCore.aar"))
import ai.thestage.qlip.TheStageAI

// Inside a coroutine (initialize / start_model / infer are suspend).
TheStageAI.registerContext(context)
TheStageAI.initialize(api_token = "your-api-token")

TheStageAI.start_model(
    model_name = "tts",
    engines_path = "TheStageAI/neutts-multilingual",
    config = mapOf("voice_id" to "paul", "language" to "english")
)

val json = TheStageAI.infer(
    model_name = "tts",
    input_json = mapOf("text" to "Hello, world!")
)
val audio       = json[0]["audio"]                          // 24 kHz mono PCM
val sampleRate  = json[0]["sample_rate"] as Int             // 24000

The direct Kotlin pipelines load from a local (already-downloaded) engines directory — they do not download, so prefetch_model first:

import ai.thestage.qlip.models.neutts.NeuTTSMultilingual
import ai.thestage.qlip.models.neutts.NeuTTS

val dir = TheStageAI.prefetch_model(repo_id = "TheStageAI/neutts-multilingual")

val tts = NeuTTSMultilingual(
    engines_path = dir,
    voice_id = "paul",
    language = "english"
)

val result = tts.generate(text = "Hello, world!")
val audio = result.audio              // FloatArray, 24 kHz mono
val sample_rate = result.sample_rate  // 24000

The English-only nano variant follows the same shape:

val nanoDir = TheStageAI.prefetch_model(repo_id = "TheStageAI/neutts")
val nano = NeuTTS(engines_path = nanoDir, voice_id = "dave")

Flutter — JSON path:

import 'package:thestage_android_sdk/thestage_android_sdk.dart';
import 'dart:typed_data';

await TheStageFlutterSDK.initialize(api_token: 'your-api-token');

await TheStageFlutterSDK.start_model(
  model_name: 'tts',
  engines_path: 'TheStageAI/neutts-multilingual',
  config: {'voice_id': 'paul', 'language': 'english'},
);

final result = await TheStageFlutterSDK.infer(
  model_name: 'tts',
  input_json: {'text': 'Hello, world!'},
);
final audio       = result[0]['audio']       as Float32List;
final sampleRate  = result[0]['sample_rate'] as int; // 24000

Configuration

Inputs / Outputs

Direction	Type	Description
input text	String	Text to synthesize.
input config.seed	Long?	Deterministic sampling.
output NeuTTSResult.audio	FloatArray	24 kHz mono PCM, samples in [-1.0, 1.0].
output NeuTTSResult.sample_rate	Int	Always 24000.
output NeuTTSResult.audio_duration	Double	Seconds of audio.
output NeuTTSResult.rtf	Double	Real-time factor (duration / wall time).
output NeuTTSResult.tokens_per_second	Double	Decode speed.

Streaming hyperparameters

NeuTTSStreamConfig controls codec-side audio chunking and crossfading. Defaults match what the SDK ships with — only override these when you need to trade latency against smoothness.

Field	Default	Description
frames_per_chunk	25	Codec frames decoded per emitted audio chunk after the first. Larger = fewer, longer chunks.
first_frames_per_chunk	25	Frames in the first chunk; pass a smaller value (or null to fall back to frames_per_chunk) to lower time-to-first-audio.
lookforward	5	Future frames decoded together with each chunk to stabilise the seam (overlap-add window).
lookback	50	Past frames re-decoded for context when bridging chunks; reduces audible boundaries.
overlap_frames	1	Frames of crossfade between consecutive chunks.
realtime_gate	true	Paces streaming synthesis toward real-time to keep NPU heat/power down; false synthesizes as fast as possible.
output_sample_rate	0	Resample output to this rate in Hz; 0 keeps the native 24 kHz.
chained_chunks	model default	Keep decoder state continuous across a request’s chunks for smoother joins.

Generation knobs (temperature, top_k, …) live on TTSGenerationConfig and are independent of these.

Kotlin — pass stream_config: to open_streamer or infer_stream:

val streamer = tts.open_streamer(
    stream_config = NeuTTSStreamConfig(
        frames_per_chunk = 25,
        first_frames_per_chunk = 12,   // smaller first chunk → faster first audio
        lookforward = 5,
        lookback = 50,
        overlap_frames = 1
    )
)

val stream = tts.infer_stream(
    text = "Hello, world.",
    stream_config = NeuTTSStreamConfig(first_frames_per_chunk = 12)
)

The same knobs are exposed through the singleton:

val streamer = TheStageAI.open_tts_streamer(
    model_name = "tts",
    stream_config = NeuTTSStreamConfig(first_frames_per_chunk = 12)
)

Flutter — drop a stream_config map into input_json:

final stream = TheStageFlutterSDK.infer_stream(
  model_name: 'tts',
  input_json: {
    'text': 'Hello, world.',
    'stream_config': {
      'frames_per_chunk': 25,
      'first_frames_per_chunk': 12,
      'lookforward': 5,
      'lookback': 50,
      'overlap_frames': 1,
    },
  },
);

Unknown keys are ignored; defaults are kept for any field you omit.

Voices and languages

Voices live under voices/{voice_id}/ inside the bundle. Pass the language at construction time (it can be overridden per-voice default). The nano variant is English-only and ignores the parameter.

Streaming

open_streamer is push-based: collect streamer.output concurrently with streamer.send(...) so audio plays the moment each sentence is ready. Typical use case is piping LLM tokens straight into TTS.

Kotlin:

val streamer = tts.open_streamer()

val consumer = launch {
    streamer.output.collect { chunk ->
        val pcm = chunk.audio
        if (pcm != null) player.enqueue(pcm)
    }
}

streamer.send("Hello, world. ")
streamer.send("This sentence streams as it synthesizes.")
streamer.finish()   // flush remaining buffer + close `output`
consumer.join()

If you already have the full text up-front, infer_stream(text) does the same thing in a single call. Use streamer.cancel() instead of finish() to abort an in-flight turn (e.g. on barge-in) — that drops the buffer and closes immediately.

Flutter — push-based streaming uses infer_stream + send + finish_stream against a stable stream_id. Open the stream with empty text first, then push sentences as they become available:

const streamId = 'tts-utterance-1';
final player = TheStageAudioPlayer(sampleRate: 24000)..start();

// 1) Open the stream + start consuming chunks concurrently.
final consumer = () async {
  final stream = TheStageFlutterSDK.infer_stream(
    model_name: 'tts',
    input_json: {'text': ''},   // empty = wait for `send`
    stream_id: streamId,
  );
  await for (final chunk in stream) {
    final audio = chunk['audio'] as Float32List?;
    if (audio != null && audio.isNotEmpty) player.enqueue(audio);
    if (chunk['is_final'] == true) break;
  }
}();

// 2) Push sentences (e.g. from an LLM token stream).
await TheStageFlutterSDK.send(stream_id: streamId, text: 'Hello, world. ');
await TheStageFlutterSDK.send(
  stream_id: streamId,
  text: 'This sentence streams as it synthesizes.',
);

// 3) Signal end-of-input → flush remaining buffer, close the stream.
await TheStageFlutterSDK.finish_stream(stream_id: streamId);
await consumer;

For an already-known string, just call infer_stream with the full text and skip send / finish_stream.

Output contract

• 24 kHz mono FloatArray, samples in [-1.0, 1.0].

• Batch: NeuTTSResult.audio is the full utterance.

• Streaming: each InferenceStreamChunk.audio is one sentence-sized PCM slice; chunk.sample_rate is 24000. The streamer applies overlap-add crossfading between sentences, so consumers can concatenate slices end-to-end.

• If your playback path runs at 16 kHz to match VAD/ASR, set output_sample_rate: 16000 to have the SDK resample for you, resample the output yourself, or drive your TheStageAudioPlayer at 24 kHz (the bundled player defaults to 24 kHz).

See TheStage Android SDK (Audio I/O Contract) for the shared format used across VAD / ASR / TTS.

Lifecycle

1. TheStageAI.registerContext(context) + TheStageAI.initialize(api_token) — must have succeeded before any TTS call.

2. start_model (auto-downloads the bundle), or prefetch_model followed by a direct constructor (local-only, no network).

3. Call infer / infer_stream or open a streamer.

4. stop_model (singleton) or close() (direct pipelines) when done.

Usage Guides

Play synthesized audio

A batch result is 24 kHz mono PCM with samples in [-1.0, 1.0] — hand it to any float-PCM player at the documented rate.

Kotlin — play the FloatArray with a standard Android AudioTrack:

import android.media.AudioAttributes
import android.media.AudioFormat
import android.media.AudioTrack

val json = TheStageAI.infer(
    model_name = "tts",
    input_json = mapOf("text" to "Hello, world!")
)
val audio = json[0]["audio"] as FloatArray        // 24 kHz mono PCM
val sample_rate = json[0]["sample_rate"] as Int   // 24000

val track = AudioTrack.Builder()
    .setAudioAttributes(
        AudioAttributes.Builder()
            .setUsage(AudioAttributes.USAGE_MEDIA)
            .setContentType(AudioAttributes.CONTENT_TYPE_SPEECH)
            .build()
    )
    .setAudioFormat(
        AudioFormat.Builder()
            .setEncoding(AudioFormat.ENCODING_PCM_FLOAT)
            .setSampleRate(sample_rate)
            .setChannelMask(AudioFormat.CHANNEL_OUT_MONO)
            .build()
    )
    .setTransferMode(AudioTrack.MODE_STATIC)
    .setBufferSizeInBytes(audio.size * Float.SIZE_BYTES)
    .build()

track.write(audio, 0, audio.size, AudioTrack.WRITE_BLOCKING)
track.play()

Flutter — enqueue the Float32List into TheStageAudioPlayer at 24 kHz (see Streaming):

final result = await TheStageFlutterSDK.infer(
  model_name: 'tts',
  input_json: {'text': 'Hello, world!'},
);
final audio = result[0]['audio'] as Float32List;   // 24 kHz mono

final player = TheStageAudioPlayer(sampleRate: 24000)..start();
player.enqueue(audio);
await player.drain();
await player.stop();

If your playback path runs at a different rate, set output_sample_rate or resample yourself (see Output contract).

Custom voices

Beyond the voices baked into a bundle, an app can supply its own. Each custom voice is a folder with a single voice.json; the SDK merges them with the bundle’s built-in voices, so they show up in the voices list returned by start_model and are selectable by voice_id exactly like the built-ins.

Custom voices ship as assets inside your app. The custom_voices_assets config key is the name of the asset folder that holds them; the SDK extracts that folder out of the APK on load and merges the voices in. Inside it, each voice is its own subfolder with a single voice.json:

neutts_voices/
  my_voice/
    voice.json

Where that folder lives depends on the app type. Pass the same folder name (neutts_voices) either way — the SDK looks in both locations, so one config value serves a Flutter app, a native app, or one that carries both:

App	Put the folder at
Flutter	assets/<folder>/ in your project, declared under flutter: assets: in pubspec.yaml
Native Android	src/main/assets/<folder>/

Flutter — declare each voice file in pubspec.yaml, then pass the folder name:

flutter:
  assets:
    - assets/neutts_voices/my_voice/voice.json

await TheStageFlutterSDK.start_model(
  model_name: 'tts',
  engines_path: 'TheStageAI/neutts-multilingual',
  config: {
    'voice_id': 'my_voice',
    'language': 'english',
    'custom_voices_assets': 'neutts_voices',
  },
);

Native Android (Kotlin) — drop the folder at src/main/assets/neutts_voices/ and pass its name:

TheStageAI.start_model(
    model_name = "tts",
    engines_path = "TheStageAI/neutts-multilingual",
    config = mapOf(
        "voice_id" to "my_voice",
        "language" to "english",
        "custom_voices_assets" to "neutts_voices",
    )
)

voice.json shape

A voice.json is small (~4 KB) — reference text, the codec codes for that reference audio, and per-voice generation defaults (ref_codes truncated here; a real file holds the full code sequence for the reference clip):

{
  "name": "my_voice",
  "language": "english",
  "ref_text": "A short sentence spoken in the reference clip.",
  "ref_codes": [ [ 1234, 5678 ] ],
  "temperature": 0.7,
  "top_k": 50,
  "max_length": 1024,
  "use_lang_token": true
}

Field	Required	Description
name	yes	Display name; usually matches the folder.
language	yes	One of the supported languages (nano ignores it).
ref_text	yes	Transcript of the reference audio the codes came from.
ref_codes	yes	NeuCodec codes for the reference audio (authored offline).
temperature, top_k, max_length, min_new_tokens, use_lang_token	no	Per-voice generation defaults; omit to use the model defaults.

Note

ref_codes are authored offline. The on-device SDK ships the NeuCodec decoder only, so it cannot turn a reference .wav into codes on the phone. Produce the codes ahead of time with the NeuCodec encoder and paste them into voice.json.

To ship a voice with your app, drop its voice.json under your asset folder (see the table above for where that folder lives), point custom_voices_assets at that folder, and it appears in the voices list at load.

Troubleshooting

Symptom	Cause / Fix
Flutter audio glitches / NaNs	PCM promoted to Float64List or player rate mismatch. Keep audio as Float32List across the platform channel and match the player to the output rate.
Direct constructor can’t find engines	The direct Kotlin pipelines load from a local (already-downloaded) engines directory — they do not download. Call prefetch_model first.
Playback wrong speed on a 16 kHz path	Player rate ≠ 24 kHz output. Set output_sample_rate: 16000 to have the SDK resample, resample yourself, or drive TheStageAudioPlayer at 24 kHz.

Load Progress / Prefetch / Cleanup

Load progress

Download / extract / load progress is reported through the singleton and Flutter entry points via an optional on_load_progress handler that fires through four phases with a monotonic fraction in 0.0 .. 1.0:

TheStageAI.start_model(
    model_name = "tts",
    engines_path = "TheStageAI/neutts-multilingual",
    config = mapOf("voice_id" to "paul"),
    on_load_progress = { p ->
        println("[${p.model}] ${p.phase} ${(p.fraction * 100).toInt()}%")
    }
)

The same on_load_progress parameter is accepted by TheStageAI.prefetch_model(...). For the full event contract see TheStage Android SDK (Load Progress).

Flutter:

TheStageFlutterSDK.on_progress.listen((event) {
  if (event['model_name'] != 'tts') return;
  final phase    = event['phase']    as String?;   // downloading | extracting | loading | ready
  final fraction = event['progress'] as double?;   // 0.0 ... 1.0, monotonic
  print('[tts] $phase ${(fraction ?? 0) * 100}%');
});

await TheStageFlutterSDK.start_model(
  model_name: 'tts',
  engines_path: 'TheStageAI/neutts-multilingual',
  config: {'voice_id': 'paul', 'language': 'english'},
);

Prefetch engines

val engines_dir = TheStageAI.prefetch_model(
    repo_id = "TheStageAI/neutts-multilingual"
)

// Later — instant load, no network:
val tts = NeuTTSMultilingual(
    engines_path = engines_dir,
    voice_id = "paul"
)

Cleanup

The direct pipelines expose close() to release them. When you used the singleton API:

TheStageAI.stop_model(model_name = "tts")

Flutter:

await TheStageFlutterSDK.stop_model(model_name: 'tts');

• TTS (NeuTTS)
  • Supported models
  • API surface
    • Singleton (TheStageAI)
    • Direct constructors
  • Quick start
  • Configuration
    • Inputs / Outputs
    • Streaming hyperparameters
    • Voices and languages
  • Streaming
  • Output contract
  • Lifecycle
  • Usage Guides
    • Play synthesized audio
    • Custom voices
    • voice.json shape
  • Troubleshooting
  • Load Progress / Prefetch / Cleanup
    • Load progress
    • Prefetch engines
    • Cleanup