Speech and Transcripts

Not implemented in this build. Would run Demucs stem separation on-device, producing 4 tracks: vocals, drums, bass, other.

Example prompt: separate the stems on track 1

Returns: node_id, stem track IDs

Inference ships as a stub, so this currently returns an error whatever you configure — there is no stem model to download and no environment variable that changes that. Only htdemucs is a recognised model id.

Turn one track and a set of speaker segments into a track per speaker, so each voice gets its own gain, EQ and noise treatment. Writes no new audio — every speaker track points at the same source file, which is why combined playback is unchanged. Audio no segment covers goes to an 'unassigned' track rather than being dropped, and overlapping turns are awarded to whoever was listed first and reported back.

Example prompt: split track 1 by speaker: Priya 0-12s, Sam 12-30s

Returns: node_id, tracks, speakers, unassigned_samples, overlapping_segments

Segments can come from anywhere — a diariser, a transcript that names speakers, or a cue sheet you type. Speaker turns land as ordinary clips, so you correct a boundary by dragging it.

Transcribe speech to text on this computer and attach the words, with times, to the session as a new version, replacing the earlier transcript. Give `track` (word times are then session seconds) or `path` for a file that is not in the session (the file's times), and optionally `language` as a two-letter code; leave it out to detect it. The audio never leaves the computer. It needs the Whisper base speech model the user downloads once in Settings → Account; the tool never downloads it. Runs on the CPU, usually in under half the audio's length, and the user can cancel.

Example prompt: transcribe track 1

Returns: node_id, word_count, language, duration_sec, cached, replaced_earlier_transcript, summary, and the words as [text, start_sec, end_sec] (the position is the word number cut_words takes), or sentence segments with word numbers for long audio

The listing is bounded to 6,000 bytes for each call, and it is what the AI provider receives: the text of what was said, never the audio, and with Ollama it stays on your computer. A transcript records the track it is of; cut_words and remove_fillers cut that track and refuse another. If the model is not downloaded or cannot be loaded, the result starts with a bracketed code and the window shows a card with the fix. No speech found changes nothing. duck_under_speech and select_region's speech-passage mode do not need a transcript: they can key on a voice track's audio.

Delete a span of transcribed words and the audio underneath, closing the gap. Indices are into the session transcript from `transcribe`. The remaining word timings shift so they still line up with the audio, and the whole thing is one undoable node. Needs a transcript (see transcribe), and cuts the track that transcript is of.

Example prompt: delete the bit where he repeats himself

Returns: node_id, removed_words, removed_text, removed_sec

The span runs from the first word's start to the last word's end — cutting from the first word's end would leave a clipped syllable behind.

Find filler words in the transcript and, when asked, remove them and their audio in one undoable edit. Reports by default without changing anything — this is a destructive edit across a whole track. Hesitations (um, uh, er) go wherever they appear; discourse markers (like, actually) only where they stand alone between pauses, because speech with every hesitation stripped sounds rushed. Needs a transcript (see transcribe), and cuts the track that transcript is of.

Example prompt: how many ums are in this?

Returns: found, would_save_sec, per-word list; node_id when applied

Leaves a short pause where each filler was, so the result does not sound spliced. The word list can be replaced — fillers are language- and speaker-specific.