Voice notes

A voice meeting goes through five steps: a member invites the bot, the bot listens to each speaker, a transcription service (automatic speech recognition, ASR) turns the speech into text, the text goes to the pipeline that reads uploaded transcripts, and the summary is posted where everyone who heard it can read it. For the member’s view, see Voice notes. The summary, owner matching and ticket steps are the same as for uploads and are on Meeting transcripts.

Settings

Voice notes are switched on with VOICE_NOTES_ENABLED, which also needs MEETING_TRANSCRIPTS_ENABLED and GEMINI_API_KEY, because Gemini still writes the summary. The transcription hosts, maximum meeting length, per-host timeout, shortest utterance and transcription concurrency are on Configuration. A blank value falls back to the default. See When it is off.

The parts

Part	What it does
Notes manager	Holds the one session state, answers /notes, enforces permissions, exclusions and the privacy rule, posts the notice, and ends the session
Voice session	Joins the channel, listens to each speaker, queues utterances and builds the transcript
Opus decoder	Turns each utterance’s Opus packets (Opus is the audio format Discord voice uses) into 16 kHz mono WAV, in a worker thread that is replaced when it fails
ASR client	Sends one WAV to the transcription fleet (the group of ASR hosts), choosing a host at random and failing over to the next host on an error
Meeting pipeline	The same extract, match, post and notify steps as an upload
Recovery	At startup, finishes meetings a restart interrupted

The voice connection uses Discord’s voice library with end-to-end encryption (DAVE). Decoding uses a WebAssembly Opus decoder, so there is no native build and no ffmpeg.

Starting

A member starts a session with /notes start, answered privately, or with Discord’s Invite to voice. That invite reaches the bot as a direct message carrying a discord.gg link. The bot reads only the first link in a message, allows 5 invites a minute per member, and starts the same session after checking that the invite is for a voice channel of this server and that the sender is in it and has Use Application Commands there. The checks below then apply to both, and the invite answers in the DM with the same texts.

The bot checks these in order and stops at the first that fails. The replies are on Voice notes.

1. The caller is in a voice channel. The bot reads the caller’s current voice channel from the server.
2. No session is running. There is one session for the whole bot. While one is starting, running or stopping, a second start is refused as busy. The channel is named only if the caller can see it.
3. The bot has the permissions. Connect, View Channel and Send Messages in that voice channel, checked live.
4. The channel is not excluded. An excluded channel refuses members. A member with Manage Server may still start there, because starting is the invitation. If the lookup fails, the start is refused and nothing is assumed.
5. The target is chosen. See Where the summary goes. This is the step that can create the summary channel, and only for a channel that is open to everyone.
6. The bot joins. Muted and not deafened, with a 30 second limit to become ready. A connection error is caught and logged as [voice] connection error: <name>, so it can never take the process down. Each connection state change is logged as [voice] connection <from> -> <to>, and the join as [voice] ready after N ms. A join that never becomes ready logs [voice] join did not become ready (last state <state>) after N ms: <name>. Only state names and timings are logged, nothing from the channel.
7. The notice is posted. In the voice channel’s text chat, with mentions off. If it cannot be posted, or the bot lost the connection while posting it, the bot leaves and refuses the start. Nothing is transcribed without it. The notice names the starter (cut to 40 characters), the channel the summary goes to, the retention in days, and whether the bot just created the summary channel or could not find or create it. Its wording is on Voice notes.
8. The session is active and the maximum-length timer starts.

The channel option for exclusions accepts voice and stage channels.

Listening

Discord sends one audio stream per speaker, so the bot knows who is speaking without guessing.

• Who is listened to. Anyone who starts speaking in the channel. Other bots, such as a music bot, are never listened to. Whether a speaker is a bot is checked once per person.
• One utterance. A stream ends after one second of silence. A person who talks for more than 30 seconds without a pause is cut into 30 second pieces.
• Dropped pieces. An utterance shorter than VOICE_MIN_UTTERANCE_MS (800 ms by default) is dropped, by both the length the packets claim and the length after decoding. Packets over 1,275 bytes or empty are skipped, because one oversized packet can corrupt the decoder.
• A stream that stops mid-speech. If the stream is destroyed while the person is still speaking, the bot listens again, up to three times in a row with nothing received.
• Names. A speaker is named by their server display name, collapsed to one line and cut to 40 characters. When the bot cannot find the member it uses User and the last four characters of their id.

Nothing is written to disk. The packets stay in memory until the utterance has been decoded and sent.

Decoding

Decoding happens in a worker thread, so it never blocks the bot. A 30 second clip takes about 90 ms. Each Opus packet is decoded to 48 kHz stereo, then the two channels and every three frames are averaged into 16 kHz mono and wrapped as a WAV file. If the decoder’s memory is corrupted, the worker is replaced and a clip queued behind it is tried once more on the new worker. A clip that takes over 15 seconds to decode retires the worker too.

Transcribing

Each utterance is sent to one ASR host with POST <url>/transcribe/file as a multipart upload named audio_file, with no key. The answer is JSON with a transcript and an optional language.

Rule	Value
Host choice	Weighted random among the hosts not yet tried for this utterance
Failover	On any error, the next host. Each host is tried at most once
Timeout	ASR_TIMEOUT_SECONDS per attempt
What counts as a failure	A timeout, a network error, a non-success status or an answer with no transcript text. The log line names the model and the class only, such as status 5xx
All hosts failed	The utterance counts as failed
Three failures in a row	Transcription pauses for 30 seconds. Utterances that arrive then count as failed and are not sent
Waiting utterances	Up to 60 wait for a free slot, ASR_CONCURRENCY (5 by default) run at once. The sixty-first is dropped and counted as failed
Which model answered	Stored with the transcript

The fleet is shared with other PlaySafe services and is sized without spare capacity. It answers 503 when it is saturated, which is why the bot pauses instead of retrying. Speech never falls back to Gemini.

The first pause in a meeting posts one notice in the voice chat, The transcription service is having trouble, so parts of this meeting may be missing from the notes. and logs [voice] ASR unavailable, pausing: AsrUnavailableError. The notice is not repeated for that meeting.

The transcript

When the session ends the bot waits for the utterances in flight, up to two minutes, then builds the text:

• One line per run of speech: Display Name: words, ordered by when each utterance began.
• Two lines from the same person are joined into one when the second starts no more than 5 seconds after the first ended.
• Empty answers are dropped.
• There are no time stamps.

The people who produced at least one line are the meeting’s participants. The list of models that answered is kept with the transcript.

Ending

Reason	Set when	Voice chat line starts with
Stop	/notes stop	Nothing
Empty	The last person who is not a bot left	Everyone left, so I stopped taking notes.
Maximum	VOICE_NOTES_MAX_MINUTES passed	The maximum meeting length was reached, so I stopped taking notes.
Disconnected	The bot was moved or disconnected, or the connection was gone for more than 5 seconds	I was disconnected from the voice channel, so I stopped taking notes.
Shutdown	The bot received SIGTERM or SIGINT	The bot is restarting, so I stopped taking notes.

Anyone in the call, the person who started the notes or a member with Manage Server may stop it. A channel being excluded while it runs also stops it. A session that ends for any reason other than /notes stop also sends the result to the person who started it as a direct message, limited by REMINDER_ALLOWLIST. After /notes stop, that person gets a direct message only when the write-up fails, see Writing the notes.

Ending stops listening at once, leaves the voice channel, waits for the transcription still in flight, writes the notes and then posts the result line in the voice chat. The bot is already out of the call while the notes are written.

Writing the notes

The meeting goes to the same pipeline as an uploaded transcript, see Meeting transcripts. The differences:

Step	For a voice meeting
Too short	A transcript under 200 characters, speaker names included, ends the run with no notes. Nothing is claimed
Claim	A row with source voice is inserted, keyed by the SHA-256 of the transcript, the voice channel id and the start time. The same meeting finishing twice finds that row and answers These notes were already posted.
Limits	The same two counters as uploads. One take from the starter’s allowance of 5 per 24 hours, then one from the shared daily allowance. They are in memory, so a restart resets them
Read	There is no file. A transcript over 400,000 characters is cut and ends with [transcript truncated]
Summary	The same prompt and limits. The model is told the lines are Name: words
Attendees	The people who spoke, not the names the model read
Owners	A name that matches exactly one person who spoke is that person, even if the bot does not otherwise know them, in which case the item has no assignee. In a call whose summary goes to the voice chat, nobody else is matched, so no one outside the call is offered an item. In a call whose summary goes to the shared channel, other names are matched to members by name as for uploads
Post	Not a reply, no ping, to the target channel
DMs	The same action item messages, linking to the summary

A failed run sends the starter a direct message, limited by the allowlist, after /notes stop too. The wording for each reason is on Voice notes.

If the AI provider is down, the run ends as ai_unavailable, pauses uploads and voice notes for 60 seconds as for uploads, and gives its limit slots back. Two cases follow:

• The meeting ends during an earlier pause. It is saved at once as failed with the reason aborted. The recovery sweep, started about 62 seconds after the pause began, takes every voice meeting saved that way and tries it once.
• The meeting’s own run hit the outage. It is marked failed with ai_unavailable. The transcript was saved before the AI was called, but reclaimVoice takes only aborted or stale processing rows, so the sweep lists the meeting and skips it. It is never written up, although the starter’s text says it will be. The transcript text is deleted at the retention time.

Both cases send the starter The AI service is unavailable right now. I kept what was said and will write up the voice meeting notes automatically in a few minutes. The only other text is for a meeting that ended during the pause with under 200 characters, so nothing was saved: The AI service is unavailable right now, so I could not write up the voice meeting notes. Please try again in a minute. The result line in the voice chat is the same in every case.

Where the summary goes

The rule is applied when the meeting starts, from live permissions, and it fails closed.

1. Does @everyone have both View Channel and Connect on the voice channel? If so, anyone in the server could have been in the call, so the target is the shared summary channel.
2. If not, or if the bot cannot read the permissions, the target is the voice channel’s own text chat. Its audience is exactly the people who could have been in the call.

The target is stored on the meeting as its channel_id, so the link in the action item messages and any recovery go to the right place. Recovery never falls back to the shared channel for a private chat that is gone.

For the shared channel the bot uses the name an admin set, or MEETING_CHANNEL_NAME when none was set. See The summary channel.

/notes status and the busy refusal name a channel only to a member who has View Channel on it, so a private meeting is never revealed by asking.

The summary channel

The shared summary channel is not created at startup. It is created the first time a meeting that goes to the shared channel starts.

• The bot looks for a text channel with the current name. If one exists it is used and nothing is created.
• Otherwise it creates the channel with the topic Upload a meeting transcript (.txt, .vtt, .srt, .md) for a summary and action items, or run /notes start in a voice channel. and no permission overwrites, so everyone in the server can read it. This is a single-server assumption.
• Creating it needs Manage Channels. If it cannot be created, the summary stays in the voice chat and the notice says so. The log shows [meetings] channel lookup failed: <class> or [meetings] channel create failed: <class>, with the Discord error code when there is one. Success is logged as [voice] created the summary channel #<name>.
• The current name is meeting_settings.summary_channel_name, loaded once at startup. A failed read stops startup, because running on the default could post to a channel the admin moved away from. /notes-admin summary-channel set stores the name first and updates the running bot only when the write succeeded.
• Uploads are read only in a channel with the current name. Until one exists, an upload is not read.
• Renaming never moves or deletes the old channel.

With voice notes off, nothing creates the channel

Only a voice meeting creates the summary channel. An install that uses uploads only has to create the channel by hand with the configured name. /notes-admin is registered whenever meeting transcripts are on, with or without voice notes, so an admin can still set the name with /notes-admin summary-channel set.

Restart and recovery

A meeting must not be lost when the bot restarts.

On shutdown the bot ends a running meeting first, before anything else stops. It leaves the voice channel at once, cancels the transcription still in flight and keeps what exists. Nothing waits for the AI. If the transcript has at least 200 characters it is saved with the meeting marked failed with the reason aborted. The voice chat gets The bot is restarting, so I stopped taking notes. What was said is saved and the notes will be finished when the bot is back. The shutdown has up to 8 seconds for this, inside the 25 second limit.

At startup, once Discord is ready and the channels are cached, the bot looks at up to 20 unfinished voice meetings, oldest first:

Found	What happens
A transcript saved with failed and reason aborted	Taken over and finished once
A processing meeting with a transcript that has not changed for 10 minutes	Treated as a crash and finished once
A posted meeting from the last 7 days, not touched for 10 minutes, whose action item messages are still unsent	The messages are sent. Only with a Linear connection

A row is taken with a compare-and-set (an update that succeeds only if the row still has the status the bot last read), so two processes cannot both recover it. The notes are posted in the channel the meeting was going to use, and the person who started the meeting gets A restart interrupted your voice meeting notes, so I finished them now: and a link, limited by the allowlist. Recovery takes one slot from the shared daily allowance when one is free, but is never refused by the daily or the personal limit. The people who spoke are restored from the attendee names saved with the meeting, otherwise from the names at the start of each transcript line.

One attempt per meeting. If it fails, for example because the AI is still down, the meeting is marked failed with the reason recovery_failed and is not tried again. A meeting marked ai_unavailable is listed but not taken. If the channel is gone it is recovery_failed too, and the starter is told The meeting notes channel is not available, so I could not post the voice meeting notes.

The startup logs [voice] recovery: N finished, M failed when anything was recovered.

Exclusions

An admin blocks a voice or stage channel with /notes-admin exclude. The block is checked at every start. It refuses members, not admins: a member with Manage Server can still start notes there, which is the explicit invitation. Blocking a channel with a running meeting stops that meeting. See Voice notes for admins.

When it is off

Situation	What happens
VOICE_NOTES_ENABLED is false	Logs [voice] disabled. /notes is not registered, and /notes-admin is registered only while meeting transcripts are on. The bot never joins a channel. ASR_HOSTS is not read
On, but MEETING_TRANSCRIPTS_ENABLED is off or there is no GEMINI_API_KEY	Logs [voice] off: needs MEETING_TRANSCRIPTS_ENABLED and GEMINI_API_KEY. The same as off
On with both	Logs [voice] enabled, N ASR hosts, max N min, explicit invite only

The commands appear in the server only after the bot next starts. Turning voice notes off removes them at the next start.

Next

• Voice notes for what members see.
• What the bot stores for what is kept.
• Data for the tables.
• Meeting transcripts for the shared pipeline.