Skip to content

Voice meetings#

A Lens Prism agent can join a Google Meet call and take part by voice. It listens, speaks, and uses its tools while the call is happening: ask it a question out loud and it answers out loud, running a lookup first if the answer needs one. Ask it in chat to join a meeting:

Join https://meet.google.com/abc-defg-hij

It joins signed into its own Google account, a host admits it like any other participant, and it leaves on request — or on its own once everyone else has gone.

Speaking is the agent's only channel from a call. It cannot post to the meeting's chat panel, to Slack, or to your chat session, and nothing said in the call is recorded or transcribed for later. Google Meet is the only supported platform.

Warning

A meeting is not an authenticated channel: anyone with the link can speak to the agent, and it cannot verify who they are. Anyone who addresses it by name has its full toolset — including the ability to change the agent itself. Share a meeting link only with people you would trust with the agent's shell and configuration. See What the agent will do.

Requirements#

Voice meetings need three things, and the agent's meeting tools appear only when the first two are present:

  • The container image. It ships the browser and audio toolchain (Chromium, Xvfb, PulseAudio) the agent drives to join a call. A native install has no browser, so voice meetings are container-only.
  • OPENAI_API_KEY. The in-meeting voice runs on OpenAI's Realtime API — the one model call that does not route through the configured provider (Bedrock, Azure, or Bedrock Mantle) or the Nexus gateway, because none of them serves a realtime voice surface. The socket connects straight to api.openai.com. The agent's chat model is unaffected.
  • A Google account for the agent (GOOGLE_BOT_EMAIL and GOOGLE_BOT_PASSWORD). Effectively required — see Why the agent needs its own Google account.

An egress-filtered deployment must allow api.openai.com along with Google's meeting and accounts hosts, including WebRTC UDP.

See Configuration for every variable and its default.

Why the agent needs its own Google account#

Google auto-denies automated anonymous guests: a join request without a signed-in account is refused about a second later, and the host is never shown a request to admit. This is policy, not bot detection — the same browser signed into an account joins normally. So the runtime does not attempt an anonymous join; asked to join with no account configured, it says so immediately.

Give the agent a dedicated account, not a person's:

  • It signs in once, and the browser profile is kept under the data directory, so later meetings reuse the session.
  • If the session goes stale, the agent signs in again on its own.
  • The account's display name is what participants see in the call.
  • The join tool takes an optional display name, but Google Meet ignores it — participants see the account's own name, and there is no setting for it.

An account inside the meeting's own Google Workspace is the smoothest case: invite it to the calendar event and it can skip the waiting room.

Warning

GOOGLE_BOT_PASSWORD must be the real password in plain text. Unlike other secrets, it cannot be a managed-credential sentinel — the agent types it into Google's sign-in form rather than sending it as a request header, so the container has to hold the actual value. Give the agent an account with nothing else attached to it. Accounts with a phone or hardware-key challenge, or enforced SSO, cannot be signed in unattended.

Joining a call#

The agent joins as its own signed-in participant, so a host has to admit it. The join tool returns as soon as the request is in rather than waiting — you get a note that it has requested to join, then a second note when it is admitted. It joins with no camera, appearing as a name and an avatar tile.

Some meetings will not admit it. A tenant can restrict a meeting to members of its own organization, and the agent reports what Meet told it rather than retrying. A meeting that has ended looks the same as one that refuses a guest — Meet shows one catch-all wall for both — so if a join fails, first confirm the meeting is still running with a host in it.

The agent handles one call at a time. Asking it to join a second call while it is already in one returns an error rather than dropping the first.

In the call#

The agent hears the room as one mixed audio channel with no speaker labels, so how much it speaks up depends on how many people are present:

  • One to one. It answers everything you say.
  • Anyone else present. It stays quiet unless called by name — every time, including follow-ups. There is no grace period after it speaks, because in a real room those seconds are mostly people beginning to talk to each other.

It switches between these as people join and leave, within about ten seconds. To override it for a given call, say so when you ask it to join:

Join https://meet.google.com/abc-defg-hij and only speak when someone says your name

The agent answers to its name as set in its workspace, so renaming the agent changes what summons it. The name is not matched literally — live transcription mangles names, so a check allows for a clear mishearing and tells the difference between talking to the agent and talking about it. The agent's own name and the other participants' names are passed to the transcriber as hints and refreshed as people come and go.

Where you put the name affects response speed. Say it early — "Prism, can you check the deploy?" — and the reply starts as soon as you stop talking. Leave it to the end and the agent adds a second or two of silence while it decides.

Other behavior in the room:

  • English only. The call's audio is transcribed as English so the agent can tell what was aimed at it. Another language degrades that rather than failing outright; the agent is told to say so and offer to continue in a chat session.
  • You can interrupt it. Start talking and it stops — a real interjection cuts it off, a short "mm-hm" or a cough does not, and it picks up from what it actually heard.
  • It keeps answers short. One to three sentences, spoken, with no markdown read aloud. For a command or a long list, it says what the thing does and offers to walk through it.

What the agent will do#

Inside a call the agent keeps every tool except file sharing, which is withheld in code because a meeting does not run the chat machinery that delivers a file. Nothing else is held back, so the guardrail is an instruction in the agent's prompt rather than the tool list: it announces what it is about to do and gets an explicit "yes" in the room before anything that deletes, deploys, spends money, grants access, or changes its own configuration.

That includes changing itself. Renaming itself, or rewriting its Soul, user profile, or memory, are things it will do on a spoken request the room agrees to. A sentence agreed in an unauthenticated meeting can therefore change the agent for every future turn on every channel, not just for the call. An instruction is a strong default, not enforcement — for a real restriction, start the agent with a narrower tool surface.

The meeting tools are chat-only: a heartbeat or scheduled task cannot dial into a call on its own.

Realtime audio is billed per audio token in both directions, and a meeting bot hears everything, so a quiet hour still costs. Expect roughly $0.10–0.30 per minute with the default model. Two things reduce it:

  • OPENAI_REALTIME_MODEL=gpt-realtime-mini is materially cheaper.
  • In a call with several people, staying quiet unless addressed cuts the expensive half — spoken responses — down to the turns meant for the agent. This happens on its own.

Every call is capped by MEETING_TIMEOUT_MINUTES (60 by default) so a meeting nobody ended cannot run up a bill. OpenAI's Realtime sessions also end at 60 minutes regardless of that cap; when a session ends, the agent drops out of the call rather than winding down, so ask it to rejoin if the meeting is still going.

Tell the other participants that the agent is in the call and what it is for. It introduces itself when admitted, but a bot listening to a conversation is something people should know about and agree to — in many places recording or processing a call without consent is a legal question, not only an etiquette one. Nothing is recorded by the runtime; whether OpenAI retains anything is governed by your OpenAI account settings.

Troubleshooting#

The agent says it cannot join meetings, or does not offer to. The meeting tools register only when the browser and audio toolchain and OPENAI_API_KEY are both present. Check the startup log for Voice meetings available or Voice meetings unavailable, and confirm you are running the container image.

It waits to be admitted and then gives up. Nobody admitted it, or the meeting blocks anonymous guests. It gives up after MEETING_ADMISSION_TIMEOUT_SECONDS and says so.

It says the meeting had ended, but the link was live. Meet shows one wall for an ended meeting, a dead link, and a refused guest. A rejection now has to survive three reads before it counts, and being in the call outranks anything on the page. Try joining again with a host present.

It says it could not sign in to its Google account. Google is challenging the account — usually a "verify it's you" prompt. Sign in once by hand using the persisted browser profile under the data directory, or clear that profile and let the next join sign in fresh. A screenshot of a failed join is saved under the data directory.

It is in the call but silent, or cannot hear. Almost always the audio devices — check the runtime log for parec or pacat warnings, and confirm the other participants are not muted. A heavily constrained container can starve Chromium of shared memory.

It left mid-call saying it lost its voice connection. The Realtime socket dropped. The agent leaves rather than sitting in the room unable to hear or speak, and it does not reconnect, because a fresh session would rejoin with none of its context. Ask it to join again; recurring drops are worth checking against egress filtering on api.openai.com.