Audio capture and short recordings

Verify the environment with Audio Capture Lab#

1. Build the complete toolkit using the quickstart. In Studio, select the phone's Audio Capture Lab and the matching glasses audio_capture_lab component. Keep the example manifest's strict pairing declarations.
2. Open the example's capability information and confirm unified audio-stream support. Use desktop simulated input to test capture, stopping, and playback logic.
3. Record for a few seconds and stop. Check duration, frame count, and data size, then switch to streaming mode and confirm that data continues to arrive.
4. Confirm that the App is connected to the glasses, scan and run the same ZIP, grant the requested permissions, and repeat capture, stop, and playback.
5. Test repeated starts, cancellation, disconnection, screen locking, and exit separately. Starting again after stopping should work, and the page must show the actual result.

• Scanning and Bluetooth runtime steps
• Match SDK and host versions

Check which API family the current host supports#

This page uses the current PhoneSDK unified-stream API: openCapture(options) takes no mode parameter, and getCapabilities().audio directly exposes transport, payload, profiles, and related fields. Older hosts using audio.modes.recording / stream follow a different contract.

If audio capabilities are entirely absent, first check that audio.capture is declared and authorized, then confirm host capture support. Investigate an API-version mismatch only when an old modes structure or an explicit mode-parameter error is returned.

If the capability structure is old or the error says “audio capture mode must be recording or stream”, switch to a Studio/App version matching the SDK, then rebuild the example. Adding mode to the new SDK is also rejected. Do not treat this as a microphone permission issue.

Even with identical Bridge version numbers, check the actual capability structure and matching release versions. Verify with getCapabilities() and an unmodified example operation; method-name existence alone is insufficient.

Choose between streaming and short recordings#

Consume the stream, then end it with the stop button#

Add this function to an existing plugin.js and use it after gm.ready(). The start button saves the stop function returned by await startAudioMeter(gm); the stop button calls it later. Disable duplicate starts while handling the button, catch errors, and update page state.

This example captures for at most 10 seconds and only logs chunk information. Read stream immediately after starting, and drain the stream tail when stopping. Business uploads need a bounded queue and timeout handling; do not accumulate unlimited audio.

export async function startAudioMeter(gm) {
  const { audio } = await gm.runtime.getCapabilities();
  if (audio?.transport !== 'message-port' ||
      audio?.payload !== 'binary-envelope-v1') {
    throw new Error('A matching Host with unified audio streaming is required');
  }
  const capture = await gm.audio.openCapture({
    profile: 'interactive', pickupMode: 'frontBalanced',
    noiseReduction: true, maxDurationMs: 10000,
  });
  const reader = capture.stream.getReader();
  let streamError;
  const consuming = (async () => {
    try {
      while (true) {
        const { value, done } = await reader.read();
        if (done) break;
        console.log(value.sequence, value.data.byteLength,
          value.frameLengths, value.discontinuity);
      }
    } catch (error) {
      streamError = error;
      await capture.stop().catch(() => {});
    } finally {
      reader.releaseLock();
    }
  })();
  return async function stop() {
    try { await capture.stop(); } finally { await consuming; }
    if (streamError) throw streamError;
  };
}

Audio formats, limits and playback#

Stopping, errors and background operation#

• Listen through gm.audio.onCaptureState and use sessionId to distinguish the current session from old ones. Handle Host duration-limit stops and errors, and remove the listener when finished.
• The current audio path supports only one native capture operation at a time. For repeated starts or contention with other audio features, end the old operation or ask the user to try later.
• A normal stop retains tail data in the bounded queue. Cancellation, disconnection, or stream errors cannot guarantee complete audio. Tell the business layer explicitly whether the result is usable.
• Active capture while the phone is locked differs from exiting the plugin. Background policy depends on the App version and OS. Create a new session when re-entering after exit, reload, or session stop.
• A desktop microphone cannot validate glasses pickup quality. If native capture is denied or fails, do not silently switch microphones and report success.

• Background operation and lifecycle
• Common errors and recovery steps

Reference files in the toolkit#

• PhoneSDK/packages/web-sdk/src/index.js
• PhoneSDK/packages/web-sdk/src/index.d.ts
• PhoneSDK/packages/bridge-contract/src/index.js
• PhoneSDK/examples/audio-capture-lab/plugin.js
• PhoneSDK/examples/audio-capture-lab/manifest.json
• PhoneSDK/examples/talking-pet/README.md