Skip to content

Using the Web SDK

Create the API object with createGMPlugin() and await gm.ready() before calling phone or glasses capabilities.

Where to find code and type definitions#

  • Public implementation: PhoneSDK/packages/web-sdk/src/index.js.
  • Parameters and return type: PhoneSDK/packages/web-sdk/src/index.d.ts.
  • Permission format and method mapping: PhoneSDK/packages/bridge-contract/src/permission-policy.js.
  • For a minimal runnable project, follow “Your first Web plugin”. Do not copy handshake code from older native Bridge examples.

Find APIs by feature#

FeatureJavaScript entry pointDeclaration
Initialization and Capability Querygm.ready()、gm.runtime.getCapabilities()、getBridgeVersion()Complete the handshake first, then use returned capabilities to decide which features to show.
Glasses pagegm.display.createPage / updateText / updateImage / closePagedisplay; use with GM Web Bridge in Studio.
Device Statusgm.device.getInfo()device.info。
Button and head-gesture eventsgm.device.subscribeEvents / unsubscribeEvents；onButton / onGesture / onRawImu / onConnectiondevice.events, with scope.types declared.
Application datagm.storage.get / set / remove / clearstorage; use the host's isolated storage.
User filesgm.files.pick / list / stat / openRead / getUsage / deletefiles.user-selected; let the user select files instead of reading arbitrary system paths.
Phone–glasses messagesgm.plugin.sendMessage / onMessagedevice.messaging; declare request and response channels. Requires a matching GMP.
Recordinggm.audio.openCapture / openRecording / stopCapture / onCaptureStateaudio.capture; manage the returned sessions. Do not use the old startRecording API.
Foreground locationgm.location.getCurrentPosition / watchPosition / clearWatch / onPositionlocation.foreground; Studio may return simulated location values.
Networking and playbackStandard browser APIs such as fetch and audionetwork / audio.playback; system, CORS, CSP, and host-state restrictions also apply.

Initialize before starting application logic#

gm.ready() means Web has completed its host handshake; it does not mean the glasses GMP is running. The current App prepares the Web session and starts the glasses component in parallel. Glasses business communication and runtime results become available only when both sides are ready.

Do not wait for glasses display, a plugin reply, or a business network response before calling ready. Initialize the page and complete the handshake first, then run business logic according to user actions and host state. Handle temporary unavailability and timeouts.

Call gm.close() when an SDK instance is no longer needed. Before closing, stop recording, cancel event/location subscriptions, and release your own resources. Create a new instance when re-entering the page.

Subscriptions and listeners are different#

< / >
const stopListening = gm.device.onButton(event => {
  console.log('button', event);
});
const { subscriptionId } = await gm.device.subscribeEvents(['button']);

// When this feature stops:
await gm.device.unsubscribeEvents(subscriptionId);
stopListening();

Files and audio#

  • Files use fileId and controlled binary streams; short recordings and real-time capture use sessions. Follow the guides below for starting, consuming, stopping, and cleanup.
  • Desktop microphones and simulated location are development tools. Verify actual input, permissions, and background behavior on the target phone and glasses.

Where data is stored#

In the current App, gm.storage is isolated by account and installation instance. Same-name updates retain installation ownership. Do not assume that switching accounts or installing under a new name grants access to old data.

Desktop Studio stores gm.storage locally on the computer by Web manifest.id, preserving it across refreshes and reopening Studio. A copied example with the same ID may read the original example's saved data; changing only name does not create separate simulated storage. Change the ID for an independent application. For an upgrade of the same application, keep the ID and test migration of old saved data.

Source and static assets are delivered in the ZIP. Runtime saved data, user-selected files, and local authorizations remain under the current installation's ownership. Design explicit export/import functionality when data migration is needed.

Location: one-time reads and continuous listening#

  • Use gm.location.getCurrentPosition({ timeoutMs }) for a one-time location read. For continuous updates, register onPosition / onError first, then call watchPosition and save the watchId.
  • When finished, await gm.location.clearWatch(watchId), then remove local event listeners. Removing listeners alone does not release the host's location resources.
  • Results include WGS84 longitude and latitude, accuracy, timestamp, and precision. Do not present reduced precision as precise location or treat WGS84 values as another map coordinate system.
  • Query getCapabilities().location for actual limits. The current App allows at most one watch, at most one update per second, and a default timeout of 15 seconds. Location still depends on foreground state and system authorization.

Sending large Web images in tiles#

Split a large frame into independent tilesA tightly packed 600 × 350 GRAY_4 image uses 105000 bytes. To display multiple tiles together, call beginFrame, compress each tile independently with LZ4, and await sends in tileIndex order. The full frame appears only after the final tile succeeds. On failure, rebuild with a new frameId.MEMOMIND / FRAME TILESSplit a large frame into independent tilesPrepare each tile; do not split compressed streams arbitrarilyConvert to GRAY_4Split by imageregionRecord coordinatesand uncompressedsizebeginFrameSpecify frameId /tileCountConfirm host supportfirst.Await each tilesend in ordertileIndex：0, 1, …updateFrameImageLz4Show the frame when the last piece succeedsFinal tileconfirmedAll tiles in thisframe are presentDisplay the completeframeA tile failsStop assembling thisframeRebuild with a newframeIdAvoid these twomistakesSending one frame'stiles concurrentlyTreating PNG bytesas GRAY_4600 × 350 × 4 ÷ 8 = 105000 bytes is only the tightly packed pixel size. Tile sizes mustalso satisfy message-header, compression-format and host limits.
Split a large frame into independent tilesA tightly packed 600 × 350 GRAY_4 image uses 105000 bytes. To display multiple tiles together, call beginFrame, compress each tile independently with LZ4, and await sends in tileIndex order. The full frame appears only after the final tile succeeds. On failure, rebuild with a new frameId.MEMOMIND / FRAME TILESSplit a large frame intoindependent tilesPrepare each tile; do notsplit compressed streamsarbitrarilyConvert to GRAY_4Split by image regionRecord coordinates anduncompressed sizebeginFrameSpecify frameId / tileCountConfirm host support first.Await each tile send inordertileIndex：0, 1, …updateFrameImageLz4Show the frame when the lastpiece succeedsFinal tile confirmedAll tiles in this frame arepresentDisplay the complete frameA tile failsStop assembling this frameRebuild with a new frameIdAvoid these two mistakesSending one frame's tilesconcurrentlyTreating PNG bytes asGRAY_4600 × 350 × 4 ÷ 8 = 105000 bytes isonly the tightly packed pixel size.Tile sizes must also satisfymessage-header, compression-formatand host limits.
Send independent tiles in order. Present the frame only when all its tiles succeed.
  1. Start with standard gm.display APIs and verify a small image first. Do not pass PNG/JPEG bytes directly as GRAY_4 data.
  2. A tightly packed 600 × 350 GRAY_4 bitmap uses 105000 bytes, exceeding the current single Scene-message limit. Split it into independent tiles in the SDK format. Compress each tile separately with LZ4 and retain its uncompressed size.
  3. To display multiple tiles together, first check support for display.beginFrame and display.updateFrameImageLz4, then call beginFrame({ frameId, tileCount }).
  4. Starting at tileIndex 0, await updateFrameImageLz4 for each tile in sequence and check each result. The complete frame appears after the last tile succeeds.
  5. If a tile fails, rebuild the entire frame with a new frameId. Do not skip failed tiles or send tiles of the same frame concurrently with Promise.all.

Reference files in the toolkit#

  • PhoneSDK/packages/web-sdk/src/index.d.ts
  • PhoneSDK/examples/permission-debug
  • PhoneSDK/examples/audio-capture-lab
  • PhoneSDK/docs/web-plugin/runtime-and-lifecycle.md