Skip to content

Phone and paired interactions

The phone handles complex interactions; the glasses provide immediate feedback. Track connection, plugin runtime, and business-task states separately to report accurate results after disconnection, backgrounding, and restart.

App UI design reference#

App UI design guidelines · FigmaView design references for App UI and interactions.

App UI guidelines#

Use the phone for long input, settings, file selection, and result management unsuitable for the glasses. Show the current application, device, and task states before presenting actionable primary controls.

Page areaRecommended practiceVerification
Equipment and application statusShow disconnected, plugin-starting, and business-running states separately. One green indicator must not imply that everything succeeded.Disconnect Bluetooth, stop the GMP, and cancel a business task separately. Check that the messages distinguish these cases.
Settings and savingOrganize fields by user goal. Explain where settings are saved and when they apply to the glasses.Preserve unsubmitted input if saving fails. Confirm synchronization from actual replies.
Files and authorizationExplain the purpose first, then let the user initiate selection or authorization. Explain the impact of denial.After canceling file selection or denying authorization, users must be able to return to the page without repeated prompts.
Long-running tasksShow the object and processing stage, with stop or cancel actions that are actually supported.Leave and return to the page while awaiting a reply, upload, or processing. Check whether the result can be queried again.
Failure and recoveryKeep usable content and emphasize one primary recovery action, such as reconnecting or querying status.Button behavior must match its label. Retrying must not repeat uncertain side effects.

Distinguish these three states#

Status ObjectSource of truthHow to present it
Device Connectionconnected from gm.device.getInfo(), plus subscribed connection events.When disconnected, explain that the user must connect the glasses in the App. A Bluetooth connection does not confirm completion of a business operation.
Web runtimegm.ready(), gm.runtime.getLifecycleState() and runtime.lifecycleChanged.Stop unnecessary work when the host suspends. Reinitialize after runtime recreation.
Your business taskConfirmation comes from your business protocol or server. The SDK does not define your application's task states.Show waiting, running, completed, failed, canceled, or result-unconfirmed states separately.

When a command counts as successful#

gm.plugin.sendMessage() returns sent, channel and payloadBytes, indicating that the message has been sent to the transmission path. It does not generate business completion confirmation for your custom GMP. Display Control Lab registers a pending request before sending it, then completes that request using requestId and statusCode in the glasses reply.

After sending, wait for the business resultRegister the pending requestId before sending. After transport succeeds, wait for the glasses reply and update the result by requestId and status. A timeout means the result is unconfirmed; query first before deciding to retry.MEMOMIND / COMMAND RESULTAfter sending, wait for the business resultSend and confirmRegister thepending requestfirstGenerate requestIdShow processing inthe UISubmit the messagesendMessage returnssentBusiness completionis still notconfirmedReceive a businessreplyMatch requestId /statusUpdate UI from theglasses resultHandle each failure separatelySending failedEnd this waitShow Transfer ErrorBusiness errorreceivedShow specificreasonsKeep user inputWait timed outResult not yetconfirmedQuery beforedeciding to retryThis is suggested business-protocol design. The SDK does not generate business-completionacknowledgments for custom GMPs automatically.
After sending, wait for the business resultRegister the pending requestId before sending. After transport succeeds, wait for the glasses reply and update the result by requestId and status. A timeout means the result is unconfirmed; query first before deciding to retry.MEMOMIND / COMMAND RESULTAfter sending, wait forthe business resultSend and confirmRegister the pendingrequest firstGenerate requestIdShow processing in the UISubmit the messagesendMessage returns sentBusiness completion isstill not confirmedReceive a business replyMatch requestId / statusUpdate UI from the glassesresultHandle each failureseparatelySending failedEnd this waitShow Transfer ErrorBusiness error receivedShow specific reasonsKeep user inputWait timed outResult not yet confirmedQuery before deciding toretryThis is suggested business-protocoldesign. The SDK does not generatebusiness-completion acknowledgmentsfor custom GMPs automatically.
Change “Processing” to “Complete” only after receiving a matching business result.
  • Register pending requests before sending so a fast reply cannot arrive before its handler is ready.
  • Match results by protocol, operation type, requestId, and status. After timeout, query the actual state before deciding to retry.

Add safeguards to your protocol#

  • Validate message channel, length, protocol version, and field ranges before changing business state. Copy borrowed C callback data if you need it later.
  • Use correlatable request IDs for commands and results. Start waiting before sending so fast replies have a receiver.
  • Disable an in-progress action or coalesce settings for the same target on repeated clicks. Implement deduplication for side-effecting operations; do not assume the SDK guarantees exactly-once business execution.
  • Bound pending queues for real-time updates. Ordinary state may retain only the latest value; handle stop, exit, and acknowledgment messages separately.
  • Message channels must be declared in the manifest's device.messaging scope.channels. Platform display or event channels also require the corresponding permissions.

Handle disconnection, timeout and background operation#

SituationImmediate actionAction after recovery
Disconnected before or during a runDisable glasses-dependent actions and show connection state. Preserve forms and confirmed data that do not depend on the glasses.Confirm that the App is connected and the plugin can still run, then query the glasses' current state.
Request TimeoutEnd this wait and show “Result not yet confirmed”. Do not keep accumulating identical requests.Query the actual result before retrying. Check whether the operation already caused side effects.
Web enters suspendedStop unnecessary refreshes and subscriptions. Manage active resources according to each capability's lifecycle.Query host and device state. If the page was recreated, call gm.ready() again; do not reuse the old session.
One End Stops or ExitsEnd waits associated with that session, remove listeners, and release unused resources.On restart, build the page from current state. Do not let replies from a previous run overwrite the new page.
Permission denied or revokedStop the affected feature and explain the impact. Do not repeatedly request authorization in a loop.Recheck capabilities after the user authorizes again. Preserve features that do not depend on that permission.

Replace generic failures with actionable messages#

What to explainSuggested messageImplementation requirement
What is unfinishedThe glasses have not yet confirmed the settings result.A timeout only means no acknowledgment arrived; it does not prove the device did not execute the operation.
What remains availableYour current input has been preserved.Say this only if the input was actually preserved. Provide a recovery mechanism before page recreation.
What to do nextConnect the glasses in the App, then query the state again.Provide an action that actually works. “Retry” must not repeat an uncertain business operation.
Why is the operation not available?Microphone access was not authorized, so recording cannot start.Use the actual permission error. Do not classify device contention or disconnection as missing authorization.

Privacy and data explanations#

Data usedWhat the UI should explainImplementation checks
User Selected FilesWhy the file is selected, whether it is uploaded, and how to remove it.Use only selected files' fileId values. Cancellation must allow a normal return.
Audio & LocationWhen use starts, whether it is still active, and how to stop it.Match actual permission, foreground/background, timeout, and exit-cleanup behavior.
Content sent to business servicesProcessing purpose, recipient, and how results are saved and deleted.Send only data needed for the current feature. Keep long-lived service credentials on the server.
Configuration and task recordsWhere data is saved and whether it survives restart or upgrade.Provide recovery for migration, read failures, and user cleanup.

Support different languages and input methods#

  • Use clearly named Web elements such as button and label. Label forms, preserve keyboard focus, and show loading state.
  • Do not rely only on color, sound, or an icon for success and failure. Text and actions must mean the same thing on phone and glasses.
  • Allow room for translation expansion and translate complete sentences. Format dates, numbers, and units for the user's language.
  • GMP reads language tags through host->locale_get and maintains its own translations. Web can follow the example i18n directory. Do not assume the SDK translates business text automatically.
  • Verify icons, images, and Emoji with the actual fonts and display path. Users must be able to identify key results from text or state.

Reference files in the toolkit#

  • PhoneSDK/examples/display-control-lab/plugin.js
  • PhoneSDK/examples/display-control-lab/display-control-protocol.js
  • PhoneSDK/examples/display-control-lab/manifest.json
  • PhoneSDK/packages/web-sdk/src/index.d.ts
  • PhoneSDK/docs/web-plugin/runtime-and-lifecycle.md
  • GlassSDK/include/gm_plugin.h