When Robots, Glasses and Phones Share a Map

Setting the Stage: Collaborative Spatial Computing

We opened this update from the Hong Kong lab with a simple idea: “A phone like this or a pair of glasses like these are actually robots with no arms and legs.”

That’s the core of what we do: collaborative spatial computing. We teach phones, glasses and robots to share the same understanding of physical space.

• The real world web (Auki network) lets devices browse physical locations the way browsers visit websites.
• Devices don’t need a hard-coded local map. They connect to a domain, download the spatial context, and can immediately navigate and coordinate.

If you’ve seen our earlier demo of Terri navigating a conference he’d never seen before by using a map built from phone video, that’s the same idea in action.

Recap: Terri, Gotu and App Clips

To ground it, we started with a handheld demo:

1. Scan a QR on the floor with an iPhone.
2. iOS opens an App Clip for our Gotu app clip (no install needed).
3. Scan again and the phone loads:
   • The local domain’s map
   • Points of interest (like “Nils’s desk”)
   • AR navigation as dotted lines on the floor, plus anchored AR content (e.g. in-store ads).

The phone is talking to our network, pulling down a shared map of the lab and navigating within it.

We’ve shown similar things with Terri before:

• Terri connects to the same domain, downloads the map, and can be told “go to this location” without ever having seen the space before.
• That’s our earlier large-scale autonomous humanoid navigation milestone.

This time, we wanted to go beyond “phone and one robot share a map” to many devices orchestrating together.

New Milestone: App- and Device-Agnostic Spatial Orchestration

The main goal of this session was to show app agnostic, device agnostic, spatial orchestration.

In practice, that meant getting:

• Mentra Live smart glasses
• Terri (the Unitree humanoid)
• An X3 warehouse-style robot

…all to act in the same shared space, triggered from different apps, made by different vendors.

We had a few live-demo gremlins (and beer), but we got three key behaviours working:

1. Glasses call a robot over

From a video recorded through Mentra glasses, you can hear Arshak saying: “Hey Terri, come here.”

What’s happening under the hood:

• The glasses stream their view into our network.
• Our reconstruction + localization stack pins his exact pose down to the centimeter.
• That pose is shared on the domain.
• Terri subscribes to that and receives a “go to glasses location” command.

Terri then walks across the lab to the person wearing the glasses. It doesn’t matter that the glasses and robot are built by different manufacturers, running different software – they meet in the same shared coordinate system.

2. The other robot does it too

We repeated the same behaviour with the Padbot X3:

• Glasses say “come here.”
• X3 gets the glasses' pose via the domain.
• X3 autonomously drives over to the glasses.

Again, no direct app-to-app integration. The only shared “language” is:

• Pose + map on the real world web
• A small network message: “go to this pose”

3. Robots navigate to each other

Then we flipped things around:

• We sent the X3 to a known point of interest (“merch store”) in the domain.
• From Terri's web interface, we hit “Go to robot”
• Terri used the shared domain data to navigate to the X3’s current location, not just a static waypoint.

He did it with a weird sideways crab walk (that’s a Unitree / ROS Nav2 issue, not our logic), but he got there.

So in one session, we showed:

• Glasses → robot (Terri → glasses)‍
• Glasses → robot (X3‍ → glasses)‍
• Robot → robot (Terri → X3)

All without bespoke wiring between vendors or apps. That’s the point: orchestration lives in the network, not in one monolithic stack.

Under the Hood: Domains as “Data Brokers”

We also talked a bit about how we’re evolving our design for shared spatial data.

In the early days we had a service (Odal) where everyone had to upload assets to a common CDN to be interoperable. Now we’re moving toward:

• Domain servers as data brokers
  • Each app (Gotu, McKenna, etc.) can store its own assets on its own infra.
  • The domain holds pointers and metadata: “these assets live over there, in that app, in this coordinate system.”
  • Other apps can query the domain and discover those assets without copying them into a central store.

As Nils put it on the call: “You don’t actually have to upload your assets to Auki to make them visible across applications. You just have to register with a specific domain where to go get it.”

This keeps things:

• Interoperable – multiple apps can render the same objects and anchors.
• Decentralizable – data can live where its owner wants it.
• Aligned with our DePIN model – venues and developers stay in control of their data.

Where This Goes Next

We also shared some near-term context:

• A trip to IROS in Hangzhou, followed by meetings in Shanghai with a major Chinese robotics partner. They’ve already talked about tailoring their next-gen robot to be “Auki-ready out of the box” and offering us partial exclusivity in certain geographies and industries.
• Ongoing work with Mentra on the glasses side, and with builders like Mika (OneShot copilot, pizza demo) on top of our network.
• A steady stream of robotics and AR teams who want to integrate with the real world web to improve deployment speed and economics.

Nils ended the call with the simple take-home: “You honestly saw a bit of robot history today… app agnostic, device agnostic, spatial orchestration. We did it. It happened here.”

There’s still a lot of polish and robustness work ahead, but the core behaviour is real: phones, glasses and robots can now coordinate in physical space through our network, without sharing a vendor or a codebase.

‍

Watch the whole update on X.

About Auki

Auki is making the physical world accessible to AI by building the real world web: a way for robots and digital devices like smart glasses and phones to browse, navigate, and search physical locations.

70% of the world economy is still tied to physical locations and labor, so making the physical world accessible to AI represents a 3X increase in the TAM of AI in general. Auki's goal is to become the decentralized nervous system of AI in the physical world, providing collaborative spatial reasoning for the next 100bn devices on Earth and beyond.

X | Discord | LinkedIn | YouTube | Whitepaper | auki.com