Wander
Indoor navigation for blind and low-vision people: a chest-worn iPhone, a scanned 3D map of the building, and a voice guide.
- Role
- Team of 4, Hack the North 2026. My part: navigation backend, agent tools, Modal pipeline, assistant chat.
- Stack

Problem
GPS stops at the door. A white cane finds the ground in front of you, but it can’t say which hallway leads to room 3B. We wanted to add the layer the cane was never meant to cover, and let someone you trust help from anywhere.
What I built
A sighted helper scans the building once. The scan becomes a Gaussian splat for the web app and a Niantic VPS map that the phone localizes against. The backend proposes a walkable waypoint graph from the mesh, and the helper reviews and publishes it.
The wearer talks to a voice agent (“guide me to room 101”). The model decides what to do; routing is plain A* over the waypoint graph, so no position or path ever comes from a model. Obstacle detection runs on the phone with LiDAR and never touches the network.
My part was mostly the backend. I wrote the first version of the FastAPI navigation backend with the OpenAI Responses agent and its function tools, and the Elasticsearch retrieval layer behind it. I automated the Modal scan pipeline, added self-hosted visual localization and Gaussian splat training on Modal, and set up CI that tests the backend and deploys to Modal when main is green.
In the web app I built the Ask tab: a streaming chat with the building assistant that grounds “what’s in view” in the web camera and has a mobile voice-call mode. I also made detected scene objects turn into searchable note pins.
Caleb Bae led the web app, Alan Roybal led the iOS app and the Niantic SDK integration, and Ayro Escobar built the marketing site.
Results
- Built by four people in one weekend; the helper web app is live.
- End to end: scan, align, propose the graph, publish, localize at about 5 Hz, route, and speak cues while walking.
