Auki is on track to deploy robots in retail stores this year. But before those robots can flag an empty shelf, scan a price tag, or generate a nightly task list, they need to recognize what they're looking at. And for that, they need training data.

We're asking the community to help train our robots to read retail shelves. The task is straightforward: take photos in a store, then annotate where ESLs, paper price tags, or empty shelf spaces appear in each image.

What the robots need it for

As you may recall, we're focused on shipping robots that do perception tasks, for example shelf audits. Each night, robots drive around the store capturing camera data, and Cactus generates a task list: move products around, restock empty shelves, fix planogram compliance issues, to name a few.

Eventually, the robot will even be able to map the store. It just needs to record video while driving through the aisles, send those videos to reconstruction servers, and scan the barcodes on the shelves. Just like we do with our phones today when we set up stores. Mapping a store's full product inventory onto its shelves is the step that takes the longest during setup, and it's something the robot will do autonomously.

What Cactus needs to learn

Before any of that happens, we need the robot's vision model to reliably detect three things on any shelf, in any store, under any lighting:

• ESL (Electronic Shelf Label). The small E-Ink price tags clipped to shelf edges in more modern stores. They update wirelessly.
• Paper price tag. The printed tags with a barcode, usually in a plastic holder on the shelf edge. Most stores still use these.
• Empty shelf space. A gap beside or above products large enough that something should be there. Missing stock.

These three classes are deceptively hard. ESLs vary by manufacturer, screen state, and how they're clipped on. Paper tags get crumpled, faded, partially obscured by stock, or hung at odd angles. Empty space is the trickiest of the three — Cactus has to learn the difference between a deliberate gap, a product pushed back on the shelf, and a real out-of-stock. The only way through is volume and variety of real-world examples.

How to Contribute

Next time you visit a supermarket, convenience store, pharmacy, or DIY store, take some photos of the shelves. Vary the distance and angle — close-ups and wider shots, slightly above or below eye level, are more useful than a dozen nearly identical frames from the same spot. Then upload the photos and mark where the above three things appear in each image.

A note on photography in stores

Before you start, check whether there are any legal restrictions in your country — particularly around GDPR or private property rules. Obey any "no photography" signage. If a store employee asks you to stop, stop.

In most places, photographing products and shelf layouts in a public-facing retail space is fine. But the rules vary, and it's your responsibility to check.

Detailed instructions

1. Take up to 50 images per supermarket, convenience store, pharmacy, or DIY store. Note: synthetic images are not valid for this initiative.
2. Go to https://modifiable-presymphysial-alisia.ngrok-free.dev/user/signup/
3. Create an account (feel free to use a fake email address like username@auki.com — this will be visible to everyone who participates)

4. Open the Shelf Data project

5. Click the "Import" button to upload your images

6. Drag your photos into the modal and click "Import" to upload them

7. Click on your first image and begin annotating:
   
   
   • Select the label type at the bottom, and draw a rectangle around the item. Please annotate the following:
     
     • ESL - Electronic Shelf Label, an E-Ink price tag
     • Paper price tag - A printed price tag with barcode
     • Empty shelf space - A space either to the side, or above products big enough to be filled (aka missing stock)
   • Continue until all instances of the above are annotated (you will need to select the label each time)
   • Click "Submit"
     

8. Move on to the next image and repeat.
9. Fill in the Google form with your details and wallet address on Base: https://forms.gle/wwZncsNNLVD9BbVi6

Rewards

Please submit up to 50 photos per location, and up to 300 total photos per person.

Contributors will be rewarded 15 $AUKI tokens per valid annotation.

Training data is the bottleneck

The robot we've described — connected to the real world web, running Cactus, doing shelf audits and later store mapping — is a $100/day value proposition that can be deployed this year.

But first we need Cactus to reliably read a shelf. A vision model that handles every variant of ESL, every battered paper tag, every kind of gap, in any store the robot rolls into. The only way to get there is a lot of labelled images from a lot of different stores.

That's what your photos are for.

Questions? Come find us in Discord.

About Auki

Auki is making the physical world accessible to AI by building the real world web: a way for robots and digital devices like smart glasses and phones to browse, navigate, and search physical locations.

70% of the world economy is still tied to physical locations and labor, so making the physical world accessible to AI represents a 3X increase in the TAM of AI in general. Auki's goal is to become the decentralized nervous system of AI in the physical world, providing collaborative spatial reasoning for the next 100bn devices on Earth and beyond.

X | Discord | LinkedIn | YouTube | Whitepaper | auki.com