Household Perception Data

Background

 

In the pursuit of truly autonomous cleaning, the industry faces a "Long-Tail" problem. While robots can easily navigate around furniture, they often lack the ability to distinguish the nature of the debris on the floor. Is it a harmless dust bunny, a wet juice spill, or a sticky dried stain? Current datasets typically lack this granular variety. To bridge this gap, Robotin is building a massive, community-contributed repository of Real-World Floor Contaminants. By tapping into community contributions, we capture the messy, unstructured reality of daily life—data that laboratory simulations simply cannot replicate.

 

 

Core Features of the Dataset

 

Unlike traditional robotic datasets captured by specific sensors, this dataset embraces the diversity of real-world capture conditions.

・Community-Contributed Realism (Smartphone RGB):

Data is captured via users' smartphones, providing high-fidelity images under natural, uncontrolled lighting. This approach ensures a vast diversity of textures, reflections, and floor types (wood, tile, carpet).

・Multi-View Flexibility:

  1. Robot-Centric View: Users are guided to shoot from low angles (10–40 cm) to mimic the perspective of standard vacuum robots.
  2. Human-Centric View: Includes supplementary high-angle shots, offering 3D context useful for humanoid robots and spatial reasoning tasks.

・Rich Semantic Metadata:

Beyond visual data, each sample is tagged with user-described categories (e.g., "coffee," "pet hair," "mud"). This allows models to learn not just where the dirt is, but what it is.

・Scalable Annotation (Bounding Box):

We provide efficient 2D Bounding Box annotations to support rapid object detection training.

・Dynamic "Before & After" Paired Data:

A unique feature of this dataset is the inclusion of pre-cleaning and post-cleaning comparison shots. These pairs are critical for training "Reward Models" in Embodied AI, helping robots learn to verify if a surface is truly clean.

 

 

Potential Applications

 

・Smart Organization Strategy (Sorting & Manipulation)

Enables robots to classify clutter types effectively—identifying soft items like clothing to pick up, or spotting dense clusters of items to trigger sorting logic rather than simple vacuuming.

・Visual Verification & Closed-Loop Control

Leveraging the "Before/After" data, robots can self-assess their tidying performance. If out-of-place objects are still detected after a task, the system can autonomously trigger a secondary organizing cycle.

・ Hazard & Fragile Item Detection

Improves the robot's ability to distinguish between movable clutter and hazardous/delicate items (e.g., power strips, glass cups, fragile decor), ensuring safe navigation and manipulation in complex home environments.

 

 

Market & Research Value

 

・ Academic Value:

Provides a challenging benchmark for fine-grained classification and anomaly detection in unstructured environments.

・ Industrial Value:

Offers critical "corner case" data (transparent liquids, sticky residues) needed to reduce failure rates in consumer cleaning robots.

・ Community-Driven Growth:

As a community-driven dataset, it continuously evolves, capturing regional and seasonal variations in household environments.

 

Vision: Building a global embodied intelligence data asset through community-contributed, real-world complexity

This dataset shifts the focus of robotic training from simple spatial navigation to granular environmental understanding. By capturing the chaotic, unscripted reality of household surfaces, robots can move beyond merely "avoiding" obstacles to meaningfully "interacting" with and maintaining their environments. This is what separates robotic training data that generalizes from data that merely covers the easy cases.