When we think of autonomous navigation, we usually picture cars navigating highways. But some of the most challenging physical AI deployments today are happening on sidewalks.

As last-mile delivery robots become more common in urban environments, many robotics companies are discovering that sidewalks present a completely different set of perception challenges than roads.

One delivery robotics company encountered this firsthand as it expanded from controlled suburban testing environments into dense city centers. While the robots performed well in testing, real-world deployments revealed a critical challenge: Pedestrian Chaos.

Here’s how Datum AI helped address the perception gaps that were limiting deployment performance.

The Challenge: The “Horizon” Bias in Autonomous Data

Many perception models used in robotics are heavily influenced by datasets originally created for autonomous driving.

The problem is that a delivery robot experiences the world very differently from a car.

Vehicle-mounted cameras are positioned several feet above the ground and focus on traffic, vehicles, road markings, and signs. Delivery robots operate much lower to the ground and encounter a completely different set of objects and obstacles.As the robots entered busy urban environments, several challenges emerged:

• The Micro-Mobility Problem : Fast-moving objects such as e-scooters, bicycles, and skateboards frequently moved through the robot’s field of view, creating tracking and prediction challenges.
• Sidewalk Clutter : The robots struggled to consistently distinguish between shadows, puddles, temporary obstacles, debris, and permanent infrastructure.
• Small Hazard Detection : Thin and difficult-to-detect obstacles, such as leashes, cables, and narrow barriers, often went unnoticed by perception models optimized for larger objects.

To maintain safety, the robots frequently triggered conservative stopping behaviors, leading to unnecessary pauses and slower delivery times.

The Solution: Datum AI’s Custom Low-Angle Collection Pipeline

To improve performance, the robotics company required training data that accurately reflected the robot’s real-world operating environment.

Datum AI designed a custom collection and annotation program focused specifically on sidewalk navigation.

1. Low-Angle Multi-Sensor Collection  

Custom capture systems were deployed at approximately the same height as the delivery robots, combining high-frame-rate cameras with LiDAR sensors to replicate the robot’s perspective.

2. Targeted Edge-Case Collection  

Rather than collecting generic urban footage, data collection focused on environments known to generate perception challenges, including:

• Busy pedestrian corridors
• Transit stations
• Bike lanes
• Shared mobility zones
• High-density intersections

This approach captured large numbers of interactions involving e-scooters, cyclists, skateboarders, and pedestrians.

3. Dense 3D Semantic Annotation  

Datum AI’s annotation teams created detailed scene understanding datasets using semantic segmentation and 3D labeling techniques.

Special attention was given to:

• Traversable vs. non-traversable areas
• Small obstacles and hazards
• Dynamic objects
• Urban infrastructure
• Environmental edge cases

The Impact

After retraining their perception stack using the newly collected data, the robotics company observed significant improvements during urban deployments.

1. Reduced Unnecessary Stops  

Improved scene understanding helped reduce false obstacle detections and unnecessary stopping events.

2. Better Micro-Mobility Awareness  

The addition of targeted high-speed interaction data improved the robot’s ability to track and respond to bicycles, scooters, and other fast-moving objects.

3. Improved Route Efficiency  

With more accurate perception and fewer unnecessary interruptions, route completion performance improved across dense urban environments.

The Takeaway

Physical AI systems perform best when trained on data that closely matches their deployment environment.

A sidewalk robot does not experience the world the same way as a passenger vehicle. Differences in sensor placement, operating conditions, and environmental complexity all influence model performance.

By combining custom data collection, sensor fusion, and expert annotation, organizations can build perception systems that perform reliably in real-world environments.

Datum AI supports robotics and physical AI teams with custom data collection, annotation, and dataset development services designed for deployment-specific challenges.

Explore Datum AI’s Custom Collection Services : https://datumdata.ai/custom-collection/