Most mobile, logistics, agricultural and other robots can now work autonomously between 90 and 98% of the time, however, they are not able to fully function without human support when they encounter a situation out of the norm.
In talking to over 800 roboticists and companies, 68% flagged stable teleoperation is mission critical at some level to remotely deploy fleets of robots. Being able to remotely control your robots stably matters. And, it is very hard to do.
Companies have requested operators being able to:
- Override mission planning and send a robot to a location
- Take over driving to get a robot out of a safety situation
- Enable manipulation and control of lifts and arms
- Manually execute the pick of an object if it can not autonomously complete it
- Seamlessly switch between control modes of a robot (Driving vs controlling an arm or elevator)
With the goal of quickly enabling a robot to return to automatic operation quickly. Without this, we have had several companies report that their entire production line has gone down repeatedly for minutes at a time due to being starved. This is because a single robot got into a situation it could not autonomously navigate and it took a significant amount of time to safely reset it and deliver the missing parts.
In complex environments, AMRs can easily get into situations where human level intelligence is needed.
We are excited to announce the release of our Operator Teleop, or colloquially known as Pilot 1.5, which builds on our June initial release of Freedom Pilot.
This major release enables Freedom Robotics’ users to intervene and tele-operate robots in poor connectivity environments. It takes into account significant real-world feedback and feature requests over the last few months from Freedom users who have deployed fleets which rely on Pilot. Additionally, it has been optimized with hundreds of hours of testing and tuning in poor cellular connected areas on a variety of robotic platforms.
The new release introduces a much faster, smoother, and more responsive teleoperation experience, several bug fixes and UI improvements and adds support for using physical game controllers for advanced mobile robot and robotic arm control.
A Game Changer in Remote Teleop
To be able to confidently control a robot from anywhere in the world, a fast, smooth, and stable connection is needed. In areas with poor connectivity and limited bandwidth, the connection algorithm has to properly scale down video properties that matter the least to ensure a connection that guarantees safety and a smooth driving experience. This has always been our focus and today we are announcing a major step forward in this direction, after very significant testing and tuning of the Freedom Agent and underlying transport infrastructure.
In extremely poor connectivity environments, latency can spike through dead spots in wifi or cellular connectivity. Through multiple infrastructure iterations which increased the resilience of the connection and optimized bandwidth, areas of poor control have been almost eliminated.
Poor connectivity doesn’t just happen outdoors on cell networks. In many logistics facilities, where they can be up to a quarter mile long, there are major weak or dead spots in connectivity which robots have to navigate.
Huge increase in performance in poor environments
With the improvements in Pilot 1.5, in real-world, constrained, cellular connectivity, environments, you will see significant increases in stability during teleoperation:
- 48% increase in frame rate with the same settings, resulting in enhanced control at all speeds
- 35% lower control latency with a complete elimination of latency spikes due to sudden drops in bandwidth
- 61% more stable bit-rate for improved bandwidth usage
- 16x fewer disconnects in low-connectivity areas
This is a short video showing the smoothness of webrtc control. Watch the correlation between the movement of the joystick and the viewport.
100s of hours of A/B testing in harsh environments
In addition to customer beta testing through this process, A/B testing was done on many platforms on wifi, cellular connectivity and with computer resources as small as low-power, ARM systems up to NVIDIA-enabled, 300 lbs mobile platforms.
Platforms like the Rover Robotics and Fetch Robotics platforms need high-quality, low latency remote control.
A key to remote teleoperation is stable connectivity, either through wifi or, more regularly, cellular modems. Definitely check out our previous blogs on 8 Common Mistakes when Connecting your Robot with a Cellular Modem and 18 Ways to Stabilize Robotics Compute Resources.
Please see the Freedom Robotics documentation for how to set up and tune teleoperation to your use case.
Additional Pilot 1.5 Features
Fast and Powerful Gamepad control
We now support full gamepad or joystick buttons and axes through webRTC, enabling real-time control of robot arms, hybrid control of complex systems, and fast command issuing. The message sent through is formatted to work like ROS’ sensor_msgs/Joy message. We support most major gamepads, including the PS and Xbox controllers. For a full list, see here.
Here's a video of us controller a Fetch Robot through Freedom Robotics' teleop feature with a standard Xbox controller
For more information, please visit our documentation on how to set up gamepad control.
Improved UI and Map View
With this update, the settings page was overhauled to make it easier to find the right settings and be more clear. Additionally, maps are sent through in higher quality and now support visualization of known obstacle-free space, making it easier to use for navigation.
It is extremely easy to set up game controllers by going to DEVICE -> SETTINGS -> PILOT and changing from a twist to a joy message.
We’re always looking to hear your feedback. Shoot us an email at hello@freedomrobotics.ai.
