Researchers at the University of Bristol, led by Professor Nathan Lepora, have made a significant breakthrough in robotic dexterity with their AnyRotate system.
They have developed a four-fingered robotic hand with artificial tactile fingertips that can rotate objects in any direction and orientation, even when the hand is upside down. This “gravity-invariant” in-hand manipulation capability marks an essential step towards robots with human-like dexterity.
What does gravity-invariant mean?
In robotic manipulation, “gravity-invariant” refers to a system’s ability to perform manipulation tasks consistently and robustly regardless of the direction of gravity relative to the robot’s frame of reference. In other words, the robot can manipulate objects successfully, whether upright, upside-down, or in any other orientation concerning gravity.
Why is gravity-invariance challenging?
Gravity plays a significant role in manipulation tasks. It affects the stability of grasps, the forces required to lift and move objects, and the dynamics of the manipulated objects. When a robot’s orientation changes relative to gravity, the gravitational forces acting on the robot and its manipulating objects also change.
For example, consider a robotic hand holding an object. When the hand is palm-up, gravity pulls the object into the hand, making the grasp more stable. However, when the hand is palm-down, gravity pulls the object away from the hand, making the grasp less stable and requiring more precise control and greater gripping forces to maintain.
Traditional robotics approaches often rely on precise models of the robot and environment, including the direction of gravity, to plan and control manipulation tasks. These models become inaccurate when the robot’s orientation changes, leading to poor performance or failure.
How is gravity-invariance achieved?
Achieving gravity-invariant manipulation requires a combination of advanced sensing, adaptable control strategies, and robust learning algorithms. Some critical approaches include:
1. Tactile sensing: High-resolution tactile sensors can provide detailed information about contact forces and object motion relative to the hand. This allows the robot to detect and respond to changes in grasp stability caused by gravity without needing explicit models.
2. Adaptive control: Control strategies that can adapt to changes in the system dynamics, such as those caused by changing gravity direction, are essential for gravity-invariant manipulation. This could involve techniques like adaptive impedance control or model-predictive control.
3. Machine learning: Learning-based approaches, such as deep reinforcement learning, can allow robots to learn manipulation skills directly from experience without relying on precise models. By training on a wide range of orientations and gravity conditions, the learned policies can be robust to these variations.
4. Sim-to-real transfer: Simulation training allows robots to experience various conditions, including different gravity directions, safely.
In-hand manipulation with multi-fingered robotic hands is a challenging problem due to the high degrees of actuation, fine motor control, and environmental uncertainties. Previous attempts at in-hand manipulation, such as the notable work by OpenAI in 2019, mainly used systems that relied on vision, requiring multiple cameras to capture the object from different angles. However, these vision-based approaches often needed help with occlusion, where the robotic hand would block the cameras’ view of the object being manipulated, making it difficult for the system to perceive and control the object’s position and orientation accurately.
AnyRotate’s goal was to achieve similar dexterity with a more efficient approach by leveraging the sense of touch. Lepora’s team, alongside researchers from MIT, UC Berkeley, and Columbia, have now demonstrated impressive in-hand manipulation skills on basic hardware setups with tactile sensing.
Artificial Tactile Fingertips
The critical enabler for AnyRotate was the integration of high-resolution tactile sensing into the robotic fingers. Inspired by the human sense of touch, the researchers designed artificial tactile fingertips with soft, deformable skin.
The skin consists of a 3D-printed mesh of thin pin-like structures called papillae on the inside of the soft outer surface. These intricate structures are printed with advanced multi-material 3D printers that can mix soft and rigid materials to mimic the mechanical properties of human skin.
Underneath this skin, a tiny camera captures how the papillae deform when in contact with an object. Machine learning models are then used to infer rich contact information, such as the pose (position and orientation) and force at each finger, from these “internal” images.
To learn a general policy for multi-axis object rotation, the team formulated the problem as repeatedly reorienting the object to a moving target pose. Goal-conditioned reinforcement learning was used to train control policies continuously rotating an object about any given axis.
The training process involved two stages. First, a “teacher” policy was trained in simulation using privileged information like object pose. Then, a “student” policy was trained to imitate the teacher using only proprioceptive data (joint angles) and tactile feedback, mimicking what would be available in the real world.
The simulation environment played a vital role in this process. The team used NVIDIA’s Isaac Gym to simulate the robotic hand and object interactions at high fidelity and speed.
Domain randomization was applied to simulation parameters like object size, shape, mass, and friction to improve the sim-to-real transfer.
Deep Dive: What is Domain Randomization?
Domain randomization is a technique used in robot learning, particularly in sim-to-real transfer, where a control policy is learned in simulation and then deployed on a real robot. The goal of domain randomization is to improve the robustness and generalization of the learned policy by exposing it to a wide range of variations in the simulation environment during training.
In a typical robotics simulation, the environment is defined by a set of parameters, such as the physical properties of objects (e.g., size, shape, mass, friction), the lighting conditions, and the robot’s dynamics. These parameters are usually fixed or vary within a narrow range to closely match the real-world setup.
However, these parameters can vary significantly due to factors like manufacturing tolerances, wear and tear, and environmental conditions. If a policy is trained on a fixed set of parameters, it may not generalize well to the real world, where these parameters differ from the simulation.
Domain randomization addresses this issue by randomly varying the simulation parameters during training. Instead of training in a single, fixed environment, the policy is exposed to various environments, each with different randomized parameters. For example:
- Object properties like size, shape, mass, and friction coefficients are sampled from predefined ranges in each training episode.
- Lighting conditions, such as the position, intensity, and colour of light sources, are randomly varied.
- The robot’s dynamics, such as joint friction, motor torque limits, and sensor noise, are perturbed.
By experiencing this wide range of variations in simulation, the policy learns to be robust to these differences and can generalize better to the real world, where the exact parameters are unknown and may change over time.
Mathematically, domain randomization can be seen as learning a policy π that maximizes the expected return over a distribution of environments p(E):
max_π 𝔼_{E∼p(E)} [𝔼_{τ∼π(E)} [R(τ)]]
Here, E represents an environment with randomized parameters, τ is a trajectory (sequence of states and actions) sampled from the policy π in environment E, and R(τ) is the cumulative reward of the trajectory.
The critical challenges in domain randomization are defining the appropriate ranges and distributions for the randomized parameters and ensuring that the simulation remains realistic and informative for the learning process. If the randomization is too extreme, the simulation may become realistic, and the policy may need help to learn valuable behaviours. On the other hand, if the randomization is too narrow, the policy may overfit the simulation and fail to generalize to the real world.
In practice, domain randomization has been successfully applied to various robotic tasks, from object grasping and manipulation to locomotion and navigation. It has been shown to improve sim-to-real transfer and enable policies learned purely in simulation to work effectively on real robots with minimal fine-tuning.
In the case of the AnyRotate system, domain randomization was used to train the control policy to handle variations in object properties (size, shape, mass, friction) and the robot’s dynamics. This allowed the policy to generalize to novel objects and maintain robust performance under different hand orientations and dynamic conditions when deployed on the real robot.
As research in robot learning advances, techniques like domain randomization will play an increasingly important role in enabling robots to learn flexible, adaptive skills that can be effectively transferred from simulation to the real world. By training in a wide range of simulated environments, robots can be prepared to handle the complexities and uncertainties of real-world tasks, bringing us closer to the goal of genuinely autonomous and versatile robotic systems.
Robust Manipulation with Tactile Sensing
The resulting control policy demonstrated remarkable dexterity and robustness. Not only could it rotate objects around any axis, but it could also maintain a stable grip regardless of the hand’s orientation relative to gravity. The hand could keep a firm hold on the object whether the palm was facing up, down, or anything in between.
This gravity invariance is essential for object manipulation in real-world situations where the hand may need to move dynamically.
The researchers conducted ablation studies involving removing or altering specific components to understand their impact. These studies revealed that policies using more detailed tactile feedback, mainly information about the contact pose (where the object touches the finger) and force (how hard it presses), performed better than those using more straightforward feedback like binary contact (just knowing if the object is touching or not).
The high-resolution tactile data allows for implicit slip detection, meaning the policy can learn to sense slight object movements and adjust its grasp accordingly to prevent the object from slipping out of hand. This is achieved without the need for a separate slip detection module.
The learned policies could generalize to new objects not encountered during training, such as everyday items with different shapes and sizes. This generalization ability results from the sim-to-real approach, where the policy is trained in a simulated environment and then transferred to the real world, as well as the high-fidelity tactile sensing that provides detailed information about the object’s interaction with the hand.
Future work could explore even richer tactile representations, like using full tactile images rather than compressed features or fusing visual and tactile data to infer additional object properties. Continued advancements in robotic hand design, especially in sensing and actuation, will also be vital in expanding dexterous manipulation capabilities.
AnyRotate represents a significant milestone in robotic dexterity. The successful integration of high-resolution tactile sensing, sim-to-real learning, and goal-conditioned reinforcement learning provides a blueprint for advancing in-hand manipulation.
While complex multi-fingered hands have traditionally been challenging to control, this work demonstrates that they can achieve stable, flexible, and robust manipulation skills with the proper sensing and learning approaches. The gravity-invariant rotation capabilities open up new possibilities for robots to manipulate objects in dynamic, unstructured environments.gy enabling the sense of touch.
TLDR:
- Researchers created a robotic hand with artificial tactile fingertips to rotate objects in any direction, even upside down.
- High-res tactile sensing using tiny cameras and 3D-printed skin with deformable papillae was critical to this breakthrough.
- The control policy was learned using goal-conditioned reinforcement learning in simulation with domain randomization.
- The tactile sensing allows robust in-hand manipulation of objects regardless of the hand orientation relative to gravity.
- This brings robotic hands a step closer to human-like dexterity, with potential applications in manufacturing and recycling.
