Soft Robotic Dynamic In-Hand Pen Spinning
Yunchao Yao, Uksang Yoo, Jean Oh, Christopher G. Atkeson, Jeffrey Ichnowski
2024-11-20
Summary
This paper presents FlipSketch, a system that allows users to turn their static drawings into animated sketches by simply describing how they want the animation to move.
What's the problem?
Creating animations traditionally requires a lot of skill and time, as artists need to draw many key frames and specify motion paths. Existing automation methods still require significant artistic effort, making it hard for casual users to create animations easily.
What's the solution?
FlipSketch simplifies the animation process by letting users draw their ideas and describe the desired movements. The system uses advanced techniques from text-to-video models to generate smooth animations. It includes features like fine-tuning for sketch-style frames, a reference frame mechanism to keep the original drawing intact, and a dual-attention system that ensures fluid motion while maintaining visual quality. This method captures the expressive freedom of traditional animation without needing extensive artistic skills.
Why it matters?
This research is important because it makes animation accessible to anyone who can draw a simple sketch. By combining easy-to-use tools with advanced technology, FlipSketch encourages creativity and storytelling through animation, allowing more people to express their ideas visually.
Abstract
Dynamic in-hand manipulation remains a challenging task for soft robotic systems that have demonstrated advantages in safe compliant interactions but struggle with high-speed dynamic tasks. In this work, we present SWIFT, a system for learning dynamic tasks using a soft and compliant robotic hand. Unlike previous works that rely on simulation, quasi-static actions and precise object models, the proposed system learns to spin a pen through trial-and-error using only real-world data without requiring explicit prior knowledge of the pen's physical attributes. With self-labeled trials sampled from the real world, the system discovers the set of pen grasping and spinning primitive parameters that enables a soft hand to spin a pen robustly and reliably. After 130 sampled actions per object, SWIFT achieves 100% success rate across three pens with different weights and weight distributions, demonstrating the system's generalizability and robustness to changes in object properties. The results highlight the potential for soft robotic end-effectors to perform dynamic tasks including rapid in-hand manipulation. We also demonstrate that SWIFT generalizes to spinning items with different shapes and weights such as a brush and a screwdriver which we spin with 10/10 and 5/10 success rates respectively. Videos, data, and code are available at https://soft-spin.github.io.