- > Anatomically Correct Testbed (ACT) Robotic Hand Mechanisms and Control
- > Life-size Safe Brake-Actuated Manipulator for Rehabilitation, Exercise, and Realistic Haptic Display
- > Distorted Feedback Robotic Rehabilitation
- > Human Hand Neuromuscular Structure and Control
- > Characterization of Multi-Finger Twisting Motion
- > Reverse-Engineering the Human Brain’s Ability to Control the Hand (in collaboration with Francisco Valero-Cuevas and Emo Todorov)
- > Human Haptic Exploration Strategy (in collaboration with Ed Colgate)
- > Electrocorticographic Brain-Machine Interface (in collaboration with Rajesh Rao and Jeff Ojemann)
- > Sensory-Feedback Training to Affect Balance (in collaboration with Sally Westcott McCoy)
- > Neurosecurity (in collaboration with Yoshi Kohno)
- > Development of Home-Based Virtual Robotic Rehabilitation Environment
- > Assessment of Fine Motor Control for Parkinson’s
- > Silicon/Neuron Interface
- > Hand Exoskeleton
- > Portable Wearable Movement Tracking System
- > Sensory Transfer Systems for the Sensory Impaired
ANATOMICALLY CORRECT TESTBED (ACT) ROBOTIC HAND MECHANISMS AND CONTROL
The Anatomically Correct Testbed (ACT) Hand has been under construction for several years. Its purpose is to discover mechanisms and control salient features in human hands that allow robust, versatile, dexterous movements and rich object/world exploration. Currently these human hand’s structural and functional features are being uncovered using cadaveric and computational models that are difficult to emulate realistic object interactions. An anatomical robotic tool will allow advancement in biomechanics and neuroscience as well as for current prosthetic, industrial, and personal assistance robotic hand design and control.
1) As an experimental testbed to investigate the complex neural control of human hand movements for both healthy and disabled conditions,
2) As the first prosthetics/telemanipulator that is capable of executing any human tasks autonomously or with direct and natural neural signals,
3) As a way to discover the neuromusculoskeletal details of a human hand that allows highly dexterous movements,
4) As a working physical model of the human hand for neuro- and plastic-surgeons to test new surgical reconstruction techniques for impaired hands, and
5) As an educational tool for K12 outreach.
This hand, unlike many anthropomorphic robotic hands, incorporates neuromusculoskeletal aspects of the anatomy that are functionally crucial in order to use control signals that resemble the neural commands. For example, the tendon insertion points, general bone shapes, the extensor mechanisms, musculotendon passive viscoelasticity, muscle contraction behavior, joint axes locations, joint range of motion, and the general size and weight are preserved. anatomical prosthetic hand.↑ Back to Top
LIFE-SIZE SAFE BRAKE-ACTUATED MANIPULATOR FOR REHABILITATION, EXERCISE, AND REALISTIC HAPTIC DISPLAY
If a dissipative physical human-robot interaction (pHRI) device can produce the transparency and controllability of an active device, it would dramatically change the safety of such interactions. A 2m^3 workspace (life-size) brake-actuated manipulator (BAM) is developed with sophisticated controllers that utilizes human neural and perceptual science findings.
Motor-actuated devices are widely used in haptics research and advanced teleoperator masters - capable of creating a rich set of virtual environments in the former and recreating physical environments in the latter. Motors, however, introduce the risk of a device striking or overpowering users within reach. With rare exception, safety concerns limit motor-actuated haptic devices to small workspaces.
The replacement of energetic servo-motors with passive actuators such as brakes improves the inherent safety of the system, and expands the range of motion while still allowing us to simulate virtual objects (i.e. to constrain the user's motion inside an object). Safety is essential for acceptance in new applications such as medical procedures, quantitative rehabilitation, advanced exercise training, and entertainment. Even if the device experiences a power, hardware, or software failure, a brake-actuated device is inherently incapable of exceeding the kinetic energy that a user supplies in each motion. Using currently available engineering technology, the workspace of brake-actuated haptic devices can expand to cover whole-body movements without risking a user's safety. With the expanded workspace, a variety of large movements can be trained and quantitatively analyzed.
We constructed a brake actuated manipulator (BAM) with 2 meter-cubic workspace based on spherical kinematics with a non-actuated 3DOF handle serving as the end-effector. The device can withstand a user’s force up to 222N. A novel passive gravity compensation mechanism has been designed using the similar triangles principle to balance the weight of the prismatic joint, increasing the devices transparency. The 3DOF handle is dynamically balanced with an asymmetric design. The asymmetries allow for a large range of motion in which gimbal lock is mechanically impossible. The dynamic balancing ensures no off axis rotations will be felt by the user during operation.
The system of the BAM was fully characterized using an Unscented Kalman Filtering technique. A variety of path guidance paradigms are being explored to expand the capabilities of the BAM to facilitate rehabilitative applications.↑ Back to Top
DISTORTED FEEDBACK ROBOTIC REHABILITATION
A robotic rehabilitation environment with distorted feedback introduces an undisclosed discrepancy between the actual movement and the virtual fisual feedback subjects receive about their movement. Feedback distortion targets individuals with chronic stroke, traumatic brain injury, and others with mobility difficulties who have capacity to make functional improvements but may have self0imposed limits on their performance due to entrenched habits , learned-non-use, or psychological barriers. Our study has shown that the finger’s range of motion can increase by more than 50%, spasticity measure to be reduced, and functional scores (such as AMAT score) to improve from a 6-week therapy for subjects 2-8 years post injury.
Robotic therapy has been shown to increase the range of motion, strength, and velocity of arm movements in people with chronic stroke, traumatic brain injury (TBI), and other neurological trauma. Robotic therapy may improve the condition of such patients because it focuses on intensive, repetitive movements, which have been shown to counteract the detrimental effects of a habitual decrease in movement.
We use a virtual robotic environment to combine the repetitive movements of robotic therapy with visual feedback distortion. By distorting visual feedback of their performance, we showed that people result in producing larger and stronger movements without their awareness or increased physical effort level. In experiments with young and elderly subjects, we have shown that gradual visual distortion leads to increases in force production by 30.0% (young subjects) and 72.5% (elderly subjects). These increases occurred without detection of the distortion by subjects even though they are 2-3 times larger than the Just Noticeable Differences (JNDs).
Preliminary therapeutic work with several people with chronic TBI/stroke used gradual visual distortion of up to 92.0% without the detection of the distortion. The rehabilitation program led to significant increase in the range of motion and maximum force during the 6-week therapeutic training.↑ Back to Top
HUMAN HAND NEUROMUSCULAR STRUCTURE AND CONTROL
Human psychophysical experiments are conducted to understand neuromuscular control of the hand. We record EMGs, joint angles, forces, and stiffness from subjects to understand what humans optimize as they select a neural control strategy from redundant/overactuated structure. In addition, Constructing and controlling the ACT hand has resulted in further understanding about the tendon shapes and routing, bone shapes, non-linear moment arms, and neural control optimization.
We are interested in understanding the neuromuscular control of human hands as a way to restore function to those with paralyzed or amputated hand. The Neurobotics laboratory focuses on the lower-level neuromusculoskeletal details that allow highly dexterous multi-finger movements. These investigations allow both (1) to further development of the ACT hardware and the controller, and (2) to allow the ACT hand to be controlled by the natural neural signals. We focus on the following issues:
- Development of mathematical foundation to describe the neuromuscular control redundant space for any given finger posture, force, and stiffness.
- Neural optimization under redundant control space
- Utilization of passive properties
- Discovery of biomechanical and neural control synergies
- Understanding human task execution strategies
- Comparison of physiological (human hand) and engineering (ACT hand) neuromuscular control strategy differences
- Understanding cortical/peripheral neural signals
With the goal of developing biologically-inspired manipulation strategies for an anthropomorphic hand, we use a generic concept called the iso-effector space to describe muscle actuation to achieve different tasks. The iso-effector space is the space of actuation solutions that produce a specific force and stiffness for a given manipulator and configuration. Using computer simulations and human-subject experiments, we have shown that humans utilize small portions of the available muscle actuation space to achieve a task. This indicated that humans exploit specific neural synergies to simplify decision making when performing a task. Finally, we recently have explored how humans utilize the hand's redundant neuromusculoskeletal biomechanics to modulate stiffness and force. We explore different hypotheses about how the central nervous system chooses paths in muscle actuation space to transition between tasks. We use force sensors and electromyographical recordings from the muscles and explore how such insight into human-muscle control strategies will help develop human-like robotic manipulation.↑ Back to Top
CHARACTERIZATION OF MULTI-FINDER TWISTING MOTION
In order to work on diagnostic and rehabilitation paradigms for dexterous finger movements, the finger coordination strategies must be understood. Twist-cap motion is chosen as a simple multi-finger coordinated movement to study. We characterize hand movements from subjects who are healthy (younger and elderly) and those who have difficulty producing these movements.↑ Back to Top
REVERSE-ENGINEERING THE HUMAN BRAIN’S ABILITY TO CONTROL THE HAND (in collaboration with Francisco Valero-Cuevas and Emo Todorov)
In collaboration with USC, we record hand-object interactions and neuromuscular activity during real-world manipulation tasks; use the experimental data to approximately infer the underlying sensorimotor control laws; refine these control laws in physically realistic simulations via optimal control methods; implement, test and adapt them on the ACT hand; and use the engineering insights obtained from the synthetic system to better understand its biological counterpart.↑ Back to Top
The ultimate goal of this project was to create chronically implantable neural interfaces that may be used to control external devices and replace lost sensory input. We investigated conductive polymer coatings to promote more intimate connections between cells and metal electrodes, leading to a device that would be safe for long-term use and that would produce low-noise chronic single-unit recordings.
Our long-term goal is to construct an integrated device based on these coatings and the CMU CMOS-MEMS process that will entrap living cells within the device structure and use the processes of those cells as the connection to a host nervous system. To this end, we studied improvements to existing methods of controlling neurons in vitro such as microcontact printing of cell adhesion molecules.
We evaluated the biocompatibility and performance of a candidate polythiophene. Our results demonstrated that primary dissociated rodent neurons can grow and survive for periods of days or longer on conductive polymer self-assembled monolayers when those monolayers are doped with the neural cell adhesion molecule L1. Even the limited survival observed to date enables in vitro electrophysiology to further evaluate candidate polymers.↑ Back to Top
The goal of this project was to design and construct a lightweight, comfortable orthotic device for the hand (exoskeleton) to be used by upper spinal cord patients suffering from a loss of precise control in the upper limbs and hands. The main mechanical focus was to provide three simple tasks: pinching, pointing and full hand grasping, through the direct manipulation of the users joints. The device was controlled by the users own muscle signals via the use of surface electromyography (EMG) sensors attached to the patients arm. Our system was tested with a quadriplegic student and he was able to manipulate objects that were too large, heavy or squishy otherwise!↑ Back to Top
PORTABLE WEARABLE MOVEMENT TRACKER
The goal of this project was to construct a completely portable and wearable sensor system that tracks human limb movements accurately and continuously records people's movements to capture abnormal neuromuscular control, biomechanical disorders, sports movements and injuries. The applications include elderly management, neurodegenerative disorder prediction, injury prevention, and movement analysis.↑ Back to Top
SENSORY TRANSFER SYSTEM
The purpose of this project was to construct a sensory transfer system to assist and enhance the manipulation abilities of the sensory impaired. This system consisted of two parts, an array of sensors worn on the arm and an array of actuators worn on the face or neck. The system sensed the pressure exerted on the hand by various objects and transfers that pressure to a location from which a patient still receives sensory information.↑ Back to Top