The project

Design decisions#

A few more choices shape the whole project:

• Printed in PLA or PETG. Anyone with an ordinary 3D printer can make the parts.
• Simulation first. A virtual copy of the hand in MuJoCo (a free physics simulator) runs on any laptop. It lets us test ideas before touching the hardware, and later train AI safely.
• The same code for sim and real. A script written for the simulated hand runs unchanged on the real one.
• Angles, not motor steps. The computer always talks in joint angles. Only the ESP32 knows about motors, so swapping steppers for servos doesn't change anything above it.

Roadmap#

1. Phase 0

   Done

   Phase 0: Foundation

   Set up the project and get the hand model moving in simulation.

   • Project structure, public repo and licenses
   • Checked the Fusion 360 export and wrote down its problems
   • Converter from the CAD export to a MuJoCo simulation model
   • Simulation viewer with a slider for every joint
   • Simulated motions checked against the real hand

2. Phase 1

   In progress

   Phase 1: V0 prototype, thumb and index

   A first start: make the 8-joint thumb and index prototype move cleanly on stepper motors, controlled from the computer.

   • Firmware with smooth speed-up and slow-down for all 8 motors (tested on the computer)
   • Motors switch off when idle to stay within the power supply
   • Python control library that works on the simulation and the real hand
   • Digital twin software: move a slider on screen and the real joint follows (not yet tried on the hand)
   • Still to do: first power-on, motor direction checks and calibration of each joint
   • Still to do: basic motions like open, close, pinch and point

3. Phase 2

   In progress

   Phase 2: Tendra Hand V1, the first full hand

   Five fingers and 20 joints, moved by 16 smart servos that report their own position (each fingertip joint follows its middle joint, like in a human finger). This is the first real Tendra Hand.

   • Middle, ring and little fingers, copied from the index and sized like a human hand (designed in Fusion 360)
   • A thumb with 4 joints (a fifth was tried and dropped)
   • All 16 servos in the forearm, with every tendon in its own channel through the palm and wrist
   • Full-hand simulation model: 20 joints and 36 tendons
   • Firmware for the servo bus, ready for all 16 servos, with the same Python code as V0
   • Still to do: electronics and a power supply big enough for 16 servos
   • Still to do: wire all 16 servos on one shared data cable, through an FE-URT-1 adapter
   • Still to do: print a test section of the tendon channels, then print and assemble the whole hand
   • Still to do: read back position, load and temperature, and show the measured positions live in the digital twin
   • Still to do: compare the V0 steppers with the V1 servos for speed, precision, force and noise

4. Phase 3

   Planned

   Phase 3: Mechanics and a realistic simulation

   Make V1 robust and the simulation behave like the real thing.

   • Study tendon routing, friction and tension, and model the tendons in simulation
   • Soft TPU fingertip pads
   • Measure real speed, friction and play, and tune the simulation to match
   • Weigh the printed parts and update the model

5. Phase 4

   Planned

   Phase 4: Sensing

   Give the hand a sense of touch and a camera.

   • Touch and force sensing in the fingertips
   • A camera to see the hand and what it holds
   • Sensor data in the Python library and the simulation

6. Phase 5

   Planned

   Phase 5: AI control

   Teach the hand to move by itself.

   • Control the hand by moving your own hand in front of a webcam
   • Learn from recorded demonstrations
   • Train in simulation on cloud GPUs, then transfer to the real hand
   • Optional: larger-scale training in NVIDIA Isaac Sim or Isaac Lab

7. Phase 6

   Planned

   Phase 6: Seeing and grasping

   Use the camera to find objects and handle them.

   • Detect objects with the camera and plan how to grasp them
   • Harder tasks: turning objects in the hand, using tools, everyday objects
   • Publish benchmarks and results

8. Phase 7

   Planned

   Phase 7: Understanding tasks

   Go from "do this motion" to "do this job".

   • Tell the hand what to do in plain words, and it works out the actions from what the camera sees
   • Break a long task, like making a sandwich, into steps and pick the right skill for each
   • Notice when something goes wrong, like a dropped object, and try again
   • Learn a new task from a few human demonstrations
   • An open dataset of hand demonstrations for household tasks

9. Phase 8

   Planned

   Phase 8: Arm, body and real chores

   Take the hand out of the workshop and into a home.

   • Put the hand on a robot arm, then use two hands together
   • Kitchen tasks: cut, stir, pour, crack an egg, cook a simple meal
   • Chores: fold laundry, load the dishwasher, tidy up
   • A body that can move from room to room
   • Safe around people: gentle forces and stopping on unexpected touch