RobotKraft, imitation learning on an SO-ARM101 arm

Context

RobotKraft is an AI and robotics challenge, held in Illkirch from 2 to 4 October 2026. Each team had an SO-ARM101 kit (leader and follower arms), the LeRobot framework, a wrist camera and a depth camera. In my team, “les Bras cassés” (the Broken Arms), I handled the data and the model.

What I did

• A complete pipeline in official LeRobot commands, from teleoperation to evaluation, driven by a Makefile.
• About 50 cycles recorded per mission by teleoperation, on missions with several actions.
• Automatic check of each dataset before upload to the Hub.
• Fine-tuning of SmolVLA, with an ACT model as a fallback.

What I noticed and learned

• Around fifty demonstrations are not enough for SmolVLA to generalize on chained tasks.
• Inference matters as much as training. On our CPU, SmolVLA took 5 s to produce 1.7 s of motion.

My mistakes

• The trained SmolVLA models did not work in real conditions, for lack of sufficient data.
• Like the other teams, we fell back on algorithmic control and teleoperation replay.

Results

• 3rd place in the challenge.
• A public repository that documents the whole pipeline, from installation to testing on the robot.
• Next step, getting the same kit to continue at home.

Tech stack