The simulator got every direction right, and almost everything else wrong.
In Part 1 I made the case for building the whole system inside a digital twin before letting a metal arm loose on a wooden board. The simulator was working, the trajectories were tested and the collision clearances were validated. All that was left was to plug in the real LeArm and let Kit play.
That is not quite how it went. The short version: the simulator got the direction of every joint right and almost everything else about the real world wrong. The sizes, the heights, the claw, even which USB device the arm turns up as. This post is the honest account of what survived contact with the hardware, what did not, and why I have decided to bolt a depth camera above the board.
Key lesson: A simulator tells you which way things move. Only the real arm tells you how far, how high and how wide. Plan for a calibration phase, and give the robot eyes.
One More Round in the Simulator
Before plugging anything in, I made the digital twin earn its keep one more time. We had the simulator play ten complete games, two at each of the five experience levels, driven through the real browser UI while recording the screen. An in-page collision audit sampled points on every arm mesh against the board, the graveyard tray, the table, every standing piece and Reachy Mini on every frame, sweeping between frames so a slow render could not hide a hit.
The results were reassuring. All ten games were legal, the 3D board matched the chess engine after every move, the turn lock rejected every human move sent while the arm was busy, and there were no collisions with the board, tray, table or Reachy Mini. (The one game that came back with errors turned out to be me clicking New Game in my own browser halfway through the run. A test rig is only as clean as the person using it.)
Bug 7: The claw closed before it reached the piece. Watching the recordings back, I noticed the claw was almost shut all the way down to the piece. The “open” position I used for approaching was only 1 mm wider than the piece’s neck, which looks fine in a render and is impossible on a real arm. The claw now opens wide while it hovers and descends, closes only once it is at the grasp height, stays closed while carrying, and opens again to release.
The problem simulation could not fix. The audit also showed the claw brushing neighbouring pieces by up to 11 mm on almost every pick. The stock LeArm claw is roughly 60 mm wide and my simulated squares were 18 mm. No amount of software fixes that. It was the first hint that the real board would have to be different from the simulated one, and the real board turned out to be very different indeed.
The Wrong Device and a udev Rule

Figure 1: The controller board.
The arm showed up on the USB bus as an HID device called “Hiwonder” (a MindMotion MM32F103RB, USB ID 0483:5750). Before that I had been talking to /dev/ttyACM0, a USB serial port, assuming it was the arm. It was not. It was the controller board inside Reachy Mini. A handful of read-only query bytes went to the wrong robot before we noticed. It shrugged them off, but it is a good reminder to check what you are talking to before you send it anything.
Linux made the HID device root-only, so a one-line udev rule fixed the permissions:
SUBSYSTEM=="hidraw", ATTRS{idVendor}=="0483", ATTRS{idProduct}=="5750", MODE="0666"
The protocol is delightfully simple: 64-byte HID reports that start with 55 55, then a length and a command. Command 15 reads the battery voltage and command 3 moves servos with a duration, so every move carries its own speed. The first thing the arm ever said back to me was 7.7 V. It felt like a handshake.
One Servo at a Time
With real servos under my fingertips, I did not run the planner. I moved one joint at a time, in slow small steps, and described to Claude what I saw and heard after each one. It sounds painfully slow, but it is the only safe way to learn what a mechanism actually does.
The good news: every direction matched.
| Servo | Joint | Larger pulse moves it… |
|---|---|---|
| 6 | Base pan | to the left |
| 5 | Shoulder lift | forward, toward the board |
| 4 | Elbow | backward |
| 3 | Wrist flex | backward |
| 2 | Wrist roll | to the left |
| 1 | Claw | closed |
Every one of these agreed with the URDF axes from Part 1. The sign fixing from Bug 1 paid off: the geometry in the digital twin really is the real arm’s geometry. Sending 1500 to the lift, elbow, wrist flex and roll stood the arm perfectly upright, which is exactly what the simulator calls zero.

Figure 2: The laptop webcam’s view of the rig. It turned out to be a very useful pair of eyes.
The bad news: the claw. The simulator assumed a claw that opens to about 90 mm at pulse 1100 and closes around 1750. The real claw is nothing like that:
| What | Simulator assumed | Real arm |
|---|---|---|
| Fully open | pulse 1100, about 90 mm gap | about 1450, roughly 2 inches (50 mm) |
| Open end stop | not modeled | clicks violently at 1250. Never go below about 1400 |
| Closing | linear, done by 1750 | the gap barely changes until about 1900, then closes to about 2400 |
| Grip on a pawn’s neck | about 1620 | about 2350 |
Key lesson: If a servo is clicking, it is pushing against an end stop and chewing its gears. I only found the open limit because I was stepping slowly and listening. Back off at the first click, and put a hard clamp on every servo in the driver.
The driver now clamps the claw to 1400-2400, and it refuses to move any servo whose safe range I have not measured.
The Board Is Too Big for the Arm
Then I put a real chessboard in front of the arm: 12 inches across, with 1.5-inch squares and the near edge 5 inches from the centre of the base. Those squares are twice the size of the simulator’s, which is great news for a claw that is 60 mm wide. The bad news is reach.
The LeArm can stretch about 290 mm from its base at table height. The far edge of the board is 432 mm away.
rank 8 ######## reachable (black side, nearest the arm)
rank 7 ######## reachable
rank 6 ######## reachable
rank 5 .######. reachable except the a and h corners
rank 4 ........ out of reach
rank 3 ........ out of reach
rank 2 ........ out of reach
rank 1 ........ out of reach
Moving the board closer does not help, because the arm also cannot grasp anything much nearer than about 100 mm from its base. For this arm, a full game needs a board with squares of about 24 mm, and pieces to match. For now I have kept this board and let Kit work on black’s half only. That is enough to calibrate pick and place and to start training, even though it is not enough for a full game.
First Real Pick and Place
Pulse 1500 on the base pointed the arm 90 degrees away from the board. The base servo only travels about 90 degrees either way, so this could not be fixed in software without putting half the board out of range. I switched the board off and physically turned the arm’s base a quarter turn.
After that came the process I now think of as “go down a little further”. We hovered the claw above the black king, lowered it in 5 to 10 mm steps, and I told Claude what I could see. A lot of what looks like precise calibration in the table below was me saying “close it slightly more” while holding my phone camera over the board.
Figure 3: Pads on the king’s base. Too low.
Figure 4: Pads on the stem. Closer, but loose on one side.
Three things went wrong, and each one gave us a number:
- My estimate of how high the board sat relative to the arm was off by close to 100 mm. A photo of the pads sitting on the king’s base gave the real figure.
- A grip height that suits the king (pads 20 mm above the board) scraped the top of the pawn. Pawns need the pads at 12 mm, the lowest I will let the arm go.
- The claw’s approach opening has to fit between neighboring pieces. At 50 mm the pads hit the pieces on either side, so I narrowed it to roughly 20 mm for the descent.
Then the king came up in the claw, level and firm, and went back down on e8. After the king, a pawn: d7 to d6. The pawn landed about a third of a square too far toward the back of the square. That is a classic open-loop error. Tiny mistakes in link lengths and servo scale add up the further the arm stretches, so each rank farther from the base is a little worse. We applied a 13 mm correction and tried again, and it did not fix it. That is a teething problem for another day. Guessing the error by eye is exactly the kind of thing a camera should do for me, and it is a big part of why I decided to add one.
| Measured on the real arm | Value | How |
|---|---|---|
| Pawn / king height | 38 mm / 64 mm | ruler |
| Pad height for a king’s neck | 20 mm above the board | stepping down, with a photo at each step |
| Pad height for a pawn’s neck | 12 mm above the board | same, after scraping the top once |
| Claw opening for the approach | about 20 mm (pulse 2230) | fits between neighbors |
| Grip: king / pawn | pulse 2380 / 2360 | closed in steps until firm |
| Carry height | at least 80 mm above the board | clears the king at 64 mm |
Why I Am Adding a Depth Camera
Look at how I calibrated everything above. Every number came from a person standing next to the arm, watching it, and telling a computer what they saw. That works for one pawn on one square. It will not work for sixty-four squares, six piece types, a board that gets nudged and a claw that wears.
The arm needs eyes. I had an Intel RealSense D435 depth camera to hand. I plugged it into the laptop and installed pyrealsense2 in a separate Python virtual environment so I would not disturb the simulator. Within a few minutes I had depth and colour frames. It reports distance in millimetres at 640 x 480 and 15 frames a second, and the first thing it told me was that the surface it was pointing at was 197 mm away.
Why not mount it on the wrist? My first instinct was to fix it above the grip, so it could look straight at the piece. We ruled that out for three reasons:
- Minimum range. The D435 cannot measure anything closer than about 20 to 28 cm. On the wrist it would be 5 to 15 cm from the piece at the moment I most need to know, and the depth would be blank.
- Weight. It is 70 to 100 g with its cable, hanging off a small wrist servo at full stretch.
- The cable. A stiff USB 3 cable pulling on the wrist would disturb the very calibration I am trying to build.
If I ever want a camera on the arm itself, the Intel D405 is designed for that, with a minimum range of about 7 cm.
A fixed overhead camera. So the D435 goes on a fixed stand above the board, 500 mm over the surface, directly above the board’s centre line and 186 mm in front of the base axis, lenses pointing straight down, with the long side of the camera running along the line from the arm to the far edge of the board. At that height the colour camera covers a 687 x 384 mm patch, which takes in the arm’s base and the whole board with a margin. I drew a diagram so I can build the bracket.
Figure 5: The mounting diagram. 500 mm up, 186 mm in front of the base, lenses straight down, support coming in from the side.
Precision matters less than you might think. The camera will calibrate itself from the squares of the board and a plane fitted to the depth. It just has to be rigid.
A live view in the simulator. Of course it needed a window into the simulator. A small service on the laptop streams the camera over HTTP as MJPEG, and a new box at the top left of the simulator shows it live. It has a toggle between colour and a false-colour depth view, and clicking anywhere on the image reads out the depth in millimetres at that pixel. The service runs under systemd with the same restart behaviour as the rest of the stack, so it comes back when the laptop does. (The only hiccup was a port clash: the one I wanted was already used by Reachy Mini’s software, so it lives on 8790.)

Figure 6: The camera box in the simulator, top left, live at 15 fps.
Here is what I want the camera to do, once it is mounted:
- Find the board squares in the image and work out the true transform between the camera and the arm, replacing the numbers I tuned by hand.
- Measure where each piece actually stands and correct the claw position before every grasp.
- Check the height of the piece top, so the pads stop at the right place for each piece.
- Later, watch the board to see the human’s moves.
None of that exists yet. Right now I have a camera that reads depth, a window for it in the simulator and a bracket to build.
What Survived and What Did Not
| Survived contact with the hardware | Did not |
|---|---|
| Every joint axis and sign from the URDF | The claw’s open and closed pulses |
| Pulse 1500 as the upright zero pose | The board size and layout (only ranks 5 to 8 are reachable) |
| The planner’s structure: hover, descend, grasp, lift, carry | Every hover and grasp height, all re-measured |
| Piece-specific top grasps | Position accuracy at the far end of the reach |
| Clamping and stepping as a safety habit | Assuming the board and the arm sit at the same height |
Lessons So Far
- Check which device you are talking to before you send it a single byte.
- Move one joint at a time, in small steps, and listen.
- Put a hard clamp on every servo in the driver, and refuse to move any servo you have not measured.
- Measure the claw and the board before you trust the maths.
- A human with a phone makes a great calibration tool, right up until you need to do it twice.
What Comes Next: Part 3
In Part 3 I will mount the depth camera, calibrate it against the board and let it close the loop: look, correct, grasp. If that works, Kit will play black’s half of a real game on the wooden board. I also need to make a decision about the board: a smaller one so the arm can reach every square, or a different arm.














