What is the Lerobot skill?
Lerobot is a ai engineering skill by arpitg1304 for Claude Code, Cursor, Codex and other AI coding agents. Hugging Face LeRobot end to end: install, hardware bring-up, dataset recording, training, and deploying robot learning policies.
That is the main branch. The newest PyPI release on that date was 0.7.0. LeRobot changes quickly between minor versions, so check what is installed before trusting any flag in this file (see Check what you have). This file marks what was read in the source or docs as verified. It marks recommendations that come from field experience as advice.
What the SKILL.md covers
- Contracts Between Stages
- Check What You Have First
- Install (verified)
- SO-101 Bring-Up (follower + leader, wrist + overhead camera)
- 1. Ports
- 2. Motor IDs and baudrate (once per arm, when new or reused)
- 3. Calibration
- 4. Teleoperate (check calibration before adding cameras)
- 5. Cameras
- Recording Datasets
- Author
- arpitg1304
- Pack
- arpitg1304/robotics-agent-skills (17 skills)
- Category
- AI engineering, #440 of 610
- Alyn Score
- 60 / 99
- Files
- 11, includes references
- License
- Apache-2.0
- Last updated
- 2 days ago
- Works with
- Claude Code, Claude, Cursor, Codex, Gemini CLI, Copilot
Lerobot compared to similar skills
| Skill | Alyn Score | GitHub stars | Updated |
|---|---|---|---|
| Lerobot · arpitg1304 | 60 | 403 | 2 days ago |
| Trl Training · Hugging Face | 72 | 11k | 2 days ago |
| Fine Tuning Expert · Jeffallan | 78 | 12k | 4 days ago |
| Domain Modeling · Matt Pocock | 98 | 328k | 3 days ago |
Install Lerobot
Pick your agent. Every command is ready to paste.
npx alyn-cli add lerobotDetects your agents (Claude Code, Cursor, Codex and 70+ more) and installs to the right folder. Add -g to install globally.
npx skills add arpitg1304/robotics-agent-skills --skill lerobotOr use the open skills CLI directly.
01
Install it
Run the command above, or upload the folder in Claude.
02
Ask naturally
Describe the task. Your agent loads lerobot when the request matches its description.
03
Name it if needed
If it does not trigger, say “use the lerobot skill”. In Codex, type $lerobot.
LeRobot Skill
Verified against LeRobot 0.7.1 (commit 7e57397), 2026-11-18.
That is the main branch. The newest PyPI release on that date was 0.7.0. LeRobot changes
quickly between minor versions, so check what is installed before trusting any flag in this file
(see Check what you have). This file marks what was read in the
source or docs as verified. It marks recommendations that come from field experience as
advice.
Use a sibling skill instead for: choosing an IL algorithm, action representation or data
budget (imitation-learning). Auditing demo quality, outliers or label noise (robot-data-quality).
RL theory and reward design (robot-reinforcement-learning). Building MuJoCo or Isaac scenes
(mujoco, isaac-sim). Full scope list, pipeline diagram and draccus flag syntax:
overview.md.
Flags (verified): every CLI is a draccus dataclass. Nested fields are dotted flags
(--robot.port, --policy.chunk_size), type fields pick a registered subclass
(--robot.type=so101_follower), and dicts and lists are passed as YAML/JSON strings.
<command> --help is the ground truth for the installed version.
Contracts Between Stages
Six things tie the stages together. When one changes, the next stage breaks:
| Contract | Set where | Breaks when |
|---|---|---|
--robot.id / --teleop.id |
every hardware command | a different id loads a different calibration file, or none |
| Calibration file | $HF_LEROBOT_HOME/calibration/... |
you recalibrate after recording, so positions shift |
| Camera names | --robot.cameras dict keys |
they become observation.images.<name>, and the policy expects exactly those |
Camera resolution and fps |
camera config, --dataset.fps, --fps |
deployment differs from training, or the loop cannot keep the rate |
Joint units (use_degrees) |
robot/teleop config | dataset in one unit, robot in another |
| Normalisation stats | dataset meta/stats.json, copied into checkpoint processors |
stats missing, or taken from the wrong dataset |
Check What You Have First
lerobot-info # LeRobot, Python, torch, CUDA/MPS, ffmpeg, torchcodec versions
pip show lerobot | head -3 # installed version
lerobot-record --help | grep -A2 streaming # does this build have the flag I'm about to use?
# Registered types. A draccus ChoiceRegistry only knows the modules that have been imported, so import the CLI module first:
python -c "import lerobot.scripts.lerobot_record; from lerobot.robots import RobotConfig; print(sorted(RobotConfig.get_known_choices()))"
python -c "import lerobot.policies; from lerobot.configs.policies import PreTrainedConfig as P; print(sorted(P.get_known_choices()))"
Changes that make older tutorials wrong (verified in 0.7.x docs and source):
- Policy deployment moved from
lerobot-record --policy.pathtolerobot-rollout. In 0.7,lerobot-recordneeds a teleop and rejects dataset names starting witheval_. The cheat sheet sayslerobot-rolloutdoes not exist in v0.5.x and earlier. - Weights & Biases was replaced by Trackio. Use
--trackio.enable=trueinstead of--wandb.enable=true. Old checkpoints with awandbsection are migrated when loaded. - Python >= 3.12 is required (
requires-python = ">=3.12"). lerobot-recordappends_YYYYMMDD_HHMMSSto--dataset.repo_idunless you pass--dataset.no_stamp=true.- The dataset format is v3.0. A v2.1 dataset must be converted before 0.4+ can read it.
- Feetech and Dynamixel buses go through
rustypot(extrasfeetech/dynamixel).
Install (verified)
conda create -y -n lerobot python=3.12 && conda activate lerobot
conda install ffmpeg -c conda-forge # TorchCodec video decoding; system ffmpeg is OK only with torch >= 2.10
pip install 'lerobot[core_scripts,training,feetech]' # SO-100/101: record + train
# Policy extras as needed: smolvla | pi (pi0/pi0.5/pi0-FAST) | diffusion | groot | xvla | ...
pip install 'lerobot[smolvla]'
# From source (for reading or patching code):
git clone https://github.com/huggingface/lerobot && cd lerobot && pip install -e ".[core_scripts,training,feetech]"
core_scripts=dataset+hardware+viz(record, replay, calibrate, Rerun/Foxglove).training=dataset+accelerate+trackio.intelrealsenseis needed for RealSense cameras. Sim envs are extras too:pusht,aloha,libero(Linux only),metaworld.- Linux torch wheels from PyPI are CUDA 13 builds and need NVIDIA driver >= 580.65. For an older
driver, install torch from the
cu126index first (seedocs/source/installation.mdx). - Apple Silicon: MPS needs macOS 15+. On macOS 14, LeRobot falls back to CPU.
- Load policies.md for the policy-to-extra table.
SO-101 Bring-Up (follower + leader, wrist + overhead camera)
Do these steps in order. Each step relies on the one before it. Keep the shared values in one env file (advice) so every command uses the same ports, ids and camera dict:
# so101.env (source it in every shell: `source so101.env`)
export FOLLOWER_PORT=/dev/ttyACM0 LEADER_PORT=/dev/ttyACM1 # fill in from step 1
export FOLLOWER_ID=so101_follower_01 LEADER_ID=so101_leader_01 # never change after calibrating
1. Ports
lerobot-find-port # unplug the board when prompted; it prints that board's port
# macOS: /dev/tty.usbmodem5A7A0185331 Linux: /dev/ttyACM0
sudo usermod -aG dialout "$USER" # Linux, then log out/in (docs show `sudo chmod 666 /dev/ttyACM0`, which resets on replug)
ls -l /dev/serial/by-id/ # Linux advice: stable names; ttyACM0/1 can swap after reboot or replug
Advice: put the /dev/serial/by-id/... path in your variables. Then the leader and follower
cannot swap ports without you noticing.
2. Motor IDs and baudrate (once per arm, when new or reused)
lerobot-setup-motors --robot.type=so101_follower --robot.port=$FOLLOWER_PORT
lerobot-setup-motors --teleop.type=so101_leader --teleop.port=$LEADER_PORT
The script asks for one motor at a time, starting with gripper (ID 6) and ending at
shoulder_pan (ID 1). Only that motor can be connected to the board. It writes the ID and
baudrate to EEPROM. Waveshare jumpers, gear ratios: so101-hardware.md.
3. Calibration
lerobot-calibrate --robot.type=so101_follower --robot.port=$FOLLOWER_PORT --robot.id=$FOLLOWER_ID
lerobot-calibrate --teleop.type=so101_leader --teleop.port=$LEADER_PORT --teleop.id=$LEADER_ID
Procedure (verified from SerialMotorsBus.record_calibration):
- Torque turns off. Move every joint to the middle of its range and press ENTER. That pose becomes the zero (half-turn homing offset).
- Sweep each joint through its full range, then press ENTER.
wrist_rollis treated as a full-turn motor and is not swept. - The result is written to the motors and saved as JSON:
- follower:
~/.cache/huggingface/lerobot/calibration/robots/so_follower/<robot.id>.json - leader:
~/.cache/huggingface/lerobot/calibration/teleoperators/so_leader/<teleop.id>.json
- follower:
Commit or back up these JSON files next to your datasets. A policy is only valid with the calibration its data was recorded under. Dir overrides, reuse prompt: so101-hardware.md.
4. Teleoperate (check calibration before adding cameras)
lerobot-teleoperate \
--robot.type=so101_follower --robot.port=$FOLLOWER_PORT --robot.id=$FOLLOWER_ID \
--teleop.type=so101_leader --teleop.port=$LEADER_PORT --teleop.id=$LEADER_ID
Things to check (advice): the follower tracks the leader with no constant offset. The gripper opens
and closes fully. Nothing buzzes at rest. lerobot-teleoperate loops at --fps=60 by default and
prints a cadence summary when it exits.
5. Cameras
lerobot-find-cameras opencv # lists ids + default profile, saves frames to outputs/captured_images
lerobot-find-cameras realsense # needs lerobot[intelrealsense]; macOS may need sudo and is unstable
Define the camera dict once and reuse it in every command. Names and resolution must stay the same from recording to deployment:
# append to so101.env. Linux paths shown; on macOS use the integer ids from lerobot-find-cameras (index_or_path: 0 / 1)
export CAMERAS='{ wrist: {type: opencv, index_or_path: /dev/video2, width: 640, height: 480, fps: 30, fourcc: MJPG}, top: {type: opencv, index_or_path: /dev/video0, width: 640, height: 480, fps: 30, fourcc: MJPG} }'
lerobot-teleoperate ... --robot.cameras="$CAMERAS" --display_data=true # Rerun viewer
- Mount both cameras rigidly and light the scene evenly. A wrist camera mount that moves between sessions is a silent distribution shift.
- Camera fields, stable
/dev/v4lnames, USB bandwidth, RealSense: so101-hardware.md. Other robots (SO-100, bimanual, Koch, OMX, LeKiwi, OpenArm, ...): robots.md.
Recording Datasets
hf auth login # write token
HF_USER=$(NO_COLOR=1 hf auth whoami | awk -F': *' 'NR==1 {print $2}')
lerobot-record \
--robot.type=so101_follower --robot.port=$FOLLOWER_PORT --robot.id=$FOLLOWER_ID \
--robot.cameras="$CAMERAS" \
--teleop.type=so101_leader --teleop.port=$LEADER_PORT --teleop.id=$LEADER_ID \
--dataset.repo_id=${HF_USER}/so101_cube_to_bowl \
--dataset.single_task="Pick up the red cube and place it in the bowl" \
--dataset.num_episodes=50 --dataset.episode_time_s=30 --dataset.reset_time_s=15 \
--dataset.fps=30 \
--dataset.streaming_encoding=true --dataset.encoder_threads=2 \
--display_data=true
- Keys (verified):
→/nends the episode or reset early.←/rre-records the episode.ESC/qstops, encodes and uploads. - The saved name is
${HF_USER}/so101_cube_to_bowl_YYYYMMDD_HHMMSS. Find it withls ~/.cache/huggingface/lerobot/${HF_USER}and use that full name for resume, train and replay. - Resume an interrupted session or add episodes. Here
num_episodesmeans additional episodes:
On resume, robot_type, fps and the feature dict must match the existing dataset. A camera rename or resolution change failslerobot-record ...same robot/teleop/cameras... --resume=true \ --dataset.repo_id=${HF_USER}/so101_cube_to_bowl_20261118_143012 \ --dataset.root=$HOME/.cache/huggingface/lerobot/${HF_USER}/so101_cube_to_bowl_20261118_143012 \ --dataset.num_episodes=10sanity_check_dataset_robot_compatibility. - Read the cadence summary printed at the end of each episode and of the session. Frame
timestamps come from the frame index, so a loop that ran at 25 Hz still produces a dataset that
claims 30 Hz. If
observetakes most of the work time, the cameras are the bottleneck. "pacing headroom" near zero means the loop is saturated. - Replay to check repeatability:
lerobot-replay --robot.type=so101_follower --robot.port=$FOLLOWER_PORT --robot.id=$FOLLOWER_ID --dataset.repo_id=<full_name> --dataset.episode=0
Dataset Format (v3.0)
<root>/ # default $HF_LEROBOT_HOME/<repo_id> = ~/.cache/huggingface/lerobot/<repo_id>
meta/info.json # codebase_version "v3.0", fps, robot_type, features (dtype/shape/names), path templates
meta/stats.json # mean/std/min/max (+ quantiles like q01/q99) per feature -> normalisation
meta/tasks.parquet # task string <-> task_index
meta/episodes/chunk-000/file-000.parquet # per-episode length, tasks, offsets into shared files, per-episode stats
data/chunk-000/file-000.parquet # many episodes per file: action, observation.state, timestamp, indices
videos/observation.images.top/chunk-000/file-000.mp4 # many episodes per mp4, one stream per camera
SO-101 features (verified pattern): action and observation.state are float32 vectors with
names [shoulder_pan.pos, shoulder_lift.pos, elbow_flex.pos, wrist_flex.pos, wrist_roll.pos, gripper.pos].
Images are observation.images.<camera_name>. Bookkeeping columns, Python loading, visualiser,
lerobot-edit-dataset ops, v2.1 conversion and finalize(): dataset-format.md.
Training
lerobot-train \
--dataset.repo_id=${HF_USER}/so101_cube_to_bowl_20261118_143012 \
--policy.type=act \
--policy.device=cuda \
--policy.repo_id=${HF_USER}/act_so101_cube_to_bowl \
--output_dir=outputs/train/act_so101_cube_to_bowl \
--job_name=act_so101_cube_to_bowl \
--batch_size=8 --steps=40000 --save_freq=5000 \
--trackio.enable=true
- Size
--stepsfrom epochs, not the 100k default. The docs say 5–10 epochs is typical. 50 episodes × 30 s × 30 fps = 45k frames. At batch 8 that is about 5.6k steps per epoch, so 28k–56k steps. When you shorten the run, also shorten--save_freq. For policies with a decay schedule (smolvla, pi0/pi05, ...), set--policy.scheduler_decay_steps≈--steps. Otherwise the LR never decays.
Resume, checkpoints, devices, SmolVLA fine-tune (camera renaming) and ACT defaults: training.md. Policy choice: policies.md.
Deployment on the Real Robot (lerobot-rollout)
# Quick look, no recording
lerobot-rollout --strategy.type=base \
--policy.path=${HF_USER}/act_so101_cube_to_bowl \
--robot.type=so101_follower --robot.port=$FOLLOWER_PORT --robot.id=$FOLLOWER_ID \
--robot.cameras="$CAMERAS" \
--task="Pick up the red cube and place it in the bowl" --duration=60 --display_data=true
- Action chunks: ACT runs
n_action_stepsactions open-loop. At the default 100 steps and 30 fps that is 3.3 s blind. For reactive tasks, lower it, or use temporal ensembling with--policy.temporal_ensemble_coeff=0.01 --policy.n_action_steps=1, which queries every step. For slow VLAs (pi0/pi0.5/SmolVLA), use--inference.type=rtcto compute the next chunk while the current one executes. You can also move the model to a GPU server withlerobot-policy-server+--inference.type=remote. That needslerobot[remote]on both machines, and ACT must have temporal ensembling disabled. Readdocs/source/remote_inference.mdxbefore configuring it. - Advice: for the first run of a new policy, set
--robot.max_relative_target=10. It caps how far any joint target can jump in one step (units followuse_degrees). Keep a hand on the follower's power switch.
Recorded episodic evaluation, other strategies, overrides, Python inference loop, processors:
deployment.md. Sim eval and HIL-SERL:
sim-and-rl.md.
Common Failure Modes
| # | Symptom | Cause | Fix |
|---|---|---|---|
| 1 | Follower holds a constant offset from the leader, the gripper never fully closes, or calibration runs again on every command | Different --robot.id/--teleop.id between commands (each id is its own JSON), calibration not done at mid-range, or files deleted |
Use one id everywhere (put it in env vars). ls ~/.cache/huggingface/lerobot/calibration/*/*/. Recalibrate both arms, moving to true mid-range |
| 2 | A policy that worked yesterday is now off by a few degrees per joint | Recalibrated after recording, so the new zero no longer matches the data | Treat calibration JSON as part of the dataset: back it up, never recalibrate mid-project. If you must, restore the old file (ENTER at the prompt writes it to the motors) or re-record |
| 4 | Cadence summary below target, jerky video, or duplicate frames | Slow cameras or USB saturation. get_observation() uses read_latest(), which returns a buffered frame up to 500 ms old, so a stalling camera duplicates frames silently |
Fix it before recording 50 episodes: fourcc: MJPG, 640×480, separate USB ports, --display_data=false, --dataset.streaming_encoding=true. Check "Encoder queue full ... dropped N frame(s)" warnings |
| 5 | FrameTimestampError: One or several query timestamps unexpectedly violate the tolerance while training |
A video stream is shorter than or out of sync with the parquet (dropped encoder frames, crash during encoding, edited file) | Open that episode in the visualiser. Remove it with lerobot-edit-dataset --operation.type delete_episodes. Raising --tolerance_s (default 1e-4) only hides the problem |
| 11 | Visual feature mismatch between policy and robot hardware at rollout, or a feature mismatch at train |
Camera names differ (front vs top, or camera1 for SmolVLA base) |
Reuse the exact $CAMERAS string. Otherwise add --rename_map='{"observation.images.top": "observation.images.camera1"}' (observation keys only) |
| 12 | Deployed policy behaves well for about a second, then drifts or overshoots | Open-loop chunk execution (ACT 100 steps), or sync inference stalls at chunk boundaries | --policy.n_action_steps=20–50, temporal ensembling, RTC, or remote inference. Check that infer in the rollout cadence fits the budget |
| 14 | Incorrect status packet! or intermittent read failures |
Loose 3-pin daisy chain, weak supply, many joints moving at once | The SO configs retry num_read_retries=2 times. Reseat the cables, check power. Raise --robot.num_read_retries only once the wiring is known good |
| 15 | Repository not found / dataset not found at train time |
Used the unstamped repo_id |
Use the _YYYYMMDD_HHMMSS name, or record with --dataset.no_stamp=true |
Row numbers match the full table in troubleshooting.md, which has the other rows (units, crash recovery, v2.1/v3, q01/q99 stats, overheating, old flags, task strings), more anti-patterns and the debugging playbook.
Anti-patterns:
- Recording 50 episodes before checking one. Record 2, replay one, open it in the visualiser, read the cadence summary, then scale up.
- Changing camera placement, resolution or names between sessions of one dataset. Resume rejects the feature change, and a policy cannot learn from a moving mount.
Experiment Checklist
-
lerobot-infosaved. LeRobot version and commit recorded alongside each dataset and checkpoint - Ports use stable names.
FOLLOWER_ID/LEADER_ID/CAMERASdefined once in an env file - Both arms calibrated at true mid-range. Calibration JSONs backed up with the dataset
- Teleop has no offset. Gripper opens and closes fully. No buzzing or heat after 5 minutes
- Cameras rigidly mounted, even lighting, 640×480@30 with MJPG, cadence ≥ 99% of target
- Task string written down and reused verbatim. Episodes visualised. Bad ones deleted
-
--stepsfrom 5–10 epochs,save_freqand scheduler decay adjusted. Trackio enabled - First deployment uses
max_relative_target,--duration, and a hand on the power switch - Evaluation recorded with
episodic, with a fixed number of trials and a success criterion written down beforehand
Related Skills
imitation-learning: algorithm choice (ACT vs diffusion vs VLA), action chunking theory, data scaling, evaluation protocolrobot-data-quality: auditing recorded episodes, outliers, sync and label QA before trainingrobot-reinforcement-learning: SAC/HIL-SERL concepts, reward classifiers, safety during online RLmujoco,isaac-sim: building simulated SO-101 scenes and envs (LeRobot EnvHub/IsaacLab Arena consume them)robot-bringup,robotics-testing: general hardware bring-up discipline and test strategyrobot-app-harness: closed-loop development of scripted robot apps against a simulated SO-101 (ARMATURE). It has no LeRobot hardware backend yet, so it does not drive a real arm
Reference Files (load on demand)
- so101-hardware.md: when an SO-101 motor, calibration or camera step misbehaves (gear ratios, calibration dirs, camera fields, USB bandwidth).
- troubleshooting.md: when a symptom is not in the short table, or to debug from the bus up (18 failure rows, anti-patterns, playbook).
- training.md: before resuming, fine-tuning SmolVLA, training off-GPU or on HF Jobs.
- deployment.md: for recorded evaluation, other rollout strategies, a custom Python control loop, or processor/normalisation questions.
- sim-and-rl.md: for
lerobot-evalin simulation or HIL-SERL. - cli-reference.md: when building a non-trivial command (flag tables and defaults for the main CLIs).
- robots.md: for any robot other than a single SO-101, and for teleop devices.
- dataset-format.md: for loading from Python, editing, merging, converting, streaming, transforms or custom loaders.
- policies.md: when choosing a policy or its extras, VRAM and policy-specific flags.
- overview.md: for the pipeline diagram, full scope list or draccus dotted-flag syntax.
Tips for Lerobot
- • Start a fresh session after installing so your agent picks up the new skill.
- • It bundles reference files. The agent reads them only when needed, so they cost no context until then.
- • It is part of a pack with 17 skills. Install only the ones you need with --skill to keep triggers precise.
- • If two skills overlap, mention the one you want by name.
Questions about Lerobot
What does the Lerobot skill do?
Hugging Face LeRobot end to end: install, hardware bring-up, dataset recording, training, and deploying robot learning policies.
How do I install Lerobot?
Run "npx alyn-cli add lerobot" in your project. The Alyn CLI installs it for Claude Code, Cursor, Codex and 70+ other agents. In Claude on the web, upload the skill folder as a ZIP under Customize > Skills.
Does Lerobot work in Cursor and Codex?
Yes. Lerobot uses the open SKILL.md format, so it works in Claude Code, Claude, Cursor, OpenAI Codex, Gemini CLI, GitHub Copilot and other agents that support agent skills.
What is inside the Lerobot skill?
The SKILL.md covers Contracts Between Stages, Check What You Have First, Install (verified), SO-101 Bring-Up (follower + leader, wrist + overhead camera), 1. Ports, 2. Motor IDs and baudrate (once per arm, when new or reused), and the skill folder has 11 files.
What is a good alternative to Lerobot?
Trl Training by Hugging Face is the highest-ranked similar skill from another creator, with an Alyn Score of 72 compared to 60 for Lerobot.
Is Lerobot free?
Lerobot is open source on GitHub under the Apache-2.0 license. It is free to install; check the license before commercial use.
Who made Lerobot?
Lerobot is published by arpitg1304 in the arpitg1304/robotics-agent-skills repository, together with 16 other skills.



