What is the Isaac Sim skill?
Isaac Sim is a science skill by arpitg1304 for Claude Code, Cursor, Codex and other AI coding agents. NVIDIA Isaac Sim and Isaac Lab for robot learning: install (uv, pip, binary, Docker, cloud), RTX GPU and driver requirements, headless runs and livestreaming, OpenUSD assets, URDF/MJCF import, ArticulationCfg and actuators, manager-based and direct envs, configclass, gymnasium task registration, training and playing with rsl_rl, rl_games, skrl or sb3, parallel envs, tiled cameras, contact sensors, ray casters, IMU…
Verified against Isaac Lab 3.0.0 (tag v3.0.0) / Isaac Sim 6.2.0, 2026-11-18. Not executed: examples are checked against source, not run.
What the SKILL.md covers
- When to Use This Skill
- Reference Files
- Ecosystem Map and Requirements
- Three choices you make separately
- Setup
- Recommended: uv from a checkout
- Verify, smallest layer first
- Headless, visualizers, livestream
- Assets and Import
- Robot config
- Author
- arpitg1304
- Pack
- arpitg1304/robotics-agent-skills (17 skills)
- Category
- Science, #195 of 221
- Alyn Score
- 53 / 99
- Files
- 10, includes references
- License
- Apache-2.0
- Last updated
- 2 days ago
- Works with
- Claude Code, Claude, Cursor, Codex, Gemini CLI, Copilot
Isaac Sim compared to similar skills
| Skill | Alyn Score | GitHub stars | Updated |
|---|---|---|---|
| Isaac Sim · arpitg1304 | 53 | 403 | 2 days ago |
| Omd Lab 02 Design Harness · kwakseongjae | 57 | 594 | 5 days ago |
| Cellprofiler · K-Dense | 80 | 58k | 5 days ago |
| Digital Worked Example Sequence · GarethManning | 62 | 1k | 18 days ago |
Install Isaac Sim
Pick your agent. Every command is ready to paste.
npx alyn-cli add isaac-simDetects your agents (Claude Code, Cursor, Codex and 70+ more) and installs to the right folder. Add -g to install globally.
npx skills add arpitg1304/robotics-agent-skills --skill isaac-simOr use the open skills CLI directly.
01
Install it
Run the command above, or upload the folder in Claude.
02
Ask naturally
Describe the task. Your agent loads isaac-sim when the request matches its description.
03
Name it if needed
If it does not trigger, say “use the isaac-sim skill”. In Codex, type $isaac-sim.
Isaac Sim and Isaac Lab
Verified against Isaac Lab 3.0.0 (tag v3.0.0) / Isaac Sim 6.2.0, 2026-11-18. Not executed:
examples are checked against source, not run.
The 3.0 release is a break. Isaac Lab is now multi-backend: it installs Newton (Warp)
physics by default and treats Isaac Sim as an optional extra. Quaternions are XYZW, asset data
is a ProxyArray, and one isaaclab train command replaces the per-library scripts. Most
Isaac Lab code that agents learned before 2026 is wrong for 3.0. When a detail matters, trust
the installed checkout over memory, in this order: --help output, then source under
source/, then the docs under docs/source/.
An Isaac Lab checkout also ships NVIDIA-maintained agent skills in .agents/skills/isaaclab-*
(installing, migrating 2.x to 3.x, presets, sensors and actuators, events, multi-GPU). When
you work inside that repo, read the matching one. This skill is the portable overview.
When to Use This Skill
- Installing Isaac Sim or Isaac Lab, or choosing between pip, binary, Docker and cloud
- Importing a URDF or MJCF robot, or writing an
ArticulationCfgwith actuators - Writing a manager-based or direct RL environment and registering it with gymnasium
- Training or playing a policy with
isaaclab train/isaaclab play - Adding cameras, contact sensors, ray casters, IMUs or frame transformers to a task
- Domain randomisation, actuator models, observation noise and latency for sim-to-real
- Teleoperating, recording demonstrations and multiplying them with Isaac Lab Mimic
- Connecting Isaac Sim to ROS 2
- Fixing exploding articulations, slow training, black camera images or a broken upgrade
Out of scope: RL algorithm choice, reward design and sim-to-real theory
(robot-reinforcement-learning); policy architectures and BC/diffusion training
(imitation-learning); dataset format and QA (lerobot, robot-data-quality); MuJoCo and
MJX outside Isaac Lab (mujoco).
Reference Files
Load these only when the task needs the detail:
references/install-and-setup.md: non-uv installs, Docker gotchas, version pairing, first launch, visualizers and livestream.references/assets-and-import.md: USD, schema fragments, robot configs, importer gotchas. Load before converting or debugging an asset.references/environment-examples.md: full manager-based task with registration, direct skeleton. Load when writing an environment.references/sensors-and-rendering.md: sensor table, camera example, image memory. Load when adding a sensor or camera.references/randomisation-and-actuators.md: events example, PhysX vs Newton differences, actuator models, sim-to-real hooks.references/debugging.md: physics settings and the full symptom table. Load when a symptom is not in the short table below.references/migration-2x-to-3x.md: porting Isaac Lab 1.x/2.x, Isaac Gym oromni.isaac.*code; old-name to new-name tables; also when an error names a removed API.references/config-reference.md: config field tables, event functions, all train/play flags, training notes (logging, outputs, custom scripts,num_envs).references/demonstrations-and-ros2.md: teleop → Mimic → robomimic, HDF5 layout, GR00T/RLinf, ROS 2 bridge.
Ecosystem Map and Requirements
| Component | Current (2026-11-18) | Notes |
|---|---|---|
| Isaac Sim | 6.2.0 (tag v6.2.0; v7.0.0a1 alpha exists) |
Omniverse Kit + OpenUSD app. Source on GitHub isaac-sim/IsaacSim, Apache-2.0; the build downloads the Kit SDK and extensions |
| Isaac Lab | 3.0.0 (tag v3.0.0, 2026-10-19) |
BSD-3-Clause (Mimic is Apache-2.0). main targets Isaac Sim 6.2 and has already removed some 3.0 deprecated aliases |
| Newton | 1.6.1, on Warp 1.17 | Default Isaac Lab physics. Solvers: MJWarp, Kamino, VBD, MPM. Needs no Isaac Sim |
| OvPhysX / OVRTX | 0.6.4 / 0.5.1 | PhysX and RTX rendering without Kit. Cannot share a process with Isaac Sim |
| Python / PyTorch | 3.12 / torch 2.11.0 + CUDA 13.0 | CUDA 13 requires Turing or newer (compute 7.5+) |
Isaac Lab 3.0 rejects Isaac Sim 5.1 and older. Full version pairing:
references/install-and-setup.md.
Requirements:
- OS: Ubuntu 22.04+ x86_64, Linux aarch64 (DGX Spark), Windows 11 x86_64. macOS is not supported by Isaac Sim or Isaac Lab; a Mac can only edit code or drive a remote or cloud machine. Isaac Sim marks Ubuntu 22.04 as deprecated, and its pip wheels need GLIBC 2.35+. Below that (for example Ubuntu 20.04), use the downloaded binary package.
- Driver: Isaac Lab needs 580.65.06+ on Linux and 580.88+ on Windows, set by PyTorch CUDA 13. Isaac Sim 6.2 was tested with 595.58.03 (Linux) and 595.97 (Windows).
- GPU for Isaac Sim: RTX with RT cores. The minimum is an RTX 4080 with 16 GB VRAM and 32 GB RAM. A100/H100 are not supported for Isaac Sim (no RT cores). Headless Newton simulation without cameras needs no RTX hardware.
- Disk/network (first-launch downloads, EULA env var):
references/install-and-setup.md.
Check what is actually installed before giving version-specific advice:
uv run python -c "import isaaclab; print(isaaclab.__version__)"
uv run --extra isaacsim python -c "from isaaclab.utils.version import get_isaac_sim_version; print(get_isaac_sim_version())"
uv pip show isaacsim isaaclab newton warp-lang torch | grep -E '^(Name|Version)'
nvidia-smi --query-gpu=name,memory.total,driver_version --format=csv,noheader
Three choices you make separately
physics= isaacsim_physx | physx (auto) | ovphysx | newton_mjwarp | newton_kamino ...
renderer= newton_renderer | ovrtx | isaacsim_rtx | rtx (auto) (camera tasks only)
--viz none | newton_gl | newton_rtx | kit | rerun | viser (what you look at)
Kit (Isaac Sim) is loaded only when something needs it: physics=isaacsim_physx,
renderer=isaacsim_rtx, --viz kit, livestreaming, or a direct isaacsim.* import. Defaults
are per task. Cartpole defaults to newton_mjwarp, and other tasks may default to
PhysX. Always list what a task supports:
uv run isaaclab list_envs --show_presets --keyword Cartpole
uv run isaaclab train --task Isaac-Cartpole --help # groups physics= / renderer= / presets=
Setup
Recommended: uv from a checkout
curl -LsSf https://astral.sh/uv/install.sh | sh
git clone https://github.com/isaac-sim/IsaacLab.git && cd IsaacLab
git checkout v3.0.0 # pin; main moves daily
uv run isaaclab train --task Isaac-Cartpole --num_envs 16 --viz newton_gl physics=newton_mjwarp
uv run --extra isaacsim isaaclab train --task Isaac-Cartpole-Direct physics=isaacsim_physx
uv run --extra ovphysx isaaclab train --task Isaac-Cartpole-Direct physics=ovphysx
uv run syncs the environment on every call. Put --extra <name> before isaaclab. The
extras are isaacsim, ov/ovphysx/ovrtx, rsl-rl, rl-games, skrl, skrl-jax,
sb3, rlinf, torchrl, rerun, viser, mimic, teleop, video, importers,
tetrahedralization and leapp. --extra all adds the curated ov, RL libraries, rerun
and viser, but not isaacsim, mimic or teleop.
Other install paths and Docker gotchas: references/install-and-setup.md.
Verify, smallest layer first
uv run python -c "import isaaclab; print('ok')" # import
uv run isaaclab zero_agent --task Isaac-Cartpole --num_envs 4 # task builds and steps
uv run isaaclab train --task Isaac-Cartpole --max_iterations 1 # runner plumbing
uv run --extra isaacsim isaaclab -p scripts/tutorials/00_sim/create_empty.py --viz kit # Kit opens
Headless, visualizers, livestream
- In 3.0, no
--vizmeans headless.--headlessand--enable_cameraswere removed: the 3.0 parser rejects them (seetest_kit_launcher.py), and cameras enable themselves. A few 3.0 doc pages still show these flags, so trust--help.
Visualizers and livestream: references/install-and-setup.md.
Assets and Import
USD, schema fragments and SI units/scale: references/assets-and-import.md.
Robot config
rigid_props and articulation_props are the schema-fragment lists from
references/assets-and-import.md.
import isaaclab.sim as sim_utils
from isaaclab.actuators import ImplicitActuatorCfg
from isaaclab.assets import ArticulationCfg
MY_ARM_CFG = ArticulationCfg(
spawn=sim_utils.UsdFileCfg(
usd_path="/assets/my_arm/my_arm.usda",
activate_contact_sensors=True, # required for any ContactSensor on this robot
rigid_props=rigid_props,
articulation_props=articulation_props,
),
init_state=ArticulationCfg.InitialStateCfg(
pos=(0.0, 0.0, 0.0),
rot=(0.0, 0.0, 0.0, 1.0), # XYZW in 3.0 (was WXYZ in 2.x)
joint_pos={"joint[1-6]": 0.0, "finger_.*": 0.04}, # regex keys, radians/meters
),
actuators={
"arm": ImplicitActuatorCfg(
joint_names_expr=["joint[1-6]"],
stiffness=400.0, damping=40.0, armature=0.01,
joint_effort_limit=87.0, # solver clamp (was effort_limit_sim)
joint_velocity_limit=2.6, # (was velocity_limit_sim)
),
"gripper": ImplicitActuatorCfg(joint_names_expr=["finger_.*"], stiffness=2e3, damping=1e2),
},
soft_joint_pos_limit_factor=0.95,
)
# in a scene: robot = MY_ARM_CFG.replace(prim_path="{ENV_REGEX_NS}/Robot")
URDF and MJCF import
Convert once, commit the USD, and spawn with UsdFileCfg. UrdfFileCfg and MjcfFileCfg can
convert at spawn time, but that hides conversion errors and costs startup time.
uv run --extra importers python scripts/tools/convert_urdf.py robot.urdf /out/dir \
--fix-base --merge-joints --joint-stiffness 100.0 --joint-damping 1.0 --joint-target-type position \
--ros_package_path my_pkg /path/to/my_pkg --viz newton_gl # NAME PATH; repeatable
uv run --extra importers python scripts/tools/convert_mjcf.py robot.xml /out/robot.usd \
--merge-mesh --self-collision --collision-type "Convex Hull"
Ready-made robot configs, importer gotchas and mesh conversion: load
references/assets-and-import.md before converting.
Environment Design
Manager-based (ManagerBasedRLEnv) |
Direct (DirectRLEnv) |
|
|---|---|---|
| Define | Config classes of terms: observations, actions, rewards, terminations, events, commands, curriculum | One class with _pre_physics_step, _apply_action, _get_observations, _get_rewards, _get_dones, _reset_idx |
| Best for | Most new tasks, ablations, Hydra overrides of single terms, sharing MDP terms | Custom step logic, fused reward kernels, MARL (DirectMARLEnv), Isaac Gym ports |
| Randomisation | events |
events config also works in direct envs |
Both are driven by a @configclass config (a dataclass with mutable defaults handled for you;
use cfg.replace(...) or isaaclab.utils.replace(cfg, ...) to copy with changes). The scene is
declared on cfg.scene and accessed as env.scene["name"] in both workflows.
A complete manager-based task, its gym.register call and isaaclab.tasks entry point, and
a direct skeleton with the 3.0 write_*_to_sim_index and joint-target APIs: load
references/environment-examples.md.
Rules that example encodes:
sim.dt * decimationis the control period. Setrender_interval = decimationso cameras and visualizers don't render every physics step.- Mark time limits
time_out=Trueso they are truncations, not failures. The RL side is inrobot-reinforcement-learning. - Every term function returns a
(num_envs,)or(num_envs, k)torch tensor onenv.device. No Python loops over envs and no.item()in the step path, because each sync stalls the GPU. - Resolve joint/body names once through
SceneEntityCfg(..., joint_names=...). Do not hard-code indices, because joint order can differ between PhysX and Newton.
Training and Playing
uv run isaaclab train --task Isaac-Velocity-Rough-AnymalD --rl_library rsl_rl \
--num_envs 4096 --seed 42 --max_iterations 1500 # task default physics
uv run isaaclab play --task Isaac-Velocity-Rough-AnymalD --checkpoint latest --num_envs 32 --viz newton_gl
uv run --extra isaacsim isaaclab train --task Isaac-Velocity-Rough-AnymalD physics=isaacsim_physx --viz kit
uv run isaaclab train --task Isaac-Cartpole --video --video_length 200 --video_interval 2000
uv run isaaclab train --task Isaac-Cartpole env.scene.num_envs=1024 env.sim.dt=0.005 agent.seed=7
uv run isaaclab train_multigpu --task Isaac-Cartpole # experimental
uv run isaaclab benchmark runtime --task Isaac-Cartpole # env-step throughput
--rl_librarytakesrsl_rl(default for most tasks),rl_games,skrl,sb3,rlinfortorchrl.rsl_rlships in the default uv env. The others need their extra (--extra rl-games,skrl,sb3, ...), and the task must register a<lib>_cfg_entry_point.--checkpointtakes a path,latest,bestorpretrained. It replaces 2.x--resume --load_run. Keep the samepresets=when loading, because an observation preset changes tensor shapes.- Hydra overrides use
env.<path>=andagent.<path>=(rl_games usesagent.params.seed). Overrides do not re-run__post_init__. If you changeenv.sim.dtorenv.decimation, also overrideenv.sim.render_intervaland every sensorupdate_periodderived from them. - Outputs, TensorBoard/W&B, custom scripts,
num_envssizing:references/config-reference.md.
Sensors and Rendering
Renderers: Newton Warp (renderer=newton_renderer) is fastest and lightest, with direct
lighting and no motion vectors. OVRTX (renderer=ovrtx) is the recommended kit-less RTX
path. Isaac RTX (renderer=isaacsim_rtx) is legacy, for runs that must stay in Isaac Sim.
Sensors, camera example, image memory and conventions: references/sensors-and-rendering.md.
Randomisation and Actuators
Event modes (EventTermCfg.mode): prestartup authors USD before sim start (scale, some
materials). startup runs once after sim start (mass, friction buckets). reset runs per
episode. interval runs during an episode, with interval_range_s.
Events example, backend differences, actuator models (including the 3.0 double effort
clip), sim-to-real hooks: references/randomisation-and-actuators.md.
Which parameters to randomise, by how much, and why: robot-reinforcement-learning.
Demonstrations and Imitation Learning
- Mimic needs task-space (end-effector) actions. It replays subtask segments under rigid
transforms relative to a reference object, so convert joint-space data with FK first.
Mimic task IDs still end in
-v0.
Commands, HDF5 format, GR00T: references/demonstrations-and-ros2.md. Dataset QA:
robot-data-quality; conversion: lerobot; policy choice: imitation-learning.
ROS 2 Bridge
Supported distros (Jazzy on 24.04, Humble on 22.04), DDS, OmniGraph, /clock and policy
deployment: references/demonstrations-and-ros2.md. ROS side: ros2, docker-ros2-development.
Performance and Debugging
Physics settings and the full debug table: references/debugging.md.
| Symptom | Likely cause | Fix |
|---|---|---|
unrecognized arguments: --headless (or --enable_cameras) |
Removed in 3.0 | Omit --viz (headless is the default), or use --viz none. Cameras enable themselves |
| Robot spawns rotated or upside down after an upgrade; goals wrong | WXYZ quaternions in 2.x code (3.0 is XYZW), or a USD Gf.Quat (real-first) passed straight in |
python scripts/tools/find_quaternions.py --path my_project/ (never on Isaac Lab itself), --fix-identity-only first, review the rest. WARN_ON_TORCH_QUATF_ACCESS=1 flags runtime reads |
'ProxyArray' object has no attribute 'clone' / detach |
3.0 data properties are ProxyArray |
Use .torch (or .warp) |
| Articulation explodes or NaNs on the first steps | Spawn overlap, joint positions outside limits, zero/tiny link inertia, stiff explicit PD at large dt | Clamp resets to soft limits, fix inertias, use implicit PD or smaller dt, add armature, raise position iterations |
| Robot sags or ignores targets | Zero drive gains from import (target_type none), effort limit too low, wrong action term |
Set actuator stiffness/damping, joint_effort_limit >= actuator_effort_limit, check the action term's joint regex |
| Contact sensor reads zeros | activate_contact_sensors=False on the spawner, or wrong body regex |
Enable it on the spawner. Print sensor.body_names |
| Camera images black or garbage at scale | RTX atlas limit, wrong offset convention, no light | Smaller images or fewer cameras, or newton_renderer. Fix convention. Add a light. Inspect the last env |
| Training slow, low GPU use | Visualizer on, cameras rendering every physics step, too few envs, CPU device | --viz none, set update_period, benchmark runtime to size num_envs, --device cuda:0 |
Debug order: reproduce with --num_envs 1-4, a fixed --seed, and zero_agent/random_agent.
Fix every [PhysX]/Newton warning before reading RL curves. Then profile with
isaaclab benchmark or Nsight Systems (docs/source/how-to/profile_with_nsys.rst). Kit logs
are written to the kit_<timestamp>.log path printed at startup. Add --info/--verbose for
more detail.
Anti-Patterns
- Copying 2.x snippets into 3.0. That means
--headless,scripts/reinforcement_learning/rsl_rl/train.py,-v0task IDs on core tasks, WXYZ quaternions, and.clone()on data. Grep for them after any migration. - Assuming Isaac Lab needs Isaac Sim. In 3.0 the default install is Newton-only. Adding
--extra isaacsimeverywhere costs startup time and RTX requirements you may not need. - Assuming one physics backend. Defaults are per task. Benchmark numbers and tuned gains from PhysX don't carry over to Newton unchanged.
- Tuning RL before the asset is right. Bad inertias, colliders and gains look like reward
problems. Run
zero_agentand look first. - Per-env Python loops,
.item(),.cpu()in rewards or observations. Each one stalls the GPU pipeline every step. - Rendering cameras at the physics rate and requesting buffers the policy never reads.
- Hard-coded joint indices, or time limits marked as failures. Use
SceneEntityCfgnames andtime_out=True. - Randomisation hidden in reward functions or the training script. Put it in
eventsso it is visible and overridable. - Mixing a binary Isaac Sim with conda/uv envs, or OVRTX/OvPhysX with Kit in one process.
- Training against
mainwithout pinning a tag. APIs change weekly.
Checklist
- Versions recorded: Isaac Lab tag/commit, Isaac Sim, Newton/Warp, torch, driver, GPU
- Platform supported (Linux/Windows 11, RTX for Isaac Sim, driver at least the minimum). Not macOS
- Asset converted to USD and inspected: scale, mass/inertia, colliders, joint limits, fixed base
- Actuator config covers every actuated joint, with effort/velocity limits and armature set
- All hard-coded quaternions are XYZW. Data reads use
.torch -
sim.dt,decimation,render_intervaland sensorupdate_periodare consistent - Physics backend chosen on purpose (
physics=) and listed inlist_envs --show_presets -
zero_agentandrandom_agentrun warning-free at 4 envs, then at fullnum_envs - Contact, camera and ray-caster outputs inspected for env 0 and the last env
- Randomisation in
eventswith conservative ranges. Time-outs markedtime_out=True - Training run reproducible: seed, presets, Hydra overrides and the command line saved with the logs
- Policy exported (
exported/policy.onnx) and checked in a second backend or a held-out randomisation - For demos: task-space actions, annotations checked, Mimic success ratio reviewed, dataset QA passed
Related Skills
robot-reinforcement-learning: algorithms, reward design, randomisation strategy, sim-to-real theoryimitation-learning: BC, diffusion and VLA policies trained on Isaac Lab or Mimic datalerobot: LeRobot dataset format, conversion and training stackrobot-data-quality: validating demonstration and rollout datasets before trainingmujoco: MuJoCo/MJX/MJWarp outside Isaac Lab, and comparison with Newton's MJWarp solverros2,docker-ros2-development: the ROS 2 side of the bridge and containerised deploymentrobotics-testing: CI smoke tests for environments and exported policiesrobot-perception: camera models and perception pipelines for sim-rendered images
Tips for Isaac Sim
- • Start a fresh session after installing so your agent picks up the new skill.
- • It bundles reference files. The agent reads them only when needed, so they cost no context until then.
- • It is part of a pack with 17 skills. Install only the ones you need with --skill to keep triggers precise.
- • If two skills overlap, mention the one you want by name.
Questions about Isaac Sim
What does the Isaac Sim skill do?
NVIDIA Isaac Sim and Isaac Lab for robot learning: install (uv, pip, binary, Docker, cloud), RTX GPU and driver requirements, headless runs and livestreaming, OpenUSD assets, URDF/MJCF import, ArticulationCfg and actuators, manager-based and direct envs, configclass, gymnasium task registration, training and playing with rsl_rl, rl_games, skrl or sb3, parallel envs, tiled cameras, contact sensors, ray casters, IMU…
How do I install Isaac Sim?
Run "npx alyn-cli add isaac-sim" in your project. The Alyn CLI installs it for Claude Code, Cursor, Codex and 70+ other agents. In Claude on the web, upload the skill folder as a ZIP under Customize > Skills.
Does Isaac Sim work in Cursor and Codex?
Yes. Isaac Sim uses the open SKILL.md format, so it works in Claude Code, Claude, Cursor, OpenAI Codex, Gemini CLI, GitHub Copilot and other agents that support agent skills.
What is inside the Isaac Sim skill?
The SKILL.md covers When to Use This Skill, Reference Files, Ecosystem Map and Requirements, Three choices you make separately, Setup, Recommended: uv from a checkout, and the skill folder has 10 files.
What is a good alternative to Isaac Sim?
Omd Lab 02 Design Harness by kwakseongjae is the highest-ranked similar skill from another creator, with an Alyn Score of 57 compared to 53 for Isaac Sim.
Is Isaac Sim free?
Isaac Sim is open source on GitHub under the Apache-2.0 license. It is free to install; check the license before commercial use.
Who made Isaac Sim?
Isaac Sim is published by arpitg1304 in the arpitg1304/robotics-agent-skills repository, together with 16 other skills.

