Component
Training Workflows
Problem Statement
RL training and SIL evaluation run on the digest-pinned nvcr.io/nvidia/isaac-lab:2.3.2 image. That image runs Python 3.11, so training/rl pins Python 3.11 and, for Isaac Sim ABI compatibility, NumPy 1.26.4. As a result:
Proposed Solution
Move the shared Isaac Lab runtime to the digest-pinned nvcr.io/nvidia/isaac-lab:3.0.0-beta2-post1 image:
- Point
DEFAULT_ISAAC_LAB_IMAGE, the Azure ML environment pins, and the direct OSMO workflow fallbacks at the new image.
- Move
training/rl to Python 3.12 and NumPy 2.5.1, regenerate training/rl/uv.lock, and update the Dependabot NumPy guard and the setup-dev Isaac Lab commit pin.
- Keep the image-provided Torch and CUDA packages when installing locked RL dependencies. Install into a writable overlay when the interpreter's site-packages are read-only, and report runtime provenance from that overlay.
Alternatives Considered
- Stay on Isaac Lab 2.3.2 until a stable 3.0 release ships. This keeps
training/rl on Python 3.11 and NumPy 1.x and leaves its NumPy updates blocked.
- Offer 3.0 only through a
DEFAULT_ISAAC_LAB_IMAGE override. The RL lock resolves for a single Python version and NumPy ABI, so the image default and training/rl/uv.lock must move together.
Additional Context
| Consideration |
Impact |
| Breaking change |
Isaac Lab-based RL training and SIL evaluation jobs default to Isaac Lab 3.0 with Python 3.12, and training/rl moves to NumPy 2.5.1. Custom tasks or extensions built for Isaac Lab 2.3.2 may need updates. |
| Validation |
The target is a beta tag and hosted CI cannot run Isaac Sim, so the change needs an RL training smoke run on a GPU node. |
| Azure services |
None added |
| Network modes |
Unaffected |
| Personas |
ML engineers and robotics developers running RL training or SIL evaluation |
Related: #114 tracks a container upgrade that removes the simulation_shutdown.py workaround. This issue keeps that workaround in place.
Component
Training Workflows
Problem Statement
RL training and SIL evaluation run on the digest-pinned
nvcr.io/nvidia/isaac-lab:2.3.2image. That image runs Python 3.11, sotraining/rlpins Python 3.11 and, for Isaac Sim ABI compatibility, NumPy 1.26.4. As a result:training/rlstays below the repository's Python 3.12 minimum from build: standardize on Python 3.12 as repository minimum #540.training/rl(build: dependabot: guard /training/rl numpy + packaging against Isaac Sim ABI-breaking bumps #1029).Proposed Solution
Move the shared Isaac Lab runtime to the digest-pinned
nvcr.io/nvidia/isaac-lab:3.0.0-beta2-post1image:DEFAULT_ISAAC_LAB_IMAGE, the Azure ML environment pins, and the direct OSMO workflow fallbacks at the new image.training/rlto Python 3.12 and NumPy 2.5.1, regeneratetraining/rl/uv.lock, and update the Dependabot NumPy guard and thesetup-devIsaac Lab commit pin.Alternatives Considered
training/rlon Python 3.11 and NumPy 1.x and leaves its NumPy updates blocked.DEFAULT_ISAAC_LAB_IMAGEoverride. The RL lock resolves for a single Python version and NumPy ABI, so the image default andtraining/rl/uv.lockmust move together.Additional Context
training/rlmoves to NumPy 2.5.1. Custom tasks or extensions built for Isaac Lab 2.3.2 may need updates.Related: #114 tracks a container upgrade that removes the
simulation_shutdown.pyworkaround. This issue keeps that workaround in place.