Skip to content

Latest commit

 

History

History
92 lines (66 loc) · 4.89 KB

File metadata and controls

92 lines (66 loc) · 4.89 KB

T2 — Pilot: Cloud Training, Registry, and Shared Catalogs (Recommended)

The recommended production path. This is the tier where cloud training genuinely becomes the default rather than an option: one site, several robots, real training scale, and a team collaborating. You add an AzureML workspace, a model registry, and shared MLflow on top of the T1 storage account, and still run no Kubernetes, no Arc, and no fleet control plane.

Note

T2 is the recommended starting point for teams who have outgrown a single GPU box. The full training lifecycle is still achievable here with manual deployment. Robots are hand-updated with docker pull. For canonical tier definitions, see the tier model and the architecture tier detail.

🧱 Minimum Infrastructure

Concern What you need
Hardware One site, several robots, a collaborating team.
Edge infra None beyond Docker. No Kubernetes, no Arc, no Flux, no fleet plane.
Cloud infra AzureML workspace + Blob storage + model registry + MLflow. ACSA optional.
Tooling Azure CLI (az login), deployed infrastructure (Terraform), optionally OSMO.
Tracking Managed MLflow on AzureML (hosted), with model versioning load-bearing.

The delta from T1 is that cloud GPU training, a model registry, and a hosted catalog become the default, not an occasional reach.

🚀 Steps

Step 1: Deploy the cloud infrastructure

Provision the AzureML workspace, storage, and registry. Follow the Getting Started hub and the Quickstart for the clone-to-first-job path. This recipe assumes that infrastructure is deployed rather than duplicating its steps; for the infrastructure reference see Infrastructure.

Step 2: Prepare the dataset in cloud storage

Land and validate datasets in Blob as at T1 — Lab, following Preparing Datasets for Training and the Blob storage structure.

Step 3: Train on cloud GPU (default)

Submit a LeRobot behavioral-cloning job to AzureML or OSMO: multi-GPU, queued jobs, multiple people, VLA scale. Use the existing recipes rather than re-deriving the commands here:

Reference docs: AzureML training, OSMO training, LeRobot training.

Step 4: Track and register

Managed MLflow on AzureML is the default tracking backend, and the model registry becomes load-bearing. Trained checkpoints are registered and versioned automatically at job completion. See Experiment tracking and MLflow integration.

Step 5: Curate with the hosted dataviewer

The dataviewer is deployed as a shared web app rather than localhost, so the whole team browses and annotates the same catalogs.

Step 6: Validate

Validate registered models directly from the registry. The local entry point (run-local-lerobot-eval.py) accepts --model-name / --model-version to pull a registered model, and Evaluation covers the cloud and batch evaluation paths.

Step 7: Run on robot (manual docker pull)

Deployment is still manual: hand-update a handful of reachable robots with docker pull. This stays tractable at one site with several robots. Declarative GitOps deployment is the T3 — Production concern.

🎓 Graduate When

  • The number of robots or the update cadence makes hand-updating each robot error-prone, and version skew across robots becomes a real problem, while all robots are still at one reachable site: T3 — Production.

🔗 Related Documentation