security: secure Hugging Face credentials across cloud runtimes

[algattik]

Problem

Cloud submission paths pass the raw Hugging Face token in job or workflow definitions:

Runtime	Current transport
Azure ML pi0/pi0_fast training	environment_variables.HF_TOKEN in the submitted job definition
Azure ML pi0/pi0_fast evaluation	environment_variables.HF_TOKEN in the submitted job definition
OSMO GR00T training	hf_token workflow parameter rendered into HF_TOKEN

Users with sufficient job or workflow read access may retrieve these values from persisted metadata. Omitting the token from configuration previews and application logs does not protect the submitted definition.

The LeRobot OSMO training and evaluation workflows already use the OSMO credentials.huggingface injection mechanism and are not part of the raw-parameter migration.

Desired outcome

Required Hugging Face credentials are resolved inside the workload through a platform-native secret mechanism. Raw token values never appear in CLI arguments, job properties, workflow parameters, generated manifests, logs, or configuration previews.

Scope

• Define one shared credential contract for Hugging Face access across cloud training and evaluation.
• For Azure ML, pass only Key Vault location metadata and retrieve the secret at runtime with the job's managed identity.
• Grant the minimum Key Vault data-plane permission to the workload identity.
• For OSMO GR00T, replace the raw workflow parameter with the existing OSMO Hugging Face credential injection mechanism.
• Fail before model or dataset access when the required secret reference, identity, or permission is unavailable.
• Keep client and operator errors sanitized while recording enough context to diagnose identity and secret-resolution failures.

Acceptance criteria

• Azure ML pi0/pi0_fast training does not place the raw token in the job definition or submission command.
• Azure ML pi0/pi0_fast evaluation does not place the raw token in the job definition or submission command.
• OSMO GR00T training does not place the raw token in workflow parameters or the rendered workflow specification.
• Workloads retrieve the token only at runtime through managed identity or OSMO credential injection.
• Configuration previews show secret references and identity configuration, never secret values.
• Automated tests reject raw-token submission and cover missing reference, denied access, and successful retrieval.
• Live Azure ML and OSMO smoke tests confirm gated model access without exposing the token through job or workflow readback.
• Documentation describes setup, least-privilege access, credential rotation, troubleshooting, and rollback.

Related work

• test(training): remove HF token requirement from PI0 e2e after next LeRobot release #1487 removes the Hugging Face token requirement from the synthetic PI0 E2E test after a published LeRobot release includes configurable tokenizer sources. It does not address production credential transport for gated models.
• feat(evaluation): add pi0 replay evaluation and VLA schema v1 #1564 requires the evaluation path that exposed the Azure ML metadata risk.