Skip to content

SYCL: Intel GPU kernels for pi0 and pi0.5 - #42

Open
sandlbn wants to merge 1 commit into
VinRobotics:mainfrom
sandlbn:main
Open

sandlbn wants to merge 1 commit into
VinRobotics:mainfrom
sandlbn:main

Conversation

@sandlbn

@sandlbn sandlbn commented Oct 16, 2026

Copy link
Copy Markdown

New ggml-sycl ops behind an in-tree extension hook (src/sycl/vla_sycl_ext.cpp), built as vla_sycl_ops on oneDNN:

  • BF16 activations: BF16 matmul (BF16 or F32 result), norm, rms_norm, scale.
  • FLASH_ATTN_EXT on oneDNN's fused SDPA, operands read in place (F16/BF16, arbitrary strides, batched); default on SYCL for pi0 and pi0.5.
  • Fused elementwise nodes (layers/fused_ew.h): adaRMS, gated residual, geglu, LayerNorm, RoPE + K/V cache write, GEMM with bias/GELU/residual epilogue.
  • W8A8 linear with per-token INT8 activations (layers/qlinear.h), weights quantized at load; opt-in for pi0.5 (VLA_PI05_W8A8, VLA_PI05_W4).

pi0.5 uses them for an in-place F16 K/V cache, fused projections and fused SigLIP blocks with all camera views in one pass. No CUDA, BitVLA or Evo-1 files change.

Test: test_bf16_sycl_ops.

  • [ x] Builds clean under -Wall -Wextra (first-party code)
  • [ x] ctest passes
  • [ x] Numeric output unchanged (vla_predict_check diff), or the change is
    meant to move it and a LIBERO sweep is below

Archs and backends tested:

New ggml-sycl ops behind an in-tree extension hook (src/sycl/vla_sycl_ext.cpp),
built as vla_sycl_ops on oneDNN:

- BF16 activations: BF16 matmul (BF16 or F32 result), norm, rms_norm, scale.
- FLASH_ATTN_EXT on oneDNN's fused SDPA, operands read in place (F16/BF16,
  arbitrary strides, batched); default on SYCL for pi0 and pi0.5.
- Fused elementwise nodes (layers/fused_ew.h): adaRMS, gated residual, geglu,
  LayerNorm, RoPE + K/V cache write, GEMM with bias/GELU/residual epilogue.
- W8A8 linear with per-token INT8 activations (layers/qlinear.h), weights
  quantized at load; opt-in for pi0.5 (VLA_PI05_W8A8, VLA_PI05_W4).

pi0.5 uses them for an in-place F16 K/V cache, fused projections and fused
SigLIP blocks with all camera views in one pass. No CUDA, BitVLA or Evo-1 files change.

Test: test_bf16_sycl_ops.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant