llama-cpp: support every Linux prebuilt backend (CUDA, ROCm, SYCL, OpenVINO)

[cosmincc]

Requested by Cosmin

✍️ Description

var_backend in the llama.cpp script now covers every Linux prebuilt llama.cpp publishes. Each backend gets the runtime that upstream's .github/workflows/release.yml builds against:

• auto (the default): Vulkan when a GPU is passed through, otherwise CPU.
• cpu and vulkan: as before.
• cuda: picks the CUDA 12 or 13 build from the host driver (driver below 580 gets 12, which is x64 only). The matching cudart-llama-* tarball is also deployed into /opt/llama-cpp.
• rocm: ubuntu-rocm-<ver>-x64, plus the same ROCm release of AMD's TheRock wheels (rocm[libraries]==<ver>.* from stable.repo.amd.com/rocm/whl-next/) in a uv venv, which is added to LD_LIBRARY_PATH. On update, the runtime follows when upstream moves the build to a newer ROCm.
• sycl-fp16 / sycl-fp32: intel-oneapi-runtime-libs from Intel's oneAPI apt repo (via setup_deb822_repo), with its lib dirs registered in ld.so.conf.d. Also installs the Level Zero loader (libze1_*u24.04_amd64.deb from oneapi-src/level-zero, as in release.yml) and libze-intel-gpu1 from intel/compute-runtime. The persistent SYCL kernel cache is written commented out, because it stopped models from loading on an Intel GPU in testing.
• openvino: ubuntu-openvino-*-x64 with libtbb12, ocl-icd-libopencl1 and Level Zero. GGML_OPENVINO_DEVICE=GPU is set when /dev/dri/renderD* exists, otherwise CPU.

Choosing the build: when var_backend isn't set, ct/llama-cpp.sh asks on the Proxmox host with a whiptail menu titled "LLAMA.CPP BUILD", with auto preselected and no timeout. The installer itself runs under lxc-attach with no terminal, so a prompt there could never be answered. rocm, sycl and openvino are offered only on amd64 hosts. Cancel exits; with no terminal, the menu is skipped and auto is used. The choice is exported into the container and recorded in /opt/llama-cpp_data/.backend, and updates keep it.

Release channels: llama.cpp now flags every bNNNN build (the only releases with binaries) as a prerelease, and its stable vX.Y.Z releases ship only nightly-tag.txt, which names the build they were cut from.

• By default, the install reads that file from the latest stable release and deploys that exact build.
• With GH_INCLUDE_PRERELEASE=1, it follows the newest bNNNN build using the b tag prefix.
• The channel is recorded in /opt/llama-cpp_data/.prerelease. Updates follow it unless GH_INCLUDE_PRERELEASE is set for that run. Installs from before this change count as nightly, so they aren't downgraded.

Other changes:

• The default model is now ggml-org/gemma-4-E2B-it-GGUF:Q4_0, because the old default didn't load on OpenVINO. It can still be changed with var_model_repo.
• LLAMA_ARG_MODELS_DIR=/opt/llama-cpp_data/models is written to the env file. Once LLAMA_ARG_HF_REPO is commented out, llama-server runs in router mode and you can switch between the models in that folder.
• var_backend, var_model_repo, var_ctx_size, var_offload_layers, var_api_key and var_port are now exported from the CT script. Before this they were declared in app_vars but never reached the container.
• Removed the outdated "no Linux ROCm/CUDA build" comments and the rocm→vulkan remap. An old install that recorded rocm but has no ROCm runtime keeps updating as Vulkan.
• JSON: new var_backend options and the new default model. The notes were re-fitted to upstream's 3-note limit after the recent "jsons" cleanup.

🔗 Related PR / Issue

Link: #2124

✅ Prerequisites (X in brackets)

• Self-review completed – Code follows project standards.
• Tested thoroughly – Changes work as expected.
• No breaking changes – Existing functionality remains intact.
• No security risks – No hardcoded secrets, unnecessary privilege escalations, or permission issues.

"Tested thoroughly" is left unticked because not every backend was tested on hardware; see "Tested on" below.

---

🏗️ arm64 Support (X in brackets)

• arm64 supported - Tested and supported on arm64.
• arm64 not tested - Assumed to work on arm64, but testing has not been done.
• arm64 not supported - Confirmed upstream dependencies or binaries do not support arm64.

var_arm64 is left as it was. cpu, vulkan and cuda (13 only) have arm64 assets. rocm, sycl and openvino are x64 only: the host menu hides them on arm64, and the install stops with a clear error if one is set anyway.

---

🖥️ Tested on (X in brackets)

• Proxmox VE – Created on a Proxmox VE host and the app works.
• Incus – Created on an Incus host and the app works.

Proxmox VE, Debian 13 LXC: fresh installs of auto (Vulkan), sycl-fp16 and openvino; each installed with the right dependencies. cpu, cuda, rocm and sycl-fp32 were not tested on hardware.

---

🛠️ Type of Change (X in brackets)

• 🐞 Bug fix – Resolves an issue without breaking functionality.
• ✨ New feature – Adds new, non-breaking functionality.
• 💥 Breaking change – Alters existing functionality in a way that may require updates.
• 🆕 New script – A fully functional and tested script or script set.
• 🌍 Website update – Changes to website-related JSON files or metadata.
• 🔧 Refactoring / Code Cleanup – Improves readability or maintainability without changing functionality.
• 📝 Documentation update – Changes to README, AppName.md, CONTRIBUTING.md, or other docs.

---

🔍 Code & Security Review (X in brackets)

• Follows CODE-AUDIT.md & CONTRIBUTING.md guidelines
• Uses correct script structure (AppName.sh, AppName-install.sh, AppName.json)
• No hardcoded credentials
• No Docker / Docker Compose – The application is installed bare-metal; Docker is not used.
• No git pull – Updates use fetch_and_deploy_gh_release, fetch_and_deploy_codeberg_release, fetch_and_deploy_gl_release, or fetch_and_deploy_from_url instead of git pull.

---

🤖 AI Assistance (X in brackets)

• No AI used – Scripts were written without AI assistance.
• AI was used – I confirm the scripts were built using AGENTS.md and .github/agents/pve-script-creator.agent.md as guidance, and the output has been reviewed and corrected to match those guidelines.

Claude Code (Claude Opus, low reasoning effort) wrote the changes. It checked helper signatures against community-scripts/core and asset names against the llama.cpp releases. I tested on Proxmox VE and reported issues, which were then fixed: the in-container prompt, the Level Zero asset match, the SYCL cache, prerelease handling and the default model. shellcheck shows only the source warnings every script in the repo has.

---

📋 Additional Information (optional)

• The Level Zero asset is matched as libze1_*u24.04_amd64.deb, because GitHub writes the + in the download URL as %2B. The previous libze1_*+u24.04… never matched, so the engine fell back to libze-dev.
• Asset names checked against b11669 / b11685: ubuntu-cuda-12.8-x64, ubuntu-cuda-13.4-{x64,arm64} (+ cudart-), ubuntu-rocm-10.0-x64, ubuntu-sycl-{fp16,fp32}-x64, ubuntu-openvino-*-x64, ubuntu-vulkan-{x64,arm64}, ubuntu-{x64,arm64}.

---

📦 Application Requirements (for new scripts)

⚠️ Do not remove this section.
It is used by automated PR validation checks.
If this PR is not a new script submission, leave the checkboxes unchecked.

• The application is at least 6 months old
• The application is actively maintained
• The application has 600+ GitHub stars
• Official release tarballs are published
• I understand that not all scripts will be accepted due to various reasons and criteria by the community-scripts ORG

🌐 Source

• https://github.com/ggml-org/llama.cpp (.github/workflows/release.yml for the per-backend build dependencies)
• https://github.com/oneapi-src/level-zero
• https://github.com/intel/compute-runtime
• https://stable.repo.amd.com/rocm/whl-next/

🤖 Generated with Claude Code

[github-actions]

Try this script

ct/llama-cpp.sh, run in the Proxmox VE shell or on an Incus host:

COMMUNITY_SCRIPTS_URL=https://raw.githubusercontent.com/cosmincc/PVE-DevScripts/feat/llama-cpp-backends \
bash -c "$(curl -fsSL https://raw.githubusercontent.com/cosmincc/PVE-DevScripts/feat/llama-cpp-backends/ct/llama-cpp.sh)"

COMMUNITY_SCRIPTS_URL is not optional for ct/. Fetching the ct/ script from a
branch does not tell the engine where that branch is — with bash -c "$(curl …)"
there is no file on disk for the scripts root to be derived from, so it would fall
back to upstream main and look for the install script there.

Against a core branch as well

Add COMMUNITY_SCRIPTS_CORE_URL=https://raw.githubusercontent.com/OWNER/core/BRANCH
to test an engine change at the same time. The two resolve independently.

Useful flags while testing

dev_mode=net logs every fetch with status and duration, so you can confirm the
branch is really being used. dev_mode=keep stops a failed build from deleting
the container along with the evidence.

[asylumexp]

Not yet tested on a Proxmox host.

Go test it.

[cosmincc]

I'd love to test it by I haven't been able since the script is disabled:

This script is currently disabled in community-scripts.

I've been looking on how to circumvent that locally, but haven't yet found a way.

[MickLesk]

var_ignore_disable='yes' before bash call

[cosmincc]

Successfully tested sycl-fp16 and openvino backends for llama.cpp. The bXXXX releases are prereleases so initially had to use GH_INCLUDE_PRERELEASE=1. I fixed the stable release fetching by reading the nightly-tag.txt and using that.

With OpenVINO the default model was not loading (likely due to being unquantized).