docs(ai): add garak LLM security scanning on Workbench - #848
Merged
Merged
Conversation
Prebuilt Workbench image with garak 0.17.0 and its offline detector models, datasets and NLTK corpora, so an isolated cluster can scan a published inference service without reaching PyPI or Hugging Face. Covers obtaining the image, importing the WorkspaceKind the console needs to offer it, preparing the scan configuration, running a scan and reading the reports, plus adapting the chat template for non-Qwen models and using an LLM judge detector. Verified against Qwen3.5-0.8B on vLLM: 21 probes in 775s, reports written to the Workspace volume, detectors loading from the image with the container offline. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- drop the verified-versions line from Environment - any registry the cluster can pull from, not specifically the platform Private Registry - remove the Image contents section - condense image retrieval to the address plus the push commands, and keep section anchors only where another section links to them Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The shipped sample posts to /v1/completions because the chat endpoint of the verification service did not respond, but chat is the endpoint most services expose and it lets the server apply the model's own chat template. Present both plugins sections, chat first, and keep the template-adaptation section for the completions fallback only. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The chat endpoint is what services expose for instruct models and what applications actually call, and the server applies the model's own chat template. Scanning through /v1/completions needed a hand-built template per model, and the rest generator drops all but the last turn, which silently degrades the multi-turn probes; drop that path and the template adaptation section with it. Fix the thinking-mode setting: chat_template_kwargs has to be nested under extra_body, otherwise the OpenAI client rejects it as an unexpected keyword argument. Add a section on what to scan: the model endpoint measures the model, while the risk that matters lives in the application in front of it, so cover passing the production system prompt and pointing the rest generator at the application's own API. Results table re-measured through the chat endpoint (21 probes, 701s), plus a note on the report corruption that can break HTML generation on long parallel runs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The target does not have to be the in-cluster predictor Service: a gateway, an ingress or any other endpoint serving /v1/chat/completions works, so describe it as a base URL and mention the API key case. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- reports go to ~/garak/report via an absolute reporting.report_dir; a relative path lands under ~/.local/share/garak, which is awkward to open from the file browser - remove the verification-results section and the report-corruption note - refresh the smoke test output for the chat generator Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a Solution article for scanning a published inference service with garak from Alauda AI Workbench.
docs/en/solutions/AI/LLM_Security_Scanning_with_garak_on_Workbench.mdWhy
Installing garak on site is impractical in an isolated environment: it pulls packages from PyPI and detector models and datasets from Hugging Face. The article uses a prebuilt Workbench image with garak 0.17.0 and all of its offline assets, so the scan runs with no internet access.
Contents
scan.yaml, smoke test, running a scan, reading the reportsVerification
Everything in the article was executed end to end against
qwen3-5-0-8b(Qwen3.5-0.8B on vLLM):--net none)Open item
ProductsVersionis not set in the frontmatter. The verification cluster runsaml-serverv2.8.0-beta.4 with Workbench chart 2.0.0, which I could not map to a released AML version; the versions are stated in the Environment section instead. Happy to add the field if a reviewer can name the right version.