Repository navigation
feature/CSTACKEX-237: Pilot changes for scalebenmark framework - #85
Open
piyush5netapp wants to merge 5 commits into
Open
piyush5netapp wants to merge 5 commits into
piyush5netapp wants to merge 5 commits into
Conversation
sandeeplocharla
left a comment
sandeeplocharla
left a comment
Collaborator
There was a problem hiding this comment.
Overall, the pattern looks good but try to optimize the loggers and add more guardrails to the agent.
There was a problem hiding this comment.
Pull request overview
Adds a downstream-only benchmarking toolkit under private-cicd/benchmark/ontap to measure CloudStack ONTAP-plugin operations (storage pools and VM instances), log raw/summary CSVs, and render Confluence-ready markdown reports.
Changes:
- Introduces a minimal CloudStack session-key API client and shared helper modules for storage-pool and VM-instance benchmarks.
- Adds sequential/concurrency benchmark runners for
createStoragePool/deleteStoragePoolanddeployVirtualMachine/destroyVirtualMachine, including cleanup helpers. - Adds documentation, example configuration, requirements, and a report renderer to generate Confluence-aligned tables.
Reviewed changes
Copilot reviewed 14 out of 15 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| private-cicd/benchmark/ontap/cloudstack_client.py | Minimal CloudStack HTTP client with session-key auth and async job polling. |
| private-cicd/benchmark/ontap/storage_pool_common.py | Shared storage-pool benchmark helpers (params, logging, cleanup, config). |
| private-cicd/benchmark/ontap/vm_instance_common.py | Shared VM-instance benchmark helpers (deploy/destroy, logging, cleanup, config). |
| private-cicd/benchmark/ontap/benchmark_storage_pool_sequential.py | Sequential scale benchmark driver for storage pools. |
| private-cicd/benchmark/ontap/benchmark_storage_pool_concurrency.py | Concurrency benchmark driver for storage pools. |
| private-cicd/benchmark/ontap/benchmark_vm_instance_sequential.py | Sequential scale benchmark driver for VM instances. |
| private-cicd/benchmark/ontap/benchmark_vm_instance_concurrency.py | Concurrency benchmark driver for VM instances. |
| private-cicd/benchmark/ontap/benchmark_vm_instance_combined.py | Convenience wrapper to run VM sequential + concurrency in one run. |
| private-cicd/benchmark/ontap/render_report.py | Converts run CSVs into Confluence-ready markdown tables + results-log rows. |
| private-cicd/benchmark/ontap/config.example.yaml | Example config template for lab environment and benchmark parameters. |
| private-cicd/benchmark/ontap/requirements.txt | Python dependencies for the benchmark scripts. |
| private-cicd/benchmark/ontap/README.md | Design/usage documentation for storage-pool + VM-instance benchmarks. |
| private-cicd/benchmark/ontap/USAGE.md | Quick “what to run” cheat sheet for scripts and flags. |
| private-cicd/benchmark/ontap/.gitignore | Ignores local config, venv, caches, and generated results artifacts. |
| private-cicd/benchmark/ontap/results/.gitkeep | Keeps the gitignored results directory present in the tree. |
Suppressed comments (1)
private-cicd/benchmark/ontap/README.md:148
- This example uses a hyphenated run id (RUN-20260722-101500), but elsewhere the tooling generates underscore-separated run ids (RUN_...). Keeping examples consistent helps avoid copying an ONTAP-invalid run id into the storage-pool scripts.
python3 render_report.py --run-id RUN-20260722-101500 \
--cloudstack-build 4.23.0.0-SNAPSHOT --ontap-version 9.15.1
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
🔴 Test Coverage Grade:
|
| Metric | Value |
|---|---|
| Line coverage | 24.74% |
| Branch coverage | 18.87% |
Grade Scale
| Grade | Line Coverage | Meaning |
|---|---|---|
| 🟢 A | ≥ 80% | Excellent - this code sleeps well at night 😴 |
| 🟡 B | 60-79% | Good - almost there, don't stop now 😉 |
| 🟠 C | 40-59% | Acceptable - your code is wearing a seatbelt, but no airbags 😬 |
| 🔴 D | 20-39% | Marginal - boldly shipping where no test has gone before 🖖 |
| ⛔ F | < 20% | Failing - tests? what tests? 🔥 |
Branch coverage is shown as a secondary signal. Grade is determined by line coverage.
View full Actions run
sp16743
force-pushed
the
feature/CSTACKEX-237
branch
from
October 2, 2026 06:07
8395e72 to
ad32ad4
Compare
🔴 Test Coverage Grade:
|
| Metric | Value |
|---|---|
| Line coverage | 24.73% |
| Branch coverage | 18.86% |
Grade Scale
| Grade | Line Coverage | Meaning |
|---|---|---|
| 🟢 A | ≥ 80% | Excellent - this code sleeps well at night 😴 |
| 🟡 B | 60-79% | Good - almost there, don't stop now 😉 |
| 🟠 C | 40-59% | Acceptable - your code is wearing a seatbelt, but no airbags 😬 |
| 🔴 D | 20-39% | Marginal - boldly shipping where no test has gone before 🖖 |
| ⛔ F | < 20% | Failing - tests? what tests? 🔥 |
Branch coverage is shown as a secondary signal. Grade is determined by line coverage.
View full Actions run
Author
|
@sp16743 CLGTM. Approved from my side. |
🔴 Test Coverage Grade:
|
| Metric | Value |
|---|---|
| Line coverage | 24.72% |
| Branch coverage | 18.85% |
Grade Scale
| Grade | Line Coverage | Meaning |
|---|---|---|
| 🟢 A | ≥ 80% | Excellent - this code sleeps well at night 😴 |
| 🟡 B | 60-79% | Good - almost there, don't stop now 😉 |
| 🟠 C | 40-59% | Acceptable - your code is wearing a seatbelt, but no airbags 😬 |
| 🔴 D | 20-39% | Marginal - boldly shipping where no test has gone before 🖖 |
| ⛔ F | < 20% | Failing - tests? what tests? 🔥 |
Branch coverage is shown as a secondary signal. Grade is determined by line coverage.
View full Actions run
sandeeplocharla
left a comment
sandeeplocharla
left a comment
Collaborator
There was a problem hiding this comment.
I'm good with rest of the code
🔴 Test Coverage Grade:
|
| Metric | Value |
|---|---|
| Line coverage | 24.84% |
| Branch coverage | 19.00% |
Grade Scale
| Grade | Line Coverage | Meaning |
|---|---|---|
| 🟢 A | ≥ 80% | Excellent - this code sleeps well at night 😴 |
| 🟡 B | 60-79% | Good - almost there, don't stop now 😉 |
| 🟠 C | 40-59% | Acceptable - your code is wearing a seatbelt, but no airbags 😬 |
| 🔴 D | 20-39% | Marginal - boldly shipping where no test has gone before 🖖 |
| ⛔ F | < 20% | Failing - tests? what tests? 🔥 |
Branch coverage is shown as a secondary signal. Grade is determined by line coverage.
View full Actions run
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR...
Types of changes
Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale
Bug Severity
Screenshots (if appropriate):
How Has This Been Tested?
How did you try to break this feature and the system with this change?