Skip to content

Support VPC public gateway rate throttling, built on NIC/network rate persistence and precedence fixes - #13325

Open
sudo87 wants to merge 44 commits into
apache:mainfrom
shapeblue:networkThrottling
Open

sudo87 wants to merge 44 commits into
apache:mainfrom
shapeblue:networkThrottling

Conversation

@sudo87

@sudo87 sudo87 commented Jun 3, 2026 •

Copy link
Copy Markdown
Contributor

Description

Adds an operator-configurable data transfer rate for a VPC's public/internet-facing gateway, independent of the per-tier rates that network offerings already control.

Precedence:

VPC offering  rate >  vpc.public.network.throttling.rate (default unlimited)

In addition to vpc, there is a change in precedence order for network rate for NIC and VRs being added in this PR.

For NICs:

Default used to select network rate based on below:

Bandwidth from compute offering  > "vm.network.throttling.rate" config

Other NICs use:

Bandwidth from Network offering > "network.throttling.rate" config

Going forward Default NIC precedence will be used for all NICs

For VR's guest interface:

Old precendence:

Bandwidth from Network offering > "network.throttling.rate" config

New one:

Bandwidth from System offering > Network offering > "network.throttling.rate" config

Also persists the effective network rate per NIC and Network and exposes the effective network rate (bandwidth throttling) configured for NICs and guest networks in the API responses and UI.

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • Build/CI
  • Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

How Has This Been Tested?

Verified end-to-end in a lab: API responses, the actual libvirt bandwidth configuration on the router, and measured live throughput across a multi-tier VPC confirming tiers correctly share one capped public-gateway.

How did you try to break this feature and the system with this change?

@sudo87 sudo87 changed the title persist and expose effective network rate for NIC, Network and compute offering Persist and expose effective network rate for NIC, Network and compute offering Jun 3, 2026
@sudo87
sudo87 marked this pull request as draft June 3, 2026 09:56
@codecov

codecov Bot commented Jun 3, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 17.06485% with 243 lines in your changes missing coverage. Please review.
✅ Project coverage is 19.93%. Comparing base (510d0ec) to head (6438d86).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
...in/java/com/cloud/upgrade/NetworkRateBackfill.java 0.00% 151 Missing ⚠️
...src/main/java/com/cloud/api/ApiResponseHelper.java 0.00% 11 Missing ⚠️
...java/com/cloud/upgrade/dao/Upgrade42300to2400.java 0.00% 10 Missing ⚠️
...tack/engine/orchestration/NetworkOrchestrator.java 30.00% 7 Missing ⚠️
...pache/cloudstack/api/response/NetworkResponse.java 0.00% 6 Missing ⚠️
...rg/apache/cloudstack/api/response/NicResponse.java 0.00% 6 Missing ⚠️
...main/java/com/cloud/network/vpc/VpcOfferingVO.java 0.00% 6 Missing ⚠️
...ngine/schema/src/main/java/com/cloud/vm/NicVO.java 0.00% 6 Missing ⚠️
...ain/java/com/cloud/network/vpc/VpcManagerImpl.java 66.66% 5 Missing and 1 partial ⚠️
...m/cloud/api/query/dao/DomainRouterJoinDaoImpl.java 0.00% 4 Missing ⚠️
... and 12 more
Additional details and impacted files
@@             Coverage Diff              @@
##               main   #13325      +/-   ##
============================================
+ Coverage     19.91%   19.93%   +0.02%     
- Complexity    20194    20239      +45     
============================================
  Files          6373     6374       +1     
  Lines        577230   577517     +287     
  Branches      70696    70751      +55     
============================================
+ Hits         114942   115138     +196     
- Misses       449722   449791      +69     
- Partials      12566    12588      +22     
Flag Coverage Δ
uitests 3.70% <ø> (-0.01%) ⬇️
unittests 21.20% <17.06%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@bernardodemarco
bernardodemarco requested a review from hsato03 June 3, 2026 12:23
@github-actions

github-actions Bot commented Jun 8, 2026

Copy link
Copy Markdown

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

@weizhouapache weizhouapache added this to the 4.24.0 milestone Jun 29, 2026
@DaanHoogland DaanHoogland moved this from Backlog to conflict/waiting for author in CloudStack Testing Aug 31, 2026
# Conflicts:
#	server/src/main/java/com/cloud/vm/UserVmManagerImpl.java
#	ui/src/config/section/network.js
- Add network_rate column to nics table (schema-42300to42400.sql)
- Add DB upgrade path: Upgrade42300to42400 registered in DatabaseUpgradeChecker
- Add network_rate field and getter/setter to NicVO
- Set network_rate on NicVO in NetworkOrchestrator.allocateNic() where rate
  is already computed, eliminating secondary per-NIC update calls
- Add getNetworkRate() to Nic interface so ApiResponseHelper.createNicResponse
  can call result.getNetworkRate() without casting or extra DB queries
- Add nic_network_rate to user_vm_view and UserVmJoinVO so listVirtualMachines
  reads rate from the join without extra per-NIC findNicById calls
- Update UserVmJoinDaoImpl to use uvo.getNicNetworkRate() directly
- Expose network_rate in NicResponse as Integer (null = unlimited)
- Refresh NIC rates on VM start via refreshNicNetworkRates in UserVmManagerImpl
@sudo87

sudo87 commented Sep 4, 2026

Copy link
Copy Markdown
Contributor Author

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@sudo87 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19103

@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

@sudo87

sudo87 commented Sep 7, 2026

Copy link
Copy Markdown
Contributor Author

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@sudo87 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19139

Copilot AI review requested due to automatic review settings September 29, 2026 05:24

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

A critical upgrade-path issue and multiple unresolved persistence, update, and UI correctness issues remain.

Review effort: Lite
Findings: 3 High severity · 3 Medium severity · 1 Low severity

Open (7)
Previously missed (3)

In code that hasn't changed since last review

Medium severity Refresh NIC rates after network offering changes

server/​src/​main/​java/​com/​cloud/​network/​NetworkServiceImpl.java:3608

When a network offering changes, this refreshes only the network-level snapshot. Existing NIC rows keep their previous network_rate, while the replug path is limited to running VMware user VMs; other attached VMs/routers can therefore continue to expose the old rate through the new NIC API (and retain it until a later allocate/prepare). Recompute and persist the effective rate for each attached NIC as part of the offering update.

Medium severity Pass explicit zero rate when creating VPC offerings

ui/​src/​views/​offering/​AddVpcOffering.vue:735

A value of 0 is the documented explicit unlimited setting, but this truthiness check omits it from createVPCOffering. On a zone with a nonzero vpc.public.network.throttling.rate, entering 0 therefore applies the zone cap instead of the requested unlimited offering. Test for presence rather than truthiness so zero is sent.

Low severity Document -1 representation for unset offering values

api/​src/​main/​java/​org/​apache/​cloudstack/​api/​response/​VpcOfferingResponse.java:110

The response mapper now normalizes both an unset offering value and explicit 0 to -1, but this description still promises null for an unset value. That contradicts the actual API payload and the UI's -1 handling; document the -1 representation and clarify that an unset value may fall back to the zone/global default.

Comment thread engine/schema/src/main/resources/META-INF/db/schema-42300to2400.sql Outdated
- importNic: persist the computed effective network rate on the NIC
  instead of leaving network_rate NULL for imported NICs.
- restartVpc: only refresh the persisted public network rate snapshot
  after a restart actually succeeds, not before either restart path
  runs, so a failed restart can't leave a stale/premature snapshot.
- VpcOfferingResponse: fix stale doc string; the response normalizes
  unset/unlimited to -1, not null.
Copilot AI review requested due to automatic review settings September 29, 2026 05:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Unresolved moderate issues can leave persisted effective rates and API/UI values inconsistent after offering changes.

Review effort: Lite
Findings: 1 High severity

Open (1)
Resolved since last review (6)
Previously missed (1)

In code that hasn't changed since last review

Medium severity Recompute NIC rates when network offering changes

server/​src/​main/​java/​com/​cloud/​network/​NetworkServiceImpl.java:3608

Updating the network offering changes the persisted network-level rate here, but it never recomputes nics.network_rate for NICs already attached to this network. This leaves API responses and the applied VR bandwidth stale; in particular, a router guest NIC whose system offering has no rate should now fall back to the new network-offering rate, but will continue to expose/use its old persisted value. Update the affected NICs as part of the offering change or make the reconfiguration path persist the recomputed rate.

@sudo87

sudo87 commented Sep 29, 2026

Copy link
Copy Markdown
Contributor Author

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@sudo87 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19344

Comment thread api/src/main/java/org/apache/cloudstack/api/response/NetworkResponse.java Outdated
- Remove publicnetworkrate from updateVPCOffering; it is set only at
  create time
- createVPCOffering accepts -1 or 0 (unlimited, stored as -1) or a
  positive value; values below -1 are rejected
- Make vpc_offerings.public_nw_rate a signed int; NULL means the VPC
  uses the zone setting vpc.public.network.throttling.rate
- Omit publicnetworkrate in the VPC offering response when not set
- Default vpc.public.network.throttling.rate to -1 and reject values
  below -1
- Use since 24.0 and consistent API descriptions
- migrateVPC: copy VPC details after the tiers are migrated and skip
  publicnetworkrate, which left a stale rate on the migrated VPC
- UI: allow -1 on the Add VPC offering form, drop the edit field
- Add unit tests
Copilot AI lite review requested due to automatic review settings October 5, 2026 12:12
@sudo87

sudo87 commented Oct 5, 2026

Copy link
Copy Markdown
Contributor Author

thanks @weizhouapache for the review. I have addressed the comments, please take a look.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Several moderate issues remain around stale effective-rate persistence, inconsistent API values, migration backfill performance, and integer validation.

Review effort: Lite
Findings: 1 High severity

Open (1)
Previously missed (1)

In code that hasn't changed since last review

Medium severity Update NIC rates when the network offering changes

server/​src/​main/​java/​com/​cloud/​network/​NetworkServiceImpl.java:3608

Updating a network's offering refreshes only the network_details value here; existing NIC rows keep the old network_rate. For a VR guest NIC without a system-offering rate (and for other NIC types using the network offering), a subsequent API read still reports the old rate and the persisted NIC value can continue to be used after the offering change. Recompute and persist the effective rate for each non-removed NIC on this network as part of the offering update.

A VPC restart without cleanup does not recreate the VR, so its public
NIC keeps the rate it was prepared with. Refreshing the stored rate
there made the API report a rate that is not enforced. Refresh it only
when the VR is recreated (cleanup or makeredundant).

Also update the vm.network.throttling.rate description, as the rate
applies to every NIC of an instance, not only the default one.
Copilot AI lite review requested due to automatic review settings October 5, 2026 13:41
@sudo87

sudo87 commented Oct 5, 2026

Copy link
Copy Markdown
Contributor Author
  1. Fresh-install schema

Not needed: a fresh install applies all upgrade scripts after the base schema, so these columns are created there too.

  1. NIC rates on network offering change

Not reproducible (KVM): updating the offering re-prepares the router NICs. nics.network_rate and the libvirt bandwidth follow the new offering without a manual restart, for both isolated networks and VPC tiers.

  1. Refresh on every VPC restart

A restart without cleanup doesn't recreate the VR, so refreshing the stored rate there reported a rate that wasn't enforced. It is now refreshed only after a restart with cleanup or makeredundant.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Critical migration and network-detail persistence issues can leave API-reported rates stale or inconsistent.

Review effort: Lite
Findings: 2 High severity

Open (2)
Resolved since last review (1)
Previously missed (1)

In code that hasn't changed since last review

Medium severity Changing network offering leaves NIC rates stale

server/​src/​main/​java/​com/​cloud/​network/​NetworkServiceImpl.java:3608

When a network offering is changed, existing virtual-router guest NICs can get a different effective rate (unless the system offering overrides it), but this only refreshes network_details. The new NIC API fields read nics.network_rate, so those NICs retain the old value and the API/UI no longer matches the rate recomputed by getNetworkRate() at runtime. Recompute and persist the rate for each non-placeholder NIC after changing the offering.

The copy already gets its own networkrate from its offering, and the
migrated network updates it when it moves to the new offering.
Copilot AI lite review requested due to automatic review settings October 5, 2026 14:38

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

Review effort: Lite
Findings: 2 High severity · 2 Medium severity

Open (4)
Resolved since last review (1)

Comment thread server/src/main/java/com/cloud/api/ApiResponseHelper.java Outdated
- NetworkRateBackfill takes the connection of the upgrade, like the
  other upgrade steps
- In the network response, take the network rate from the details
  already loaded for admins instead of querying it again
@sudo87
sudo87 requested review from weizhouapache and a lite review from Copilot October 5, 2026 17:29

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

Review effort: Lite
Findings: 1 High severity

Open (1)
Resolved since last review (4)

Comment on lines +75 to +80
private final ConfigurationDao configurationDao = new ConfigurationDaoImpl();

public NetworkRateBackfill(Connection conn) {
this.conn = conn;
}

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants