Skip to content

Fix External-DHCP APIPA/Link-local address on L2 networks - #14333

Open
nvazquez wants to merge 2 commits into
apache:4.22from
shapeblue:l2-external-dchp-ip-fix
Open

nvazquez wants to merge 2 commits into
apache:4.22from
shapeblue:l2-external-dchp-ip-fix

Conversation

@nvazquez

@nvazquez nvazquez commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Description

This PR an issue observed on L2 Networks with external DHCP server:

On an L2 guest network with an external DHCP server, CloudStack learns a Windows VM's IP address as the self-assigned (APIPA / link-local) address 169.254.149.120. It recorded that as the NIC's IP and never refreshed it. More than 30 minutes later, listVirtualMachines still reports 169.254.149.120, while the guest really has 172.25.16.236 (DHCP). The QEMU guest agent reports 172.25.16.236, and SSH on 172.25.16.236:22 is reachable.

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • Build/CI
  • Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

@nvazquez

nvazquez commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@nvazquez a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@codecov

codecov Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 59.37500% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 18.02%. Comparing base (2974af8) to head (52c4e0d).
⚠️ Report is 5 commits behind head on 4.22.

Files with missing lines Patch % Lines
.../src/main/java/com/cloud/vm/UserVmManagerImpl.java 0.00% 11 Missing ⚠️
...ls/src/main/java/com/cloud/utils/net/NetUtils.java 86.66% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##               4.22   #14333    +/-   ##
==========================================
  Coverage     18.02%   18.02%            
- Complexity    16250    16285    +35     
==========================================
  Files          5936     5936            
  Lines        535823   536425   +602     
  Branches      65612    65708    +96     
==========================================
+ Hits          96582    96716   +134     
- Misses       428242   428683   +441     
- Partials      10999    11026    +27     
Flag Coverage Δ
uitests 4.04% <ø> (-0.01%) ⬇️
unittests 19.10% <59.37%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@weizhouapache weizhouapache left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code lgtm

@blueorangutan

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19442

@nvazquez

nvazquez commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

@blueorangutan test

@blueorangutan

Copy link
Copy Markdown

@nvazquez a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Legacy APIPA values can remain exposed, and malformed IPv6 zone input can throw unexpectedly.

Review effort: Balanced
Findings: 2 Medium severity

Open (2)
What changed in this PR

Fixes stale APIPA/link-local addresses reported for externally DHCP-managed L2 VM NICs.

Changes:

  • Adds IPv4/IPv6 link-local detection utilities.
  • Filters link-local addresses from KVM guest-agent results.
  • Retries NICs containing previously recorded APIPA addresses.
File Description
NetUtils.java Adds link-local address helpers.
NetUtilsTest.java Tests link-local detection.
UserVmManagerImpl.java Handles and retries APIPA NIC addresses.
LibvirtGetVmIpAddressCommandWrapper.java Ignores link-local guest addresses.
LibvirtGetVmIpAddressCommandWrapperTest.java Tests APIPA and DHCP parsing.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread server/src/main/java/com/cloud/vm/UserVmManagerImpl.java Outdated
Comment thread utils/src/main/java/com/cloud/utils/net/NetUtils.java Outdated
@blueorangutan

Copy link
Copy Markdown

[SF] Trillian test result (tid-17089)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 49968 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr14333-t17089-kvm-ol8.zip
Smoke tests completed. 149 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File

nic.setIPv4Address(null);
_nicDao.update(nicId, nic);
}
} else if (NetUtils.isLinkLocalIp4(vmIp)) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

on xenserver this still counts as a failed try, so after about 30 min we stop checking and a lease that comes later is never picked up. should this skip the count like kvm does now?

@nvazquez
nvazquez requested a balanced review from Copilot October 13, 2026 15:13

@Damans227 Damans227 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @nvazquez, fix looks right. LGTM

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

4 open findings
2 resolved since last review

🧠 Review effort: Lite

Comment on lines +811 to +821
} else if (NetUtils.isLinkLocalIp4(vmIp)) {
// a link-local (APIPA) address means the guest has not obtained a DHCP lease yet; do not record it,
// clear any previously recorded link-local address and keep retrying without consuming the retry budget
// (consistent with KVM, which reports this case as a failed answer), so a late DHCP lease is still picked up
logger.debug("Instance [ID: {}, UUID: {}, name: {}] - ignoring link-local IP {}, DHCP lease not obtained yet",
vmId, vmUuid, vmName, vmIp);
decrementCount = false;
if (nic != null && NetUtils.isLinkLocalIp4(nic.getIPv4Address())) {
nic.setIPv4Address(null);
_nicDao.update(nicId, nic);
}
Comment on lines +133 to +146
// Skip link-local addresses (e.g. Windows APIPA 169.254.0.0/16 before a DHCP lease is obtained),
// they are not routable and must not be reported as the instance IP
if (ipFamily.equals("ipv4")) {
ipv4 = ipPart;
if (NetUtils.isLinkLocalIp4(ipPart)) {
logger.debug(String.format("Ignoring link-local ipv4 %s with mac address %s", ipPart, macAddress));
} else {
ipv4 = ipPart;
}
} else if (ipFamily.equals("ipv6")) {
ipv6 = ipPart;
if (NetUtils.isLinkLocalIp6(ipPart)) {
logger.debug(String.format("Ignoring link-local ipv6 %s with mac address %s", ipPart, macAddress));
} else {
ipv6 = ipPart;
}
Comment on lines +61 to +77
// Windows guest that has not obtained a DHCP lease yet and self-assigned an APIPA address
private static String VIRSH_DOMIF_OUTPUT_WINDOWS_APIPA = " Name MAC address Protocol Address\n" + //
"-------------------------------------------------------------------------------\n" + //
" Ethernet 02:01:00:d0:00:87 ipv6 fe80::df84:2984:e396:8ac3%6/64\n" + //
" - - ipv4 169.254.149.120/16\n" + //
" Loopback Pseudo-Interface 1 ipv6 ::1/128\n" + //
" - - ipv4 127.0.0.1/8\n";

// Same Windows guest once the DHCP lease has been obtained
private static String VIRSH_DOMIF_OUTPUT_WINDOWS_DHCP = " Name MAC address Protocol Address\n" + //
"-------------------------------------------------------------------------------\n" + //
" Ethernet 02:01:00:d0:00:87 ipv6 fd00:ac19:0:1910:0:d3fc:4177:2c32/128\n" + //
" - - ipv6 fe80::df84:2984:e396:8ac3%6/64\n" + //
" - - ipv4 172.25.16.236/22\n" + //
" - - ipv4 169.254.149.120/16\n" + //
" Loopback Pseudo-Interface 1 ipv6 ::1/128\n" + //
" - - ipv4 127.0.0.1/8\n";
Comment on lines +2536 to +2542
// a link-local (APIPA) address recorded before the guest obtained its DHCP lease is not a valid instance IP,
// clear it so it is no longer exposed and the NIC is retried as one without an IP
if (NetUtils.isLinkLocalIp4(nic.getIPv4Address())) {
logger.debug("Clearing link-local IP {} recorded for NIC {}", nic.getIPv4Address(), nic);
nic.setIPv4Address(null);
_nicDao.update(nic.getId(), nic);
}
@nvazquez

Copy link
Copy Markdown
Contributor Author

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@nvazquez a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19542

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants