Repository navigation
Conversation
troglobit
force-pushed
the
laguna-bug
branch
3 times, most recently
from
October 12, 2026 06:20
ddcf690 to
3ff4ae9
Compare
troglobit
marked this pull request as draft
October 12, 2026 13:30
troglobit
force-pushed
the
laguna-bug
branch
3 times, most recently
from
October 13, 2026 08:43
f3593b5 to
73c0229
Compare
Refreshing the patch series for the current kernel version removed the patches it had just regenerated, since the script always ran the upgrade steps: kernel-refresh.sh -k ~/src/linux -t v6.18.55 -o 6.18.55 With -o equal to the new version, the rm of the old patch directory hit the new one. Without -o it was worse, patches/linux/ itself went. Skip the directory removal, defconfig bump and tarball checksum when no old version is given or it matches the new one. A plain refresh also no longer downloads the kernel tarball. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
RAUC cannot read the boot state on a Tactical 1000 laid out with provision-tactical: rauc: Failed to get boot state of 'rootfs.0': uboot backend: fw_printenv failed with exit code: 1 # fw_printenv Cannot open /mnt/aux/uboot.env: No such file or directory The script creates the aux filesystem but never writes uboot.env, which the stock image gets from image-itb-aux and a regular install from prod/provision. Write it the same way, with the default BOOT_ORDER and one boot attempt per slot. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The minimal builds drop bash, and BusyBox ash does not read /etc/bash.bashrc even when started as /bin/bash, so the log and follow helpers were gone from the shell there. Move them to /etc/profile.d/log.sh as plain sh functions, read by both ash and bash login shells, including the CLI shell command. The bash completions for them stay in bash.bashrc. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The management port on Tactical 1000 locks up now and then, and the only way to see what the switch is doing is to read its registers. Add Microchip's symreg tool, which reads and writes LAN969x registers by symbolic name and dumps the MAC, VLAN, VCAP and stream tables, via the symreg debugfs driver added with patch 0080. The package enables the driver in the kernel, and each Laguna board selects the package and carries the device tree node in its Infix overlay. Builds without a Laguna board never see it. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The shared Laguna directory had no README. Describe what it holds, what a new Laguna board needs for the symreg tool, and how to use the tool to chase a port fault, with the management port as the example. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
PTP ports on the EV23X71A copper ports drop to the faulty state a few seconds after reaching master or slave: ptp4l: timed out while polling for tx timestamp ptp4l: port 1 (e20): send sync failed ptp4l: port 1 (e20): MASTER to FAULTY on FAULT_DETECTED (FT_UNSPECIFIED) The LAN8814 PHYs on the board do the timestamping, and the kernel picks them over the switch core as timestamp provider. The PHY returns the transmit timestamp from its interrupt handler over MDIO, a bus shared by 25 PHYs and polled by phylib, so it sometimes takes longer than the 10 ms ptp4l waits by default. Set tx_timestamp_timeout to 100 ms in the generated ptp4l configuration whenever hardware timestamping is used. On the EV23X71A, 20 ms already clears every fault, and a MAC timestamper answers within microseconds, so the extra headroom costs nothing. This covers any port that ends up timestamping in a PHY. The Laguna ports move to the switch clock in the next commit, which the Tactical 1000 needs since its PHY timestamps never arrive at all.
Every IEEE 802.1AS test on the Tactical 1000 rig fails: the ports stay in the listening state and never exchange Announce, while the same units synchronize fine with the IEEE 1588 profile. ptp4l reports a mean link delay of 1.5 us on these ports. Port statistics show the peer delay exchange running in both directions, but the generated configuration sets neighborPropDelayThresh to the 800 ns of the automotive profile, so neither port becomes asCapable and neither sends Announce. The switch core time stamps behind the PHY, so the delay it measures includes the PHY latency, 1.5 us on a LAN8814 at 1 Gbit/s and more at 100 Mbit/s. Default the threshold to 10 us with hardware time stamping, keep ptp4l's own 20 ms default with software time stamping, and honor mean-link-delay-thresh from the gPTP port data set when set. With 10 us on both ends of the link, the receiver on the rig reaches the time-receiver state within seconds, offset within a few ns. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
provision-tactical lays out the aux partition and seeds uboot.env on it, the natural place to also give the unit its identity. Ask for a base MAC address and a serial number and write them as an ONIE TLV to vpd.bin there, which the system reads as the product VPD. The README documents the file and how to write it on a provisioned unit. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Sync patches with kkit-linux-6.18.y. On LAN969x, taking any standalone port down, or adding it to a bridge, removed the multicast MAC table entries it shared with every other standalone port. The management port then dropped IPv6 all-nodes and mDNS traffic until cycled, which is how it disappeared from ixll and the test rig. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The IP address of a bridge is unreachable from the bridge ports on the LAN969x boards, while forwarding between the ports works. Every bridge test on the Tactical 1000 rig fails on this. The sparx5 driver stands in hardware VLAN 1 for a VLAN-unaware bridge, and confd creates bridges with vlan_default_pvid 0, which adds VLAN 1 at creation and removes it right after. That forgets the VLAN 1 broadcast entry, so ARP from the ports never reaches the CPU, and the bridge MAC entry, so unicast to the bridge is flooded to the ports. Taking a bridged port down also forgets its MAC on the bridge VLAN, which cuts the bridge off whenever its first port, whose MAC the bridge uses, is disabled. Three kernel patches, 0104 to 0106, synced from kkit-linux-6.18.y. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The Link Aggregation test fails on the Tactical 1000 rig at the failure mode check, and every test that uses the aggregate ports after it fails too, until the units are rebooted: tunnels, NTP, PTP, static routing and DHCP between the two units all time out. The bond gives both member ports the address of the first one, and the sparx5 driver keeps one CPU entry per address. Taking one member down forgets the entry the other member still needs, and deleting the aggregate restores the members' own addresses in an order that forgets the first member's entry for good. That port then drops all unicast sent to it until it is disabled and enabled again, which the later tests never do. One kernel patch, 0107, synced from kkit-linux-6.18.y. Same fix as the standalone multicast entries got in 0103: the port addresses share a reference counted list, so an entry is forgotten with its last user. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
PTP never works on the copper ports of the Tactical 1000. A port
drops to the faulty state on its first Sync:
ptp4l: timed out while polling for tx timestamp
ptp4l: port 1 (e5): MASTER to FAULTY on FAULT_DETECTED (FT_UNSPECIFIED)
The PHYs read 0x8804 from the LAN8814 SKU register: they are LAN8804s,
the variant without the 1588 unit, with the same chip ID as a LAN8814.
The 6.18 driver takes them for full LAN8814s, creates a PTP clock per
quad and tells the kernel the PHY can time stamp, so ptp4l waits for a
transmit timestamp the silicon never produces.
Backport mainline 61b7ade9ba8c ("net: phy: micrel: Add support for non
PTP SKUs for lan8814"), patch 0108. It checks the SKU register and
creates no PTP device on these parts, and the kernel then time stamps
them in the switch core on its own. Boards with real LAN8814s, like
the EV23X71A, keep stamping at the PHY.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The copper PHYs on the Tactical 1000 and the EV23X71A run with the power-on defaults for registers that Microchip's silicon errata, their PHY API, and their BSP kernel all change on every port: the power management modes for center-tapped magnetics (module 4), the EEE wake timer (module 9), and the PLL trim. Mainline applies the first two to the LAN8842 only, and the PLL trim to neither. The Tactical units read silicon revision C1, which all three apply to. Patches 0109 and 0110 apply them from config_init, reusing the LAN8842 tables. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Copper ports on the Tactical 1000 at times come up with one direction dead: the switch counts the frames it sends at 1000 Mbps, the link partner receives none. A QSGMII soft reset of the PHY package followed by a new autonegotiation on the port brings it back. Microchip's PHY API runs that sequence on every LAN8814 port, after the MAC has configured its end of the QSGMII. The kernel driver resets the QSGMII before that, when the PHY is attached. Patch 0111 resets the package QSGMII once after the MAC is set up, and renegotiates each port once after its first 1000 Mbps link. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Two more LAN8814 errata handled the way Microchip's PHY API does: - Module 24: spurious EEE low power idle after a cable unplug. The PHYs advertise EEE out of reset and two of them negotiate it on every unit-to-unit link, although the switch has no LPI support. Patch 0112 disables EEE. - Module 20: autonegotiation can get stuck with the partner present. Patch 0113 restarts it once after three seconds of signal without a link. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Three more LAN8814 errata handled the way Microchip's PHY API does: - Module 13: 10BASE-T transmitter settings, patch 0114. - Module 11: late collisions in half duplex. Rate adaptation FIFO thresholds on 10/100 half-duplex links, FIFO reset on link down, patch 0115. - Module 22: a forced 100BASE-TX link can fail to train. Soft reset of the DSP every three seconds while a partner is detected without link, patch 0116. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The sparx5 driver numbers its ports from the switch base address, base + port + 1, so a board cannot hand out addresses in front panel order or give each port its own nvmem cell, as the BPI-R3 does from its VPD. On the Tactical 1000 the copper ports run backwards within each quad. Patch 0117 uses the address in a port's device tree node when there is one, and marks it permanent. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The Tactical 1000 and the EV23X71A have no MAC address in OTP, so every boot gets a random base MAC, random port addresses, and new link-local addresses. Same scheme as the BPI-R3: U-Boot copies /vpd.bin from the aux partition to reserved memory, the device tree describes it as an nvmem-rmem node with an ONIE layout, and the switch and each port reference the base MAC with an offset. The ports get front panel order, e1 at base + 1 up to e28, and the management port base + 29. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
A bridge with no VLANs configured on a LAN969x board forwards nothing between its ports. confd creates bridges with vlan_default_pvid 0, so the ports are never added to the hardware VLAN the switch puts their untagged traffic in, and every frame is dropped. Patch 0118 makes every port in a VLAN-unaware bridge a member of that VLAN. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
A VLAN interface on top of a port in a VLAN filtering bridge, e.g. e1.10, never receives anything on a LAN969x board. The switch has no MAC entry sending the port address or broadcast in that VLAN to the CPU, since the driver never learns about the VLAN interface. Patch 0119 adds them when the VLAN interface is created, without adding the port to the VLAN, so its traffic stays local and is not bridged. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
The IGMP VLAN test cannot reach the other unit's VLAN interface on the bridge over the trunk between them. A frame sent by the CPU out a port with no untagged VLAN gets an extra outer tag with VID 0: 802.1Q vlan 0, 802.1Q vlan 55, ARP who-has 10.0.1.2 Patch 0120 has the switch tag all frames except VID 0 on such ports. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Bridging a VLAN interface on top of a LAN969x port fills the console with "Controlling non-bridged port 7?" and applies the bridge settings of the VLAN interface to the port below it, which is in no bridge. Patch 0121 only takes an attribute for the port itself, or for the bridge the port is in. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Bridge STP Basic and Link Aggregation Basic fail on every Tactical run. The LAN969x switch forwards frames to 01-80-C2-00-00-0X like any other multicast, so a bridged port floods BPDUs and LACPDUs to the other ports, and each bridge sees BPDUs from bridges that are not its neighbours. Reproduced on tact3: BPDUs from tact2 crossed tact3's bridge to tact4 with STP enabled on tact3. Patch 0122 redirects that address block to the CPU on every port, still discarding pause frames. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
Link Aggregation Basic fails on every Tactical run: a bond in a bridge does not forward. The LAN969x driver does not offload link aggregates, so the bond is a software port in an offloaded bridge, and the switch never hands its traffic to the CPU. Patch 0123 adds LAG offload, based on the LAG support by Daniel Machon, and an MDB fix by Horatiu Vultur, in Microchip's BSP kernel, adapted to the mainline 6.18 driver. Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes and debug aids for the Laguna (LAN969x) boards, found while bringing the Tactical 1000 into the new test system.
Kernel, from the
laguna-bugbranch of the kernel tree, on top of kkit-linux-6.18.y:vlan_default_pvid 0, which adds VLAN 1 at creation and removes it right after. That forgot the VID 1 broadcast entry, so ARP from the ports never reached the CPU, and the bridge MAC entry, so unicast to the bridge was flooded to the ports. A third fix stops the driver from forgetting a bridged port's MAC on link down, which cut the bridge off whenever its first port, whose MAC the bridge uses, was disabled. Traced withsymregon the EV23X71A.vlan_default_pvid 0. In a VLAN-unaware bridge the switch classifies untagged traffic to hardware VID 1, but a port only joins that VLAN when the bridge adds VID 1 to it, which never happens without a default PVID, so the VLAN mask stays empty and every frame between the ports is dropped. Bridge forwarding, STP, IGMP, static multicast filters and LAG-in-bridge tests have failed on every Tactical run for this reason. Reproduced ontact3withip link add br0 type bridge vlan_filtering 0 vlan_default_pvid 0and confirmed withsymreg: adding the two ports toANA_L3_VLAN_MASK_CFG[1]by hand made it forward. 0118 makes every port in a VLAN-unaware bridge a member of that VLAN.e1.10with the bridge carrying only VLAN 1), never received anything: sparx5 had nondo_vlan_rx_add_vid, so nothing sent the port address or broadcast in VID 10 to the CPU. 0119 adds those entries when the VLAN interface is created, without making the port a member of the VLAN, so the VLAN stays local and is not bridged, as in software. The bridge's own per-VLAN broadcast entry now shares the same reference count. Proven ontact3with the two MAC entries written by hand withsymreg: ping overe5.10went from 0/5 to 5/5, and the peer on the other bridge port saw no VLAN 10 traffic.vlan 0, vlan 55. 0120 uses the "tag all except VID 0" mode on such ports; switched traffic in a VLAN filtering bridge never has VID 0. Proven ontact1andtact3e7 withsymreg: ping across the trunk went from 0/4 to 4/4 with a single VID 55 tag on the wire.tact1andtact2: LACP converged and tact1 answered ARP, but the reply never reached tact2. Link Aggregation Basic has failed on every Tactical run for this reason. The adaptation keeps the registers in the mainlineregs->gaddrform, folds the LAG source and destination masks intosparx5_update_fwd(), applies the MDB fix to the MDB code insparx5_switchdev.c, drops the HSR, MRP, XDP and QoS parts of those commits, and lets the bond's switchdev attributes through on its member ports. It includes the BSP's later "allow foreign devices in bridge" (c92187b2b2a5): without it the driver vetoed every non-switch device, veth, VLAN, VXLAN, GRE, WireGuard, from joining any bridge, which failed 22 tests on the first run with 0123. A second regression from the same rework:switchdev_handle_fdb_event_to_device()reaches the driver only through a bridge's member ports, so a bridge address set before any port joins, which is how confd builds bridges, never got its CPU entry and the bridge IP was unreachable. Reproduced ontact2: the MAC set before the port joined gave 0/3 pings, after gave 3/3. 0123 now installs the bridge's own addresses for the CPU directly, as the driver did before.ANA_CL_CAPTURE_BPDU_CFGonly discarded pause frames, so every other address in 01-80-C2-00-00-0X was forwarded like ordinary multicast. On a bridged port BPDUs and LACPDUs were flooded to the other ports, so in the STP test's ring each bridge saw BPDUs relayed from bridges that are not its neighbours; Bridge STP Basic and Link Aggregation Basic have failed on every Tactical run. Reproduced ontact3: tact2's BPDUs crossed tact3's bridge to tact4 even with STP enabled on tact3, and stopped once the block was redirected to the CPU withsymreg. 0122 redirects the whole block to the CPU on every port and keeps discarding pause frames. The storm from the unbroken loop is also the likely source of theFlush timeout port 5. SRC-MEM queue not emptyseen when the STP test tore its bridge down.e5.10made the switchdev helper pass the bridge port's STP state, flood flags, VLAN filtering and ageing down toe5, which is in no bridge, and the console filled withControlling non-bridged port 7?. 0121 only takes an attribute for the port itself or for the bridge the port is offloaded to.symregontact1in the rig.ixlland the test rig until the port was cycled. Fixed by reference counting the shared entries.ndo_set_rx_moderuns undernetif_addr_lock_bh, and the MAC table access took a mutex. Now a spinlock with an atomic poll, as lan966x does. Upstream fixed it via the newndo_set_rx_mode_async, which 6.18 does not have.config_anegand restarts aneg once per port on its first 1G link. Not yet verified on the rig: the fault is rare and needs a full run to judge.Infix side:
0x8804, the LAN8804: a LAN8814 without the 1588 unit and with the same chip ID. 6.18 takes them for full LAN8814s, creates PHY clocks and advertises PHY timestamping that never produces a timestamp, so every PTP port faulted on the first Sync. Backport of mainline 61b7ade9ba8c, which checks the SKU register and creates no PTP device on these parts; the kernel then uses the switch core timestamper for them on its own. Boards with real LAN8814s, the EV23X71A, keep stamping at the PHY. On the EV23X71A those PHY timestamps arrive over MDIO later than the 10 ms ptp4l allows, so confd now allows 100 ms with hardware timestamps.mean-link-delay-threshfrom the gPTP port data set is honored per port. The default matters on the Tactical, where the switch core stamps behind the PHY and the measured link delay is 1.5 µs; on the EV23X71A the PHY stamps at the wire and the delay is a few hundred ns./vpd.binfrom the aux partition into reserved memory, now also on Laguna, and the device tree describes it as an nvmem-rmem node with an ONIE layout. The switch and every port take their address from it with an offset. sparx5 only numbered ports from the switch base, base + port + 1, so 0117 lets a port's own device tree address override that; the Tactical copper ports run backwards within each quad, and the ports keep the front panel order the rig already uses, e1 at base + 1 and mgmt at base + 29. The addresses are permanent, so confd restores them when a custom MAC is removed, which fixes the custom MAC test.provision-tacticaloffers to write the file. This needs the Infix FIP from this branch on each unit.symregpackage, selected by the Laguna boards, with the device tree node in each board's Infix overlay. Usage notes, including how to chase a port fault, inboard/aarch64/microchip-lan969x/README.md.aarch64_minimal_defconfiglacked the Tactical 1000, EV23X71A and Vero W6m board packages, so PR CI images had no DTB for them and hung afterStarting kernel ....provision-tacticaldid not seeduboot.env, so RAUC could not read any boot state on provisioned units.kernel-refresh.shdeleted the patch directory it had just regenerated when run for the current kernel version.logandfollowmoved to/etc/profile.d/, so they work in BusyBox ash on minimal builds too.Risk analysis
Bridge fixes (0104–0106): the VID 1 broadcast entry is now permanent, like the VID 0 one for standalone ports. Standalone ports never classify to VID 1 and
PGID_BCASTholds only bridged ports, so it is inert without a bridge. The shared-entry count only covers entries the driver installed itself (locked); auto-learned entries behave as before. Skipping learn and forget on up/down for bridged ports leaves those entries to the bridge FDB notifications the driver already handles; join still forgets the standalone entry and leave still restores it. Upstream has the same three bugs, hidden by the defaultvlan_default_pvid 1.Lock conversion (0079): nothing sleeps or takes another lock under
sparx5->lock, it nests inside the mutexes and never the reverse, and every caller is process context; the atomic switchdev notifier defers FDB work and never touches the MAC table. A softirq caller would already have been taking a mutex from softirq, so the spinlock tolerates exactly the contexts the mutex had to. Same fix lan966x took in 77bdaf39f3c8. Only behaviour change: a wedged MAC table command busy-waits up to 100 ms instead of sleeping.Shared CPU entries (0103, 0107): list touched only on group join and leave, port up and down, bridge join and leave, and address change, held across the hardware learn/forget so two ports cannot race on the same entry. Lock order is always
mc_cpu_lockthensparx5->lock.PHY errata (0109–0111): register values are identical to MEPA's after-reset sequence and Microchip's BSP kernel, and mainline already writes the same tables on the LAN8842. 0111 runs once per boot: the package QSGMII reset at the first
config_anegin the package, and one extra aneg round per port at its first 1G link, so each copper port's first link-up after boot takes about three seconds longer. Later ifups are unchanged. At boot the quad's first port, which owns the switch serdes, attaches before its three siblings.EEE off (0112): no behaviour change for traffic, since the switch never used LPI. Ports stop advertising EEE, so partners that relied on it to save power no longer can.
Aneg watchdog (0113, 0116): runs only while a port is started, link down and autoneg on. An uncabled port costs one MDIO read every five seconds. One restart per link-down period, re-armed on the next link change. Forced 100 Mbps ports are watched too, and get a DSP reset every three seconds while a partner is detected without a link. The work checks the PHY state under
phydev->lockand is cancelled on remove.10BASE-T and half duplex (0114, 0115): values from MEPA's after-reset and link-state code. The half-duplex thresholds only apply to 10/100 half-duplex links, and the FIFO reset on link down restores the defaults. Full-duplex links at any speed are unaffected apart from that reset.
symreg: debugfs only, root only; the driver never probes without its device tree node, so it is inert on non-Laguna boards. The tool bypasses the driver when writing, documented in the README.
Checklist
Tick relevant boxes, this PR is-a or has-a:
🤖 Generated with Claude Code