Six layers behind \"the Wi-Fi in my room isn't working\"

A guest calls the front desk. The Wi-Fi in their room isn't working.

That sentence is the entire fault report, and it's compatible with at least six unrelated failures across four different systems. Behind it sits a fiber access platform I inherited with no formal training and no integrator left on site: a Zhone MXK 319, a GPON OLT feeding every room's wireless over passive fiber.

Not a switch with fiber ports#

The instinct is to treat an OLT like a switch, because it racks like one and speaks VLANs like one. It isn't. The useful mental model is a service-mapping platform: it takes VLANs from the core network and maps each one onto a per-subscriber logical channel carried over shared fiber. A switch moves frames between ports. An OLT translates between an Ethernet world and a PON world, holding state per subscriber to do it.

That distinction explains why its CLI looks nothing like the Comware or IOS dialects a networking background prepares you for, and it's the reason a single room's Wi-Fi is actually six layers deep:

SSID
  │
CPE WLAN service          subscriber wireless object on the ONT
  │
RG / bridge mode          how the ONT treats the traffic
  │
GEM port                  logical GPON transport channel
  │
VLAN bridge               Ethernet service identity
  │
GPON fiber → OLT uplink → core → DHCP → gateway → Internet

Each layer fails independently. Each looks completely different in the CLI. A guest can't tell you which one is broken, and neither can the symptom on its own: "Wi-Fi doesn't work" is consistent with a dead radio, a bad credential, a missing DHCP lease, or a perfectly healthy room sitting behind a broken upstream gateway.

The identifier that isn't what it looks like#

Almost every serious mistake on this platform comes from misreading an identifier, and the one that matters most is the GEM port: the logical GPON transport channel for one service on one subscriber's fiber connection. Not a VLAN. Not the Wi-Fi network itself.

A GEM number decomposes into a service-class prefix and the subscriber's own ID. For a guest VLAN carried across four adjacent rooms, the numbering mostly follows the ONT ID:

Room 21   VLAN 400 → GEM 721
Room 23   VLAN 400 → GEM 723
Room 24   VLAN 400 → GEM 724
Room 22   VLAN 400 → GEM 522     ← different class

Three rooms out of four follow the pattern. That's worse than no pattern at all, because no pattern forces you to check every time. A near-pattern lets you skip checking and be right most of the time, which is exactly the condition that produces an occasional expensive mistake.

I found that out by making it. Troubleshooting Room 22, I moved its service from GEM 522 to GEM 722, reasoning that the neighbouring rooms all carried that VLAN on the 7xx class, so 722 looked like the obvious value. The bridge state for the entire room went down. Restoring GEM 522 brought everything back up immediately, and running the actual GEM inspection command afterward showed why: 522 was the real, provisioned channel on that subscriber's connection, and 722 simply didn't exist there. There was nothing for the bridge to attach to.

One command, run first instead of last, would have prevented the entire detour. It's the single most important diagnostic command on the platform for exactly this reason: it returns the GEMs actually provisioned on a subscriber's connection, rather than the ones a pattern says should be there.

The lesson generalises past GPON, and I've caught myself starting to trust exactly this kind of near-pattern elsewhere since: switch-port-to-room mappings that mostly follow a scheme, IP allocations that mostly increment, naming conventions that mostly hold. The habit that actually holds up is reading state before writing it, even when the answer looks obvious. Especially then.

A second trap, hiding in a label#

The same platform has a quieter version of the same mistake: traffic profiles, which control the bandwidth ceiling on a GEM, are commonly named after their own bandwidth in kbps. Two out of three profiles I checked on one room matched their names. The third didn't:

GEM (guest VLAN)   traffic profile "512"   enforcing 30.720 Mbps

Someone reused a profile ID, or edited one, without renaming it. The label said one thing; the enforced ceiling said another. Read the field that's actually enforced, never the name attached to it. The rule applies well beyond bandwidth profiles.

The diagnostic ladder that survives a bad guess#

Fixing the actual mistake above cost minutes once the right command was run. The lesson that stuck was the order to run things in, which now doesn't change no matter how confident the first guess feels:

1. Capture the current state before touching anything: it's the only route back if a change goes badly. 2. Is the subscriber's whole service set present, and not merely the one VLAN in question? 3. Are the GEMs actually provisioned, and on which channel? 4. Is the expected VLAN on the expected GEM, checked against that room's own history, never a neighbour's. 5. Is the wireless service itself configured and enabled? 6. Is the bridge actually learning client traffic? 7. Only if all of that is healthy does the fault sit upstream, in DHCP or beyond.

A guest holding an address in the 169.254.x.x range, for instance, means the bridge is healthy and learning, and the fault is squarely in the DHCP path. That also means the GEM and VLAN layers below it are exactly right and shouldn't be touched.

The part that has nothing to do with the CLI#

The vendor behind this platform went through a corporate bankruptcy and asset sale inside the last two years, continuing under the same brand name with a different owner. That kind of transition reliably disrupts support contracts, RMA paths and firmware availability, and it's easy to miss entirely if you only ever interact with the box through its command line.

The useful response has nothing to do with GPON specifically: confirm whether a support contract still exists and with whom, hold a cold spare of the line card carrying the most subscribers, take an offline configuration backup while the management path still works, and archive the vendor documentation while it's still reachable. None of that shows up in a CLI reference, and all of it matters more than any single command on the list.

Command reference, by trust level#

The single organising idea that made this platform usable under pressure: group commands by how much they're trusted, not by what they do. <cpe> below stands for the platform's subscriber address, in the form <slot>/<pon-port>/<onu-id>; <gem> for a specific GEM port.

Tier 1 - verified, safe to run for diagnosis.

help
swversion                     software identity
slots                         card inventory and status
card show

cpe wlan show all             every subscriber WLAN system-wide
cpe wlan show <cpe>           all WLANs on one subscriber
cpe wlan show <cpe>/<wlan>    one specific WLAN

cpe wlan modify <cpe>/<wlan> admin-state down
cpe wlan modify <cpe>/<wlan> admin-state up
                               reversible - the safest first test for a
                               radio-side fault

gpononu gemports <cpe>        the GEMs actually provisioned on a subscriber -
                               the single most important diagnostic command
                               on the platform

bridge show <cpe>/gpononu     all bridges on one subscriber
bridge show <gem>/gponport    one GEM's bridge, including learned MAC/IP

ping <target>

cpe wlan show output can include the WLAN encryption key in plain text. Redact it before pasting the output anywhere.

Bridge creation and deletion are also Tier 1 in the sense that they're verified working, but both are service-impacting and belong in a change window, not a diagnostic pass:

bridge add <gem>/gponport gtp <profile> downlink vlan <vlan> tagged wlan <n> rg-bridged maxUnicast <n>
bridge delete <gem> vlan <vlan>

Tier 2 - present in the CLI's own help output, syntax not yet verified. Confirm with help <command> before relying on any of these:

bridge showall     bridge showdetail    bridge stats
bridge rates       bridge igmp          bridge igmpstat
bridge igmpver     bridge capture       bridge flush
bridge unblock     bridge modify

interface show     interface stats

ip ifsum    ip ifstat    ip stats      ip icmpstat
ip udpstat  ip tcpstat   ip ipstat     ip arpshow
route show  route list   route help

alarm show
alarm show summary
log

upgrade show        upgrade log        (read-only members of upgrade)

ip arpshow is the natural next step when a client holds a valid address but has no connectivity. bridge flush and bridge unblock read like diagnostics but are likely disruptive, so treat them as Tier 3 until proven otherwise. alarm show is worth running on a schedule rather than only during an incident: a degrading optic tends to alarm before a guest notices.

Tier 3 - change window and a written rollback plan only.

cpe delete              whole-subscriber scope, not one WLAN
set2default              configuration reset
systemreboot              chassis reboot
upgrade set
upgrade continue
upgrade continue -force

cpe delete is worth naming specifically: it is not how you remove one unwanted SSID, and the narrower syntax for that on this firmware was never established during this work. Confirm with help cpe wlan rather than guessing at delete syntax on a production access platform. And given the vendor history above, don't reach for upgrade casually either: a rollback image may not be easy to source.

Documented by the vendor, not yet exercised on this platform. Recorded for the day a replacement ONT or a new service actually needs one of these: ONU discovery and activation (gpononu show to list discovered serial numbers, then binding one with onu set), traffic-profile creation (new gpon-traffic-profile with its DBA parameters), the uplink-side bridge and bridge-path commands that have no downlink-side equivalent documented here yet, WLAN creation (cpe wlan add, confirming the addressing scheme above), and two alternate residential-gateway modes alongside the bridged one used throughout this piece.

What actually generalises#

An OLT holding per-subscriber state across six layers is an unusually deep example, but the pattern isn't unique to fiber access platforms. Any system with identifiers that are "almost regular enough to guess at" will eventually reward the habit of checking over the habit of pattern-matching. The expensive failures come from the identifiers that are right three times out of four, not the ones that are wrong every time.