Skip to content

ADR-0002 — pymobiledevice3 tunneld runs inside the macOS VM, not on the Linux host

Status

accepted

Context

The iosmux development environment is a Linux host with a macOS VM (havoc) running Xcode and CoreDeviceService. The iPhone is physically plugged into the Linux host. Somewhere in this chain a process has to hold the lockdown tunnel to the iPhone and expose the tunneld HTTP API that CoreDeviceService queries to discover the device.

The first iteration placed pymobiledevice3 remote tunneld on the Linux host and forwarded its HTTP port into the VM. That was architecturally simpler: the iPhone is physically on the host, so the host should own the USB relationship. CoreDeviceService inside the VM would fetch the tunnel metadata over the forwarded port and proceed.

That design was wrong, in a way that took a few sessions to diagnose. Two empirical observations eventually forced the reverse:

  1. Host-side tunneld hands out a tunnel address that only routes inside the host's network namespace. CoreDeviceService in the VM would get that address back from the HTTP call and then fail to actually reach the tunnel endpoint — the IPv6 address for the tunnel was not routable from inside the VM.
  2. Even with careful L2 bridge setup so that the VM and the host shared the correct layer-2 segment, CDS's internal expectation was that the tunnel address in the JSON response would be locally bindable, which it is not when tunneld is on a different host.

The fix was to reverse the placement: move pymobiledevice3 remote tunneld inside the macOS VM, and relay the USB-level lockdown traffic from host to VM at a lower layer (USB pass-through plus an L2 bridge that gives the VM direct access to the iPhone-side usbmux channel).

Decision

pymobiledevice3 remote tunneld runs inside the macOS VM. The Linux host is responsible only for the L2 bridge setup and the iPhone's raw USB path; tunneld itself is launched by scripts/iosmux-restore.sh via ssh havoc-root and the tunnel JSON it serves has a local tunnel address that CoreDeviceService can reach natively.

Consequences

Wanted:

  • CoreDeviceService gets a routable tunnel address and can open its own nw_connection to it without any forwarding hops.
  • Tunneld and CDS share a loopback interface, so the address CDS receives is trivially local from its own process namespace.
  • The L2 bridge becomes a one-time setup concern on the host; the interesting state lives on the VM.

Accepted as cost:

  • Startup complexity: scripts/iosmux-restore.sh now has to drive both the host (bridge + USB) and the VM (tunneld) in the right order.
  • Extra ssh hop for operating tunneld. We accept this because the alternative (running tunneld on a misplaced host) is worse.
  • Losing the ability to use tunneld directly from the Linux host for ad-hoc research — we have to go through the VM. In practice this has been zero-cost because all the interesting research targets (CDS, Xcode, devicectl) also live on the VM.

Evidence

  • architecture/connection.md — current ground-truth layout with the "Historical note: why host-side tunneld was wrong" section.
  • scripts/iosmux-restore.sh — the idempotent post-reboot script that implements this layout, visible in the repo.
  • Session-⅞ bring-up logs on havoc showing the IPv6 tunnel address from host-side tunneld being unreachable from the VM.

References

  • ADR-0001 — why we use pymobiledevice3 at all.
  • ADR-0004 — the shim layer that now sits between CDS and the VM-local tunneld.