Skip to content

Connection Architecture

Status: verified — 2026-04-13

Matches the landed scripts/iosmux-restore.sh flow and the S1.A-D inject chain on havoc. Any drift from this page should be treated as a bug in the page, not in the code.

This document describes how the iPhone, the Linux host, the macOS VM (havoc), and the CoreDeviceService inject are wired together at runtime. It replaces scattered notes about tunneld placement and provides a single authoritative diagram.

Status after Phase S2.B (April 2026)

The wiring below is correct for everything up to and including Stage S1.D: the L2 bridge, tunneld on the VM, the inject loading, and CDS being able to surface the device in devicectl list devices all work exactly as drawn. The Pair-button smoke test in Phase S2.B, however, established that one edge in the diagram — "CDS talks RemoteXPC to the iPhone directly through the tunnel" — does not work the way the diagram implies. CDS opens nw_connection to the tunnel RSD endpoint and receives an immediate TCP RST because the client-side protocol CDS expects is different from the one pymobiledevice3 tunneld produces. pymobiledevice3's own Python client speaks the correct dialect and works end to end (verified: rsd-info, mounter, DVT root-fs listing).

Everything CDS currently "knows" about the iPhone it learned from our inject serving cached handshake data via MDRemoteServiceSupport interposers. No active CDS → iPhone traffic completes today. Stage S2 (Option δ) will insert a CDS → pymobiledevice3 backend shim so that CDS's requests are translated into pymobiledevice3 operations on its existing working tunnel, instead of trying to speak to pymobiledevice3 tunneld natively.

The diagrams below are kept in their original form so readers can see the intended/naive topology; inline notes mark the edges that are aspirational vs. currently working, and docs/research/s2b-pair-attempt-log.md has the full post-smoke-test analysis.

Inject capability inventory (pre-D.6.6-impl)

This is the starting point for Phase D.6.6-impl work. Verified by direct review of inject/iosmux_inject.m plus grep for the negative-inventory items.

The current iosmux_inject.dylib does:

  • Synthesise OS_remote_device (binary-set ivars) + DeviceInfo (Swift sret + per-field setters) + ServiceDeviceRepresentation (Swift __allocating_init with R13=metatype), and register it via the canonical install(browser:) discovery-closure path (S1.D, hook on CoreDevice + 0x27d7d0).
  • Interpose seven RemoteServiceDiscovery / RemoteXPC C functions via DYLD_INTERPOSE: remote_device_copy_service_names, remote_device_copy_service, remote_device_copy_property, remote_device_heartbeat, remote_service_create_connected_socket, remote_service_connect_socket, xpc_remote_connection_create_with_remote_service.
  • Hook serviceDeviceRepresentations(forDeviceIdentifiedBy:) at the SDM level so PairAction's UUID lookup resolves to our SDR.
  • Hook three CDS-binary call sites: CDS+0xCCB0 (witness-table bypass), CDS+0x5E2D0 (callq replacement → withRSDDeviceWrapper short-circuit), CDS+0xB896 (S1.B passthrough trampoline on invoke(anyOf:usingContentsOf:)).
  • Swizzle ServiceSidePairingSession.alloc/init for tracing.
  • Stand in for the missing usbmuxd path on the VM (verified by iter-18 Run B: removing the active inject collapses the synthetic device entirely and CDS falls through to 127.0.0.1:62078 with Connection refused).

The inject does not:

  • Intercept Mercury Codable XPC messages. There is no xpc_connection_set_event_handler interpose, no mangledTypeName recogniser, no Codable Output fabrication. The CDS+0xB896 hook is intentionally a no-op tail-call into the original invoke — patch in place as a future hook point only.
  • Publish _remotepairing._tcp.local Bonjour adverts. Not in any iosmux process. The iPhone's own adverts get rejected by remotepairingd per the iter-16 mechanism (no peer-record match for the rotating identifier/authTag pair).
  • Stand up a synthetic AcquireDeviceUsageAssertion responder. The ~15 "static-success" actions enumerated in ADR-0009 §Consequences are not handled — Xcode reaches them through Mercury but no reply is fabricated.

The Mercury Codable interceptor is sub-task 3 of Phase D.6.6-impl per ADR-0009 §Consequences and stage2.md Phase D.6.6-impl. Sub-task 1 (lockdown-over-TCP [::1]:50367 handler) and sub-task 2 (Mercury Codable Output byte capture) precede it.

Backend self-bootstrap chain (Phase D.6.6-impl sub-task 1)

Sub-task 1 landed in commits 06ceda8 (initial) and b2da3f6 (dynamic port discovery — closes Q-D66-13). At backend startup iosmux-backend serve runs a deterministic three-step chain before binding any listener; on any failure the backend exits cleanly per ADR-0006.

sequenceDiagram
    autonumber
    participant be as iosmux-backend (havoc)
    participant td as pymd3 tunneld (havoc, stock mode)
    participant ip as iPhone (over utun)

    be->>td: GET http://127.0.0.1:49151/
    td-->>be: {UDID:[{tunnel-address, tunnel-port}]}
    Note over be: ResolveTunnelAddress<br/>internal/lockdown/tunneld.go

    be->>ip: TCP dial [tunnel-address]:tunnel-port
    be->>ip: HTTP/2 preface + SETTINGS + WINDOW_UPDATE
    be->>ip: HEADERS(s1) + DATA(s1, empty XPC dict)
    be->>ip: DATA(s1, sync flags=0x201)
    be->>ip: HEADERS(s3) + DATA(s3, INIT_HANDSHAKE)
    ip-->>be: SETTINGS, WINDOW_UPDATE, HEADERS(s1), HEADERS(s3)
    ip-->>be: DATA(s1, 14 KB big Handshake — Properties + Services)
    Note over be: FetchServices<br/>internal/lockdown/rsd_discovery.go<br/>62 entries decoded

    Note over be: FindServicePort("com.apple.mobile.lockdown.remote.trusted")<br/>returns per-session port

    be->>ip: TCP dial [tunnel-address]:<discovered-port>
    be->>ip: RSDCheckin {Label, ProtocolVersion:"2"}
    ip-->>be: ack frame
    ip-->>be: unsolicited StartService push
    be->>ip: QueryType
    ip-->>be: Type="com.apple.mobile.lockdown"
    be->>ip: GetValue
    ip-->>be: Value=<88-key device dict>
    Note over be: FetchDict<br/>internal/lockdown/client.go<br/>dict cached in-memory

    Note over be: bind synthetic listeners<br/>greeting [::1]:34719 + lockdown [::1]:50367<br/>"iosmux-backend serve starting" log line

Properties of this chain that matter for downstream sub-tasks:

  • No port hardcoded. iOS 17+ allocates Services-dict ports per tunneld session — empirically verified (Q-D66-13 §Resolution log Round 1 + Round 2). Both the tunnel-port (RemoteXPC greeting) and the lockdown service port are read from live API responses at startup.
  • Stock tunneld is required during bootstrap. The discovery chain talks to a live iPhone through tunneld's utun. SPIKE mode (scripts/iosmux-tunneld-mode.sh spike) advertises a fake tunnel-address pointing at our own backend — useful for CDS-side experiments after bootstrap completes, but breaks any restart that needs the live iPhone. Operationally, run bootstrap → swap to SPIKE → run smoke tests; never the other way round.
  • Bootstrap window. Default --bootstrap-timeout 30s (scripts/iosmux-d66-deploy.sh overrides to 45s); covers tunneld discovery latency (~5–7 s observed) plus the two round-trips (RSD greeting and lockdown 3-verb, ~1 s combined). Live experiments have consistently completed in 6–7 s end-to-end, with headroom.
  • No state file. Each restart re-runs the full chain. The 88-key dict and the discovered port are held in process memory only; nothing is written to disk, no caches survive process exit. This is the right shape for an apparatus where every reboot or iosmux-restore.sh rotates the relevant identifiers.
  • Cleanly exits on failure. If tunneld returns an empty device list, or RSD greeting GOAWAY-s, or the lockdown port refuses, the backend wraps and surfaces the error and exits with a non-zero status. ADR-0006 forbids fallback constants; the apparatus operator is expected to fix the upstream issue rather than have the backend run with stale or guessed state.

Goal

Make an iPhone that is physically connected to the Linux host visible to Xcode inside a macOS VM as if the VM owned the USB connection. The VM never sees USB lockdown traffic — everything flows through a layer-2 network bridge and a RemoteXPC tunnel terminated inside the VM.

Physical and logical layers

flowchart TB
  subgraph iPhone["iPhone (iOS 17+)"]
    ios_lock["lockdown daemon<br/>(default USB config)"]
    ios_rsd["RemoteXPC / RSD<br/>(IPv6 over NCM config 5)"]
    ios_services["iOS services<br/>(screenshot, fs, debug, ...)"]
  end

  subgraph host["Linux host (op@dra)"]
    subgraph usb["USB stack"]
      usbmuxd["usbmuxd<br/>(lockdown channel)"]
      ncm_if["enp0s*c5i4<br/>(CDC-NCM data interface)"]
    end
    iosmux_bridge["iosmux bridge<br/>USB mode switch: cfg 0 → cfg 5"]
    virbr0["virbr0<br/>(libvirt bridge)"]
    vnet0["vnet0<br/>(VM tap)"]
  end

  subgraph havoc["macOS VM: havoc"]
    en0["en0 on virbr0<br/>fe80::... link-local"]
    pymob["pymobiledevice3 tunneld<br/>(VM-local, port 49151)"]
    utunN["utunN<br/>(tunnel IPv6 endpoint)"]
    cds["CoreDeviceService<br/>+ iosmux_inject.dylib"]
    xcode["Xcode / devicectl"]
  end

  ios_services --- ios_rsd
  ios_lock -. "USB<br/>default mode" .- usbmuxd
  ios_rsd == "CDC-NCM<br/>config 5" ==> ncm_if
  iosmux_bridge -. "control transfer<br/>(once per boot)" .- usb
  ncm_if ==> virbr0
  vnet0 ==> virbr0
  virbr0 ==> en0
  en0 -- "mDNS discovery<br/>_remoted._tcp" --> pymob
  pymob -- "create tunnel<br/>over IPv6 link-local" --> utunN
  utunN -- "tunnel IPv6<br/>fdXX:XX::1:port" --> ios_rsd
  cds -- "GET /<br/>(VM-local HTTP)" --> pymob
  cds -. "RemoteXPC<br/>(S2.B: RST,<br/>protocol mismatch)" .-> ios_rsd
  xcode -- "XPC" --> cds

S2.B note on the dashed edge: before Phase B we assumed CDS could speak RemoteXPC directly to the tunnel RSD endpoint that tunneld creates. It cannot — Apple-native CDS expects a transport produced by its own sibling daemons, not by pymobiledevice3. Stage 2 (Option δ) replaces this edge with cds → inject shim → pymobiledevice3 client session → ios_rsd.

Key invariants

  1. tunneld runs inside the VM, not on the host. The VM discovers the iPhone via mDNS/bonjour on the NCM bridge (_remoted._tcp on en0) and creates the utun interface locally. The host never runs tunneld — it has no business terminating a VM-side tunnel. This was an early architectural mistake corrected in session 9.

  2. The host contributes only L2 plumbing. iosmux bridge switches the iPhone to USB configuration 5 (CDC-NCM) and the admin adds the resulting enp*c5i4 interface to virbr0. From that moment the host is a dumb bridge — no tunnel state, no Python, no patches.

  3. The inject never talks to the host directly. Its tunneld client points at http://127.0.0.1:49151/ because tunneld is on the VM. iosmux_resolve_tunneld_url() in inject/iosmux_xpc_proxy.m allows overriding via the IOSMUX_TUNNELD_URL environment variable for unusual setups, but the default is loopback and works for the standard flow.

  4. No sudo on the host during steady state. Sudo is used only twice, both during setup:

  5. iosmux bridge is a regular-user process, but its required sysfs write (echo 5 > .../bConfigurationValue) needs root. The restore script runs it via sudo sh -c.
  6. ip link set <ncm> master virbr0 and ip link set <ncm> up need root for the bridge join.

Everything else (tunneld on VM, CDS inject, devicectl) runs unprivileged.

  1. No sudo on the VM at all. ssh havoc-root authenticates via key directly as root, so launching pymobiledevice3 remote tunneld on the VM does not prompt interactively.

Flow after a clean host boot

sequenceDiagram
  autonumber
  participant user as User
  participant host as Linux host
  participant vm as havoc VM
  participant phone as iPhone
  participant cds as CDS + inject

  user->>host: ./scripts/iosmux-restore.sh
  host->>phone: lsusb (detect 05ac:12a8)
  alt iPhone already in NCM mode 5
    host->>user: "Unplug and replug — script waits"
    user->>phone: unplug → replug
    host->>host: wait for lockdown reachability
  end
  host->>host: iosmux bridge (background)
  Note over host: reads "sudo sh -c 'echo 5 > ...'"<br/>from stdout, eval's it
  host->>phone: USB config 0 → config 5 (CDC-NCM)
  host->>host: sudo ip link set enp*c5i4 master virbr0
  host->>vm: ssh havoc-root tunneld (VM-local, port 49151)
  vm->>phone: mDNS discovery on en0 (virbr0)
  vm->>phone: RSD pair + tunnel create
  vm->>vm: utunN up, tunnel-address fdXX:...:1
  host->>host: restore script exits, daemons alive
  user->>vm: ssh / devicectl list devices
  vm->>cds: launchd spawns CoreDeviceService
  cds->>cds: inject dylib loads
  cds->>vm: GET http://127.0.0.1:49151/
  vm->>cds: JSON {UDID: [{tunnel-address, tunnel-port}]}
  Note over cds,phone: S2.B found: CDS cannot actually speak RemoteXPC<br/>to tunneld's RSD endpoint — immediate RST.<br/>Instead the inject serves cached handshake data:
  cds->>cds: md_proxy serves Handshake (62 services, 46 properties)<br/>from in-memory cache populated by pymobiledevice3
  cds->>cds: build SDR, register, reply CheckIn
  cds->>user: devicectl list devices shows iPhone

Why an earlier design with host-side tunneld was wrong

A briefly-held intermediate design put tunneld on the Linux host. The reasoning was "the iPhone is physically on the host, so tunneld belongs there." That is architecturally wrong for two reasons:

  • pymobiledevice3 tunneld tears down its monitor task when the iPhone disappears from usbmuxd. When iosmux bridge switches the device to CDC-NCM mode, the lockdown channel closes, usbmuxd emits a disconnect, tunneld marks the device gone and removes it from its HTTP registry. Everything after that point fails.

  • Even if the tunnel survives, the utun it creates lives on the host. The VM cannot reach a host-local utun without extra routing, forwarding, or NDP proxy work. None of that is needed if tunneld runs where the consumer lives.

Running tunneld on the VM sidesteps both problems: the VM reaches the iPhone at layer 2 via the NCM bridge, mDNS discovery works without usbmuxd, and the utun is local to the process that cares.

Historical note: why the v2 rewrite deleted the host venv/patches

Session 5 (April 2026) built an entire support stack to make pymobiledevice3 tunneld run rootless on the Linux host: a Go-managed venv, two invasive source patches to pymobiledevice3 (001-cap-net-admin-check for getpcaps-based privilege check, 002-enable-ipv6-on-tun for the TUN IPv6 sysctl dance), iosmux install to orchestrate it, and setcap cap_net_admin,cap_net_raw=eip on a copied-rather-than-symlinked Python interpreter. All of that machinery existed for one reason: running tunneld on the host without sudo.

The v2 rewrite (commit d4e0eb3) deleted this entire stack — venv management, patches, install subcommand, setcap wrapper, the whole thing — and we forgot why. Session 9 re-introduced it by mistake while trying to recover after a reboot: I saw tunneld failing with "This command requires root", remembered the old patches, ported them forward to Go, reinstalled everything, and got back to a working host-tunneld that immediately broke again on device disconnect.

The v2 rewrite was right to delete this. Host-tunneld is the wrong architecture; the patches only existed to paper over that wrong architecture; bringing the patches back did not make host-tunneld correct, it only made it fail at a different place. The moment iosmux bridge switches the device to NCM mode, host-tunneld's usbmux monitor task dies and the tunnel registry is cleared — no amount of capabilities fixes that.

Lesson for future selves: if you are tempted to re-introduce host-side tunneld, the patches, the venv manager, or the iosmux install setcap flow, read this section first. The correct location for tunneld is inside the VM, discovered via mDNS on the NCM bridge. There is no architecture where host-side tunneld ends up clean.

If you find deleted machinery in git history labelled "rootless tunneld", "apply-patches", "CAP_NET_ADMIN", or "setcap" — that is artefact of the wrong architecture. Leave it deleted.

Components and where they run

Component Where Reason
iOS device physical source of truth
iosmux bridge host USB mode switch, holds USB handle alive
virbr0 + NCM attachment host L2 bridge to VM
pymobiledevice3 tunneld VM (havoc) mDNS discovery + utun creation
iosmux_inject.dylib VM (inside CoreDeviceService) hooks RSD, injects SDR
CoreDeviceService (Apple) VM entry point clients connect to
devicectl, Xcode VM developer tooling

No Python runs on the host. No tunneld runs on the host. iosmux itself is a Go binary that does USB mode switching and L2 bridge instructions — it is not a tunnel client.

Port and address summary

  • http://127.0.0.1:49151/ on havoc — tunneld HTTP API, consumed only by the inject inside CoreDeviceService. Bound to loopback because nothing outside the VM should read it.
  • fe80::...%en0 on havoc — link-local IPv6 over the NCM bridge. Used by tunneld to reach the iPhone before the tunnel exists.
  • fdXX:XX::1:<tunnel-port> (example) on havoc — the tunnel's own IPv6 address. Used by pymobiledevice3 Python clients to reach the iPhone RSD endpoint directly. A native CDS client cannot use this endpoint: see the S2.B note in the flowchart section — Option δ will wire CDS through a local shim backed by a pymobiledevice3 session instead of having CDS touch this endpoint itself.
  • 192.168.122.0/24 (libvirt default) — the IPv4 bridge network. Used only for ssh havoc from the host; not involved in the tunnel path.