Q3 iter-2 findings — event-driven dispatch clears HTTP/2 layer, RST_STREAM(s3) on application mismatch¶

Status: verified — 2026-04-18

Live run on havoc. Real CoreDeviceService connected to the iter-2 spike listener running in IOSMUX_SPIKE_REPLAY=1 mode with the event-driven dispatcher (commit 74e8d61). Session completed one full XPC handshake exchange on the wire, then CDS closed the REPLY_CHANNEL with RST_STREAM(stream=3, error_code=5 STREAM_CLOSED) and reset the TCP connection. Every byte below was produced by that run and decoded through standard HTTP/2 framing plus pymobiledevice3's XpcWrapper.parse.

TL;DR¶

Event-driven dispatch fixed the iter-1 PROTOCOL_ERROR. CDS accepted our server HEADERS frames (no more "invalid stream_id" diagnostic), opened both ROOT_CHANNEL (s1) and REPLY_CHANNEL (s3), and sent its full three-frame XPC handshake (empty-dict, sync 0x0201, INIT_HANDSHAKE). Our replay answered it byte-for-byte from the iter-0 corpus.

But CDS then RST_STREAM'd stream 3 with STREAM_CLOSED (error code 5) immediately after our INIT_HANDSHAKE reply on s3, and terminated the TCP connection. Root cause is a dispatch ordering bug: our iter-2 table emits the 14 KB Handshake dict (replay #8) in reaction to the first client DATA(s1), which fires before CDS has opened stream 3 and sent its INIT_HANDSHAKE there. In the real Q2 pcap, the iPhone held #8 until after it received INIT_HANDSHAKE on s3. Our premature #8 lands on s1 while CDS is still mid-handshake on s3, and CDS's REPLY_CHANNEL state machine does not tolerate "big Handshake arrived before I finished INIT_HANDSHAKE".

Forward progress: recv grew from 180→217 bytes (+37, +21%), 9 client frames parsed vs. iter-1's 5. The framing layer is now invisible — the remaining gap is pure application-layer ordering.

Iter-3 plan (one-line): redirect #8 from the client DATA(s1) trigger to the client DATA(s3) INIT_HANDSHAKE trigger, and split client DATA(s1) into a counter (1^st → #4, 2^nd → #5).

The session, byte by byte¶

Setup¶

Listener bound [::1]:34719, IOSMUX_SPIKE_REPLAY=1, iter-2 dispatcher (commit 74e8d61).
Tunneld (SPIKE mode) advertising tunnel-address=::1, tunnel-port=34719 (PID 1952 from the iter-1 run, unchanged).
CDS restart via killall CoreDeviceService, devicectl list devices to drive one tunnel query.

Client → listener: 9 HTTP/2 frames, 217 bytes total¶

#	type	stream	flags	payload len	wire len	payload decoded
—	(preface)	—	—	0	24	`PRI * HTTP/2.0...SM...`
0	SETTINGS	0	0x00	12	21	MAX_CONCURRENT_STREAMS=100, INITIAL_WINDOW_SIZE=0x100000
1	WINDOW_UPDATE	0	0x00	4	13	window_increment=0x000f0001
2	HEADERS	1	0x04	0	9	END_HEADERS — opens ROOT_CHANNEL
3	DATA	1	0x00	44	53	XpcWrapper empty-dict, `message_id=0`, flags `ALWAYS_SET`
4	SETTINGS	0	0x01	0	9	ACK of our server SETTINGS
5	HEADERS	3	0x04	0	9	END_HEADERS — opens REPLY_CHANNEL
6	DATA	1	0x00	24	33	XpcWrapper sync, `message_id=0`, flags `ALWAYS_SET \\| 0x200`
7	DATA	3	0x00	24	33	XpcWrapper INIT_HANDSHAKE, `message_id=0`, flags `ALWAYS_SET \\| INIT_HANDSHAKE`
8	RST_STREAM	3	0x00	4	13	error_code=0x00000005 STREAM_CLOSED

Every DATA payload is byte-exact with the c2s frames recorded in ../Q3-iter-00-pcap-dispatch.md (pcap) and in ../Q1-decode.md (iter-0 Q1 extraction). CDS's handshake envelope did not change between iter-1 and iter-2.

The 8-frame sequence HEADERS(s1) → DATA(s1,44) → HEADERS(s3) → DATA(s1,24) → DATA(s3,24) is exactly what iter-0 predicted for the client half.

Listener → CDS: 10 HTTP/2 frames, 14326 bytes total¶

The same 10-frame corpus as iter-1 (byte-for-byte identical send buffer), but dispatched by client-frame trigger instead of upfront. Dispatch trace from results/session-01.log, reordered for clarity (trigger → emit):

client trigger	server frames emitted	cumulative wire bytes
SETTINGS(s0, non-ACK)	#0 SETTINGS(12), #1 WINDOW_UPDATE(4), #2 SETTINGS-ACK	43
HEADERS(s1)	#3 HEADERS(s1), #4 DATA(s1,44), #5 DATA(s1,24 sync)	43 + 95 = 138
DATA(s1, first)	#8 DATA(s1,14124 Handshake), #9 RST_STREAM(s1,5)	138 + 14146 = 14284
HEADERS(s3)	#6 HEADERS(s3)	14284 + 9 = 14293
DATA(s3)	#7 DATA(s3,24 INIT_HANDSHAKE)	14293 + 33 = 14326

All five triggers fired exactly once (dispatched_* flags in Session.__init__ ensured idempotency). The 14 KB #8 lands on the wire before #6 HEADERS(s3) — because the client DATA(s1) trigger preceded client HEADERS(s3) on the wire. That is the bug.

The RST_STREAM decoded¶

off=0xd0 type=RST_STREAM(0x03) stream=3 flags=0x00 len=4

payload (4 bytes):
  00 00 00 05                        error_code = 0x00000005 (STREAM_CLOSED)

No GOAWAY, no debug text. Just a targeted stream-3 reset. After the RST_STREAM, CDS closed the TCP connection (ECONNRESET observed by our recv()) — which on iOS/macOS networking stacks can be the application layer intentionally tearing down a logical channel that entered an unexpected state.

STREAM_CLOSED per RFC 7540 §7: "The endpoint received a frame after a stream was half-closed." In our case the stream was freshly opened and not closed — but that only means CDS's h2c parser is not the one raising this; it is CDS's RemoteXPC layer deciding the REPLY_CHANNEL exchange is invalid and synthesising a STREAM_CLOSED to drop it.

Why the current dispatch table fails¶

Reference: the server-side emit order observed by the real iPhone in the Q2 pcap is strict:

iPhone pcap order (s2c):
  #0 SETTINGS(s0)
  #1 WINDOW_UPDATE(s0)
  #2 SETTINGS-ACK(s0)
  #3 HEADERS(s1)              ← opens ROOT_CHANNEL response
  #4 DATA(s1, empty dict)     ← mirrors client's empty dict
  #5 DATA(s1, sync 0x0201)    ← mirrors client's sync
  #6 HEADERS(s3)              ← opens REPLY_CHANNEL response
  #7 DATA(s3, INIT_HANDSHAKE) ← mirrors client's INIT_HANDSHAKE
  #8 DATA(s1, 14124 Handshake) ← THE substantive rsd-info reply
  #9 RST_STREAM(s1, STREAM_CLOSED) ← closes ROOT_CHANNEL after reply

Our iter-2 dispatcher interleaves this with client frames as:

iter-2 wire interleave (client=C, server=S):
  C: SETTINGS(s0)
  S: #0, #1, #2
  C: WINDOW_UPDATE(s0)            ← no trigger
  C: HEADERS(s1)
  S: #3, #4, #5
  C: DATA(s1, empty dict)
  S: #8, #9                        ← TOO EARLY — #8 should wait for s3
  C: SETTINGS-ACK                  ← no trigger
  C: HEADERS(s3)
  S: #6
  C: DATA(s1, sync 0x0201)         ← no trigger (dispatched_data_s1 set)
  C: DATA(s3, INIT_HANDSHAKE)
  S: #7
  C: RST_STREAM(s3, STREAM_CLOSED) ← CDS gives up on REPLY_CHANNEL

Two concrete defects in that interleave:

#8 and #9 fire on client DATA(s1) instead of on client DATA(s3). The big Handshake lands while CDS is still opening REPLY_CHANNEL; CDS expects INIT_HANDSHAKE acknowledgement first, sees the 14 KB payload on the other stream instead, and its channel state machine de-syncs. RST_STREAM(s3) is the diagnostic.
client DATA(s1) arrives twice (empty dict, then sync). Our idempotency flag catches only the first; we never emit #5 (sync). That is strictly less wrong than #8 misfiring (sync is a message_id=0 header-only frame, not critical for rsd-info progress) but it is still a deviation from the iPhone-observed order.

Implications for iter-3¶

Minimal dispatch table fix:

client SETTINGS(s0, non-ACK)   → #0, #1, #2
client HEADERS(s1)             → #3
client DATA(s1), count==1      → #4   (empty-dict mirror)
client DATA(s1), count==2      → #5   (sync 0x0201 mirror)
client HEADERS(s3)             → #6
client DATA(s3) INIT_HANDSHAKE → #7, #8, #9

Two state changes:

Replace the single dispatched_data_s1 flag with a counter (int), keyed on the 1^st/2^nd observation.
Move #8 and #9 from the DATA(s1) trigger to the DATA(s3) trigger.

Open question for iter-3 readers: if CDS sends a 3^rd DATA(s1) after sync (unlikely per Q1+Q2 captures, but not ruled out), the counter-based scheme silently drops it. Acceptable for iter-3 — the iter-0 pcap has exactly 2 c2s DATA(s1) frames and any deviation is new information we want to surface explicitly.

Second open question: does #8 need to land before or after #7 on the wire? The pcap strict order is #7 before #8. Since both are triggered by the same client event (DATA(s3)), emitting them as [#7, #8, #9] (iter-3 proposal) matches the pcap.

Artifacts (committed, redacted)¶

All artifacts under results/ alongside this file. Same redaction policy as iter-01: text logs scrubbed for VM hostname, tunnel ULA, and USB link-local IPv6. Binary files contain no personal data — the send-side is the already-redacted iphone_replay_bytes.py rendered onto the wire, and the recv-side captures CDS's handshake envelope (no UDID inside it, per the Q1 decode).

results/session-01-recv.bin — 217 bytes, CDS's full transmission (preface + 9 HTTP/2 frames ending in RST_STREAM(s3)).
results/session-01-send.bin — 14326 bytes, identical to iter-1's send buffer (same replay corpus, different dispatch order).
results/session-01.log — 5427 bytes, human-readable per-frame log. Starts with "REPLAY mode on — iter-2 event-driven dispatcher armed, no upfront frames", ends with session aborted: [Errno 54] Connection reset by peer.
results/iosmux-listener.log — 138 bytes, listener lifecycle (start + one accept).
results/tunneld-spike.log — 571 bytes, SPIKE-mode tunneld output (tunnel ULA + USB link-local redacted).

How to reproduce¶

On havoc (assumes the iter-2 deployment from step B is still up):

# Restart CDS to drive one new tunnel query.
ssh havoc-root 'killall CoreDeviceService 2>/dev/null || true'
ssh havoc 'devicectl list devices 2>&1 | tail -15'

Pull artifacts with scp and decode session-01-recv.bin with any HTTP/2 framer plus XpcWrapper parser — 5 DATA payloads decode cleanly through pymobiledevice3.remote.xpc_message.XpcWrapper.parse (magic 0x29B00B92).

Status of the iter-2 chain¶

Step A (listener event-driven dispatcher): delivered in commit 74e8d61.
Step B (deployment to havoc): delivered, chain re-armed.
Step C (trigger CDS + capture): delivered, one clean session on record.
Step D (decode + findings): this document.

Iter-2 closes. Iter-3 (dispatch table v2: #8/#9 keyed on client DATA(s3) instead of client DATA(s1), DATA(s1) counter) is the next implementation pass.