Designing a Phone-to-Windows Gamepad That Recovers from Disconnects

The hardest problem is maintaining state

A phone can draw buttons and joysticks, but a useful Windows gamepad has to deal with several touches at once, dropped packets, and rapid changes in connection quality. Sending independent button-down and button-up messages can leave the receiver with stale input if one event never arrives.

NATSX Controller treats gamepad input as a complete state snapshot instead of trusting every event in isolation. A shared transport-independent GamepadState model lets the Kotlin Android client and the C# Windows receiver reason about the same buttons, sticks, and triggers.

Multi-touch requires independent input ownership

A player may move a joystick while holding a shoulder button and tapping a face button. A single global touch coordinate would lose track of which finger owns which control.

The Android layer keeps pointer ownership independent across interactive controls. That design supports simultaneous stick, D-pad, trigger, shoulder, and face-button input without collapsing them into one pointer stream. The input layer emits state; it does not decide how networking should carry that state.

A protocol should survive late and duplicated packets

A realtime transport may reorder or duplicate data. The Windows receiver therefore cannot assume that the most recently received packet represents the newest state. The documented protocol includes sequencing and integrity checks as part of authenticated local session semantics.

Stale or duplicate state is rejected. A watchdog can return the controller to neutral input when valid updates stop arriving, reducing stuck-button risk after a disconnect. This is a safety behavior, not merely a network performance optimization.

Selecting a transport without connection thrashing

The connection design considers USB Direct, Wi-Fi, and Bluetooth as possible carriers of the same gamepad state. The preferred order is USB Direct, then Wi-Fi, then Bluetooth, but preference alone is not enough to justify switching while play is underway.

The Smart Connection Manager also weighs latency, jitter, packet loss, silence, cooldowns, hysteresis, and failure history. Only one transport is authoritative at a time, and a candidate must be evaluated before takeover. Otherwise, small network fluctuations could trigger repeated handovers and unstable input.

• Keep input state independent of the connection implementation.
• Evaluate transport health instead of relying only on a fixed priority.
• Reject late and duplicate snapshots.
• Neutralize Windows input after loss of authoritative updates.
• Keep the virtual-controller lifecycle separate from connection reconnects.

What the source verifies, and what it does not promise

The project includes native Android input work, a .NET/WPF Windows receiver, cross-language protocol fixtures, and deterministic connection-policy testing. The documented engineering scope includes USB Direct work as well as Wi-Fi and Bluetooth connectivity.

USB integration can depend on Windows drivers and device-specific constraints. The case study describes the architecture and implemented verification scope; it is not a claim that every Android phone and every unsigned Windows driver combination will work out of the box.