Skip to content

Understanding iDevice protocol layers

Overview

In this article we're going to review how communicating to an iDevice (iOS/iPadOS/...) actually works.

In order to understand it all, we are going to review:

Once we understand each part, we'll discuss how pymobiledevice3 is structured to handle communication to all these moving parts.

usbmuxd

The usbmuxd (USB Multiplexer Daemon, though technically it supports both USB and Wi-Fi) daemon is responsible for two main tasks:

  • Detecting iDevices in both your LAN and via USB.
  • Proxying traffic to any TCP port onto the target device.

It provides this api by exposing a unix domain socket /var/run/usbmuxd.

In order to interact with usbmuxd, you may use the following commands from your shell:

# List connected iDevices
pymobiledevice3 usbmux list

# Start a TCP server listening on port 2222, transferring all received traffic to
# TCP port 22 on the device
# NOTE: you may use `-d` in order to start this process as daemonized
pymobiledevice3 usbmux forward 2222 22

# If you are using corellium, you may want to forward all the commands to use their
# remote usbmuxd, listening on the hard-coded port 5000
# As such, (almost) all of pymobiledevice3's commands, may accept an optional `--usbmux` option 
pymobiledevice3 usbmux list --usbmux 10.11.1.2:5000

The same can be accessed via python API:

from pymobiledevice3.usbmux import list_devices


async def main() -> None: 
  # Listing all connected devices locally
  usbmux_devices = await list_devices()

  # Or if using corellium, or any other remote usbmuxd
  usbmux_devices = await list_devices('10.11.1.2:5000')

  # Now may list them and establish a TCP connection to any port on-device
  for device in usbmux_devices:
      # The serial can either be the device UDID if it's connected via USB, or its Wi-Fi mac address
      if device.serial == '11223344':
          sock = await device.connect(22)  # return a pure python socket object

On a macOS workstation this daemon is builtin. On other platforms however you'll need an external tool for that:

Okay, so now that we understand usbmuxd main purpose is to simply connecting to TCP ports in an iDevice, but where will we wish to connect to? Probably to lockdownd.

lockdownd

lockdownd is a daemon that listens on the hard-coded TCP port 62078. It has 3 main purposes:

  • Query general device information (ProductVersion, UDID, ...)
  • Pairing
  • Accessing lockdown services

You may query the device information via lockdownd using LockdownClient from python:

from pymobiledevice3.lockdown import create_using_usbmux, create_using_tcp


async def main() -> None:
  # If we avoid passing the `serial` option, we'll get a `LockdownClient` instance 
  # of the first available device 
  # By default, pymobiledevice3 attempts to pair with the device, if it was not already 
  # paired (presenting a "Trust/Don't Trust" dialog). We use the `autopair=False` when 
  # we don't want to block on that operation
  lockdown = await create_using_usbmux(serial='11223344', autopair=False)

  # Corellium anyone?
  correlium_lockdown = await create_using_usbmux(serial='11223344', autopair=False, usbmux_address='10.11.1.2:5000')

  # If the device can be found in our LAN, and we know its address, we simply connect to it
  # Please note the device does not allow pairing over LAN, so we must first pair it over USB
  lockdown = await create_using_tcp('192.168.2.7', autopair=False)

  # An example for accessing a lockdown attribute
  print(lockdown.product_version)

As you may have noticed, we mentioned the iDevice can be interacted over Wi-Fi. For that, we'll need to first enable this feature over USB:

# Turn it on
pymobiledevice3 lockdown wifi-connections on

# Or off
pymobiledevice3 lockdown wifi-connections off

Now the device will use the bonjour protocol in order to announce its availability over the LAN. You may query these available devices using:

# It announces itself using the `_apple-mobdev2._tcp.local.` name
pymobiledevice3 bonjour mobdev2

Of course this can also be done in python in asyncio API:

from pymobiledevice3.lockdown import get_mobdev2_lockdowns


async def main() -> None:
  async for ip, lockdown in get_mobdev2_lockdowns():
      print(ip, lockdown.product_version)

However, as long as we don't pair, we can only access a pretty small pool of data. We won't delve into how the pairing is actually performed, since this is a top-level guide, but we'll tell you that after a key-exchange, followed by a user prompt to trust our client, we can access everything lockdownd has to offer.

If you're interested, Jon Gabilondo wrote a fantastic thorough article about all the process. You may read about it in here: https://jon-gabilondo-angulo-7635.medium.com/understanding-usbmux-and-the-ios-lockdown-service-7f2a1dfd07ae

We may interact with lockdownd using any of the lockdown subcommands:

# Pair with the device
pymobiledevice3 lockdown pair

# Or unpair
pymobiledevice3 lockdown unpair

# Or just view general information
pymobiledevice3 lockdown info

This is all nice and all, but usually the more interesting stuff can be found in the lockdown services.

Lockdown services

The lockdownd daemon can also be used to spawn other device services and access their data. It does so according to a hard-coded plist built into one of lockdownd sections. We may examine them using:

segedit /path/to/lockdownd -extract __TEXT __services /tmp/lockdown_services.plist

In that plist we'll see different service names, where each of them can be started using lockdown's StartService protocol command.

The response to the StartService command we may then connect to using usbmuxd as discussed earlier. Each of these services implements its own unique protocol. You may try to play around these services using:

# You may try any other service name in order to study and mess with its protocol messages
pymobiledevice3 lockdown service com.apple.mobile.heartbeat

And of course this is also available from code:

from pymobiledevice3.lockdown import create_using_usbmux


async def main() -> None:
  # Create the LockdownClient instance 
  lockdown = await create_using_usbmux()

  # Get a handle to the service
  service = await lockdown.start_lockdown_service(SERVICE_NAME)

  # Attempt to send and receive messages from it
  await service.sendall(b'hello')
  response = await service.recvall(20)

Many of the services exposed by lockdownd, are already implemented in pymobiledevice3.

For example consider the following examples:

# View device syslog
pymobiledevice3 syslog live

# Reboot the device
pymobiledevice3 diagnostics restart

# And the list goes on and on...

In order to access the different services, the project is structured in the following from: pymobiledevice3.services.service_name.ServiceClass.

For example:

from pymobiledevice3.lockdown import create_using_usbmux
from pymobiledevice3.services.os_trace import OsTraceService


async def main() -> None:
  lockdown = await create_using_usbmux()

  # Print all syslog line entries, whereas `OsTraceService` as a wrapper to the 
  # `com.apple.os_trace_relay` lockdown service, and the `syslog` method is a protocol operation
  # for that service
  async for entry in OsTraceService(lockdown).syslog():
      print(entry)

DeveloperDiskImage

Some of the more interesting services we can interact with for automation purposes can only be accessed from an external image, called the DeveloperDiskImage (or DDI for short). Once we mount it, lockdownd searches for services in /Lockdown/ServiceAgents, in an attempt to increase its possibilities for lockdown services.

As of iOS 15, Apple added the "DeveloperMode" option, forcing users to first enable it before they can mount the DDI. Assuming, your iDevice doesn't have a pin-code defined, you can simply enable it from CLI:

# Enable it
pymobiledevice3 amfi enable-developer-mode

# Or just query its state
pymobiledevice3 amfi developer-mode-status

As any other lockdown service, this of course can be accessed from python API:

from pymobiledevice3.lockdown import create_using_usbmux
from pymobiledevice3.services.amfi import AmfiService


async def main() -> None:
  lockdown = await create_using_usbmux()
  amfi = AmfiService(lockdown)
  await amfi.enable_developer_mode()

Once the DeveloperMode is on, we can mount the DDI. This however has very much changed in many aspects in iOS 17. For short, we'll just say you can simply use the following CLI command:

# This will automatically deduce the correct way to mount the DDI onto your device
# Please note this will require network activity for mounting on iOS 17
pymobiledevice3 mounter auto-mount

Once this is done, you may access a much wider variety of features in the device, such as process management, debugging, simulate locations and much more.

To make it clear which of pymobiledevice3 commands require the DeveloperMode to be on together with the DDI being mounted, we put it all in the developer subcommand.

For example:

pymobiledevice3 developer dvt launch com.apple.mobilesafari

DVT

One of the more interesting developer services is the one exposed by DTServiceHub. It is using DTX protocol messages, but since it mainly wraps and allows access to stuff in DVTFoundation.framework we called it DVT in our implementation (probably standing for DeveloperTools).

We don't delve too much into this protocol, but we'll say in general it allows us to invoke a whitelist of ObjC methods in different ObjC objects. The terminology used by DVT to each such ObjC object is called "channels".

To access this different object use the following APIs:

from pymobiledevice3.lockdown import create_using_usbmux
from pymobiledevice3.services.dvt.instruments.dvt_provider import DvtProvider
from pymobiledevice3.services.dvt.instruments.screenshot import Screenshot


async def main() -> None:
  # Create a LockdownClient instance
  lockdown = await create_using_usbmux()

  # Use it to create a DVT provider (DTX lifecycle is handled automatically)
  async with DvtProvider(lockdown) as dvt:
    # Use it to invoke methods on a DVT channel
    dvt_channel = Screenshot(dvt)
    open('/tmp/screen.png', 'wb').write(await dvt_channel.get_screenshot())

Looking for an unimplemented feature/channel? Feel free to play with it (and submit a PR afterward 🙏) using the following shell:

pymobiledevice3 developer dvt shell

NOTE: The full list of the methods that can be invoked on a DVT channel can be found by looking at all ObjC classes in DVTInstrumentsFoundation.framework implementing the DTXAllowedRPC protocol. There is an existing Anubis rule I use to diff new methods against the existing ones.

RemoteXPC

Starting at iOS 17.0, Apple made a large refactor in the manner we all interact with the developer services. There can be multiple reasons for that decision, but in general these refactor main key points are:

  • Create a single standard for interacting with the new lockdown services (XPC Messages, Apple's proprietary IPC)
  • Optimize the protocol for large file transfers (such as the dyld_shared_cache)
  • Perform authentication with the device only once, and connect to each service via an encrypted tunnel from different places

However, Apple implemented it in a very confusing way, and seemed to regret some of its own steps along the way, so we'll cover all the chaotic mess we now can use to connect to the device.

Firstly, we'll just say the protocol messages are layered as follows:

  • HTTP/2 messages for more efficient parallel file transfers
  • XPC messages (Apple's proprietary IPC)
  • Remote Service Discovery protocol (or RSD for short)

However, the connection broker used for this communication is remoted instead of lockdownd, using its own completely different pairing logic, leading into two different "Trust/Don't Trust" dialogs (though they appear exactly the same).

Since all this communication is IP-based, but without any additional exported TCP port from the device, usbmuxd can't help us here. Instead, starting at iOS 16.0, when connecting an iDevice, it exports another non-standard USB-Ethernet adapter (with IPv6 link-local address), placing us in a subnet with the device's remoted.

As we've said, this communication is non-standard and requires either:

  • macOS Monterey or higher
  • Special driver on your linux/Windows machine

Spoiler Alert: Apple may have regretted this, since starting at iOS 17.4, they added the CodeDeviceProxy - a new lockdown service, allowing us skip all the steps this special driver is required for.

You can use the following shell command in order to query RSD instances over bonjour (over the USB Ethernet device specifically):

pymobiledevice3 bonjour rsd

We don't delve too much into what RSD exposes. For that you may read in:

https://github.com/doronz88/pymobiledevice3/blob/master/misc/RemoteXPC.md

In short, it will allow us to both pair and start a VPN tunnel onto device, where we can access both lockdown and all the other RemoteXPC services. As we previously mentioned, starting at iOS 17.0, this is the only way to access the developer services.

You'll have to start this tunnel using a privileged process, since it requires creating a TUN/TAP device:

# This will create a QUIC VPN tunnel to the connected USB device. 
sudo pymobiledevice3 remote start-tunnel

# Apple later switched from QUIC to TCP tunnels, but my SSLPSK seemed to cause troubles to some workstations
# However, using TCP tunnels is much faster especially since the TCP stack is implemented by the OS, instead
# of the QUIC which is implemented in pure python code
# If the following command works for you, it will create MUCH faster tunnels
sudo pymobiledevice3 remote start-tunnel -p tcp

If you're using a corellium instance, since you cannot pair it first over USB, they patched the iOS platform to expose another service over remoted to allow remote pairing. So, assuming you are on the same LAN as the device, you may use the following command:

sudo pymobiledevice3 remote pair

Once we have established pairing with the iDevice's remoted, we can now also establish trusted tunnels over Wi-Fi as follows:

sudo pymobiledevice3 remote start-tunnel -t wifi

This is nice and all but, as previously mentioned, Apple may have regretted this remoted separate pairing (perhaps thanks to EU ruling because of the special drivers needed), because iOS 17.4 added a new lockdown service allowing us to just establish this trusted tunnel over our existing lockdown connection. This means no extra pairing process is required - and the cherry on top is that it's always TCP tunnels, making it MUCH faster.

To do so, simply use:

# You may also add a `--usbmux` option for working with a corellium instance
# And of course, since `lockdownd` can be accessed over Wi-Fi, this can also be done remotely
sudo pymobiledevice3 lockdown start-tunnel

Anyhow, once the tunnel has been established, you'll get an output that looks like this:

Identifier: <DEVICE-UDID>
Interface: utun5
Protocol: TunnelProtocol.QUIC
RSD Address: fdc3:16b1:5cac::1
RSD Port: 52954
Use the follow connection option:
--rsd fdc3:16b1:5cac::1 52954

You may simply add this extra --rsd option to any existing pymobiledevice3 subcommand for all its services to be available once more (including the developer ones) as follows:

pymobiledevice3 developer dvt launch com.apple.mobilesafari --rsd fdc3:16b1:5cac::1 52954

This is of course also available via a python API:

from pymobiledevice3.remote.remote_service_discovery import RemoteServiceDiscoveryService
from pymobiledevice3.services.os_trace import OsTraceService


async def main() -> None:
  # Assuming 
  host = 'fdc3:16b1:5cac::1'
  port = 52954
  rsd = RemoteServiceDiscoveryService((host, port))
  await rsd.connect()

  # Both LockdownClient and RemoteServiceDiscoveryService implement LockdownServiceProvider, 
  # meaning you can simply use this instance as any other LockdownClient instance
  async for entry in OsTraceService(rsd).syslog():
      print(entry)

All the tunnels above need root/admin, since they create a kernel TUN/TAP interface. There's one more option that needs no root at all: add --userspace to any developer command to establish the iOS 17+ tunnel in-process over a pure-python network stack (PyTCP). There's no separate tunnel process - the flag builds the tunnel inside the command and tears it down on exit:

pymobiledevice3 developer dvt ls / --userspace

See the dedicated guide for what it supports and its limitations (most notably: the device address is reachable only from that process, so it can't be handed to external tools like lldb). This is of course also available via a python API, through the UserspaceRsdTunnel handle:

from pymobiledevice3.remote.userspace_tunnel import UserspaceRsdTunnel
from pymobiledevice3.services.os_trace import OsTraceService


async def main() -> None:
  # serial=None picks the first USB device; pass a UDID to target a specific one
  # It's an async context manager (or keep the handle and use `await tunnel.aopen()` / `await tunnel.aclose()`)
  async with UserspaceRsdTunnel(serial=None) as rsd:
      # `rsd` is a RemoteServiceDiscoveryService - use it like any other LockdownServiceProvider
      async for entry in OsTraceService(rsd).syslog():
          print(entry)

Confused by all these "start-tunnel" permutations? Don't blame yourself - it's very confusing especially since there isn't only one way to achieve cross-platform for iOS 17.0-17.4.

Because of that, and because it requires starting a privileged process to each tunnel, we made this process much simpler by implementing our own version of remoted called tunneld. To start it use the following:

sudo pymobiledevice3 remote tunneld

Now tunneld will always search for newly connected devices via all available manners of connecting to them (both USB and Wi-Fi) and just start establishing tunnels to them on its own.

You can then request pymobiledevice3 to work over the existing tunnel instead by adding the --tunnel option as follows:

# You'll get a prompt asking to which device you wish to connect to, if there are several active tunnels
pymobiledevice3 developer dvt launch com.apple.mobilesafari --tunnel ''

# Or you can supply a specific UDID to not be prompted
pymobiledevice3 developer dvt launch com.apple.mobilesafari --tunnel '11223344'

This is of course also available via a python API:

from pymobiledevice3.tunneld.api import get_tunneld_devices
from pymobiledevice3.services.os_trace import OsTraceService


async def main() -> None:
  rsds = await get_tunneld_devices()

  # We can now simply use the returned list of RSDs as any other LockdownClients
  async for entry in OsTraceService(rsds[0]).syslog():
      print(entry)

Other python service examples

The best way to search for examples is via the pymobiledevice.cli module.

Each submodule represents a CLI subcommand. You can copy each subcommand implementation and simply replace the service_provider variable with any other LockdownServiceProvider (either RemoteServiceDiscoveryService or LockdownClient).