System Design and Architecture

Cloud Application Architecture

SVMP relies heavily on cloud APIs to manage the life cycle of the virtual smartphone devices. This lets us accomodate a variety of different deployment environments with ease. At a high level it resembles a traditional desktop-oriented thin-client system. It also has elements that resemble voice and video over IP services like XMPP or SIP, since we reuse several of those technologies via WebRTC for transporting the remote desktop video and audio data.

The main components are:

The client application on users’ phones or tablets
The SVMP Overseer that handles authentication, database managment, and virtual machine deployment
The SVMP Server the clients connect to
The virtual device “gold image” and per-user set of persistent data volumes
The cloud controller that serves the API of the underlying cloud

Additionally, an optional reverse proxy/load balancer can be used in the implementation.

Client Application

The front-end SVMP client application is a simple, unprivileged application installed on a user’s Android or iOS phone or tablet. This application is quite similar in scope to existing remote desktop clients for VNC or RDP servers. Users configure server connections in the client’s menu, enter their login info, and are presented with a display of the remote virtual smartphone.

The similarities end when the user begins to interact with the remote device. Instead of emulating a mouse and keyboard, the SVMP client captures native touch-screen events, sensor inputs like accelerometer and gyroscope, location information for mapping apps, and some two-way inter-app messaging for notification popups and Intents. This enables interacting with remote mobile apps in the same ways as with local apps using gestures like pinches and swipes, tilting and rotating, getting GPS location, and even letting apps launch other apps (e.g., to dial a phone number or open a URL).

All of these inputs are packaged into SVMP protocol messages and sent up to the server.

The client also creates a WebRTC peer connection with the virtual device. Video and audio output from the virtual device flows back across this link and is played back to the user, completing the loop.

SVMP Overseer

The SVMP Overseer is the first internet-facing components of the system. This is what goes on your DMZ.

It is responsible for receiving login requests from users, performing authentication against a database or external identity service, and creating and managing virtual machines on the cloud. It operates a RESTful API that clients and SVMP Servers can access. It also includes the Web Console for managing users and the HTML5 client.

SVMP Server

The SVMP Server is the second internet-facing component of the system. This also goes on your DMZ.

It is responsible for receiving connection request from users that have already authenticated with the Overseer, initiating virtual machine deployments through the Overseer, and routing input messages from client connections to the SVMP daemons running inside the virtual devices.

The Server is designed for modularity and scalability so multiple instances can be run behind a load balancer to service large numbers of simultaneous users.

Virtual Device Image

This image encapsulates the core operating system of the virtual smartphone device, including the kernel, boot files, and Android framework services. Different cloud and virtualization platforms handle this image in different ways, for example, a Glance image in OpenStack or an AMI on Amazon EC2.

User Data Volumes

The user data volumes provide the storage for users’ “roaming profiles.” These are typically backed by block storage volumes (e.g. OpenStack Cinder, Amazon EBS) provided by the cloud infrastructure.

These volumes are used for the /data filesystem tree within the Android-based virtual devices. Android has the useful property of keeping all of a user’s data, installed applications, settings, and preferences in this filesystem making it easy to package and move this data from one virtual device to another.

Cloud Controller

The cloud controller is not a part of SVMP software itelf, but performs an important role in the SVMP system. The SVMP server is configured with credentials that allow it to perform VM creation, deletion, and maintenance operations on the underlying cloud infrastructure.

When new virtual phone VMs need to be created, the SVMP server makes a request to the cloud controller to create a new VM cloned from the current device image. It also orders the controller to attach the user’s data volume to the freshly created instance. When the user has logged off and the VM fallen idle, the SVMP server finally tells the cloud controller to delete the instance to make room for future connections.

Virtual Device Structure

AOSP can compile and run on x86 processors out of the box, but it’s missing any ways to interact with it remotely. SVMP adds a set of virtual input devices, video streaming output, and some other customizations to enable a rich remote access experience.

The main components are:

The central SVMP daemon
Touch-screen input injection
Virtual sensors
Location update pub/sub
Notification and Intent forwarding and re-broadcast
Virtual framebuffer, video encoding, and streaming

SVMP daemon

Also referred to as the SVMP eventserver, this daemon is the primary entry point of client user input to the virtual device.

The code for this module resides in the SVMP/android_external_svmp_eventserver repository and checks out to the path external/svmp/svmpd in the source code tree.

Touch Input

The touch input events for the smartphone VM are generated on the client app as the user interacts with the video being shown and forwarded to the smartphone VM using protocol buffers. The code for the protocol buffer structure definitions resides in the SVMP/svmp-protocol-def repository.

The SVMP eventserver , as described above, handles these touch input events and injects them into the Android Smartphone VM as they are received. The code for this can be found in the SVMP/android_external_svmp_eventserver repository in the handleTouch() function in src/org/mitre/svmp/events/EventServer.java.

Sensors

The sensor events for the smartphone VM are also generated on the client app and forwarded using protocol buffers. The SVMP eventserver forwards them to the local listening /dev/svmp_sensors socket on the smartphone VM. The SVMP eventserver code dealing with sensors can be found in the SVMP/android_external_svmp_eventserver BaseServer.java repository in the sendSensorEvent() function in src/org/mitre/svmp/events/BaseServer.java.

The SVMP HAL module libsensors listens on the svmp_sensors socket then processes the actual sensor events. The libsensors HAL module code can be found in the SVMP/android_device_mitre_svmp repository.

Location

The location events are communicated both ways between the client app and the smartphone VM using protocol buffer messages. Any subscriptions intents requested by apps from the LocationManager on the Smartphone VM are passed back to the client app. The client then passes the Location information to the Smartphone VM.

The code for dealing with location events on the client side can be found in the SVMP/svmp-android-client repository under src/org/mitre/svmp/client/LocationHandler.java.

Similarly, the code for the smartphone VM side handling of location events can be found in the SVMP/android_external_svmp_eventserver repository under src/org/mitre/svmp/events/LocationHandler.java.

Intents and Notifications

Currently, SVMP supports exchange of intents between the SVMP client app and the smartphone VM. SVMP client application supports receiving the ACTION_DIAL intent when a phone number URI is pressed in the smartphone VM and shows any notifications received from the smartphone VM.

Additionally, the client app allows forwarding URLs with ACTION_VIEW intent to the smartphone VM allowing for opening the URL inside the VM.

The code for handling intents and notifications in the client app is part of the SVMP/svmp-android-client repository and can be found under src/org/mitre/svmp/client/NetIntentsHandler.java and src/org/mitre/svmp/client/SendNetIntent.java.

The code for handling intents and forwarding notifications on the smartphone VM is available as part of the SVMP/android_external_svmp_eventserver repository in src/org/mitre/svmp/events/IntentHandler.java.

Video

At the lowest level, the SVMP VM image displays video output to a Virtual FrameBuffer (VFB) device from the Linux kernel as opposed to a real video device. When frames are writen to the framebuffer (the VFB in this case), the Android surfaceflinger library generates a VSYNC event. When a VSYNC event occurs, frames are copied from the VFB after the screen has been fully updated. Each frame is fed into the WebRTC subsystem which is responsible for video encoding and streaming.

WebRTC is used to stream video for several reasons. First, complex network topology issues are handled very well. The inclusion of STUN, TURN and ICE protocols provides excellent support for traversing NAT and firewalls. Second, support for several resilient video codecs, such as VP8, provide adaptation to less than ideal network conditions common on mobile devices. SVMP includes a custom WebRTC video input driver that reads from the VFB for input.

On the receiving end, things are much simpler. Since the source stream from the VM appears as a standard video streaming source, nothing special is required outside of receiving a standard WebRTC video stream. This architecture provides great flexibility. Currently video streaming is functional on Android, iOS and HTML5 clients.

Versioning

SVMP version 1.0 and earlier are based on Android 4.0.4 “Ice Cream Sandwich”. Versions 1.1 through 1.4 are based on Android 4.2.2 “Jelly Bean”. Version 2.0 is based on the 4.4 or “KitKat” version.

Going forward, the SVMP major version number will increment when we upgrade from one Android “letter” series to the next. For example, and SVMP 3.x will correspond with the “L” version of Android.