Integrating Mobile Phones as Controllers Into Our Quiz Game

All Connections Lead to Askutron

Screenshot: Lobby screen of the Askutron Quiz Show game

Askutron is a “Quiz Show” PC game we are building as a classic local multiplayer game to be played with your friends in front of a big screen. As such it supports up to 8 players. 8 players who have to somehow be able to “buzz” in their answers. Our first prototype only supported keyboard and game pads, but who just happens to have 8 controllers lying around? Naturally, adding support for phones and tablets was one of the first things we started working on once the basic gameplay was working. But the development process was not as straightforward as you might expect.

First Things First: Unity 3D

Using UNET to Connect Phones to the PC Game Locally

Since Askutron itself is developed with the Unity 3D engine, using it to build the mobile app was the obvious choice. However, up until that point the entire game was taking place on one computer and one screen, it was merely the input that was coming from multiple connected controllers. Somehow the Unity app on the phone needed to communicate with the Unity game running on the PC and send the same commands as the game pads. The app would just display the same buttons that were used on the Xbox 360 controller to keep things simple.

Once more utilizing what Unity already has to offer seemed like the obvious thing to do. So for the first time I started looking into UNET, Unity’s networking system, using the tutorials provided on the official website. While initially it took some time to wrap my head around it the first working version came together quickly. I created the mobile app as just another scene in the game’s Unity project, except it contained only a canvas with the needed buttons and an input field for the player’s name.

Screenshot of the first mobile app

Thanks to UNET phones could simply join the game running on the PC directly via the IP address. To make things even easier this could be automated with the NetworkDiscovery component which uses UDP broadcast. So players just needed to open the app on the phone and would immediately join the game. Alternatively they could use the built-in QR code scanner to join by scanning the QR code displayed in the lobby screen. And if we lived in a perfect world this is where this article would end. However, the real world is a bit more complicated. Of course the setup so far only works if a a few conditions are true:

  • PC and phone can exchange TCP packets directly via network
  • Security settings on the PC allow for direct incoming connections
  • For NetworkDiscovery: App actually receives UDP broadcast packages

Surprisingly, these conditions are often not met. On Windows the player has to explicitly give the application permission via the UAC dialog. Anti-virus software likes to get in the way. Often devices are simply not on the same network. The latter especially is true when not in an environment you control like your own living room. For example, when I showcased the game at the local Unity meetup people were connected to different WLANs, in different subnets or not connected to WLAN at all. But even if the network stars align for you, sometimes it still doesn’t work! During my tests some Android phones would immediately connect, and others just refused to do so for indiscernible reasons. This makes the solution so far not very robust. So what now?

A Short Detour Through the Internet

While this sounds counter intuitive at first, the easiest solution is actually to connect phones via Internet rather than directly! It’s much more likely to find a working Internet connection on any given phone than a properly configured network route that allows direct communication with the PC. Luckily, once again Unity helps us out: Unity Multiplayer. Working seemlessly with the existing network components it adds relay servers into the mix that route TCP traffic through Unity servers when direct network connections are not available.

Integrating Unity Multiplayer (the service) is relatively straight-forward. Rather than hosting the game directly via LAN, you create a “match” and all network traffic is then automatically routed through Unity’s relay servers. This of course requires an active Internet connection and has another downside: It’s not free. To be precise, as of now it will cost you $0.49 per GB of data that goes through the relays.

How much the relay will cost for your game naturally depends a lot on the type of game and the number of players. For Askutron the amount of data is miniscule in theory, since players only send 2 or 3 answers per minute, each consisting of merely a letter (‘A’, ‘B’, ‘X’ or ‘Y’). However, it is not completely clear how exactly that traffic is counted and what data it includes, e.g. does it contain the TCP protocol overhead? Unity offers a calculator to get rough cost estimates based on your expected number of monthly active users, messages per second and message size. It’s up to the developer to decide whether it’s worth it or not.

There are alternatives to Unity Multiplayer such as Photon. Specifically, Photon’s Thunder offering looks very interesting to me and it’s compatible to UNET, so not much code needs to be adjusted. However, I found its pricing model which is based on the number of concurrent connections to be a bad fit for our game. Askutron clearly profits from Unity’s pricing model which is based solely on the amount of traffic.

Issues With the Unity App

Having sorted out the multiplayer stuff I finished the first version of the mobile app based on Unity and it worked splendidly. Until it didn’t. At some point the Android app started crashing, but only in the production build, only when not debugging. Ugh. Unfortunately Unity wasn’t much help at this point, in fact I couldn’t find the reason at all! No crash reports showed up in Google Play Console or in Unity’s own Performance Reporting. Eventually, I gave up, at least for the time being. I used this as an excuse to check out alternatives, because I like shiny toys and, well, because I can.

A Different Approach: ReactNative & WebSockets

My first thought was that writing this simple app that consisted of literally only 7 buttons and a text field (and later an QR code scanner) should be easy to pull off with native code. So, impulsive as I sometimes are, I just wrote the app from scratch for Android using Java. It took maybe half a work day to get a working native Android version of the app with one key difference of course: It couldn’t just use Unity’s Multiplayer service, at least I don’t know how. Instead it now used WebSockets to communicate with the game.

WebSockets

Why WebSockets? Mostly because I wanted to be able to use a web-based client as a buzzer for the game, as well. The idea was that it would be easier to open a website than to download an app. However, we still also wanted to have full apps, because they offer more possibilities like vibration or other haptic feedback in the future. Also browser apps have the annoying downside that they cannot prevent the phone’s screen from turning off.

Thanks to websocket-sharp adding WebSocket support to the game was a matter of maybe 50 lines of code. However, I was now back where I started. The WebSocket server only could be accessed from phones when on the same network, the port was opened and so on. Unity’s relay servers weren’t there to help. Luckily I was able to just offload this onto my brother 😀. Markus quickly developed a small WebSocket relay of sorts that I could use to solve this problem in a similar way as before, simply routing the traffic through an external server. The app first attempts to connect locally, and if this doesn’t work, uses the relay instead. Perfect! Right? Not yet.

Not relying on Unity Multiplayer or other similar services for routing the player traffic of course has the advantage of having full control and completely transparent cost. On the flipside we now have to take care of maintaining and scaling this relay ourselves. So let’s hope the hordes of players won’t overrun our single server.

But the real remaining issue was this: I now had two different apps, one for iOS based on Unity, and one for Android based on Java. They looked slightly different, and I suddenly had to maintain two code bases. One step forward, two steps back?

Enter React Native and Expo.io

React has been the best thing that happened to user interface programming for the Web in ages, in my opinion. So I’ve been wanting to play around with React Native which applies the same principles to native Apps for a while. This seemed like the perfect opportunity to give it a try because this way I could share the same code between iOS, Android and Windows Mobile.

The easiest way to get started seemed to be Expo, and easy it was! I got the ReactNative app running in no time and being able to see changes almost immediately on the device helped tremendously while still figuring out how everything works.

Best of all, Expo comes with a barcode scanner component right out of the box! This definitely saved a lot of time and worries, because I didn’t have to deal with integrating a qr code reading library myself like with the native Android app.

Finally, Expo also allows you to push code changes at any time to apps even after they already have been deployed to the app stores! Thanks to this I was able to quickly fix small bugs or improve the app without having to go through the App review process everytime.

Conclusion

So that’s where we arrived in the end. We’re now using ReactNative for our Android and iOS apps. While it wasn’t exactly a direct path that brought us here I’m very satisfied with the solution. If I had to develop another mobile “companion” app or virtual controller for a Unity game I would probably choose ReactNative again, since it ended up being easier, more convenient and also gives you more access to the phone’s features.