Re-Architecting the GameSparks Portal

Over the past few months, the GameSparks tech team and I have been working hard to bring you the next generation of our developer portal. Here’s a run down of what drove this next stage in the portal’s evolution, along with technical details of how it was implemented.

The Issues

When we initially designed the portal application, it was built as a traditional Java MVC application. This single application was responsible for authentication, database access, and building the HTML required to present the forms required to manage the GameSparks configuration.

Over time, as we’ve packed more and more features into the platform we started to find issues with this approach:

  • UI/Rest API Disparities – The REST API we ran in parallel had disparities with the portal UI. There were things you could do in the UI for which a REST endpoint wasn’t available. We have a lot of customers who integrate GameSparks into their publishing pipelines, and they need to be able to interact with the platform entirely through an API.
  • Development Overhead – Adding functionality required changes to be made in multiple locations within the code base, slowing development. For example, to add a new property to a configuration object required a change to the REST API, the portal business logic, and the HTML templates.
  • VPN’s Maintenance Overhead – We have over 20 separate runtime clusters in different cloud providers. The NoSQL REST API required a VPN to each runtime cluster to proxy the request over. This arrangement carries a significant overhead for managing these VPN’s. But it also introduces latency issues. For example, a US customer making a NoSQL query to a US runtime cluster would need traffic routing via Europe and then over a VPN back to the US to execute the query.
  • Monolithic Design – A single application approach imposes a “whole server” deployment process for updates. Our deployment process runs a comprehensive suite of tests against all components being deployed as a single block, even though much of the redeployed code had not changed.
  • Uneven User Experience – Though not inherently an issue with the platform’s architecture, we’d come to realize that with the introduction of more advanced functionality, the portal’s UI was straining to deliver a smooth and streamlined user experience and an overhaul was required:
    • Complex Configuration Forms – We identified a lot of forms that were more complicated than they needed to be, often because fields existed that were redundant for specific game configurations.
    • Segmentation – We reviewed workflows for setting up and configuring segmentation for objects and saw that this very useful and clever feature was nowhere near as clear and accessible to users as it should be.

Original Architecture

OldArchitecture

There were 3 core components in the old architecture. Each of these components was deployed within a single (clustered) web application.

  • NoSQL REST API
    • Provided a Mongo-like query interface into the MongoDB of the runtime cluster.
    • All requests were passed over the VPN between our management tier and the runtime cluster.
    • Customers will be aware of intermittent issues, where collection would not appear in the list. This was always due to some kind of VPN failure.
  • Config REST API
    • A REST API to allow the game structure to be changed.
    • Relatively new API, but one that utilizes reflection to always keep up to date with the current game object model.
  • Portal
    • The HTML UI for configuring the game through the browser.

Addressing the Issues

  • Harmonize Portal UI/REST API – We decided to remove the UI responsibility from the server and build a single page EmberJS application which would be a consumer of RESTful services. This would mean, going forward, if we wanted to add a feature to the UI, it would have to be in the REST API and any UI/API disparities could not arise.
  • Facilitate Faster Development – The config REST API already built itself based on the current object model of the server. We felt we should be able to extend this and have the Ember application build itself using the same process. This has now got us to the happy position where if we add a field to a configuration object, with the addition of an annotation, the REST API updates itself and the field is automatically added to the HTML. This will speed things up for us enormously.
  • Drop VPN’s – To remove the VPN requirement, we removed the central NoSQL proxy and created a new service that would be deployed in each runtime cluster to provide the same functionality against a local MongoDB. US customers making NoSQL queries to a US runtime cluster will no longer be liable to latency issues.
  • Introduce Micro Services – To break up the monolithic design, we introduced micro services. The NoSQL service is a good example of a micro service, where a single application performs a single function that could be deployed individually. With this step, we also identified two areas of concern that we felt should be split into separate services:
    • The user management and single sign-on functionality could be deployed in a single “Auth” application, one solely responsible for this crucial functional area.
    • Configuring the game remains in its own application, consisting of the original REST API plus a few additional endpoints to allow for some of the previous “UI Only functions”. Doing this usefully leaves the Config REST API mainly untouched – with a few additional endpoints added to allow for some of the previous “UI Only functions”.
  • Streamline User Experience – We then used EmberJS to create a single page JS application which would provide the UI for the portal:
    • The new single page UI schema affords a much more up-to-date and slick user experience that both facilitates and streamlines user workflows.
    • Careful UX design means functional purpose is made more readily accessible to users, such as when setting up complex segmentation of your game’s configuration objects.
    • The build process for this UI relies heavily on the swagger specifications that each application generates allowing it to (mainly) self-update when new apiMethods or types are created.

New Architecture

NewArch

  • Auth/Users
    • Stores all details of our customers.
    • Provides user management functions (registration, change password, and so on).
    • Provides a JWT based single sign-on solution.
  • Config
    • Uses SSO tokens provided by Auth to authenticate users.
    • Provides all endpoints required to manage a Game.
  • NoSQL
    • An instance of this application is deployed on each runtime cluster.
    • Uses SSO tokens provided by Auth to authenticate users.
    • Provides a Mongo query interface into a game’s Mongo database.
  • Ember Portal
    • Delivered as static files to the browser via CloudFront.
    • Uses all available REST services to provide the portal user experience.

This fundamental shift to separate out individual concerns across the application results in easier and quicker development of new features. By allowing the deployment of individual services without having to deploy everything together will step up our delivery time.

Secondly, with the new architecture in place, a REST client can fully interact with the platform providing the full feature set the UI provides.

Introducing the new UI

A number of new features have been added to the new portal, and there will be follow up posts to describe this functionality.

Some of the key drivers in the new design:

  • It’s important that we don’t structurally change the navigation (as much as possible), because nobody wants to feel lost in a system they have been using for years
  • Segmentation should be made simpler to understand and clearer to use.
  • Forms should be quicker and easier to complete by removing fields not relevant for the current game configuration.
  • Some customers liked the old dark theme, but others had requested a “white” version. Make the ‘skin’ customizable and provide a “light” and “dark” theme.
  • NoSQL should allow multiple collections to be queried at the same time, rather than being limited to a single collection.
  • The extremely powerful Experiments feature (our A/B testing) should be promoted to its own standing.
  • Direct queries on the API Stream should be possible without having to first create a “chart”.
  • Make the UI consistent so it’s easier to use and saves time for users. Use standardized visual cues and affordances like red buttons for destructive actions, green for create actions, all to provide an intuitive interactive experience.

Here are a few screenshots of the new portal, initial feedback has been excellent and once we’re through our final beta, we’ll be able to retire the old portal.

I hope this has given you some insight into why these changes have been made. The next generation of the GameSparks portal is now available through our open BETA release, explore for yourself at https://portal2.gamesparks.net

Thanks,

Gabriel Page, CTO GameSparks

 

Overview Page

OverviewLight

Overview Page (Dark Theme)

OverviewDark

Cloud Code Editor

CCEditor

Test Harness

TestHarness

Experiments

Experiments

  • Otto Lopes Braitback de Olivei

    Great news!
    Amazing work Gabriel and GameSparts team! The new UI is excellent.

  • Brian Allen

    looks nice guys great job!

  • Great work. So which region does a user connect to now if no specific region is specified for a snapshot?

Who uses GameSparks?