Data Persistence on the GameSparks Platform

GameSparks aims to remove roadblocks from the development process, saving developers time, money and talent whilst enabling them to deepen engagement with their players. In order to maximise every game’s potential, we are producing a series of posts to show all of the best practices involved with GameSparks, ensuring all of our users can take full advantage of our platform, and make the most of its features.

A common question that is asked by our customers is how to persist various types of data on the GameSparks platform. A guide to best-practice is given here:


Player Data


The system Player collection offers two ways to store custom data (in addition to the standard data stored for a player such as currency, virtual goods, etc.) These are designed for storing key-value pairs, although note that the “value” can be a either a simple value, or a complex JSON object, allowing structured data to be stored if required.


The first method for storing data in this way is to use scriptData. This can be set and retrieved using Cloud Code as follows:


Spark.getPlayer().setScriptData(“myCustomData”, { “subKey” : 1 } );


var myCustomData = Spark.getPlayer().getScriptData(“myCustomData”);


Any data stored in scriptData is available via the GameSparks API for authenticated players – that is, it will be returned to client devices in responses to requests including AccountDetailsRequest and ListGameFriendsRequest.


For example, sending an AccountDetailsRequest as follows:


{ “@class”: “.AccountDetailsRequest” }


Would return a response similar to:



“@class”: “.AccountDetailsResponse”,

“currency1”: 0,

“currency2”: 0,

“currency3”: 0,

“currency4”: 0,

“currency5”: 0,

“currency6”: 0,

“displayName”: “Test Player”,

“externalIds”: {},

“reservedCurrency1”: {},

“reservedCurrency2”: {},

“reservedCurrency3”: {},

“reservedCurrency4”: {},

“reservedCurrency5”: {},

“reservedCurrency6”: {},

“scriptData”: {

“myCustomData”: {

“subKey”: 1



“userId”: “5784b7a9e62308e1ec7a51c8”,

“virtualGoods”: {}



PrivateData is similar to scriptData except this data is never sent to clients via normal API requests. This is always retrieved via Cloud Code only:


Spark.getPlayer().setPrivateData(“myPrivateData”, { “secretStuff” : 1 } );


var myPrivateData = Spark.getPlayer().getPrivateData(“myPrivateData”);


So even if the player has some privateData set, only the scriptData will be returned using API calls – the result of an AccountDetailsRequest, for example, would be exactly the same as the example above even after setting privateData.


These two mechanisms are perfect for when you have a small amount of data that is associated with a player, and will be stored for the lifetime of the player (although it may be updated using Cloud Code). The data is structured JSON, so can be used in a meaningful way in code, and scriptData provides an easy mechanism to share this information about a player between clients such as a public profile containing avatar information (e.g. hair color, eye color, etc.)


Storing large amounts of data this way is not recommended however – every time a player is accessed (which is one of the most common things to do on the platform) this data is retrieved, and in the case of scriptData, sent to clients in responses. This reduces performance and increases your bandwidth usage, leading to reduced responsiveness on the client. So only data that is either small in size, or really is needed to be returned to client every time you request the Player data should be stored this way.


So what if you need to store larger volumes of data that is only accessed on-demand? In this case, you probably want to explore….


Custom Collections


Custom collections allow you to store large, complex structured data in MongoDB which can be accessed on-demand through Cloud Code.


There are two types of custom collections. Metadata collections are read-only at runtime. These are perfect for storing metadata (hence the name) in your game, such as static level configuration, descriptions of characters in the game, etc.

Runtime collections, by contrast, can be altered at runtime, and should be used for storing data generated at runtime rather than a static configuration.

Collections can be created on-the-fly in MongoDB – the first time you access a runtime collection, if it doesn’t exist it will be created, e.g.


var query = { “_id”:”12345″ };

var data = Spark.runtimeCollection(“largeData”).find(query);


will work just fine, even if the “largeData” collection has never been explicitly created (although obviously it will return a cursor with no documents in it).


Storing data in a runtime collection is also achieved in Cloud Code:


var doc = { “gameState”: { “gameType”: “deathMatch” } };



Custom collections can (and often should) be indexed for performance reasons. For example, if you always access the data by the field “gameType” then you should index the collection as follows:




which will create an ascending index on the “gameType” field. The calls to ensureIndex for your collections should be placed in the Game Published system script, to ensure they are only called once for each collection (or more specifically, once per collection per game version that is published). There would be a slight overhead placing ensureIndex calls in a regularly executed Event script, for example.


Storing data in custom collections means that the data can be queried and (in the case of runtime collections) modified easily at run-time, and provides all the power and flexibility that MongoDB has to offer.


There are two common mistakes to avoid when storing data in runtime collections. The first is how the data is populated in the first place. It is generally a bad idea to pass up the entire document from the client and persist it to the database. It is much better to either pass up data from clients representing the change in state for the data, rather than the entire data itself.


Persisting the entire document every time (rather than just altering the document to store the new data) can lead to performance problems for your game for two reasons. First, you are passing a lot of data over the network, which takes time and reduces the responsiveness of your game for your customers. Second, saving the entire document every time also means a lot of data being passed to MongoDB unnecessarily, which places unnecessary load on the underlying database. So in effect this is a double-whammy – although it may work for a small number of players, as your player-base increases, this kind of design will become a problem. Try to pass up only the changes that have been made, and persist those into the document instead.


The second mistake commonly made is to attempt to store binary data in a collection. For example, you may think that encoding some binary data using Base64 and saving it to MongoDB is a sensible idea. On the face of it, this sounds fine. However, it breaks the guidelines given above – the data is not formatted in a way that it can be queried and manipulated easily in Cloud Code (i.e. it is not JSON format) and as a result you would almost certainly have to pass the entire data from client to server every time.


If you do need to store binary data in GameSparks, such as uploading a file, you should probably be using…


Binary Assets


Binary assets fall into two categories: Uploadables and Downloadables.


Uploadables are files that, as the name suggests, are uploaded from a client device. These are perfect for storing binary data created by players at runtime, and can later be downloaded to other devices to share file-type data between devices.


To upload data, a client device would first make an API call to GameSparks to retrieve an upload URL:


{ “@class”: “.GetUploadUrlRequest” }


This will return a URL in the response:



“@class”: “.GetUploadUrlResponse”,

“url”: “”



The client device can then upload the binary data to the given URL, and receives an UploadCompleteMessage, which contains amongst other things, an uploadId:



“@class”: “.UploadCompleteMessage”,

“messageId”: “5784cee777588b670617c090”,

“notification”: true,

“playerId”: “56e91d8377588b04932481d8”,

“summary”: “Your upload is complete”,

“uploadData”: {

“fileName”: “51aada5dc51c4c7daf08a4b9a4136be5-portal.jpeg”,

“uploadId”: “51aada5dc51c4c7daf08a4b9a4136be5”,

“fileSize”: 9639,

“origFileName”: “picture.jpeg”,

“playerId”: “56e91d8377588b04932481d8”,

“fileId”: “ABC.12345”


“uploadId”: “51aada5dc51c4c7daf08a4b9a4136be5”



You would then typically store this uploadId along with whatever other data you needed (e.g. playerId, information about the level they were on, or any other relevant metadata) into your own custom runtime collection so you can retrieve it later.

Once uploaded, data can be retrieved (by either the same client or any other client connected to the game) by querying the custom runtime collection to find the uploadId, then sending a GetUploadedRequest:



“@class”: “.GetUploadedRequest”,

“uploadId”: “51aada5dc51c4c7daf08a4b9a4136be5”



The response to this request contains a URL, which the client can then use to download the data:



“@class”: “.GetUploadedResponse”,

“size”: 9639,

“url”: “”



Downloadables, on the other hand, are only available for downloading to client devices – these should be used to store static binary assets for your game, which are common between players. These can be created from within the Web Portal in the Configurator, and downloaded by players in a similar way to Uploadables via a GetDownloadableRequest (passing in the short code configured in the Web Portal for the downloadable asset):



“@class”: “.GetDownloadableRequest”,

“shortCode”: “DL1”



which returns the URL to download the asset from:



“@class”: “.GetDownloadableResponse”,

“lastModified”: “2016-03-10T16:16Z”,

“shortCode”: “DL1”,

“size”: 300109,

“url”: “”



There is one final data store available to the GameSparks platform:


Redis Data


If you want a fast way to store simple data structures as key-value pairs, you also have access to a Redis instance for your game. Redis is a fast, efficient, in-memory data store and as such is significantly faster than MongoDB for storing and retrieving small quantities of data.


Redis is the option of choice when you want to perform fast data manipulation on transient data, such as Set operations, numeric sorting of data, etc.


Access to the Redis datastore is achieved, as usual, through Cloud Code. To store a simple set of values:


Spark.getRedis().sadd(“MySet”, 1);

Spark.getRedis().sadd(“MySet”, 2);

Spark.getRedis().sadd(“MySet”, 1);


This would result in two values (the numbers 1 and 2) being stored in a set against the key of “MySet”.


Redis is very powerful but can have a bit of a learning curve in comparison to other data stores – for a guide on what Redis is capable of and how to use it, the best source is probably the official Redis website at




Hopefully this has given you an overview of the different options available to you for storing both static and runtime data in GameSparks:


Small amounts of player-related data can be stored on the Player collection in either scriptData or privateData.

Larger structured data should be stored in either metadata or runtime collections.

Binary assets and data should be stored using Downloadables and Uploadables.

Transient data that requires fast operations can be stored in Redis.


By spending a little bit of time choosing the correct storage for your data, depending on your specific requirements, you can improve the performance of your game, and avoid scalability issues further down the line. Ultimately this will lead to happier players who are more likely to return to your game again and again.


Who uses GameSparks?