Skip to main content

Data Storages

The Data Storage intended to be a light-weight database to store structured data in searchable form. It is not a full-blown relational database, but it serves most use cases for cloud and mobile-first applications. It has built-in access control.

Basics​

The Data Storage is an API for storing JSON based data on the service. A single data storage entry has:

  • Access Control section. Access can be given for administration tasks, data read, and data write operations. Access can be given to individual Management API Clients or Users or a groups of users. Access given currently applies to all data attributes in the Data section. I.e. the resolution of control is coarse. For different use cases, create separate data storages.
  • Owner. The Data Storage may be deleted for example after an User owner is deleted. The Owner does not automatically, nor necessarily have full access to the Data Storage. The owner can be a Management API Client or an User. I.e. ownership does not give permissions, and it is separate from access control. It is good practise to have proper and correct data storage owners.
  • Data section. Accessible only to those who have data read or data write access via the access control section.

A new Data Storage can be created with the creation endpoint. Using it requires a special permission: DATA_STORAGE_CREATE.

Data section​

The data section is a JSON object. It can contain any valid JSON content under unlimited number of sub-keys.

Data read and write operations​

Data can be added either by replacing the whole data section, or by replacing single keys in the data section.

Data can be read by downloading the whole data section as one JSON document, or by downloading individual key values.

Limitations​

The API is limited by the backend storage solution in the following ways:

  • Maximum Data Storage size is currently 16 MB for single datastorage. This includes all metadata. After this limitation has been reached, trying to add data will not succeed.
  • Number of Data Storages is currently not limited.
  • Data keys can contain dot ('.') characters, but they cannot be used in search filters. Therefore it is not recommended to use dots in any keys.

Shall the size limitation be observed, some re-factoring is needed. Splitting data is recommended well before maximum size for best performance.

Remarks on stored data​

  • It is not recommended to store binary data in Data Storage. However, small (couple of kilobytes) base64-encoded binaries are ok. If your needs are different, you need to experiment.

  • It is possible to store pre-encrypted data. It is normally stored in base64-encoded form.

Data Storage Configurations​

DataStorage object configurations:

  • meta
    • created (string) – Read only. Example: 2017-10-20T07:17:17.606Z
    • lastModified (string) Read only. – Example: 2017-10-20T07:17:17.606Z
    • location (string) – Read only. Resource's location URI
  • id (string) – unique ID
  • name (string) – human readable name of the data storage
  • description (string) – additional notes
  • size (integer) – size of storage in bytes
  • ownerId (string) – ID of owner user account. Storage may be removed automatically if owner is removed from the system.
  • adminAccess – list of IDs of user / user group / management api client which can write data.
  • readAccess – list of IDs of user / user group / management api client which can read data.
  • writeAccess – list of IDs of user / user group / management api client which can write data.

Examples

{
    "id": "string",
    "meta": {
        "created": "2017-10-20T07:17:17.606Z",
        "lastModified": "2017-10-20T07:17:17.606Z",
        "location": "https://example.com/resource/112233"
    },
    "name": "string",
    "description": "string",
    "size": 0,
    "ownerId": "string",
    "adminAccess": [
        "string"
    ],
    "readAccess": [
        "string"
    ],
    "writeAccess": [
        "string"
    ]
}

If you have access to multiple data storages, it may be necessary to search for one that matches a filter. The endpoint which is used to list data storages supports using 'datafilter' parameter. If it is used with a valid filter string, the endpoint will return only data storages which match the filter. The filter is given in SCIM compatible format (see RFC-7644).  Complex filters are not supported.

Time stamps filtering is not yet implemented on some servers. For the time stamp querying ISO_INSTANT Date Format must be used. The meta-object doesn't support filtering, but most of the objects have "createDate" and "lastModifiedDate" properties.

Examples:

  • filter=username eq "john" (username value must be identical for a match)

  • filter=name.firstName ne "john" (first name value must not be identical for a match)

  • filter=address.streetAddress co "katu" (address's street name must contain "katu")

  • filter=group sw "gr" (filter groups that start with "gr")

  • filter=id pr (find all objects that has non-empty or non-null ID value)

Time stamp filtering examples:

  • filter=createDate ge "2016-10-20T07:17:17.606Z" (find all objects that were created after defined date)

See the RFC-7644 for more comprehensive examples.