Collections¶

Collections provides a way to define a data structure. A collection is based on a JSON Schema schema which defines the structure of the data. With this schema, you can then insert, update and list the collection entries. These collections are stored inside a MongoDB instance.

Capabilities¶

Capability	Support	Comment
Define the collection data structure with a schema		See Defining data structure with JSON Schema
Historize the changes off a collection entry		See Historize the changes off a collection entry
Create a TTL index to remove inactive collection entries		See Create a TTL index to remove inactive collection entries
Declare an auto-increment field		See Declare an auto-increment field
Interact with a collection		See Interact with a collection
Track the changes of a collection entry with a value		See Track the changes of a collection entry with a value
Limit the access of a collection entries to an user		See Limit the access of a collection entries to an user
Limit the view of a collection form to an user		See Limit the view of a collection form to an user
Access a collection from the front-end		Widgets Collection Table and Form can visualize and modify entries from a collection
Access a collection from scripts		A Collection can be accessed and edited from any JS and Lua scripts
Access a collection from values		You can listen to changes on a collection to set a value with the lasted updated entry each time the collection is updated

Examples¶

Defining data structure with JSON Schema¶

Concept¶

Schemas define the structure of a collection and validate an entry when inserting or updating a collection. They require to follow the JSON Schema specification to be considered as a valid collection schema. Therefore, you should start by getting familiar with JSON Schema before attempting to define a collection.

Note

Some of the JSON Schema annotation keywords are used to add a specific behaviour.

Use title to define a string value that will be used whenever we need to print a schema property anywhere on the front-end. If title is not present, we display the name instead. This is mostly meaningful for the CollectionsRights form component.

Use $comment as $comment: "date" to specify that a number property is meant to represent a datetime in a milliseconds/nanoseconds format. For the CollectionsRights form component, it tells it to render this number property as a datetime picker rather than a number input.

Examples¶

Define a collection

Historize the changes off a collection entry¶

Concept¶

By default, each modification done to a collection is saved into a specific predefined collection called collections_history. This can be disabled for each collection by changing the collection schema.

Be aware that if you disable the historic of a collection which was historized previously, the historic will be completely cleared. On the other hand, when enabling historic, an history entry will be created for each current entry of the collection to serve as a base.

The collections_history collection keeps track of every operations performed on any collection. Each time an insert, update or delete request is done, a new document is added to the historic with the following values :

collectionId: id of the document inserted/updated/deleted in the other collection
schemaId: id (osp path) of the schema of the collection
modified_by: name of the user that made the request
modified_at: time when the request was made
operation: is an insert or update (delete is a soft-delete operation, therefore is marked as an update)
diffs: list of every data differences

Create a TTL index to remove inactive collection entries¶

Concept¶

In the collection schema configuration, specify a TTL higher than 0 seconds in order to automatically create a MongoDB TTL index, allowing you to fully remove an entry that was previously soft deleted.

By default, the TTL is set to 0, which implies that no entry of the collection is truly deleted without a manual operation directly on your MongoDB instance.

Declare an auto-increment field¶

Concept¶

The predefined collections_counter collection associates a counter to another collection. Each time a new entry is created within a collection that have an auto-increment field defined in its schema, we lookup in this collection to get the current counter and increment it for the next insert. Then, this value is used to set all the auto-increment fields before actually creating the new entry.

A collections_counter entry contains :

schemaId: id (osp path) of the schema of the collection
counter: current value of the counter

You can declare which fields should use this counter by editing the collection schema configuration.

Interact with a collection¶

Concept¶

You can interact with a collection by :

insert: Inserts a new entry into a collection.
update: Updates an existing entry.
delete: Soft deletes of an entry. The entry is not deleted but is rather set as inactive. By default, inactive entries are not retrieved by list/get requests.
list: Get the content of a collection.
get: Retrieves one element from a collection.
listen to changes: Listens to this collection updates to set a value with the lasted updated entry of the collection each time the collection is updated.

Examples¶

Insertion¶

When attempting to create a new entry, the data must be validated with the schema in order to ensure that it respects the structure of the collection.

When creating a new entry, these following attributes are automatically added :

__created_at: new entry creation time
__created_by: user that created this entry
__modified_at: time of last operation on this entry (same as __created_at)
__modified_by: last user that modified this entry (same as __created_by)
__is_active: flag to define if this entry is active or not. If not, the entry is not accessible (default true)

Once the entry is created, an entry is added to the history collection to keep track of this operation.

The following modules can create an entry:

Module	Can insert	How
Front-end		Collection Table, Form
Scripts		Scripts
Values

Updates¶

When attempting to update an existing entry, the data must be validated with the schema in order to ensure that it respects the structure of the collection.

There is two different approaches to update a collection:

With full data: you can update an entry by giving the full updated entry. The entry will be match with the _id that must be present in the updated entry. If this id is not present, a new entry will be created instead.
With list of updates: you can update by passing a list of updates to be applied on the given entry. Updates must follow this structure :
- field: name of the updated field
- operation: type of operations: SET, UNSET, PUSH or PULL (PUSH and PULL for arrays only).
- content: updated value

Note

When attempting to pull an object from an array, it’s recommended to have an unique id associated with each object and use this id alone to pull form the array, instead of passing the full object as the update content. Mongo pull update operation may have some issues with matching the element when using the full object as the content, mainly issues that could occur if the order of the fields doesn’t match between the database data and the update content.

Therefore, it’s safer to define an unique id and use it on the pull update. MongoDB should be able to match the correct element with only the id. Your content should look like {<id-key>: '<unique-id-value>'}.

When updating an entry, these following attributes are automatically updated:

__modified_at: time of last operation on this entry
__modified_by: last user that modified this entry

Once the entry is updated, an entry is added to the history collection to keep track of this operation.

The following modules can update an entry:

Module	Can update	How
Front-end		Collection Table, Form
Scripts		Scripts
Values

Delete¶

Deleting an entry from a collection doesn’t completely delete it. It performs a soft delete by setting the __is_active flag to false. This will prevent this entry to be accessible. Once an entry is deleted, it cannot be restored without manually changing the flag inside the Mongo database.

Warning

Since these entries are never fully deleted, the collection will keep increasing in size. All cleaning operation must be manually done directly on your MongoDB instance.

When deleting an entry, these following attributes are automatically updated :

__modified_at: time of last operation on this entry
__modified_by: last user that modified this entry
__is_active: flag to define if this entry is active or not. Set to false

Once the entry is deleted, an entry is added to the history collection to keep track of this operation.

The following modules can delete an entry:

Module	Can delete	How
Front-end		Collection Table
Scripts		Scripts
Values

List¶

You can retrieve all the entries that compose a collection, with or without applying a filter.

Warning

In order to prevent requests with a large amount of results, you will need to define a pageSize limiting the number of result and a pageNumber that you can increase to get the following results. This creates a request where we limit the number of results by pageSize and where we skip the first pageNumber * pageSize, which implies that the pageNumber starts at 0.

In addition to the entries, an additional totalCount is passed into the result. This allows to determine the number of pages by simply dividing totalCount by pageSize. This is here to help you request the rest of the entries.

The following modules offers different ways to retrieve theses entries:

Module	Can list	How
Front-end		Collection Table
Scripts		Scripts
Values

Some modules are capable of maintaining a stream of data when requesting entries of a collection. Therefore, each time an update occurs, the module receives the updated entries of the collection without having to make another request.

The following modules works with streams:

Module	Can be updated	How
Front-end		Collection Table
Scripts
Values

Get¶

You can retrieve a specific entry by using its id or a filter. With a filter, the request will return the first entry that matches the filter.

The following modules offers different ways to retrieve an entry:

Module	Can retrieve	How
Front-end		Collection Table, Form
Scripts		Scripts
Values		Collection owner

Some modules are capable of maintaining a stream of data when requesting an entry. Therefore, each time an update occurs, the module receives the updated version of this entry without having to make another request.

The following modules works with streams:

Module	Can be updated	How
Front-end		Collection Table, Form
Scripts
Values		Collection owner

Listen to changes¶

Some modules are capable of maintaining a stream of data when interacting. Therefore, each time an update occurs, the module receives the updated view of the collection.

The following modules can work with streams:

Module	Can be updated	How
Front-end		Collection Table, Form
Scripts
Values		Collection owner

Track the changes of a collection entry with a value¶

Concept¶

Collections provide a way to interact with values by using an owner. The owner will update the content of the value each time a collection entry changes, if that entry match the owner definition.

In his basic form, an owner is defined by:

schemaId: the reference of the collection we want to based this owner off
filterId: the id of one of the filters off the specified collection. The value is only updated if the collection entry match the filter

In addition to these basic and mandatory properties, you can set other parameters to change the behaviour:

watchList: List of the fields on which to trigger a change of the value when the collection changes. When an entry of the collections changes, the value is updated only if it matches the filter and if the updated or removed fields are present in watchList. If empty or not defined, every change will trigger an update of the value.
scope: Scope of the document transmitted to the value. Three options are available:
- FULL_DOCUMENT: Set the content of the value directly with the full updated document. Meaning you can access a property (i.e. property name of the collection) like content.name. Note that you have no way to know which properties changed with the last update, which is the reason behind the next scope option.
- CHANGES_ONLY: Set the content of the value with a map that contains two entries:
  - updatedFields: a map with keys that represent the properties changed by the update with their updated values.
  - removedFields: list of properties that were deleted.
- BOTH: Mixed of the two previous options. Set the content of the value with a map that contains three entries:
  - fullDocument: complete document of the collection.
  - updatedFields: a map with keys that represent the properties changed by the update with their updated values.
  - removedFields: list of properties that were deleted.
targetProperty: Target property meant to be set as the value content. Instead of setting the value content with the full document, the value is set with the content of the target property only. Target is represented by a JsonPath, letting you precisely choose which field to target. If the target is not found within the full document, an error is set to the value instead. The content is checked in order to guarantee that it matches the value type. If not, an error is thrown as well. Only works when using scope FULL_DOCUMENT, as we need it to access the target.

Note

targetProperty and watchList are not dependent on each other. You can freely watch only changes on a given property (i.e. date) and retrieve another property (i.e. status).

Examples¶

Limit the access of a collection entries to an user¶

Concept¶

In addition to limit access to a collection schema using the resources access rights management, which will prevent any request on a collection, collections rights can be set for an user, limiting the accessible entries of the collection for the specified user. Therefore, these two features have very different purposes. Access rights are meant to give access to any OnSphere Item, while collections rights are meant to limit the access to the entries of a collection.

For each user and collection, you can:

give the right to create, read, update and delete entries
limit access to specific entries by providing a list of identifiers (MongoDB document ids)
limit access to specific properties by providing a list of properties name (must be from collection schema). Updates are only accepted if they modify one or many of the properties defined in this list
define filters on collection properties. Only entries that match the given filter are available

Collection rights are handled with multiple levels.

users rights: First level which define rights for a specific user.
profiles rights: Second level which define rights for profiles. Profiles can be given rights like users and then be referenced in users rights. When rights are present both at the profile level and the user level, we take into account the rights defines in the user level. Therefore, you can override rights set in a profile for a specific user.
services rights: Third and last level which defines services. Services can also be given rights and are linked to a list of profiles. Only users that have one of theses profiles can enter the service. A service contains a list of active users and is meant to only override the rights of the user when he is active in the service.

Take note that theses three different levels are strongly linked to one another. You cannot use services rights without using profiles rights too. Simply put, you cannot enable a level without enabling every level below.

Collections rights must be enabled in collections module configuration. Furthermore, for each level enabled, you need to provide a valid collection schema. Having a schema defined in the configuration is mandatory to be able to interact with these collections from scripts or from front-end widgets, which gives you a way to create and modify entries of these collections. A default configuration for these collections is available in a specific configuration branch called origin/osp-collections-rights-configuration. Theses schemas are validated to insure that they have the required properties. The schemas validator can be found under the schemas section.

Be aware too that since each one of these three levels correspond to a collection that must be define in your configuration, you are free to add additional properties to the schema and, therefore, to the entries of the collections. For example, we verify that the users rights provided schema contains at least a username property, but you could freely add an email property. Keep in mind that all of theses extras properties won’t be utilized when verifying the rights of an user.

Examples¶

Limit the view of a collection form to an user¶

Concept¶

Using the collections rights, you can set to an user a list of forms rights in addition to the others collection specific rights discussed in the previous chapter. This implies that you need to provide the necessary schemas to enable this feature (see previous chapter).

This list of form rights consist in a basic list of names, each name representing one right. These rights are retrieved when requesting a form and used to determine which part of the form the user has access to. This implies that this feature is only usable with a form.

To see how to use them in a form, look at the form widget document.