Version 1.2.X - Bald-ox¶
Version 1.2.7¶
Release date: August 2024
Upgrading warnings¶
Warning
The aggregation process now force the following files to have a linked value.ospp
of type BOOLEAN
for :
slave.modbus
controller.onvif
broker.mqtt
New features¶
Retrieve an attribute from user preferences in a script - #1320¶
Get the value associated with an attribute from the currently authentified user preferences. If no value is found, a defaultValue
is returned instead. defaultValue
is not a mandatory parameter. If not specified, null is returned instead.
Look at scripts collection methods for further details.
Front - Support row colors object for Collection table widget - #1316¶
Using SEVERITY
render on CollectionTable
widget view, it is now possible to display rows with colors based on user input.
See colored rows for collection table example for more information.
Documentation¶
Examples improvements: Review and correct examples using
osp-alarms-web-configuration
prerequisites branchExamples improvements: Remove warning when using
osp-web-configuration
aboutalarms
folderImprove the auto-generated documentation for modules:
OPC UA/DA
BACNet
Onvif
IP-RCT
Bug fix¶
#1322: Between filters on dates where not working as wanted when targeting a field present in a subarray of the entry.
#1298: Wrong types were accepted for some values associated with connection state. Now they are forced to be boolean.
#1319: Maps updates did not display correctly when multiple people were using them at the same time.
Keycloak: Configured trusted certificate were not used.
Security fix¶
Onvif: upgrade cxf to 4.0.5
Dispatcher: upgrade docker compose to 2.29.2
Proxy: upgrade openresty to 1.25.3.2-0
Version 1.2.6¶
Release date: July 2024
New features¶
Front - Disable labels for widgets - #1309¶
It is now possible to disable label for value (Value
), gauge (Gauge
) and alarm severity (AlarmSeverity
) widgets. When given false
value, it will disable the label.
Front - JSON Form control type integrator decision - #1315¶
When using form.web
, we are able to provide a UI schema. It allows us to map it to any field in a database. Sadly, when we face a field of type ['string', 'boolean', 'number']
, it will fallback to boolean
render.
We are now able to specify for these type a render override using options
with key render
with possible values :
string
boolean
integer
number
See override basic types documentation for more information.
Documentation¶
Improve the auto-generated documentation for modules:
OPC UA
BACNet
Maps
Bug fix¶
#1312: Form widget were not detecting and handling conflicts on collections elements correctly.
Composer¶
#1294: It was impossible to rename folder because they were locked by the composer.
The non-json file generated by playbook template were not locked properly.
The monitoring of file and directory changes has been refactored and should use less resources.
#1306: OnSphere path completion was removing following characters.
Security fix¶
Version 1.2.5¶
Release date: July 2024
New features¶
Front - Improve value subscription and object browser usability - #1302¶
Value subscription (ValueSubscription
) and object browser (ObjectBrowser
) widgets have knowledge whether the user can update a value or not. Previously only BOOLEAN
type were updatable. Now all types can be updated when user has write
rights and value is managed by variable module.
Documentation¶
Milestone documentation improvement: Add example and it corresponding explanation.
Bug fix¶
#1304: Bacnet
BacnetNetworkConfigurationEntity
configured in themodule.bacnet
allow to define custom port.#1149: Bacnet output allow to define priority with 16 as default. The previous priority was always 1 (MANUL_LIFE_SAFETY) which disallow other to set the value.
#1301: Generate report action requires to send all the data the report need in the action payload. This could lead to a massive payload that the websocket can not handle.
To help with this issue, when dealing with data from a collection, you can now send only the ids of the collection entries rather than the full entry.
To do so, you can use the
collectionsListById
method inside the report to query the full entries based on a list of ids.#1302: Value subscription and object browser overflowing text no longer break layout
#1300: Form Collection input now save correctly when using freeInput.
#1296: A miss configuration was rendering the user account management inaccessible.
To solve it without updating, the following step are necessary:
Connect to the admin page of Keycloak
Go to the clients tab
Select the account-console
Edit the Home URL and Valid redirect URIs to change OnSphere to onsphere.
Edit the Client authentication to set it to false.
#1297: Communications was not connecting properly when a PASSWORD authentication was used.
#1297: Communications was not using parameters TLS/SSL properly.
#1303: Keycloak user list was limited to 100, which could cause unexpected behavior when another use the list.
The module web is getting the default dashboard, api key and other information from this list.
The module script users.list() is generated from this list.
Security fix¶
CVE-2024-6387: Update configuration dispatcher ssh server.
Update influxdb to 2.7.6
Update mysql to 8.0.38
Update openresty to 1.25.3.1-5
Version 1.2.4¶
Release date: June 2024
Upgrading warnings¶
Warning
The old persistent variable store has been removed. It is not possible to upgrade from any version prior to 1.1.0 without first upgrading to 1.1.x.
New features¶
Front - Uniformize onClick evaluation context - #1287¶
Code evaluation with AlarmSeverity
and BasicInputOutput
widgets are aligned with other widgets. They support osp.navigate(path)
and osp.evaluate(code)
. See Alarm table filter update with alarm severity widget.
Front - WebView widget supports dark/light theme variants - #1285¶
Following support for theme variant colors (#1189), we expand this feature to WebView
. The source
parameter supports string
parameter as well as dark
and light
keys with string content.
Documentation¶
Example - Creation of an hysteresis for alarm generation - #1244¶
New use case example of an hysteresis handling the problem of temperature ranging around a certain threshold. This is used to prevent flooding alarms. See Usage of a template to create an hysteresis.
Bug fix¶
#1290: In
http
script context, avoid reading body when no content is available (such as204 No Content
).#1281: Communications was not using the provider port correctly when the password authentication was used.
#1282: The default value of variables were not checked which caused wrong behaviors and errors.
#1283: The modification of
detached.scripts
and the corresponding script was causing the callback to be lost (Aka the script was not called when the callback value change).Callback/Output: Add debug log message on output trigger to show which value trigger it.
#687: Add constraints (range) on values (value.ospp) depending on their types.
Reports: Misconfigured transactions caused rare unexpected errors.
Collections: Misconfigured transactions caused rare unexpected errors.
Documentation: Add missing file and explanation on examples, add warning and note on points of interest and fix typos.
Documentation: Improve explanation of the value concept for the retention.
During Portainer service
Update
withRe-pull image
activated, API listing services was receiving a value larger than integer.
Security fix¶
Update Keycloak to 24.0.5 GHSA-69fp-7c8p-crjr: Keycloak exposes sensitive information in Pushed Authorization Requests (PAR).
Version 1.2.3¶
Release date: June 2024
New features¶
Front-end - Table export date format and timezone support - #1222¶
It is now possible to specify a date format when exporting table in Excel. Using formatterExport
in views.web
or schema.web
options will set cell formatting for Excel.
By default, export now uses the user locale timezone to export. To disable it for a column, use disableTimezoneExport
in views.web
or schema.web
options.
Bug fix¶
Slow aggregation: The aggregation time was improved by reworking a validation. It is now 7 times faster.
This was due to a regression introduced with version 1.2.0.
Version 1.2.2¶
Release date: May 2024
New features¶
Aggregator - Show messages on configuration push - #1264¶
The aggregators can now display some message during the push. For example:
Update GIT successful [
Module [modules.snmp-trap.snmp-trap-1] is successful and has some messages.
[
{
"message" : "Usm files are defined but the snmpV3 engineId is not define on 'module.snmp-trap'",
"level" : "WARNING"
}
]
]
Bug fix¶
#1272: Render
NUMBER
for table views now allow to filter numeric values correctly inside an alarm or collection table.#1273: Column filters on columns with render type
ARRAY_COUNT
now effectively filters the rows based on the field size.Video: Streaming was not working as expected since version 1.2.0. This release fixes the problem by bringing kurento (both library and docker) back to version 6.14.0.
We’re not really happy with the use of kurento for advanced functionality, and we won’t be updating this version now. A full analysis and refactoring will start soon to create a new video workflow with advanced features.
#1264: Snmp-trap aggregator display a warning when a USM is define but the engineId is missing.
#1267:
AlarmSeverity
widget was missing value context to show value properly.#1256: Onvif was not checking the door capabilities correctly
The door controller now checks door capabilities for event types (sabotage, alarms, physical state) and raises an error on failure.
Patch
add_keycloak_email_and_profile
: The patch to update therealm.keycloak
file was creating a component with a key namePROFILE_KEY
instead oforg.keycloak.userprofile.UserProfileProvider
.The patch
1.2.2
andpatch-keycloak-fix-profile-name
fix this error.
Security fix¶
CVE-2022-25647: Deserialization of untrusted data in Gson.
Minors changes¶
#1268: Improve IP-RCT alarm/restore logging when alarm is being handled by OnSphere.
Version 1.2.1¶
Release date: May 2024
Upgrading warnings¶
Warning
The Keycloak users now require an email when define on the users.keycloak file.
Warning
On some install, Keycloak will not restart after the update and show the error Cannot invoke "org.keycloak.models.ClientModel.addScopeMapping(org.keycloak.models.RoleModel)" because "accountConsoleClient" is null
.
This is due to the missing client account-console
in the realm.keycloak file. The patch #add_missing_client_on_realm_keycloak
add the missing client if necessary.
New features¶
Scripts - Generate random UUID - #1259¶
You can now generate a random UUID in a script. For more details, see generate UUIDv4 ids.
Support insecure TLS MQTT broker connections - #1257¶
When attempting TLS connection on a broker which does not comply to certificate requirements (common name or subject alternative name), the communication will be stopped.
It is now possible to set insecure
flag to true
in broker.mqtt
file. This will allow communication to a MQTT broker which does not comply to standards.
External certificate to connect to the broker are still needed in the configuration to validate the certificate.
Cancel and close options for collection table - #1259¶
It is now possible to cancel or close entry when in entry creation or edition. These button are showed on the right side of the toolbar where submit, conflicts, errors indicators are located.
Bug fix¶
#1209: If the keycloak configuration cannot be applied, the retry will not be done indefinitely. This was growing the database and did not change anything as an underlying error with the configuration needs to be changed. The module state will now report
RUNNING_WITH_ERROR
.#1252: Due to an optimization done on the messaging via websocket in the 1.2 release, the layers were sometimes not completely loaded or missing. This has been fixed and the usage of more than one layer is now working well.
#1262: The composer was using too much CPU to watch the change on files.
#1239: Layouts widgets no longer trigger re-rendering which might never load children widgets.
The dispatcher will not generate commit with message
Adding auto-generated files
without content anymore.#1259: Lazy loads user preferences to unclutter build artifact.
#1259: Sets proper z-index for modal, menu and drawer to allow combo input showing options.
#1259: Precache charts and schematics widgets (file size too large for default precache values).
#1259: Collection and journal table shows alternate color to improve readability.
#1259: Handle emptying field to transmit data as collection request to unset field.
#1260: Keycloak update was not restarting properly after the update.
#1263: Shared resources between two modules caused both to reboot when a non-common resource was changed.
For example, if two modules use the resource
data.csv
and the second module also has the resourcespecific.csv
. If thespecific.csv
changes both modules will be restarted.Modbus was not logging properly when failing.
Security update¶
Keycloak update to version 24.0.3.
Version 1.2.0¶
Status: Final version
Release date: April 2024
Upgrading warnings¶
Warning
The osp-configuration-dispatcher will clean up the /osp/run/config/tmp
folder, which is growing due to an unnoticed issue (#1205). This clean-up process may take some time on the first run if your stack exist from a long time.
Warning
The osp-help module no longer exists and has been replaced by the sphinx documentation you are currently reading.
Warning
Previously templates variables in the same file as used was not interpreted. This is not the case anymore. See issue to details.
Warning
The files beginning with a .
are not parsed anymore see changelog for details
New features¶
New documentation using Sphinx¶
We are thrilled to announce the release of our revamped documentation, providing you with a comprehensive and enhanced resource for your needs.
Complete Overhaul: We are delighted to provide you with a completely revised documentation that has undergone extensive improvements for clarity, accuracy.
Removal of Docker osp-help: As part of our ongoing efforts to streamline and optimize our services, we have removed the Docker osp-help. The module is no longer needed because the documentation is now online.
Error Corrections: Our team has diligently addressed numerous errors to ensure the accuracy and reliability of the documentation. Your feedback and our commitment to excellence have played a crucial role in identifying and rectifying these issues.
New reports template OSP method to retrieve user preferences info - #1251¶
New report template method getUserAttribute
retrieve the value of a given attribute of the given user preferences.
Report data from analytics module - #1248¶
You can now fetch data directly in a report from the module analytics, given a query and query filters. This allows you to display analytics data, like iterate over the to create a csv report, acting as an export for one or more historized values.
See report template methods to get more info on how to use this method.
New form day and time pickers - #1227¶
New day and time picker components are available. Theses two component allow to pick a day (from 0 to 6) and a time (formatted as string like ‘00:00’). More information about them in the form widget documentation.
Adding Prometheus metrics to module collections - #1092¶
Prometheus metrics are now exposed for the module collections. Look at the module documentation for the list of available metrics.
Define rights on a collection for an user - #1092¶
Rights can be applied to a collection for a specific user. These rights are stored inside collections making them updatable at any time, in contrary to access rights that are defined in the configuration. Anyway, these two features have very different purposes. Access rights are meant to give access to any OnSphere Item, while collections rights are meant to limit the access to the entries of a collection.
You can define rights for a specific user, but you can also create profiles rights
linked to one or more users. A third level called Services
represent rights that are applied on top of the user rights. Services
are meant to be entered and leaved by a user, giving him specific rights while he is active in the service.
See Limit the access of a collection entries to an user for more information.
Enable historic for a collection - #1092¶
Historic used to be enable by default and for any collection defined in your configuration. This behavior is still present by default but you now have the possibility to determine if a collection should be historize or not by setting historizeCollection
flag in schema.collections.
Note that if you disable the historic of a collection which was historized previously, the historic will be completely cleared. On the other hand, when enabling historic, an history entry will be created for each current entry of the collection to serve as a base.
See Historize the changes off a collection entry for more information.
Define a TTL for inactive entries of a collection - #1092¶
Deletes in collections doesn’t actually remove the entry from the database but instead make it inactive by setting a flag to false.
This behavior is still present but now you have the possibility to define MongoDB TTL Indexes. When defined, each inactive entry that is older than the given TTL will be fully removed from the collection.
See Create a TTL index to remove inactive collection entries for more information.
Default charset for script http request - #1113¶
HTTP requests charset defaults to standard MIME type definition if not provided (i.e. application/json; charset=iso-8859-1 will use ISO-8859-1)
Improve front-end keycloak integration - #1123¶
Previously, keycloak was integrated with external library. We now integrate keycloak-js
directly to enhance development experience.
Schematic supports variable injection in .drawio
files - #1125¶
To reuse your existing .drawio
schematics, you can now provide variables
. These are accessible globally in your .drawio
file with the $[variable-name]
format.
It is compatible with osp-variables
. Evaluation works with global variables being overridden by osp-variables
variables.
Support of Integer value for Bacnet #1183¶
Bacnet Integer value are now support for read and write.
Dashboard search behavior improvements #1161¶
Integrators can specify the display behavior of a dashboard to be shown in dashboard.web
file with:
EMPTY_SEARCH
: (default) the search bar will show the dashboard when no search text is provided or when search content matches its contentSEARCH_REQUIRED
: the search bar will only be displayed when there is a matching search textHIDDEN
: the dashboard will never be displayed in the search bar area
Better support for themes, colors and references colors in widgets and dashboards #1189¶
With 1.2 release, we did an overhaul of colors provided through dashboards.view
to support specific use cases.
A color can now be:
A HEX color string which is fixed for both theme and never changes
An object with
reference
which points to a Value allowing for dynamic color changes withfallback
value should the value not being availableAn object with
dark
andlight
supporting both previous color definition (hex color or reference color)
Each widget previously supporting colors can now immediately use this new color definition.
Severity colors support also supports either a HEX color for both dark and light theme as well as theme specific color. Value references is not allowed for severity as it could cause end-user confusion.
In order to improve integrator jobs, schematic
widget also supports theme specific schematic. Background color is also retrieved from schematic definition.
See dynamic colors example for in-depth example on what is achievable.
Snmp value conversion¶
The conversion from Snmp data to OnSphere value is improved to allow a better handling of the OctetString. The OctetString can be handle as string with are encoding or as raw binary data.
The integrator can for example directly convert 0x41CD851F
to 25.69
.
See SNMP for details.
OPC UA subscription support #838¶
OPC UA subscription is now supported by the OPC UA client module ! This new feature improve greatly the trafic on the network when reading data that do not change often.
Maps - Include zoom restriction parameter #1111¶
It is now possible to use the parameter minZoom
and maxZoom
in the mapsWidgetSettings
to determine the boundaries of the zoom parameter.
OnSphere help link¶
A link in the front-end side menu allows you to access OnSphere documentation based on your version. It can be disabled with disableHelpLink
option.
Collection table icon override #1251¶
This allows integrators to override collection table icons.
Tables export icon override #1251¶
This allows integrators to override tables export icon.
Beta features¶
Warning
Features currently under evaluation may be modified in the next version without prior notice. If you are using these beta features, we welcome your feedback and experience. Your input is crucial to refining and improving these features before their official release. Please share your thoughts and experiences with us, and contribute to the evaluation process.
New module OPC-DA - #1052¶
The OPC-DA module allows retrieving and set data. See documentation for more information.
New module Milestone - #1089¶
A new module is available to interconnect with https://www.milestonesys.com/ this interconnection. This module only support basic feature for yet.
See documentation of the module or the feature documentation
Playbook templates¶
We’re pleased to inform you that our OnSphere product now supports playbook templating. This latest update introduces compatibility with the Nunjucks format, allowing you to utilize its capabilities within the templates. The templating feature empowers you to generate configurations based on your inventory, employing straightforward rules. This enhancement provides a more dynamic and efficient method for customizing configurations, making OnSphere even more versatile to meet your specific needs.
See playbook templates for details
Theme overriding¶
In order to help integrator meets their clients requirements, we support experimental theme overriding. This allow control over many components presentation and styling.
See theme overriding for details
Prometheus metrics for script¶
The script module expose a metrics to evalute the execution time of each script.
See osp-scripts for details.
Bug fix¶
#1255: Schematic tooltip is correctly placed when toolbar is disabled.
#1241: AlarmSeverity widget properly retrieves color based on its value and not ID.
#1214: The map was actualizing its points to often when other value change.
#1238: Properly set font size upon value widget overflow.
#1236: Schematic widget
moveTo
andsetLayer
works as expected.#1226: Error when resizing the maps widget.
#1234: Now check correctly if a collection update request is valid with the user collections rights filters.
#1228: Basic I/O widget OnClick no longer needs value subscription.
#1230: Switching forms in a CollectionTable now switch the view correctly.
#1227: Form component date picker now shows the date correctly upon loading the form.
Form widgets now won’t attempt a collection request when using
Mixed
destination.Mixed
now only apply toValue
andAction
.#1220: BasicInputOutputValue widget now display the value content of a integer value without having to set a variableText.
#1191: Menu conditions have now access to user information (like user.username).
#1203: Properly check all leaf node for value/file for dashboard.view. With layout components, only top-level widgets were checked. Some were also missing from checks being done. We now use JSONPath query to retrieve all fields linked to value or file.
#1048: Viewport switch to fullscreen in light mode shows message properly.
#1104: Schematic tooltip is properly placed.
Reduce the critical concurrent period with storage of messages on disk by saving outgoing messages during save on disk task.
#731: Every possible SNMP PDU are now identified. Error message for PDU types not handled by SNMP-TRAP module has been changed to better understand why it is not handled.
#1148: Webhook value now properly transform the type to the one defines on the value.
#1166: Removed alarms were not deduplicated correctly on some rare occasion.
Fix two unnoticed java leak on module alarms / SMTP
#1110: Template are now supporting integer, decimal, boolean and object.
#1003: Form select and submit are now properly displayed and no longer relies on form handler events
Module web and maps will now properly reload the values when a change occurs on the configuration.
#1169: Using broad resource inclusion (like “root/”) was leading to the module using configuration not for him.
#1195:
alarms.untag
on the js scripts was not untagging the alarms properly.#1182:
PROMETHEUS_PORT
environment variable allow to change default prometheus port (9100) to avoid collision with remote-connector.#1200: The dispatcher timeout was not using the env variable but an hardcoded value of 30 minutes. Use the default at 15 minutes now.
#1204: The composer was not cleaning properly the inside tmp folder. This is now done at each restart of the module.
When RabbitMQ was disconnected, some messages can be lost in specific condition.
#1180: Fix of a configuration parsing validation error. The dispatcher validate only the first object, e.g. this json will not raise an exception:
{ "messagingConfiguration": { "clientId": "osp-video-1", "host": "rabbit" } } {"messagingConfiguration": {"clientId": "osp-video-1", "host": "rabbit"}, "loggingConfiguration": {"moduleLogLevel": "TRACE", "scriptLogLevel": "WARN", "externalLogLevel": "WARN"}}
This problem has now been solved and the dispatcher will issue a detailed error.
#1206: OPC UA values were not correctly transformed in OnSphere types.
#1221: The generation of the rights tree might run into a stack overflow.
A configuration with the rights with id root.test.element and root.test<something else>.element might generate an infinite loop depending on the sorting of the id.
If the sorting result in :
root.test<something else>.element
root.test.element
Everything will be fine.
If the sorting result in :
root.test.element
root.test<something else>.element
A stack overflow will be generated.
#1223: The generation of the rights was not using the correct parent for the include and exclude. This was leading to wrong right when inheritance was done.
#1233: The composer file association was missing for
stack.external-services
which was disabling the formatting and pretty printing.#1249: When changing a template file to a non-template file or removing a value from a dashboard, the dispatcher did not clean up obsolete dependencies between files. This was generating error like :
When a template is removed (No reference left on
dashboard.view
pointing to root.${LOCATION}.data as it was replace by root.corminboeuf.data):{ "type": "MISSING_FILE", "description": "A required file is missing", "fileOnError": "root/corminboeuf/dashboard.view", "missingFile": "root/${LOCATION}/data/value.ospp" }
When a value is removed (No reference left on
dashboard.view
pointing to root.data):{ "type": "MISSING_FILE", "description": "A required file is missing", "fileOnError": "root/corminboeuf/dashboard.view", "missingFile": "root/data/value.ospp" }
#1253: The access values declared in callback.ospp were not aggregated properly, causing missing value errors.
Composer
The composer has been extensively reworked to improve usability and stability.
#1176: The composer is now refreshed when schemas are updated, whether they are created or deleted. Previously, only updates of json-schema were taken into account.
#1135: Multiples improvements about the composer implementation (better usage of Vscode API for files handling)
#1058: Several fixes to improve the compositor’s ability to handle very large configurations.
File validation is now fully asynchronous and supports larger configurations.
Improved path usage for Windows compatibility.
Corrected a large number of memory leaks when reloading vscode.
Security fix¶
CVE-2023-6378: Update logback dependency to 1.4.14.
GHSA-xpw8-rcwv-8f8p: Update netty dependency to 4.1.104.
GHSA-4265-ccf5-phj5: Update apache-compress dependency to 1.26.0
GHSA-hmr7-m48g-48f6: Update jetty to 11.0.16
Configuration changes¶
File beginning with . are not parsed¶
Starting now, files that begin with a . will not be parsed and cannot be used as configuration files.
Warning
No migration script is provided as hidden file will be used later for playbook template.
Single file template improvements¶
Single file template now supports default variable definition inside the same template file, for example :
{
"name" : "",
"description" : "",
"type" : "BOOLEAN",
"templateId": "${template-id-var}",
"templateVariables" : {
"template-id-var" : "root.modbus.template"
}
}
Warning
Prior to this update, the functionality behaved incorrectly by disregarding this variable, which was not the intended behavior. Please be cautious about potential configuration changes.
See single template for details.
Minors changes¶
osp-configuration-dispatcher was verifying the certificate on SWARM but had the information locally, now the verification is done locally, this led to a small performance update (-100/150ms boot time).
Webhooks requests now support authentication using HMAC.
Version changes¶
Note
All internal libraries and dockers are updated to the latest available version of 29.10.2023 - updating to the latest version of OSP is strongly recommended for security reasons.
Internal dependency update¶
Snmp++ update to version 3.5.1.
Docker compose update to version 2.24.6
External modules update¶
Keycloak update to version 24.0.3.
Mongo update to version 5.0.24.
InfluxDb update to version 2.7.5.
Mysql update to version 8.0.36
Material Icon Fonts updated on 23.04.2024