Title: Release Notes For Version 11.0.3

0.1 Table Of Contents

1 Upgrading to 11.0

1.1 Software Requirements

To run Calpendo version 11.0, you should use:

1.2 Before Upgrading to 11.0

If you are upgrading from version 9.0.x or earlier, then:

If you are upgrading from version 10.0.x, 10.1.x or 10.2.x to 11.0, then:

Note that this version drops support for the WebDAV protocol. See API Changes for more details.

Note that this version drops support for properties called ‘count’. Such properties cause parsing problems and are no longer allowed. See Count Properties Prohibited for more details.

1.3 How To Perform The Upgrade

1.3.1 Overview

Upgrading from any version to the latest (including applying bug fix updates) should be done with the following procedure:

1.3.2 Changes to hibernate.cfg.xml

1.3.2.1 If you are using MariaDB

Exprodo Software use MariaDB for all our production and development databases. We use test systems using both MariaDB and MySQL. You will need to make modifications to your hibernate.cfg.xml file regardless of which flavour of database you use, but the changes required differ.

For use with MariaDB, you will need to change:

Change the line that reads:

<property name="hibernate.cache.provider_class">org.hibernate.cache.EhCacheProvider</property>

so that it reads:

<property name="hibernate.cache.region.factory_class">ehcache</property>

Change the line that looks like this:

<property name="hibernate.dialect">com.springsolutions.exprodo.core.server.persistence.ExprodoMySQL5InnoDBDialect</property>

and change it so it says this:

<property name="hibernate.dialect">com.springsolutions.exprodo.core.server.persistence.ExprodoMariaDBDialect</property>

Look for the line that starts like this:

<property name="hibernate.connection.url">jdbc:mysql://

and change the “mysql” to “mariadb” so that it starts:

<property name="hibernate.connection.url">jdbc:mariadb://

and finally, look for the line that starts:

<property name="hibernate.connection.driver_class">

and make sure that it reads:

<property name="hibernate.connection.driver_class">org.mariadb.jdbc.Driver</property>

1.3.2.2 If you are using MySQL

For use with MySQL, you will need to change:

but it’s also worth verifying the correct values for:

Change the line that reads:

<property name="hibernate.cache.provider_class">org.hibernate.cache.EhCacheProvider</property>

so that it reads:

<property name="hibernate.cache.region.factory_class">ehcache</property>

If you are using MySQL for your database, then it is recommended that you should use MySQL Connector/J which can be downloaded from dev.mysql.com/downloads/connector/j/. We do not distribute the MySQL driver, only the MariaDB driver. This is due to the MySQL Connector/J licence.

The jar file for MySQL Connector/J needs to be placed either in $CATALINA_HOME/lib or in the Calpendo/WEB-INF/lib directory, alongside all the other jar files we include in the distribution. It is generally better in $CATALINA_HOME/lib to make it easier to install new versions of Calpendo.

From version 8.0 onwards, the driver class name has changed. The old driver still works (at the time of writing) but is deprecated and may stop working. You should therefore switch to the new name. This is a setting in hibernate.cfg.xml that looks like this:

<property name="hibernate.connection.driver_class">com.mysql.jdbc.Driver</property>

and if using version 8.0 or later of the driver, you should use the new driver class:

<property name="hibernate.connection.driver_class">com.mysql.cj.jdbc.Driver</property>

The settings for hibernate.dialect and hibernate.connection.url that you are currently using should not need changing. They should look like this:

<property name="hibernate.dialect">com.springsolutions.exprodo.core.server.persistence.ExprodoMariaDBDialect</property>
<property name="hibernate.connection.url">jdbc:mysql://HOSTNAME/DBNAME?useUnicode=true&amp;characterEncoding=UTF-8&amp;</property>

where the URL will have appropriate values for HOSTNAME and DBNAME, and you may have different settings for the URL after the ‘?’.

1.4 Downgrading If Things Go Wrong

If something goes wrong during an upgrade, and you want to downgrade to a previous version, then it is important that you follow the right method.

If you are downgrading across patch versions, for example from 11.0.x to 11.0.y, then you can generally do that without making any changes to the database. Just change the program files to the version you want, and it will automatically handle any database changes required.

However, if you are downgrading across major or minor versions, for example from 11.0.x to 10.1.x, or from 11.0.x to 9.0.x, then it is very important that you follow the sequence described below.

If an upgrade fails for some reason, then the database may well be partially upgraded. Automatic recovery from a partial upgrade is not supported.

The general procedure for downgrading is:

As you can see, for this procedure it is essential that you took a backup of the database before the upgrade began, otherwise downgrading in the event of a failure may not be possible or may be difficult.

Examples of sequences for downgrading that are not supported and likely to result in a broken system are:

  1. Don’t bother loading a backup of the database. Just change the program files to an old version and reboot.

  2. Don’t bother dropping the database and recreate it. Just load a backup of the old database on top of an upgraded or partially upgraded database and run an old Calpendo.

Recovery from these kind of errors can be difficult and possibly labour intensive. Please avoid this by following the prescribed downgrade procedure as laid out above.

2 New Features

2.1 Workflow Database Update Phases

Workflows contain a “Database Workflow Event” which allow you to take actions when something is created, updated or deleted in the database. Previously, there was an option on the event called “Vetoable”, which was probably mostly ignored and not very well understood.

This has been replaced in version 11.0 with a new drop-down letting you choose which “phase” of the database change your workflow should execute in.

There are three phases, which are called:

Before version 11.0, there were only two phases available, and they were controlled by the “vetoable” flag. If you selected “vetoable”, then your workflow would run in what is now called the “during transaction” phase. Otherwise, it would run in the “after transaction” phase.

So what are these three phases, and when would you use each of them?

First, consider what happens when a change is made to the database as a result of a user request. First, the request arrives, and, in some cases, it has some changes made automatically to the request. For example, by incrementing a version number on the biskit being changed. Then, a new transaction is started with the database, and the biskit is saved to the database. Workflows are then triggered, and they may make other changes that are saved in the same database transaction.

If nothing goes wrong, the transaction is “committed”, and at this point everybody will see all the changes made in that transaction.

If something goes wrong during the processing, such as a rule deciding a booking cannot be allowed, or if a workflow issues a “veto” action, then the transaction is “rolled back”. In this case, everything that was saved in the transaction is reverted to its prior state.

If you create a Database Workflow Event and choose that it should operate in the “Before transaction” phase, that means:

Since changes made are not written to the database and do not trigger other workflows, it means it is much faster to execute workflows in the before-transaction phase that any other phase.

After all before-transaction updates have been made, then the whole set of changes made are merged into one. From this point on, it will then appear as if the user had requested the whole set of changes to be made themselves.

A transaction is then started, and all the merged changes are written to the database.

Any database workflow events marked for executing in the during-transaction phase will then run.

Once the during-transaction updates have completed, the transaction is committed and then after-transaction database workflow events will be executed.

For example, suppose the user creates a new booking. Suppose further that:

In this case, the before-transaction event will be triggered by the booking creation. The workflow updates the booking. However, the booking update is rolled in to the booking creation so that other events only see that the booking was created with a price already on it.

The after-transaction event would run once in this scenario, for the booking creation. Before version 11.0, this event would have been triggered twice: once for a booking create, and another for a booking update.

2.1.1 Recommendations

Some general rules of thumb about when to use which database update phase:

2.2 Repeat Bookings Behaviour Change

The way repeats are handled in version 11.0 has radically changed, with the code behind it being completely rewritten.

Previously, when you created a repeat booking, a single row would be created in the bookings table. It would include columns to say how the booking should repeat. When you ran a report or looked at the bookings calendar, the repeats would be automatically expanded so that it displayed “virtual” bookings for the later repeats.

Whenever the next repeating instance fell into the past, then a new non-repeat booking would be created, and the repeat booking would be modified so that it indicated the start date was whatever was the next instance of the repeat.

This resulted in various difficulties:

This has now all changed. When you create a repeat booking, then there are multiple rows inserted into the bookings table, one for each repeating instance up to a configurable time into the future. There’s also a single row added to a new “repeats” table, and all repeat bookings reference that repeat.

When a repeating instance falls into the past, nothing is done. There are no changes at that time. Instead, there is a process of extending the repeating sequence into the future to maintain the expansion out to the configured time into the future.

From a user’s perspective, this will be the same as before. Except that:

A “custom” repeat type is one which allows a sequence of bookings to be associated with one another without a prescribed way of identifying when the repeats should be. Instead, the dates on which repeats occur are manually assigned.

In previous versions, when a repeat instance fell into the past and a new non-repeat bookings was created, then it was assigned a value to a “sequenceId” property. This was the ID of the original repeat booking, and was a way in which one could tell that historical bookings came from a repeat.

Such bookings no longer record whether they were daily or weekly etc. Consequently, the process of upgrading to version 11.0 will automatically associate all bookings with the same sequenceId with a new repeat instance set to a “Custom” type.

Some other details of changes related to repeat bookings:

Linked Repeat Bookings

We now support linked repeat bookings. Previously, if you set up linked bookings, and allowed a linked booking to be created from a repeat booking, then the results were not good. However, this will now work, almost exactly as one would expect. Some things to note when a repeat booking caused child linked bookings:

Actual Usage (aka Resource Usage)

When recording actual usage, normally if there is a booking that the usage relates to, then the usage record would link to the booking. It was not recommended to link an actual usage record to a repeat booking, as the repeat booking’s date would continually change over its lifetime.

However, following the change to repeat bookings, it is now perfectly acceptable and sensible for an actual usage record to link to a repeat booking. Usage records should always be in the past, and historical repeat bookings are no longer modified as they used to be.

2.3 New Workflow Functions

These are the workflow functions that have been introduced with version 11.0:

2.4 Calpendo Enterprise

As mentioned in previous release notes, there are again many changes behind the scenes in support of Calpendo Enterprise. This is the ability to have a federation of Calpendo instances that work together.

Calpendo Enterprise is not yet available for general use in version 11.0, but is now available for limited use and circumstances.

With Enterprise, you can:

Further details will come later, when Calpendo Enterprise is on general release in version 12.0.

2.5 User Masquerading

We now support the ability for a user to log in, and then switch identity so that they can appear to be a secondary user.

This is here so that somebody providing support (such as Exprodo Software or a local admin person) can investigate problems reported by somebody else by seeing things from their perspective.

To support this, there’s a new permissions action called “Switch User”. You will only be able to masquerade as another user if you have “Switch user” permission.

Here’s how it works:

2.6 Automatic Deletion of Old System Events

System events account for the biggest part of disk space usage within Calpendo. However, once system events are old enough, they generally become completely uninteresting. For this reason, we have introduced and automatic system event deletion system.

In global preferences, you choose the retention period on the “General” tab, where you can find an item labelled “System Event Retention (Days)”.

This defaults to 190 days, roughly 6 months.

3 Breaking Changes

3.1 lastRepeat property removed from Booking and Template

The properties Booking.lastRepeat and Template.lastRepeat have been removed with no exact replacement. This become necessary following the changes to the way repeats work in version 11.0.

lastRepeat was a DateRange property that told you when the last instance of a repeating sequence would be. This was similar to what you get from the date property Booking.repeat.finish, but it’s not the same for two reasons:

  1. Repeat.finish is a date property and carries no time information, whereas lastRepeat was a DateRange, which means it has a start and finish, both of which were date-time properties.
  2. Repeat.finish is the date after which no repeat instances will exist, but there is not necessarily a repeating instance on that specific date. For example, if you have a booking that repeats every Wednesday, and you set the Repeat.finish to be a Friday, then the last booking would be on the Wednesday before Repeat.finish.

Repeat bookings and templates are now enumerated in the database, up to a controllable time limit which defaults to 90 days. This means there are multiple rows in the database, with each row being a particular instance of the repeat. The last instance in the database is like a “promise to repeat”, and is the template for how additional future repeat instances should be.

Since each repeating sequence results in multiple bookings being enumerated in the database, it no longer makes sense for all of them to store information about when the last repeating instance would be, and so this property has been removed.

The repeat does not store the date when the last repeating instance will occur, although it is possible a future version could provide it. However, it could not store any time or duration information since that belongs to the booking or template.

4 Changes

In the top-right corner of every page there were separate items that showed:

These have now all been replaced by a single item that shows who you are logged in as, but clicking it produces a drop-down with all the items that used to be separately displayed (as listed above).

If you are running an Enterprise system, then there will be an additional item labelled “Account Status” which will take you to the new Account Status page.

The cookie that Calpendo creates to record your session ID has changed. Every Calpendo has a “shard number” baked into it that is a number from 1 to 1023. This number is now part of the session ID. This change has been made so that when you are using the new Calpendo Enterprise version, then browsers can record a different cookie for each shard you log in to. This helps to make the process of logging in to different Calpendo instances within the federation a seamless process.

A typical session ID cookie might now be:

449571389971457-paul-54-mSjf3a0DrbPekWBa-76

where the different parts of this are:

4.3 New shardNumber Pseudo-Property

There’s now a new shardNumber biskit pseudo-property that you can select in any place where you could previously select both a biskit’s id and biskitType properties.

This property provides the number (valued from 1 to 1023) that is baked in to every Calpendo instance. In an Enterprise system, this can be used to tell whether a biskit was created in your local Calpendo, or whether it’s been imported from elsewhere. You can do this with a condition like:

Value of shardNumber not equal to MetaProperties shard.number

This means you could have different permissions apply to biskits that are imported from a foreign Calpendo.

4.4 System Event New JVMID Property

Occasionally there are problems when somebody configures multiple Calpendo instances to point at the same database. This usually happens when copying the Calpendo installation while testing out a new version, and forgetting to modify the hibernate.cfg.xml file that specifies which database to connect to.

It is not supported to have multiple Calpendos using the same database, as they tend to get in each other’s way.

When this does happen though, the evidence can be somewhat confusing.

To help with this, every system event now records an additional property called “jvmid”. This is a text string that is likely to be unique for every Calpendo instance, even those within the same Tomcat instance. If you now see different values of jvmid intermingled in the system events, this is a sure sign that there are multiple Calpendos using the same database.

4.5 Global Preferences Renamed System Settings

Global Preferences is the name used to describe the page that stores the settings that govern how everything works.

However, with the advent of Calpendo Enterprise, which allows a federation of Calpendos to work together, the settings on a single Calpendo are not “global”, since they only affect the local system.

Hence the new name “System Settings”.

New Calpendos have the System Settings menu option appear in alphabetical order on the Admin menu. Existing customers will leave the menu option in the same place it was before, but renamed.

4.6 Calpendo Enterprise

There are many changes behind the scenes in support of Calpendo Enterprise. This is the ability to have a federation of Calpendo instances that work together.

Calpendo Enterprise is not yet available in version 11.0.

With Enterprise, you will be able to:

Further details will come later, when Calpendo Enterprise is ready to be used.

4.7 CTMS Much Faster

CTMS is a Calpendo “Clinical Trials Management System” module.

One can use this to help schedule subjects on multiple studies, and to link this with the rest of Calpendo so that required resources can be automatically booked.

If one configures a complicated study, where there are many visits per subject, and many tasks required within each visit, then there can be a very large number of bookings required when enrolling a subject into a new study. We have seen a real life study with 1500 bookings required.

Calpendo was not designed to handle that kind of scale, and so it handled them very inefficiently. Consequently, there has now been a rewrite for the way updates are handled so that this can be done much faster than before.

4.8 Custom Search Page

For a long time, you’ve been able to add an item to the menu that will show a search page for a biskit of your choice, and to customise it to have search widgets as required.

For example, it can provide drop-downs for choosing some value a property on the target biskit must have.

You can also choose the report type. Typically, this would be a list report, group report or Kanban report.

When choosing a group report, you were not previously able to choose which columns should be shown by default on the report. Now you can select the columns you require.

4.9 Custom Tree Page

A custom tree page is one that shows a tree of options on the left, where it automatically expands to show nodes in the tree for biskits that it finds.

For example, you could configure it to show projects by their status, or perhaps by status and owner.

The facilities offered by this dynamic expansion of nodes in the tree has been updated:

4.10 Sort Workflow Action

When you sort a list of biskits, it’s not unreasonable that you’d then want to extract the first or last one. This is now provided automatically with the first and last biskit available in the Sort Workflow Action’s output.

4.11 Search Workflow Action

When you search for something in a workflow, you can apply conditions to the thing you want to search for. So if you’re looking for bookings, you might have a condition to find bookings with a particular resource, and maybe filter by the booking’s date/time.

Previously, the conditions you can use were only those that look at values of the biskit you’re searching for or values of other biskits in scope, and you can’t use MetaProperties conditions, which would allow you to use values from the general environment.

This has now changed so that MetaProperties conditions are allowed in this context.

There are some circumstances where this allows you to write more succinct workflows rather than having to use different actions depending on the value of MetaProperties.

4.12 Message Compression

There are some messages sent by the server to the browser that contained information in an unnecessarily vociferous format. Sometimes this was repeated information, and others it was using longer names for things that it needed to.

These have now been changed so that the messages contain the same information, but in much less space. As well as saving on the time to transfer the message over the network, it also save considerably in the time it takes to parse the message.

This helps to make things faster.

4.13 Prevent resources from being configured with iCal URL and requiring project

4.14 CSV Import Improvements To Long-Running Imports

The file import utility used to give timeouts when the import took too long. The import would continue in the background, but the user had no way to know that.

We have now change the way this works so that there will be no timeouts, no matter how long the import takes.

It also means that while an import is running, you can change your page to look at something else, and then when you come back to the import page, it will show you the status of the import that’s in progress.

4.15 Licence Management

The licence stored in Calpendo is managed differently in 11.0 from how it used to be. Previously, one would install the licence by going to global preferences (now called system settings) and adding it there.

However, when you have many systems to licence, for example as part of Calpendo Enterprise, then upgrading licences could be quite time consuming.

So licences are now managed centrally via a licence server.

Each Calpendo will send a request to the licence server at boot time and at regular intervals asking if there is an update to its licence. When somebody pays for an updated licence, Exprodo Software employees generate a new licence, and install it into the licence server. Your Calpendo will then automatically pick up the new licence.

By default, Calpendo must be able to contact the licence server at least once every 14 days to continue operating. If you are self-hosting Calpendo, and your network environment does not allow outbound https traffic, then you will need to let us know so that we can make alternative arrangements for you.

When the licence has been updated on the licence server, Calpendo will automatically fetch it when it next checks, which is done every 24 hours. If you require a licence to be pulled in immediately, there’s a new button on the system settings page for that on the licence tab labelled ‘Refresh From Licence Server’.

For those that host Calpendo on their own servers, it should be noted that this means some information is sent to Exprodo Software on a daily basis. This information includes:

The fingerprint is composed of the following items:

This data is collected for two reasons:

If you have privacy concerns relating to this data, then please contact us to discuss options.

4.16 Workflow display icons

When you look at a workflow now, there are icons added to each event and action to indicate when each one:

This helps to see where there are interesting things that may help somebody trying to understand what a workflow is doing.

4.17 CSV Import Disabling Workflows

The CSV import facility now has a setting that allows you to disable all database workflow events for the duration of the import.

This means that, when disabled, all database events will be ignored. This applies to ALL database events and not only those that are related to the import.

As such, this option must be used with great care and only root users are allowed to use this option.

This is being provided because there are sometimes occasions when workflows are not wanted. For example, when you are importing historical data. It may not conform to current restrictions that are implemented in workflows, or you may not want any changes made by workflows to these historical items.

4.18 Changed Workflow Functions

These are the workflow functions that have been modified with version 11.0:

4.19 API Changes

We have had the following APIs:

In our documentation, we have published behaviour available at /webdav/b and /webdav/q, without particularly pushing the fact that a generic WebDAV client could also connect to Calpendo.

We are now dropping support for the WebDAV protocol, while retaining support for the published behaviour on the above APIs.

Since we are dropping support for WebDAV, we are also changing the URLs to access the APIs to the following:

In order to aid transitioning to these new URLs, the old ones will continue to be supported, but we would encourage everybody to move to the new URLs as we may drop support for the old ones at some point in the future.

We have made this change mostly because our WebDAV support relied on a very old third party library that incorporated other third party libraries that had security vulnerabilities. We are also unaware of anybody using the WebDAV protocol with Calpendo, and so the most expedient solution was to remove WebDAV support while retaining the published API support.

4.20 Count Properties Prohibited

We no longer support properties whose name is ‘count’, or whose column is named ‘count’.

If there are any such properties, then when you you first boot into the upgraded version, it will automatically stop before any changes are made to the database. A message in the log will indicate that it relates to properties called ‘count’ and to seek help.

If there are any properties called count or with a column name of count, then they would need to be renamed before the upgrade takes place. You are advised to seek help from support@exprodo.com if this applies to you.

4.21 Add support for CSV import of a biskit with a property that is a set of tags

Properties defined as a set of tags could not be modified by a CSV import. However, they are now fully supported by using a field with comma separated values.

4.22 Allow CSV import to set booker, created, modified, previousStatus, cancelled

4.23 Add support for dumping stack traces to the log

4.24 Update INSTALL resources for self-installers

5 Releases

5.1 11.0.0 August 2, 2023

5.2 11.0.1 (no date)

Changes

Bug Fixes

5.3 11.0.2 October 23, 2023

Bug Fixes

5.4 11.0.3 October 23, 2023

Bug Fixes