Title: Release Notes For Version 12.0.19
To run Calpendo version 12.0, you should use:
If you are upgrading from version 9.0.x or earlier, then:
If you are upgrading from version 10.0.x, 10.1.x or 10.2.x to 12.0, then:
If you are upgrading from version 11.0.x to 12.0, then:
Upgrading from any version to the latest (including applying bug fix updates) should be done with the following procedure:
Stop tomcat
For major upgrades, you must create a backup of your database before proceeding. For minor upgrades, it is recommended to create a backup, but it is less important. After having stopped tomcat, the typical command required is:
mysqldump -u USERNAME -p calpendo > calpendo.sqlor, depending on your system:
sudo mysqldump calpendo > calpendo.sqlSave the hibernate.cfg.xml and log4j.properties files from your existing webapps/Calpendo/WEB-INF/classes directory
Take the Calpendo directory from the tar.gz download, and replace your existing webapps/Calpendo with it.
Put the saved hibernate.cfg.xml file into the new directory.
We have upgraded to Log4j2 in Calpendo 12.0 so the old log4j.properties file will no longer be used. Instead, the file to use is:
webapps/Calpendo/WEB-INF/classes/log4j2.xml.You can either use this file as it is, or customise it as you prefer. The default version will create a different file for each day, and store the date in the filename. This may mean the number of log files needs trimming periodically, but should also mean we don’t lose logs when there’s a large amount of log output on one particualar day.
If upgrading from any release before version 11.0, then you will need to modify hibernate.cfg.xml. If you already installed 11.0 and made these changes for that release, then you don’t need to do this. If you are using MariaDB, then there are three changes you must make. See the 11.0 release notes for details.
Make sure you do not have both old and new versions in the webapps directory at the same time, unless they point to a different database. Booting multiple versions of Calpendo that point to the same database will cause problems and is best avoided.
Restart tomcat
If something goes wrong during an upgrade, and you want to downgrade to a previous version, then it is important that you follow the right method.
If you are downgrading across patch versions, for example from 12.0.x to 12.0.y, then you can generally do that without making any changes to the database. Just change the program files to the version you want, and it will automatically handle any database changes required.
However, if you are downgrading across major or minor versions, for example from 12.0.x to 11.0.x, or from 12.0.x to 10.1.x, then it is very important that you follow the sequence described below.
If an upgrade fails for some reason, then the database may well be partially upgraded. Automatic recovery from a partial upgrade is not supported.
The general procedure for downgrading is:
As you can see, for this procedure it is essential that you took a backup of the database before the upgrade began, otherwise downgrading in the event of a failure may not be possible or may be difficult.
Examples of sequences for downgrading that are not supported and likely to result in a broken system are:
Don’t bother loading a backup of the database. Just change the program files to an old version and reboot.
Don’t bother dropping the database and recreate it. Just load a backup of the old database on top of an upgraded or partially upgraded database and run an old Calpendo.
Recovery from these kind of errors can be difficult and possibly labour intensive. Please avoid this by following the prescribed downgrade procedure as laid out above.
Version 12.0 sees the first production release of Calpendo Enterprise, completing the changes begun in version 10.0. A full description is beyond the scope of these release notes, and so what follows is a summary of what it is and does.
In a Calpendo enterprise system, multiple Calpendos work together in a single federation that allows (some) data and users to be shared. Each single Calpendo instance is known as a “shard”.
Each Calpendo shard in the federation would own its resources and it would control them. Some or all of the resources can be shared to other Calpendo shards.
A user can see a bookings calendar that shows resources that are owned by the current Calpendo shard, and those that have been shared to the current Calpendo shard.
They can create and edit bookings for each of the resources displayed, regardless of which Calpendo shard the resource belongs to. However, the user must have an account on the Calpendo shard that owns the resource, and it’s all the rules and permissions that apply on the shard that owns the resource that will be used to decide whether this user can make bookings or not.
From a user’s perspective, it will appear that they can just use many resources, some of of which are from other core facilities. The distinction between which core facility owns the resources does not get in their way at all, provided they have access to that core facility’s Calpendo.
For those using Calpendo projects, they can also be shared between shards. This means a user may book a resource from another facility using their own project.
Users that have accounts across multiple Calpendo shards can seamlessly move from one shard to another. That is, once they have authenticated on one shard, then they can access another that they have an account on without having to authenticate again.
There are two main use cases for Enterprise:
When you have a large number of core facilities, and they each need their own core facility management system, but they need to share some data, run cross-facility reports and generally integrate together as a single unit.
When you have a small number of facilities, it can be tempting to configure a single Calpendo that handles all of them. Calpendo is sufficiently versatile that this can be made to work. However, the complexity grows exponentially when a single Calpendo is configured to handle multiple facilities. Using a Calpendo Enterprise system means each facility can have their Calpendo working the way they want while still integrating with other facility Calpendos, in a much simpler configuration than if there were multiple facilities within a single Calpendo.
Enterprise can be configured with a top-level Calpendo shard that enforces control over many aspects of how the individual Calpendo shards work. Alternatively, there can be very little top-down control. The choice of how this should work will always reflect the politics involved at your institution.
A non-Enterprise Calpendo cannot be converted into an Enterprise system. If you want to migrate to Enterprise, then this requires a new Calpendo Enterprise that takes some configuration ideas from your old Calpendo. This will always be the case because the way one designs an Enterprise system will always be different from designing a standalone system.
If you use the API to connect to Calpendo, then you should specify the URL as one that targets one particular Calpendo shard. For example, when we host an Enterprise Calpendo, the URLs would be like this:
https://x.calpendo.com/y/api/
where ‘x’ would be the name of your institution, and ‘y’ would be the name of the particular shard within your Enterprise federation.
Further, if you use the query API, then this will search through the data on one or more Calpendo shards. By default, it will search the shard whose URL you used. For example, this would return the name of all projects on the shard called “y”:
https://x.calpendo.com/y/api/q/Calpendo.Project/status/EQ/Approved?paths=name
If you want the names of projects regardless of which shard knows about them, then you would do this:
https://x.calpendo.com/y/api/q/Calpendo.Project/status/EQ/Approved?paths=name&shards=all
whereas if you wanted projects from a particular selection of shards, then you would target them with their shard numbers (rather than names) like this:
https://x.calpendo.com/y/api/q/Calpendo.Project/status/EQ/Approved?paths=name&shards=3,56,76
Calpendo now supports two factor authentication. This applies to both Enterprise and non-Enterprise systems.
Users will need to install an authenticator app, typically on a mobile phone, or use one they already have. This authenticator app provides time-based one-time passwords (TOTP). That means the app would show a number that changes every 30 seconds or so and has to be given to Calpendo as a part of the authentication procedure.
The two factor authentication system works with all of our existing authentication methods. That mean you can use it with single sign on, LDAP authentication and local Calpendo authentication and it does not require any change to your Calpendo authentication method configuration to work.
Each user has their own two factor authentication status, stored in a property on the user account called auth2FAStatus. This status is one of the following:
| Status | Description |
|---|---|
| OFF | The user account is not configured for 2FA. They may choose to configure it. |
| ON | The user is configured for 2FA and must provide a TOTP password to log in. |
| REMIND | On logging in, the user will be reminded that they ought to configure 2FA. |
| FORCE | When the user next logs in, they will be forced to configure 2FA. |
An administrator may choose to force all users to configure 2FA by setting the status to FORCE, or they may set it to REMIND and perhaps later set those not configured for 2FA to FORCE.
If an administrator does nothing, then users can still opt in to configuring 2FA themselves. This is done by clicking their username in the top-right corner of every screen and selecting “Two Factor Authentication”. This will trigger a wizard that takes the user through the steps required.
System settings also let you specify who should be forced to configure 2FA, optionally depending on their roles. This is controlled in the Security tab, and the options of who should be forced to use 2FA are:
This means that somebody will be forced to configure 2FA if either their 2FA status is set to FORCE or if the system settings say everybody should be forced to use 2FA or if the system settings specifies roles matching the user should be forced to use 2FA.
As well as being able to specify in system settings who should be forced to configure 2FA, you can also specify who should be given a reminder to configure 2FA on login. This has the same options for who can be forced to configure 2FA. That is, everybody, nobody or those with specific roles.
Finally, if you configure an Exprodo Authentication Method, then this means that the password for a user is stored in another Exprodo database, known as the “proxy”. So if you have multiple databases, you can have one of them store the passwords for all. In this case, if a user either has 2FA configured on the proxy, or will be forced to configure 2FA next time they log in to the proxy, then they will be forced to configure 2FA when they log in to any other system that uses the proxy for authentication.
The system settings allows an admin to choose whether users can skip 2FA for a time, and if so, for how long.
On the Security/2FA section of the system settings is an option labelled “Allow 2FA to be skipped for a while”. This defaults to 7 days.
If a user with 2FA enabled authenticates, they are shown a pop-up that asks for their 2FA validation code. If validation can be skipped, then next to the text entry for the code is a checkbox that the user can tick to say they don’t want to enter the code again on this machine for 7 days (or whatever the duration is configured to be).
If they tick this, then when they log in again over the 7 days from the same browser, they will not be asked for their 2FA validation code.
This process relies on a cookie stored in the browser that records the setting that says 2FA is not required.
Many pop-ups you see can now be pinned so that they remain displayed while you continue to do other things in the system.
For example, you might go to the bookings calendar and edit a booking. When you do this, a pop-up appears, and the rest of the display is darkened to indicate you can’t click in it. You can now pin that pop-up. When you do this, the darkened effect from the rest of the display is removed, and the pop-up indicates that it is pinned. You can now use the menus or any other part of Calpendo while the pinned pop-up remains displayed and functional.
Pinned pop-ups can be:
Some pages allow you to edit text that is of a particular type or format.
For example, in the bakery, a formula is SQL (mostly). So the formula now displays in an editor that understands SQL syntax. Also, system settings lets you provide custom CSS to control the system’s appearance. That is now displayed using an editor that understands CSS syntax.
These text editors can also be used for custom properties that you create. To do this, create a layout for the biskit type, and then on the property in the layout, choose the format you want it to use.
We currently support the following content types:
In the layout editor, there are now more options for how you can display string properties.
You can choose the content type (SQL, CSS etc) to use an editor that provides syntax colouring. You can also choose whether the text should be displayed:
An example of its use is in the bakery where the space available for displaying a formula is very limited. This is now displayed using the “Always popup” mode.
In the bakery, you can now specify that a text property has a type of “Text with tags”. This then allows you to associate a separate property (on the same biskit) that stores a set of tags. The set of tags would be marked as automated.
When editing the text property, if you were to enter this:
Hello #world this is a #test of the tag system
Then the set of tags would automatically be set to contain “world” and “test”.
This means that when typing in some text, any word that you want to marked as a tag just needs to be prefixed with the # symbol.
The link between the tags and the text also feeds back from the tags to the editor. Suppose you have entered the above text and saved the biskit. Then the tag will have recorded that “world” and “test” are known tags. When you then edit another biskit, if you were to type “#wo” and press CONTROL-SPACE, then a drop-down would appear with suggestions for which tags might complete what you’ve typed. Use the mouse or keyboard to select “world” from the drop-down, and the text would be modified to say “#world”.
There are times when a user might perform one action, and it results in multiple biskits being modified. Examples where this might happen are when linked bookings create multiple bookings or a workflow that makes many changes in response to the user action.
This can mean that when you examine the audit log, it is not very clear what the user has actually done. There may be many changes, and it may not be obvious that they are tied to a single action.
To help with this, the audit log has been modified so that every change it records includes the name and number of the user request that generated the change. For example, if a user creates a booking, and a child linked booking is also created, then you get two audit log entries: one for creating each booking. But both would record that it was the same user request number and request name that caused them.
The bakery will now let you add a property that stores a set or list of integer, long, double, date, datetime, boolean and string.
Apart from a set of string tags, which was previously supported, the new options all store their data in a new table, whose name is calculated from the biskit’s table name plus the column name set on the set/list property.
For example, if you add a list of integers called “numbers” to a biskit store in table “things”, then the table name for the integers would be “things_numbers”.
The Calpendo UI has a feature that lets you dump the database, generating a SQL file.
This works on both MariaDB and MySQL, and the file generated can be loaded into either flavour of database.
However, recent versions of MariaDB have changed their command line utility for dumping the database so that the file it generates cannot be loaded into a MySQL database.
It may therefore be useful to convert the database dump code in Calpendo to something that is also callable from the command line.
Those who host their own Calpendo can find the relevant documentation in a README file in the INSTALL directory of their expanded Calpendo archive.
When anybody uploads a file, this can now be automatically scanned for viruses.
For this to work, it requires that clamav-daemon be installed somewhere on the network, and configured to accept requests on a TCP socket.
There is now a new section on the system settings’ “Security” tab that lets you specify the hostname and port number where Calpendo can access the clamav daemon. The port number defaults to 3310, and the hostname defaults to localhost.
Installing clamav-daemon on Ubuntu for this requires:
sudo apt install clamav-daemon
echo TCPSocket 3310 | sudo tee -a /etc/clamav/clamd.conf
echo TCPAddr localhost | sudo tee -a /etc/clamav/clamd.conf
sudo systemctl restart clamav-daemon.serviceSome string properties will be used in a context where HTML is allowed. In order to prevent some types of security attack, there is now support for automatic filtering of the content of string-valued properties.
In the bakery, each string-valued property now displays a new “HTML Allowed Type” option. The value you choose is from a drop-down with the following values, in decreasing order of strictness:
For example, if a string property has an HTML Allowed Type of “NONE”, then whenever any value for it is saved to the database, it will be examined to see if it contains any HTML, and the HTML parts will be removed completely.
For example, something like this:
Hello
would be modified to:
Hello
You can choose how strict the filtering should be by the drop-down choice.
We have for a long time had the ability to restrict passwords to have at least a certain length, contain upper and lower case letters, numbers and characters that are not digits or letters.
We also implemented support for haveibeenpwned.com a long time ago so that we can check whether a password somebody uses has been exposed in a previous security breach.
There are three changes with this:
The audit log history page now has a button to download the contents currently displayed.
Suppose you have a system where there are no user types created, and you run a report to show users. Then, you add a condition that looks like this:
value of userType equals Fixed
Then what appears at the end should be a widget that lets you select a user type. It should do this with a box you can type in, and it gives a drop-down of suggestions similar to what you typed, and also a button to the right that shows a drop-down of all possible values.
However, when there are no possible values, such as there being no user types that exist, then the button to the right that shows all possible values is not displayed. You only get the text box to type in to.
However, this is confusing as it says it wants you to select an item, but nothing happens when you click in it. It just looks like a text entry and nothing else.
This has now changed in two ways:
We used to use Log4J version 1.x for create Calpendo log files. This was replaced with Reload4J, which was a drop-in replacement for Log4J 1.x, but had the vulnerabilities fixed. We have now migrated to Log4J2.
Previously, log files would be rotated into a new file whenever the file grew large, with a limit on how many log files were kept. This would mean that old log files would be lost. We now shift to a standard configuration that rotates log files every day, with all files (apart from today’s) having the date in the file name.
This provides more stability for backing up log files, if required, but does add a burden of needing to delete sufficiently old log files depending on what is required to be kept.
It used to be the case that you had to remove all users from a project and save it before you would be able to delete that project. This is no longer the case.
When adding a property that is a set of string tags in the bakery, it is now possible to mark them as sorted so that they always display in alphabetical order.
If a user is manually created and assigned an authentication method that does not let them log in, then a warning will be displayed. It is possible to do this deliberately, but it’s not common that one would create a user for a disabled authentication method.
There was a time when the workflow debugger did not exist. At that point, the only option for recording any information about what was happening in a workflow was to edit the workflow and turn on recording of system events. This records a system event for each action when the action starts, and then when it finishes.
Complex workflows result in large number of system events that mean you can’t then easily view the system events.
Other problems with this are:
So there is now a new system setting, on the General tab, that lets you turn off all system event recording by workflows. When this is disabled, it overrides anything set on the workflows themselves.
This new setting is disabled by default so that we will no longer have recording of system events for workflows unless somebody manually turns it on.
Changes
On the User Search page, the ‘Roles’ dropdown isn’t very useful in Enterprise as it only looks at roles which are set at federation level.
This has now changed to use the “Effective Roles” instead, which is a logical merge of the federation-wide roles and the shard-specific roles.
The result is now much more intuitive.
Issue: 3896
In a Calpendo CTMS system, if you modify a project to indicate a task is not required for a cohort, where previously it was, then whenever you edit a subject of that cohort, any events for that task for be automatically cancelled.
This no longer applies to those events that are in the past.
Issue: 3973
Optimisations
Also here to avoid a deadlock in which multiple transactions were trying to write to linked_booking_link_conditions, even though none of the code was deliberately asking for the link or its conditions to change.
All flavours of conditions (eg on permissions, rules, reports etc) all store their child conditions using a list rather than a set. However:
All things taken into account, it seems best to remove the ordering from conditions, so that they are treated as a set and not a list.
For the end user, and the results people see, there is absolutely no difference.
It may, however, cause the conditions to display in a different order, albeit with the same semantic meaning.
Issue: 3788
A change was made in 10.1.41 for issue 3706 which meant that when handling linked bookings, we would clone all bookings before using them.
However, in 12.0, this is more aggressive than necessary. That’s because we sometimes create shallow copies of bookings when that’s all the caller requires. These shallow bookings do not need to be cloned; only those fetched from the database needed to be cloned.
Issue: 4004
Bug Fixes
BUGFIX: Prevent CTMS appointment being marked as complete when it is requested
BUGFIX: Persistence Exception adding new appointment to existing subject
BUGFIX: Shallow exception when edit CTMS Event from search page
BUGFIX: Confirming bookings reported fewer bookings than it should have done
BUGFIX: CTMS Event report comparing dateRange to appointment date range fails
BUGFIX: Cancelling CTMS appointment/event does not record app/event status
BUGFIX: Cancellation reasons from parent booking not copied to linked child
BUGFIX: Child linked bookings not cancelled when parent cancelled
As well as child linked bookings not being cancelled with their parent booking, in CTMS Appointment cancellation did not cancel Events.
Issue: 3821
BUGFIX: Cancel CTMS appointment/event does not ask for cancellation reason
BUGFIX: Update CTMS appointment from search page gives exception
BUGFIX: Tomcat restart needed when linked booking licence limits change
When you downgrade a licence from a regular licence to a Starter, then you cannot create new linked booking links and have them function.
However, pre-existing linked booking links continue to function as they did before.
A tomcat reboot is needed to make it work, but it should not require a tomcat reboot.
Issue: 3991
BUGFIX: Non-Enterprise shows “Unknown ’Enterprise Tools” on its menu
The “Enterprise Tools” page is on the menu by default, even for non-Enterprise systems.
This ought to be hidden in a non-Enterprise system, but was displayed with an error message.
Issue: 3992
BUGFIX: Confusing biskit suggestion drop-down when there are no possible values
Suppose you have a system where there are no user types created, and you run a report to show users. Then, you add a condition that looks like this:
value of userType equals FixedThen what appears at the end should be a widget that lets you select a user type. It should do this with a box you can type in, and it gives a drop-down of suggestions similar to what you typed, and also a button to the right that shows a drop-down of all possible values.
However, when there are no possible values, such as there being no user types that exist, then the button to the right that shows all possible values is not displayed. You only get the text box to type in to.
This is confusing as it says it wants you to select an item, but nothing happens when you click in it. It just looks like a text entry and nothing else.
This has now changed in two ways:
Issue: 4005
BUGFIX: Two resources with the same iCal URL means only one gets synced
Changes
When upgrading to 12.0.0, all users will have no two factor authentication status set, and so when you view a user, the status will be blank, and when you edit a user, it will say “Please select a Two Factor Auth Status”.
This is now changed so that the default is OFF.
Issue: 4025
The Calpendo UI has a feature that lets you dump the database, generating a SQL file.
This works on both MariaDB and MySQL, and the file generated can be loaded into either flavour of database.
However, recent versions of MariaDB have changed their command line utility for dumping the database so that the file it generates cannot be loaded into a MySQL database.
It may therefore be useful to convert the database dump code in Calpendo to something that is also callable from the command line.
So, if you have access to the command line on your Calpendo server, then you can use the Calpendo jar files to dump the database for you.
Those who host their own Calpendo can find the relevant documentation in a README file in the INSTALL directory of their expanded Calpendo archive.
Issue: 4181
When editing custom CSS in the system settings, it now uses an editor which will display with syntax colouring.
Issue: 4186
Many third party libraries have been upgraded. This avoids some known vulnerabilities (which may or may not have had the capacity to affect Calpendo) and also uses whatever bug fixes are available.
Issue: 4189
Bug Fixes
BUGFIX: In CTMS, cancelled events should move with appointments by default
When a CTMS appointment’s date or time is modified, then a pop-up asks which of the events within it should be changed to match.
Each event has a checkbox next to it so that the user can choose to include or exclude events as they see fit.
Currently, cancelled and denied events are unticked by default, which means they are left at their original date and time.
This can make the events seem disconnected from their appointment, and yet they still reference the appointment.
It seems that there is no obvious right thing to do here. On balance, it seems most wrong to default to the events staying where they are.
So we now automatically tick cancelled and denied events so that the default behaviour is that they be moved along with their appointment when the appointment changes date or time.
Issue: 4021
BUGFIX: In CTMS, appointment length calculation includes cancelled events
BUGFIX: No way to log out part way through 2FA configuration
If you enter your password and then find you are forced to set up 2FA, but you don’t want to or can’t, then there’s no way to back out of it and (for example) log in as a different user.
Issue: 4024
BUGFIX: Formula pop-out text editor in the bakery is not editable
BUGFIX: iCal import messages number items starting from 0 instead of 1
BUGFIX: Upgrade to 12.0.0 fails on database upgrade script “11_5_31/5/1”
Upgrading from 10.1, 10.2, 11.0 or 11.1 to 12.0.0 would give an error:
There was a problem with an automatic database upgrade. Please contact support@exprodo.com quoting 11_5_31/5/1
Issue: 4092
BUGFIX: List of tags is displayed behind the New booking window
BUGFIX: Database upgrade error on first boot into non-Calpendo 11.5.41
The database upgrade for issue 4169 and change# 744 should have checked that it was running on a Calpendo system, since the change was only relevant to Calpendo.
However, it did not do this, and so upgrades would complain as it tried to look for content inside a Calpendo-only table.
Issue: 4178
BUGFIX: iCal import fails to update bookings due to version number mismatch
When an incoming iCal feed needs to update bookings in Calpendo (because the external event has changed), then we get an exception because we don’t make sure the version number on the booking is set correctly on the updated booking.
Issue: 4183
BUGFIX: iCal import generates errors if external bookings have start == end
BUGFIX: Auth method URL not displayed when 2FA being set up for Exprodo auth
When you set up two factor authentication for an Exprodo authentication method, then one of the pages you are shown while setting it up says:
Your Calpendo account is configured via an Exprodo proxy so your password is stored on undefined.
However, “undefined” should instead be the URL for the Exprodo proxy stored in the Exprodo authentication method.
Issue: 4185
BUGFIX: Text pop-ups remain displayed and marked active on change of page and change of displayed item
Changes
Add support for automatic filtering of string-valued properties to remove some or all HTML.
See the ‘New Features’ section of these release notes for more details.
Issue: 4213
Calpendo now supports automatic virus scanning of any files uploaded.
This requires that clamav-daemon be installed, and also configured to accept requests on a TCP socket.
There is now a new section on the system settings’ “Security” tab that lets you specify the hostname and port number where Calpendo can access the clamav daemon. These default to ‘localhost’ and 3310 respectively.
For those self-hosting Calpendo, installing clamav-daemon on Ubuntu for this requires:
sudo apt install clamav-daemon echo TCPSocket 3310 | sudo tee -a /etc/clamav/clamd.conf echo TCPAddr localhost | sudo tee -a /etc/clamav/clamd.conf sudo systemctl restart clamav-daemon.service
Issue: 4222
If a user is authenticated via an Exprodo Authentication Method, and the database that stores the passwords forces that user to set up 2FA, then the user will be forced to configure 2FA on this system as well.
Issue: 4236
Security Fixes
SECURITY FIX: Login allows random HTML in username which is then parsed as HTML XSS attack
SECURITY FIX: Error pop-ups parse user input as HTML and display it (XSS)
SECURITY FIX: Set the name of a location to null and can’t edit calendar resources
SECURITY FIX: Stack traces show which is confusing & bad practice
When an exception occurs in the browser, then a stack trace would sometimes be displayed.
This is both confusing for the user and not best practice as it could information that might be useful to an attacker.
Issue: 4216
SECURITY FIX: Password hashes are sent to the browser, which is a security issue
Password hashes were previously available in the browser, but served no practical purpose there.
This made it easier for somebody to try to calculate what a password might be by trying to calculate possible passwords from the hash of the password.
Password hashes are no longer every sent to the browser.
Issue: 4217
SECURITY FIX: Session IDs accessible to all allowing possible root access
It was possible for admin users to view the session IDs of other users.
This would mean allowing people to see information that could be used as one part of an attack for escalating their privilege, potentially to allow a non-root admin user to become root.
Issue: 4218
SECURITY FIX: Other users roles and 2FA status are visible to non-admins, which they should not be
SECURITY FIX: Non-admin users can see other users’ settings
There’s no reason (in general) for one non-admin user to be able to see another user’s settings.
The security risk from this is low, but this possibility has been removed.
Issue: 4220
SECURITY FIX: Forgotten password system can be used to enumerate users
The system that lets users recover when they have forgotten their passwords could be used by an attacker to identify login names of real system users.
This could happen by guessing a login name of a user, and using that in a forgotten password request.
The response was then different depending on whether they happened to use a real user login or a fake one, thereby letting the attacker know whether they found a real login.
Responses are now identical, regardless of whether the user login or email exists or not.
Issue: 4221
SECURITY FIX: Brute force login detector fails at high rates of attack
We have for a long time had protection against brute force password hacking attempts.
That is, if somebody tries a password too often and too frequently, then they are locked out so that they cannot log in, even with the correct password, until they stop trying and wait.
The more frequently they try to log in, the longer they have to wait.
Even if they try with the correct password while locked out, this can extend the time that they must wait before the lock is removed.
However, there’s a bug with the implementation that means under some circumstances, you will be allowed in if you get the password correct within the period when you should be blocked.
Issue: 4223
SECURITY FIX: File attachments exposed session ID in download URL
Suppose you have a file attached to a project.
When you click on the link to download the attached file, the URL to contact the server and request the file to be downloaded includes the session ID.
This is bad because when you put the session ID in the URL, it means that you could be subject to a man-in-the-middle attack where somebody who manages to get themselves between the browser and the server could see this traffic and work out what your session ID is.
Once they get your session ID, then they could potentially use this to access Calpendo using your session.
Issue: 4225
SECURITY FIX: Null values are not prevented in lots of properties that should prevent null values
There are lots of properties that should never be set to null, but were never prevented from being null.
The problem with this is that somebody could, using the API, set the name of various string properties to null, and that might trigger a bug where Calpendo cannot cope with a null value.
This is possible for string values because the user interface does not normally provide a means of entering a null value; typically, you enter a real value or the empty string but not null.
Where appropriate, properties are now marked as not being compatible with a null value and null values are prevented from being saved.
Issue: 4226
SECURITY FIX: Update of biskit would not enforce bakery validation settings (min/max/required etc) if no formulaic properties exist
When you update a biskit, then it would validate that the various limits specified in the bakery are adhered to.
For example, string min/max length, integer min/max value and whether a property is required (ie must be not null).
However, when updating a biskit, this validation was only done if the Biskit being modified included a formulaic property. Otherwise, the validation was not done on update.
Issue: 4227
SECURITY FIX: A session can have its session ID modified leading to a denial-of-service attack
There is no good reason to allow anybody to modify some properties of a user session.
If you do modify the session ID of an active session, then this can lock out a user (although perhaps only temporarily).
We now prevent properties on a session being modified either by the UI or the API.
Issue: 4228
SECURITY FIX: 2FA should be enforced for privileged users
One could individually choose to use two factor authentication, and an admin could edit users to force that user to use 2FA.
However, there was no mechanism to specify that all users must use 2FA, or that all admins must use 2FA.
However, we now have a mechanism that will allow all privileged users to be forced to use 2FA.
In the System Settings page, on the security tab:
The default is that everybody with the admin role is forced to use 2FA and nobody else will be reminded
Note that individuals can have their 2FA status raised. That is, they can enable 2FA or be forced to use 2FA when the system settings say they don’t have to be forced.
But you can never have a lower status. That is, if the system settings say that admins are forced to have 2FA configured, and a particular admin user has their 2FA status set to OFF or REMIND, then when they log in, they will be forced to configure 2FA.
Issue: 4232
SECURITY FIX: Cannot configure 2FA with Exprodo Authentication when url is hidden
Suppose:
Then an exception is thrown when it tries to configure 2FA so they cannot configure it, and so they can’t log in.
Issue: 4233
SECURITY FIX: Admins are able to modify or override permissions that control what admins are allowed to do
SECURITY FIX: ical_viewer password is hard-coded fixed string
Bug Fixes
Suppose you set up permissions so that a user’s roles are hidden from the current user, and that they then look at a report of users.
The roles would show as “No roles” instead of “
Similarly, with a user’s auth2FAStatus hidden by permissions, it would show up as blank rather than give an indication that it was hidden.
Issue: 4230
Suppose a user that authenticates with an ExprodoAuthenticationMethod is forced to configure 2FA when they log in.
Then the initial message they are shown sounds rather like they had chosen to configure 2FA. It now reflects whether they are making a choice or they are being forced to configure 2FA.
Issue: 4237
Suppose:
Then when they go through the 2FA configuration, the user will see this error message:
Failed to fetch data: Something went wrong trying to verify your
code: Could not validate 2FA for Exprodo user - received status code
520 from https://proxy.exprodo.com/: Received unexpected HTTP status
code 520There is a message that tells the user they must configure 2FA on the proxy server (B in the above example) before setting it up on A.
However, if they do it on A before doing it on B, it should not give such a poor response.
This is marked as a security problem because the result you get is different depending on whether 2FA is configured on server B. This in turn means that there would be a way for an attacker to use this to glean some information about user accounts on server B, possibly resulting in account enumeration.
The response from server B will now simply indicate the code was wrong, with no indication that you have not set up 2FA yet (or indeed that the user does not exist).
Issue: 4254
Changes
Neither of the roles Guest nor User actually have any specific processing behind them in a new system.
Having these roles causes confusion for very little gain.
These roles will no longer exist in systems created in future.
This will mean admins will no longer have to assign roles when approving normal users.
Issue: 4263
There are two separate pop-ups affected by this:
In both cases, they now show the user identity in the title bar so that it is clear whose 2FA we are asking for.
Issue: 4268
Users are now allowed to skip 2FA validation of their password for a predetermined and configurable duration.
This is controlled in two ways:
This process relies on a cookie stored in the browser that records the setting that says 2FA is not required.
The cookie has an expiry date so the browser will remove it after it expires.
The cookie also contains an expiry date buried within it, and the cookie is signed by the server so that the server can check to make sure the cookie is sees is valid and has not been faked.
The cookie name is “skip-2fa-SHARD-NAME” where SHARD is the shard number and NAME is the name of the program. For example “skip-2fa-1-Calpendo” would be for a Calpendo system on shard number 1.
Issue: 4270
Bug Fixes
BUGFIX: Changes made in System Settings (Login Header) is not saving
The heading in system settings goes through some HTML filtering which gave rise to confusion.
Issue: 4258
BUGFIX: Permissions to hide user roles & 2FA status from others included admin
When upgrading a system to 12.0.2 (rather than one created as a virgin system in 12.0.2), then there are permissions created to control who can read User.auth2FAStatus and User.roles.
These permissions were meant to allow admins to read these things for other users, but they do not.
The permissions concerned do not tick the box “Use roles” on the Does-Not-Apply-To tab for permissions:
“User can’t see other people’s roles” “User can’t see other people’s 2FA status”User can’t see other people’s fragment roles "User can’t see other people’s fragment local roles
Note: The fix for this does NOT fix permissions in systems that were upgraded in 12.0.2. Rather, it means that fresh upgrades from before 12.0.2 to 12.0.3.1 or later will not make the same mistake.
Issue: 4259
BUGFIX: Two 2FA config windows displayed upon login when forced to configure 2FA and sign out and back in
BUGFIX: Clicking Cancel button of 2FA prompt allows the user to login successfully
Enter a valid 2FA verification code and then click the Cancel button allows you to log in successfully.
Once you had entered a valid 2FA code, anything that caused the text entry to lose focus would mean it was validated and then you would be logged in.
You could only log in with the right code though.
Issue: 4261
BUGFIX: Repeats can’t be expanded if booker is no longer member of project
One of the checks done when a booking is created is to make sure that the booker is a member of the project, when there is a project.
For repeat bookings, this can present a challenge.
The scenario in question is when a user makes a legal repeat booking for a project, and then some time later the user is removed from the project.
This might happen if the user leaves the institution.
Further expansion of the repeat would then see that the booker is not a member of the project, and so prevent the repeat from expanding.
Issue: 4262
BUGFIX: Log in as non-existent Exprodo user gives exception
Suppose:
Then the login would give an IllegalArgumentException with the message:
Biskit primary key is zero in a call to updateBiskitThis represents a security exception because it could be used to work out which Exprodo users exist on the proxy.
Issue: 4267
BUGFIX: 2FA system settings sent on every save regardless of whether changed
BUGFIX: Cancelling 2FA validation leaves the partial user session hanging around
BUGFIX: Missing item in menu means saving system settings gives exception
Every menu item in a menu has a “sequence” number. The sequence numbers for all siblings should be numbered contiguously starting from zero.
If there are any holes, that is to say missing sequence numbers, then whenever system settings are saved, it gives an exception like this:
Failed to save your settings: NullPointerException during UpdatePropertiesRequest: null: request=22 com.springsolutions.exprodo.core.domain.message.UpdatePropertiesRequest
Issue: 4272
BUGFIX: Bootstrap a virgin database and fails with exception if no licence
Changes
Prevent linked bookings using linkedBookingParent as link to child
The property Booking.linkedBookingParent exists so that a child linked booking knows who its parent booking is.
You may also provide a property that allows a parent booking to reference the child.
This ought to be a custom property that you add for the purpose, as we do not provide a property for it since it necessarily implies that the parent has only a single child, which may not be true.
However, the UI allowed the property linkedBookingParent to be used for the link from parent to child as well as the link from child to parent.
This is confusing and so is now prevented.
Issue: 3645,3847
Sort list reports using the sort order defined in the bakery
Display 3 letters for the month in the navigator above the calendar instead of full month name
As you clicked the arrows to move forwards or backwards a week or month, then changing from one month to another would mean the buttons would change their location because of the variable month name length.
Changing this to a constant three letters beings better stability to button locations.
Issue: 3864
Add support for a number of templates and rules in the licence to override standard starter licence limits
Currently the starter licence limits the number of templates to four and the number of rules to four, while a premium licence allows an unlimited number.
We can now alter a licence so that additional templates and rules can be added to a starter licence.
Issue: 3958
Append trailing slash to system settings Email Base URL if one not provided when configure new system
The Emailed Base URL system setting can be used by workflows to generate a URL that, when accessed, will result in a call back to Calpendo.
However, if this doesn’t have a trailing slash, and workflows don’t check for and append a slash, then URLs generated by a workflow will not work.
When you configure the initial screen for a Calpendo, it asks you for the base URL, and this now has a slash appended automatically if there isn’t already one there.
Issue: 4059
Add id to buttons and text boxes in login and new user request pages
Make the error message from linked bookings more user friendly
When a linked booking parent cannot be created or updated because an insufficient number of child linked bookings could be created, then the error message presented to the user was a bit confusing.
This language has now been made clearer.
Issue: 4175
Add licence type (full/lite/starter etc) to licence server requests
Add scripts for exporting attachments to the INSTALL directory
The Calpendo release now includes a new directory in the archive in INSTALL/export_exprodo_events for exporting attachments programmatically from a script.
Issue: 4208
Make all biskit type columns use ascii char set and related collation
Some tables store a column that references a biskit type. There can be one of two reasons for this:
In all cases, we have always enforced the requirement that the text used for a biskit type matches a simple expression that means it consists solely of upper and lower case letters, digits and the underscore.
In particular, it cannot have accented characters or any exotic non-ASCII characters.
Each string-valued column stored in the database has associated with it a character set used by that column, and also a collation that specifies how to order and compare values.
The data stored within these columns is always compatible with the database character set known as ‘ascii’, which reflects the ASCII characters.
However, the actual character set defined was left to be the default for the database. Since MySQL has changed the default in recent versions, it means that you can have a different character set being used for some of these columns than others.
The 11.0 upgrade introduces changes to the way repeats are handled.
In particular, it creates a new “repeats” table which stores a column for a biskit type, and as a part of the upgrade, values stored in this column are compared to the ‘cls’ column in the bookings and templates column.
However, since the repeats table is new, it was being assigned the new default character set, and the bookings and templates tables being old meant they could use the old default character set. This caused a failure during the upgrade when run on a MySQL database.
So now we modify all columns that store a biskit type so that they are explicitly declared with the ‘ascii’ character set, along with the default collation for that character set, which is ‘ascii_general_ci’.
Issue: 4209
Make relaxed HTML filtering allow hr and center tags, and also class and style attributes on all tags
Change the label on the “skip 2FA” to “Remember this device”
Provide different messages when forced to configure 2FA to say why
Allow virgin system config page to select whether 2FA forced on admin
It would be normal that two factor authentication is forced on all admin and root users.
However, for automated testing, it is helpful to be able to configure a new system without forcing root and admins users to configure 2FA.
So now, when you configure a new system, the first time configuration page lets you choose whether admin/root users should be forced to use 2FA, or whether they would only be reminded to do so.
Issue: 4308
Tweak the message displayed when a user gives forgotten password identifier
If a user has forgotten their password, they can provide us with an identifier, which may be a login name or an email address, and we will look for users matching that.
We now change the message displayed from this:
Please check your email for the reset code, and enter it below along
with a new password.to this:
If your email address is in our database, we will have sent you an email.
Please check your email for the reset code, and enter it below along
with a new password.Issue: 4319
Security Fixes
On every system, clicking the “Register New User” button results in a list of all registered users (their numerical id number, login name and real name) being sent to the browser.
On all Calpendo systems, it also sends the id number, project code and name of all projects.
This user and project information is not usually required, and should not be available before a user has logged in.
This represents a data breach since it is possible to enumerate all users and projects and all projects on any Calpendo.
If a user biskit references any other biskits, then all instances of whatever type is referenced is also sent to the browser. This is here so that a user that self-registers can select their user type or department etc.
However, if there were any sets of biskits attached to a user, for example their training records, then training record id numbers would also be sent, plus any name if they had one.
We now explicitly refuse to send user or project information, and do not include details of any biskits that are set as one-to-many sets on a user.
Issue: 4294
Bug Fixes
BUGFIX: Template prevents bookings 45 minutes before template start
Create a template rejecting everyone. Set it to start at 9:00.
On the calendar, if you click on any quarter-hour from 08:15, the templat rejects you outright whereas one might expect it would allow you to make a booking 08:15-09:00.
Issue: 3710
BUGFIX: Booking fields that are filtered don’t filter on first click after browser refresh
BUGFIX: Booking references from other biskits do not display name format specified in bakery
If you have a biskit with a booking field on it (ie that links to a booking and also has the little calendar icon next to it to take you straight to the calendar) then the text used for the link is:
Booking ID/start date/start time/finish time
This should be whatever is specified in the bakery as the name property or the format definition for the biskit.
Issue: 3781
BUGFIX: %dateRange% format always repeats the date even for bookings that start and end on the same day
The dateRange property on a booking is correctly displayed as:
[dateRange.start.date] [dateRange.start.time]-[dateRange.finish.time]
Except when the booking stretches over more than one day, in which case it is correctly displayed as:
[dateRange.start.date] [dateRange.start.time]- [dateRange.finish.date] [dateRange.finish.time]
However if you use %dateRange% in the format or when specifying what to show in bookings in the Calendar view, it always uses the version that repeats the date, even for bookings that start and end on the same day.
Issue: 3782
BUGFIX: Columns displayed in a list report reset to defaults after item in list is edited
BUGFIX: Workflow Search Action using a ‘Not All’ condition returns incorrect results
When a Workflow Search Action includes a ‘Not All’ condition on the search, then the results it finds are incorrect.
Issue: 3823
BUGFIX: Display is blank when viewing linked booking null parent-to-child prop
BUGFIX: Reports with null conditions cause multiple failures
Reports can have no conditions, but what that should mean is that the report has conditions that say that are empty rather than the conditions being null.
No report should ever have null for its conditions, although it has been observed.
When this happens, it resulted in an error running the report.
Issue: 3858
BUGFIX: Searching for Calpendo Config generates an IllegalArgumentException
If you try to search for Calpendo Config, then you get an error:
Failed to fetch data: could not extract ResultSet
Issue: 3909
BUGFIX: Holiday dates import not accessible to calculateWorkingHours workflow function
If you use the CSV file import utility to import holiday dates, then the workflow function calculateWorkingHours ignores the imported holiday dates.
Issue: 3923
BUGFIX: Custom Tree Page with grouping by null-valued biskit gives exception
When using a resource biskit for dynamic sub nodes on a custom tree page, if one of the resources has no location and location is used as a property to group biskits the page can’t load through menu with an error.
Issue: 3932
BUGFIX: Resource sort order ignored in Calendar-by-location
BUGFIX: Workflow sort action fails if the list of biskits contains a null value
BUGFIX: When multiple template messages should be displayed, some may not be
If you have a message on a template group, and also have a message on the template directly in the template calendar, then at times when both messages should show, only one does show.
Issue: 4031
BUGFIX: Default time when creating booking is weird when using predefined time slots
BUGFIX: Breaking booking rule on linked booking of one instance of a repeat prevents further expansion of the repeat
BUGFIX: View history of a linked booking and click on one of past versions gives an exception
BUGFIX: Click on biskit can show error saying possible permissions problem
Some permissions could result in an error being displayed incorrectly when you click on a biskit.
Issue: 4099
BUGFIX: Reports with missing columns or bad conditions generate exceptions when run
BUGFIX: Information Pages module does not set up pages in a hierarchy
Calpendo has a module called “Information Pages” which when loaded allows you to display pages about your facility.
This allows you to specify that a page has a parent and children, so that the pages display in a hierarchy using a tree structure.
However, it would always display as a flag list instead of in a tree.
Issue: 4118
BUGFIX: Cannot disable rule when over limit by more than 1 in Calpendo Starter
A starter licence allows only a limited number of bookings rules to be used.
If you are over the limit (perhaps because your licence changed from a premium system to a starter), and try to edit a rule to make it disabled, then you are not able to.
It gives a message “Failed to save Interval Rule” without any more information.
Issue: 4137
BUGFIX: Edit the schedule for a report and the start time-of-day is ignored
BUGFIX: Booking repeat displays badly if the booking layout assigns full detail tab to it
BUGFIX: Booking history fails when EXISTS permission depends on type
BUGFIX: Cannot delete bookmarks (error message says it doesn’t exist)
If you try to delete any bookmark that should be deletable, then it fails with a message to say the bookmark doesn’t exist.
Issue: 4190
BUGFIX: Workflow action displays in red box on history page
When a workflow modifies some data, the history page shows which workflow action made the change.
This is displayed on the history page in a red box, which is normally used to indicate an error.
The red box no longer displays in this context.
Issue: 4193
BUGFIX: Click on group report row shows nothing if time condition to-the-hour, minute or second
Suppose you create a group report of bookings, and add a condition of the form “dateRange.start equals DATE-TIME to the HOUR”.
Then, if the report finds any bookings, click on any of the rows displayed in the group report and it doesn’t find the bookings in that group, and so does not display them.
The same happens if you use a condition that is to-the-minute or to-the-second.
Issue: 4194
BUGFIX: Upgrade directly from 9.0.67 to 11.0 fails
BUGFIX: Tooltip on LinkedBookingLink.cancellationReason does not say when reason will be used
The tooltip on the cancellation reason on a Linked Booking Link did not make it clear when exactly the provided reason would be used, and it wasn’t otherwise obvious.
The tooltip has now been clarified.
Issue: 4201
BUGFIX: Validation errors in linked booking links are not displayed
If a linked booking link is configured to reference a property that has been deleted, then this gives an error.
However, the UI does not tell you what the error really is.
Issue: 4202
BUGFIX: Read-only calendar does not handle non-Latin characters properly
A read-only calendar displays a non-interactive view of bookings.
If any of the resource names or bookings contain accented characters, then the calendar does not display properly.
Issue: 4205
BUGFIX: Read-only calendar in systems created in 10.1 cannot cope with formatted resource IDs
Any Calpendo that was first created in 10.0 or later has a default workflow that can display an anonymously-accessed read-only non-interactive calendar of bookings. This is disabled by default.
The workflow will parse arguments in a URL so that you can, amongst other things, define the resources whose bookings should be displayed.
If you specify the resource IDs using a formatted ID (as is displayed in the Calpendo user interface) then the calendar did not display.
Issue: 4206
BUGFIX: Read-only calendar sometimes fails when specify large numeric value for resource ID
The read-only bookings calendar does not display when one of the resources being displayed has an ID greater than 2,147,483,647 and the user that is the source of displayed styles has no user setting for the resource.
Issue: 4207
BUGFIX: JSON error reported when viewing history of booking modified in 11.0.7
BUGFIX: Unable to login using different user when closing Change Password window
BUGFIX: LazyInitializationException updating booking after having modified workflow with BookingRuleWorkflowEvent
If you modify a workflow that contains a BookingRuleWorkflowEvent, then any time you create or update a booking thereafter you get an exception and are prevented from making the change.
The only work-around is to restart Tomcat after updating any workflow with a BookingRuleWorkflowEvent.
Issue: 4257
BUGFIX: “could not initialize proxy” error when adding a note to a project
BUGFIX: No maximum value defined in Allow 2FA to be skipped for a while field
BUGFIX: Configure notes in layout and then view layout shows misconfigured
BUGFIX: Booking reminders partially broken - wrong content and tries to send emails to email address “null”
BUGFIX: SQL Export of reports & timed workflow events broke start & finish
BUGFIX: TimedEvents don’t work if the start date/time is null and users don’t see the start date/time
BUGFIX: Permissions for repeat templates & bookings are overly restrictive
The change to the way repeats work in 11.0 completely changed the way bookings and templates handle repeats. One of the consequences of this is that a repeat is now a separate biskit that can be queried and modified where before it could be.
That’s because repeats used to exist only as an integral part of the booking or template that owned it, and was stored in the same database table row as the booking or template.
This meant that there was only ever one row for a repeat booking, and so all instances in a sequence had to be identical.
This is now different, and it means the repeat information now lives in a table of its own.
When you edit a repeat booking in such a way that it required a change to the repeat as well as the booking, then it was checking that you had permission to change the repeat as well as that you had permission to change the booking.
However, this doesn’t and can’t work because the repeat does not have enough context to evaluate whether you should be allowed to modify it.
For example, there is no permission one could express directly for the repeat to indicate that the resource’s manager user group should be permitted to modify the repeat because the repeat does not store a reference to the booking.
So now when you modify a repeat booking, it will only check permissions for changing the booking. If you want to limit what can be done to repeats, then create a permission on the booking or template to do so.
Issue: 4280
BUGFIX: Manually set 2FA status to ON can prevent user from logging in
BUGFIX: User names with a period in them are rejected
BUGFIX: “Remember this device” 2FA option left disabled after you paste the code into the text box
BUGFIX: Workflow does not find earlier values in context that it should
BUGFIX: Boot fails every 10,000 boots with complaint about ApproxateDate.year not being saved by its DTO
At boot time, we run automated tests to make sure that any classes that use a data transfer object (DTO) to transport their data over the network are well defined.
Occasionally, when there’s a check on the biskit ApproximateDate’s year, this will fail complaining that it put in zero and got out null with this error message:
Property ApproximateDate.year appears not to be saved by its DTO (put in ‘0’, and got out ‘null’)
This error happens on average once every 10,000 boot cycles as it depends on a random number from 0 to 9,999 coming out as zero.
Issue: 4298
BUGFIX: Misleading message given when virus scanning enabled but virus scanner not contactable
BUGFIX: Log error when file uploads fail so recorded in log & system events
BUGFIX: “Run New Process” from ProcessDef creates URL with unformatted ID
When viewing a ProcessDef, it shows a link “Run New Process”.
This triggers a URL change to something like “#process&processDef=NNN” where the number NNN is the unformatted primary key of the ProcessDef.
However, this should be a formatted ID number because we always format id numbers.
Issue: 4302
BUGFIX: Issue Tracker and Exprodo DB apps cannot skip 2FA validation
If you try to select skip 2FA validation, then it fails when the program name includes a space, because a space is an illegal character in a cookie name.
Issue: 4303
BUGFIX: Cannot log in with 2FA enabled if you have no cookies at all
If 2FA is enabled and you have no cookies at all for the server (eg logging in from an Incognito browser tab or you’ve recently cleared your cookies), then logging in fails with a NullPointerException.
Issue: 4305
BUGFIX: Manually modifying a user’s 2FA status to ON can leave the user in an inconsistent state
When somebody configures 2FA, it stores an extra secret that is used for the login verification. If a user does not have this configured yet, and somebody edits their user record to set their 2FA status to “ON”, then you would be in a situation where the user has no 2FA secret, and yet it would be configured to verify a code using that secret.
This means the user would not be able to log in.
This has now changed so that when a user is created or modified and their 2FA status is set to ON, but they have no 2FA secret configured, then we change their 2FA status to FORCE.
Issue: 4307
BUGFIX: Login with 2FA enabled and skip cookie present fails if user ID small
If the value of a user ID or an authentication method ID is small enough that it fits into a Java Integer, rather than a Java Long, then the login process registers an error when there is a cookie present that would otherwise allow the 2FA validation to be skipped, giving the user this message:
Something unexpected went wrong with checking whether we should
skip 2FA validation for USER and so we will not skip itNote that you can only get a user whose id is sufficiently small by starting with a version 9.0 or earlier system, and upgrading it to 12.0.
In that case, the user’s ID would contain no time component, and would be numerically small enough to trigger this bug.
Issue: 4309
BUGFIX: Bad message if use remember-this-device as one user, then login as another
Suppose you log in as a user with 2FA and tick the “remember-this-device” box.
Then log out and log back in as a different user that also has 2FA configured.
Then the user is allowed in (if they enter the right password and code) but they are shown this message:
Something unexpected went wrong with checking whether we should
skip 2FA validation for USER and so we will not skip itWhat it should have said was that the remember-this-device info was set up for a different user, and so it does not apply to this user.
The messaging around remember-this-device has now changed so that the user is shown good information, and we also log a system event to record everything that happens with remember-this-device, whether it’s accepted or not, so admins can see what’s going on.
Issue: 4310
BUGFIX: Log in to Enterprise as user forced to config 2FA gives exception
If you try to log in to an Enterprise system that requires 2FA, then you get a ClassCastException.
Issue: 4311
BUGFIX: Upgrades from any pre-12.0.2 system to 12.0.2/12.0.3 fail upgrade
There’s a problem with a database upgrade script related to 2FA that means the upgrade generates errors.
Issue: 4312
BUGFIX: Shallow exception on copy of workflow with GetConstants action
Create a workflow that contains a DefineConstants actions and save it.
Then edit it again and add a GetConstants action to the same workflow pointing to the DefineConstants you just created.
Now create a copy of that workflow and save it, and you get a shallow exception that prevents you from saving the copy.
Issue: 4313
BUGFIX: New user registration fails in an Enterprise system
When you try to register, it appears to work except that an error flashes up briefly after you enter your details and click the register button.
The message says something like:
Could not create fragment 956 for a child of ExprodoUser - that is probably not fragmented. We calculated the fragment type as ExprodoUserFragment956
This bug was introduced in 11.5.40 as part of the fix for issue 4164.
Issue: 4314
BUGFIX: Subroutine outputs initialised to wrong value if I/O name clash
If one of the outputs on a subroutine has the same name as one of the inputs, then it is possible that the output would be initialised incorrectly.
More precisely, the output value would be written to twice. Once with whatever the correct value is, and another time with null.
It is undefined what order these two settings would occur in.
This means it might work correctly, and it might not. It might also change behaviour in different implementations of Java.
Issue: 4315
BUGFIX: Copy self-referencing workflow gives workflow that references original workflow
BUGFIX: Partial session remains usable if user delays 2FA validation
BUGFIX: Cannot log in if login name contains the @ symbol
11.0.9 (and so also 11.5.44) introduced a limit to the characters that can be used in a login name.
However, there are pre-existing users that break the rule, and so can’t log in. Also, they don’t necessarily get a choice of login name when using a single-sign-on system.
In particular, the @ symbol being prohibited is a problem if login names look like email addresses.
This was the set of characters not allowed:
TAB
SPACE
!"£$%^&*()=+{}[]~#'@;:/?<>,|\
This has been changed to be more liberal in 11.0.10 so that the illegal characters are:
TAB
SPACE
!"&*(){}[]';?<>
Issue: 4322
Changes
When you are asked for a 2FA validation code, and click the OK button, there was no feedback to let you know that it was doing anything.
If the result came back immediately, then this would be fine.
But if there’s any delay, then it generates a little confusion.
So the buttons are now both disabled while there is a request in flight to the server to verify the code entered.
Issue: 4323
Bug Fixes
The two database upgrade scripts in 11.0.7/3 are only meant for Calpendo, but neither script specified that it was Calpendo-only and so they ran in other Exprodo apps too.
This meant that upgrading non-Calpendo apps would result in a database upgrade error, albeit a benign one that needed no remediation.
Issue: 4324
Changes
This change was created by mistake - please ignore
Allow 2FA remember-this-device information to support multiple users
When you enter your 2FA code and tick the “remember this device” checkbox, then it remembers that you don’t want to enter a 2FA code every time by storing a cookie in your browser.
This cookie is overwritten whenever anybody else asks to “remember this device” on the same browser.
However, it now remembers both the previous user(s) and new users.
Each user retains their own expiry date for the 2FA information, and the cookie content is signed by the server to prevent tampering.
Issue: 4281
Accept 2FA validation codes with spaces in them
Show all reasons why a password is rejected rather than only the first
Add password minimum entropy checks
Standard password requirements are to force a minimum length of 8 characters, along with an upper and lower case letter, number and a character that is not a number or letter.
Passwords like “Passw0rd!” are designed to pass these tests, and people use them, but they are very weak and easily guessed.
Consequently, we now incorporate something that checks the entropy in a password that shows how strong it actually is.
This is the sort of thing that would say “pwlfjtusnqy” is a strong password, despite having only lower case letters.
The system settings now let you choose a minimum password entropy, so admins can choose how complex they want passwords to be.
Issue: 4329
Show password strength as you enter it
Increase work put into password hashing & rehash existing passwords
Add ability for desktop Calpendo client to have a URL that shows a calendar for a particular resource or resources
Using the standard desktop client, you can now visit a link like this:
https://yourcalpendourl/#calendar&resources=NNN
This will show the calendar for a resource whose primary key (id number) is NNN.
You can also show a calendar for multiple resources using a link like this:
https://yourcalpendourl/#calendar&resources=NNN,MMM
This means we could generate a QR code that resolves to a URL with particular resource or resources, and it would take you straight to a calendar with that resource or resources.
Issue: 4385
Add warning on login if default project is not approved
When you log in, if you have a default project set, and that project is not approved, then display a warning.
For example, the warning for a terminated default project says:
You might want to change your default project
because its status is TerminatedNote that you cannot select a default project that is terminated, but your current default project can be modified to change its status to terminated.
Issue: 4392
Add ability for desktop Calpendo client to have a URL that shows a calendar for a particular location
You can now view a bookings calendar that will show all the resources in a particular location by using a link like this:
https://yourcalpendourl/#calendarByLocation&id=NNN
where NNN is the id number of the location whose resources should be displayed.
Issue: 4393
Add option to choose whether to allow previous/next 2FA code
When you have 2FA configured, you must provide a code from an authenticator app that provides “time-based one-time passwords” where the code changes every 30 seconds.
We now have an option in system settings to allow us to accept a code from either the current 30-second window only (which is the default), or also to accept a code from both the previous and next 30 second window.
The default is that only the code for the current 30 second window will be accepted.
This can be changed to be a little more liberal if required.
This is configured in the 2FA settings section of the system setting’s security tab.
This means we could generate a QR code that resolves to a URL to show a calendar for all resources at a particular location.
Issue: 4396
Security Fixes
If you upload a file as an attachment, then it will now reject anything that is an HTML file with a message saying:
“For security reasons, HTML files cannot be uploaded”
Issue: 4397
Bug Fixes
The two database upgrade scripts in 11.0.7/3 are only meant for Calpendo, but neither script specified that it was Calpendo-only and so they ran in other Exprodo apps too.
This meant that upgrading non-Calpendo apps would result in a database upgrade error, albeit a benign one that needed no remediation.
Issue: 4324
If you try to edit and save the special user “ical_viewer” without even making any changes, then you get an error message that tells you that it does not have an email address set and needs one.
However, this is a special user and should not have an email address set, just like special user “nobody”.
Issue: 4386
If you have 2FA configured and try to log in to a session that has timed out, then you will sometimes get an error message after entering a 2FA code to say that you took too long to enter the code, even if you entered it immediately.
Issue: 4388
If a local user tries to change their password, then they get an exception.
Issue: 4394
If you modify the 2FA system settings, and save it, and then put them back to their original values and save again, then the second save does not happen.
Instead, you are shown this message:
“Nothing has changed, so nothing saved”
Issue: 4395
Changes
Add verbose logging to track file uploads
When uploading a file and storing it in the database, there is now a variety of logging done to help track down what’s happening if things go wrong.
This is enabled by going to #verboseSearch and turning on the verbose setting for AttachmentDAO, TemporaryFileManager and ExprodoUploadServlet.
Issue: 4367
Add support for a downloadable browser log
Sometimes it is necessary for a customer to do something that generates log output in the browser itself.
We now have added support for this, and will instruct customers individually if and when this is required.
Issue: 4374
Add version number to “Boot” system events to make it easier to see
Add support for system setting to allow file upload before user is registered
When a new user registers themselves to request an account, they are offered a form including most of the properties on a user.
If there is a property that stores a file attachment, then this will let them upload a file.
By default, we now prevent file uploads before a user has logged in. This is what we refer to as an anonymous file upload because at the time the user is registering themselves as a user, we don’t know who they are.
You may or may not require that users be able to upload a file at the time they register themselves as a user, and there is now a system setting to allow the choice to be made.
This is on the “Security” tab, and is referred to as allowing anonymous file uploads.
This is disabled by default so that users registering themselves are not allowed to upload a file.
Issue: 4406
Provide indication of why session cookies do not work when they are broken
Display message when user’s 2FA status is automatically modified
Upgrade GWT from 2.12.1 to 2.12.2
Security Fixes
When you have an attachment on a biskit, and view the biskit in a read-only context, then the link to show the attachment has a URL that includes putting your session ID into the URL.
This is bad for security because it means the session ID is visible to anybody looking at server logs, which is not ideal.
When you upload a file, that can send multiple network messages to the server, some of which also include putting your session ID in the URL. This happens more if the file you are uploading is large, or if your network connection is slow (so that the browser asks for status updates while the file is uploading).
The session ID has now been removed from these things.
Issue: 4379
Temporary files are generated in several circumstances. For example:
Downloading those files would then require a key that had generated for it.
However, the download itself did not require that the user be logged in.
This would mean there’s a time window when a temporary file could be downloaded by somebody else. This would require that an attacker get a copy of the file’s key (which is very hard to guess as it contains a 16 character random text string), and they use it between the temporary file being created and deleted.
Temporary files are normally deleted either when the user that generated them logged out, or periodically Calpendo would look for temporary files older than one hour and delete them. This used to happen every 24 hours but now happens every 4 hours.
So the opportunity to exploit the weakness would be very limited.
We now require that the user is logged in, so they must have a cookie containing a valid session ID, and the temporary file must have been created for the user logged in when the download occurs.
Issue: 4380
Whenever a temporary file is generated, it is because of the action of some user. That user is recorded against the temporary file so we know who caused it to be created.
When downloading a temporary file, we not limit the download to the same user that created it. Temporary files are generated as part of a wider action by the user, and there’s no need for anybody to download a temporary file generated by somebody else.
The security benefit from this is probably quite limited because it would be quite difficult for an attacker to download a temporary file generated by somebody else.
Issue: 4382
If you upload a file as an attachment, then it will now reject anything that is an HTML file with a message saying:
“For security reasons, HTML files cannot be uploaded”
Issue: 4397
Bug Fixes
BUGFIX: Parent linked booking allowed to change to time when child is refused
BUGFIX: Enterprise export of project with a populated one-to-many property fails
BUGFIX: New user registration fails in an Enterprise system
BUGFIX: Calendar by location not filtering resources
You can add menu item called “Calendar by location”.
This adds an item to the menu that expands out to provide an entry for every location that contains at least one resource. Click on the location, and it takes you to a calendar for all the resources in that location.
However, this had stopped working so that it was instead showing all resources.
This bug was introduced in 11.0.9.3, 11.5.44 and 12.0.4 as a result of the fix for issue 3985.
Issue: 4325
BUGFIX: Edit a child linked booking and you get a shallow exception
BUGFIX: Error messages containing HTML “br” tags in linked booking error popup
BUGFIX: Cannot save a workflow function with MappedString argument
If you try add a custom workflow function that has a Mapped String as an argument, then it gives an exception when you save it.
Issue: 4350
BUGFIX: Cannot boot when custom function has mapped int arg with no mapped int
If you have a custom function in a workflow, which has an argument taking a MappedInt, but that does not specify which Mapped Int it should be, then the system will not boot.
Note that it’s not easy to generate this situation because the UI will force you to choose a Mapped Int.
However, if you use the SQL export facility, and then import that SQL to another system, it is quite possible to have no MappedInt set on a custom function argument thereby generating this error.
Issue: 4351
BUGFIX: Cannot boot when a function action references unknown biskit type
If a custom function in a workflow specifies an argument that takes a biskit, but it does not specify which type of biskit it is, then the system cannot boot.
Note that it’s not easy to generate this situation because the UI will force you to choose a BiskitDef.
However, if you use the SQL export facility, and then import that SQL to another system, it is quite possible to have no BiskitDef set on a custom function argument thereby generating this error.
Issue: 4353
BUGFIX: Validating workflows does not show errors like unknown BiskitDef
If a workflow function references a BiskitDef that does not exist, then clicking “Validate Workflows” did not show the correct error message for it. (But then, it wouldn’t boot before now either).
Also, clicking on a workflow that included such an error would show an exception pop-up instead of the normal validation error pop-up.
Issue: 4354
BUGFIX: DB schema update fails with message “wrong column type column [start] in table [report_schedules]”
BUGFIX: Uploading files logs errors for things that are not errors
BUGFIX: Update project to assign new user as owner gives ClassCastException
This only happened in limited circumstances, when some debug options were enabled and you edit a project and change the owner to some user not currently associated with the project (and not yourself).
Issue: 4369
BUGFIX: Copy biskits with attachments and new attachment says it has size zero
For example, add an attachment to a booking. Copy the booking on the calendar, and create a new booking. The attachment will be copied to the new booking, which will work.
However, when you look at what the attachment thinks its size is, it says zero.
You can see this in the UI by searching for bookings in the Bookings->Search page, run a group report, and show the attachment.size property.
However, the attachment does have a real size, and downloads properly, so it is lying about being zero length.
Issue: 4370
BUGFIX: Uploaded files from anonymous user if login name contains @
If a login name contains the @ symbol, files uploaded by them would end up not being associated with that user, but as though uploaded anonymously.
Such anonymous uploads used to be accepted, but no longer are.
Issue: 4371
BUGFIX: Updating DB schema complains about workflows.finish column type
BUGFIX: Default database workflow events on new system do not have an update phase set on database events
When a new virgin database is created, it comes with some default workflows that send emails when a user registers and other things relating to the user lifecycle.
None of the DatabaseWorkflowEvents created have a “Database Update Phase” set on them, and this is a required part of dtabase events.
This means they don’t run at all.
New systems created from now on will have this set correctly, and any pre-existing systems which have a DatabaseWorkflowEvents without a “Database Update Phase” set on them will automatically have it set to “After transaction”.
Issue: 4373
BUGFIX: File uploads sometimes end up with zero length files
When a file is uploaded as an attachment to something, then it was sometimes ending up as a zero length attachment.
Issue: 4375
BUGFIX: Repeat.ownerId shows value in reports not formatted as a primary key
Reports showing a value of Repeat.ownerId would show a plain (and large) number instead of formatting it like ID numbers are formatted elsewhere in Calpendo.
Issue: 4377
BUGFIX: History on a Holiday Date Rule gives a ClassCastException
If you have a Holiday Date Rule that contains dates, and try to view the history of changes to the rule, then clicking on any of the versions where it had some holiday dates would given an exception.
Issue: 4378
BUGFIX: When child linked booking change denied, user sees same message twice
BUGFIX: Bad msg format if booking update gives multiple linked booking errors
BUGFIX: Enterprise cannot boot when running on MySQL rather than MariaDB
BUGFIX: “Expected to find N items, but nothing found” error on some reports
If you create a group report on bookings, and then click on a group displayed in the report, you might see the error message:
“Expected to find N items, but nothing found”
This can happen depending on the columns displayed in the report.
For example, if you add a column for projectResourceSettings.costPerHour then this is likely to happen.
Issue: 4398
BUGFIX: Exception if add column on projectResourceSettings to bookings group report
If you create a group report on bookings, and then add a column that has any property that’s on projectResourceSettings, then you get an exception.
Issue: 4400
BUGFIX: Report columns “lastRepeat.start” and “lastRepeat.finish” are not deleted in upgrading to 11.0 and so give errors
The lastRepeat property was removed in 11.0 from Booking and Template, and there was a database upgrade in 11.0.5 to remove columns in reports that referenced lastRepeat.
However, it was written in such a way that it missed any column lastRepeat.foo for some pseudo-property of lastRepeat.
For example, a report that references lastRepeat.start or lastRepeat.finish or lastRepeat.start.year etc would not be deleted.
This means that after upgrade, it would still have a column referencing lastRepeat, and cause an error when such a report was run.
These columns are now automatically removed.
Issue: 4401
BUGFIX: User registration page doesn’t load if there is an attachment property
If there is an attachment property added to the definition of a user, then whenever anybody tries to register themselves as a new user, the user registration page fails to open, instead it gets immediately redirected to the login page.
Issue: 4405
BUGFIX: Project Biskit subtypes are not applied when creating new biskits in v11
If you define subtypes of project, then whenever you create a new project, you are asked which type of projject it should be.
However, the project that is displayed and saved is always the top-level type of project.
This means, for example, that any custom properties added to the subtype of project the user selected would not be displayed or saved.
Issue: 4407
BUGFIX: Uploading a file that is larger than the max file upload size results in browser appearing to hang
BUGFIX: Total time booked rule gives exception when limits are applied across all time and time masks are in use
BUGFIX: Total Time Booked Rule shows bogus error message when acceptable booking time used
When you create a total time booked rule, you can set a custom message.
It comes with placeholders that will be replaced with the total number of hours and minutes that were actually used, so you can have a message read something like:
“You used 5 hours and 30 minutes for this project, but there is a limit of 4 hours”
When the total time booked rule finds zero time is used, then the error message for the rule would be displayed. But where there are placeholders for the amount of time used, they are NOT replaced by the actual time used (zero), but left as placeholders.
This makes for a weird message displayed to the user, which is displayed as a disappearing blue info box in the bottom right corner.
The same is true when it finds there is some time used, but within the acceptable use. This is different code, but with the same problem.
In both cases, the rule should and does accept the booking, but still generates an error message, and one which is not formatted correctly.
It no longer shows an error message in these circumstances.
Issue: 4412
BUGFIX: Modules Services, CTMS and GTD cannot be loaded due to sibling_order
Changes
Process Control - add target path for secondary layout
Process Control - have a separate Writeability option for each layout
Process Control - add button definition that allows navigation between different related process controls Buttons currently allow you to go to different steps within a process control.
It would be useful to be able to define a button which will take you to the current step (or initial step) in a process control for a related biskit.
This has been implemented by adding an action type for each button. Previously, the action type was always what is now known as “Go to step”. These are the options now for what buttons may do:
This last item - edit related biskit - allows you to choose a ProcessDef for editing that biskit (in case there is more than one applicable). It is entirely possible and acceptable that this will mean editing a biskit which has no ProcessDef defined for it at all. But that would mean leaving the “edit-in-steps” user interface, with no direct way back in.
When you select “Go to step”, then it will display a step-selection drop down. When you select “Go to page”, then it will display a page selection drop down. When you select “Show help”, it will display a plain text entry When you select “Show HTML”, it will display a rich text entry When you select “Edit related biskit”, it will display a path selector so you can choose the biskit, and then a ProcessDef selector.
Process Control - remove requirement for the current step property to be called currentStep Currently you cannot create a process control for a biskit unless there is a property called currentStep.
If setting up multiple process controls for different biskits it would be nice to be able to name your current step field something else.
A ProcessDef now has a “Step Property” property that chooses which property stores the current step. So you still need the equivalent of a “currentStep” property, but now it can be anything that stores a step and is automated. This allows a biskit type to have multiple processes defined on it.
Add extra details to the system events recording uploaded files When somebody uploads a file, the system events used to record the IP address, the file’s name, its size and the user that did it. It now also records the MIME type that the server calculates it contains, as well as the MIME type that the browser told us it contains.
This is to help have evidence for when somebody malicious uploads something while specifying that it has a different type of content than it really does.
Add support for restricting which file types can be uploaded Each property that stores an attachment or set of attachments can specify what type or types of file should be selectable. This is used to give a hint to the browser so that a file selection dialog will only let you select the types you choose in the bakery.
However, this has two problems:
Instead of making this property-by-property method stronger, there is now global method. This allows you to be able to say things like “I don’t want anybody ever to be able to upload a .exe file”. When they try to do this, the file will be rejected - even if the upload is done by a malicious entity that uploads a .exe file while telling us that’s it’s really a plain text file.
The way this mechanism works is as follows:
Note: You can read about MIME types (otherwise known as content-type in HTTP headers and media types) here:
The Iana site is the definitive registered list of types, but their site isn’t as easy to read as the mozilla.org site.
Bug Fixes
BUGFIX: Process Definition - Navigation Bars - options to put buttons on left/right don’t work If you select buttons to appear at top/bottom then its fine. If you select left/right then they don’t appear anywhere.
Testing note - you can display buttons at the top and left at the same time. The problem was that only buttons on the top/bottom were ever displayed.
BUGFIX: Process Control - browser refresh causes ‘editing elsewhere’ warning that won’t go away Do a browser refresh while editing a biskit using a process control. You get a popup ‘Warning - you already appear to be editing X elsewhere’ This warning now pops up whenever you try to edit the biskit.
Testing note ————
To make this work, there are two parts to it:
When the client boots, it checks to see if there’s a clientID specified in the URL. If there is, then it uses that client ID instead of the one it calculates for itself. This means that all messages sent to the server will specify that client ID that we received when the client booted - regardless of whether future changes display a clientID or not.
When the process is displayed, it will check to see whether the current URL contains a clientID token. If it doesn’t, then it will add a new token to the history where this new one does include the client ID.
This means that when you start a new process, or do anything that renders a page in a process, the client ID will be put into the URL.
That then means any browser refresh will still have that client ID in the URL, and therefore it will use that as its client ID. Consequently, start an edit-in-steps session and refresh the browser and it will now not give a message saying it’s being edited elsewhere.
Note that if a user were to bookmark a page with a client ID in it, this would effectively disable the check for whether it’s being edited in multiple places. So there is definitely a double-edged sword in terms of the user convenience this provides.
BUGFIX: Virus scanning error is displayed with source = ‘Logging’ in the System Events Test3/0104 (12.0.4#e959ab28)
Testing note - the system event referenced above still exists. A source of “Logging” is used whenever an error is sent to the log, since a copy is created as a system event. After the fix, however, there is also a new system event with a source of “File Upload”, and a category of “Virus scanner unreachable”.
BUGFIX: “Run New Process” from Search page creates URL with unformatted ID Test3/0104 (12.0.4#e959ab28)
A. Run New Process from View Process Def window (Fixed in 4302)
B. Run New Process from Search Page
Testing note ————
Some types of biskit are marked as things that want to be notified after they are loaded from the database or before they are saved to the database. This allows them the chance to customise some things after loading the data or before saving it.
This was not being run when a report loaded. This now does so. However, it is possible that this might cause some trouble because when you run a report, you may well only load some of the data, so when a biskit does this post-load customisation, it may not handle the partial data well. There is something in place to catch that and hide it, so I don’t think we will notice anything wrong from this.
But it is possible something would go wrong when running a report on any one of the types that do this post-load and pre-save work. The full list of affected biskit types are:
Testing should run a list report on each of these to make sure they work. Also, change the list of columns to reduce it to the minimum possible in each case, because that is when they are each going to be stressed the most into misbehaving, if it will go wrong at all.
BUGFIX: Downgrade script for CTMS is not working Test3/0104 (12.0.7#bdb0baac)
Steps:
BUGFIX: When process displays current step name, it displays the previous one Each ProcessDef specifies the format to use for the current page name. This is then shown in the current page name in the top right corner of every page (the “breadcrumb”), and also in the page history page.
However, the page name displayed is always for the previously displayed step rather than the current step.
Eg create a process control with steps “step 1”, “step 2” and “step 3”, then when you’re on step 1, and click next and it takes you to step 2, the page name would show “step 1” instead of “step 2”.
Note that this is different from any layout that you choose, which may display a heading within the page. That would always display correctly. This is only about the page breadcrumb.
BUGFIX: Page name not set when first create new process from hyperlink When you create a new process from a hyperlink, then the page name (for the breadcrumb in the top-right corner and on the page history page) is calculated before the information about the process is available in the browser.
This means the displayed page name is an empty string.
It’s now calculated and displayed properly.
BUGFIX: Process Control - Add priority to ProcessDef Issue 4287 asked for the “currentStep” property to be allowed to be called something else. This was to allow multiple processes to be defined for a single biskit. Now that there are multiple processes possible for a biskit, then it means that when you start editing a biskit, there needs to be a way to choose which one would be used by default.
This is done by specifying a priority property. This is an integer value, and the ProcessDef with the highest numerical value is the one that will be chosen from amongst all that are otherwise applicable. If ProcessDefs have the same value for their priority, then it is not defined which one will be chosen.
BUGFIX: Process Control - Process uses incorrect CSS class name for biskit types with a period If you create a process for a biskit of type X, then it is assigned a CSS class name of “exprodo-Process-X”.
This was not modified in any way, so that if the biskit type X was, for example, Calpendo.Project, then the CSS class name would be “exprodo-Process-Calpendo.Project”. However, this is not a legal CSS name as periods are meaningful in CSS selectors.
This is now modified so that any periods in the biskit type are converted to underscores in the CSS class name generated, which means the above example would now be “exprodo-Process-Calpendo_Project”.
BUGFIX: Add X-Content-Type-Options nosniff header on file downloads When you download a file, there should now be a X-Content-Type-Options header on the download, with a value of “nosniff”.
This tells the browser not to look at the content to try and work out what it is, but to just take whatever the server says it contains instead.
BUGFIX: Use detected temp file content type when claimed type implausible When a file is uploaded, the browser tells the server what the content type of the file is. A malicious client could claim the upload contains one thing, when it actually contains another.
We now check to see whether the claimed content type could plausibly be correct, given the type that has been detected. The two can be different and this can be okay. For example, an XHTML file could be detected as application/xml, which is a supertype, and therefore it could plausibly be XHTML. But if we detect it as application/xml, and it claims to be an audio file, then this is definitely not plausible.
We now detect what the actual file type appears to be by looking at its contents, and if the claimed type is plausibly correct, then we use the claimed type. Otherwise we mark the temporary file as containing the type we detected.
Testing note ————
There’s a limit to what I expect will be possible with testing. You can copy any .exe or .dll file, and rename the copy to end with “.txt”, and then try uploading that. It should detect that this is not a text type, and that it really is an executable, and reject the upload.
Ideally, but probably out of scope, would be using some software to upload a file which lets you customise the content type that the server is told that it contains. Doing this by changing the file extension is probably the best I would expect.
BUGFIX: Server stack traces are given to browser whenever there is a permission denied exception Suppose a non-admin goes to a list of users, selects a user and then goes to the user groups tab.
You can then tick or untick checkboxes and click the button to ask for the user group membership to be changed.
This will result in a “Permission denied” error.
Now the UI does not display any server-side stack traces. However, if you have the browser developer tools displayed when you try to change the user groups, and then look at the network tab, you should see that the response to the change request has the server stacks in the JSON. It’s not displayed, but it’s there, which means a user could see it.
This means we are leaking information that probably should not happen.
This issue is written about permission denied exceptions, but it would apply to all exceptions.
Stack traces will still sometimes be included in responses. This will happen:
You can change a test system to be marked as “not a development server” by logging in as root, going to the bakery and the custom SQL tool, and running this command:
update config_properties set boolval='F' where name='developmentServer';You would then need to restart tomcat for this to take effect.
BUGFIX: Virus scanning a large file upload gives a “Read timeout” error
Changes
Add support for CODE93 barcodes This is an upgrade of the JsBarcode library (http://lindell.me/JsBarcode) from version 3.11.6 to 3.12.1, which has added support for displaying barcodes of type Code 93.
See https://en.wikipedia.org/wiki/Code_93 for details
Any string property can be defined in the bakery to display with a barcode (or QR code), and you select the barcode type from a drop down. This just means there is now an entry for Code 93 in the drop down, and it displays with a different flavour of barcode.
Allow condition parser to parse PK values where long expected when specify search conditions in URL If you go to https://docs.exprodo.com/railroad/ then you will see several “railroad” diagrams that define the syntax of some things used in Calpendo. One of those is known as the “Conditions URL”.
This is so you can have a URL like: https://demo.calpendo.com/#ba&type=Calpendo.CalpendoUser&action=view&condition=givenName/eq/‘Jeremiah’
and it will take you directly to the person whose given name is Jeremiah, or this: https://demo.calpendo.com/#search&searchType=Calpendo.CalpendoUser&action=view&condition=givenName/contains/‘l’
which lets you search for all users whose given name contains the letter “l”.
The format of the conditions on the URL is defined by the railroad diagram, linked above.
Now suppose you want to have a condition that checks whether a biskit’s ID value equals some number. Well you could already do that, but only with a standard unformatted number. That means you could use a condition like this: id/eq/853607537212422
But Calpendo displays ID numbers in a format, so the above number would be displayed in the UI as 20250613.121136.543.6 and there was no way to specify a condition that accepted this format.
That’s what this issue is about. So now you can specify ID numbers using the formatted notation, using a “p” or “P” prefix, so you can use a condition fragment like this: id/eq/p20250613.121136.543.6
Remove Config.bootBiskitValidation and give warning instead if there are biskit validation errors at boot The system settings property Config.bootBiskitValidation is not normally set in production, has no way to turn on in the system settings page, and is generally only used in development. When turned on, any problem found by a biskit validation at system boot time would prevent system boot.
This was a deliberate choice to make it obvious that there was a problem.
However, this was inconvenient when this is a problem that does not need to stop the boot.
The property Config.bootBiskitValidation has now been removed.
In its place, whenever there is a biskit validation failure at boot time, it now generates a warning, and this is seen when you log in regardless of whether you are in production or development, but only within an hour of booting, and in the first 10 logins since boot.
Prevent subject with null, changed or non-unique project/person This is inspired by issue 4040, which notes that it has at some point been possible to have subjects where somebody is enrolled twice into the same project.
Looking at production data, I see:
Implementing this issue:
This means that whenever something tries to create a subject without a person or project, or where the person is already enrolled into the same project, or the subject’s person or project is changed, then we will now get an error.
This is deliberate so that we can find out when this happens. I have no idea when it happens, or indeed if it still happens. It may have been a historical problem that no longer exists. But this change makes it easier to see the problem when it occurs.
I imagine there will be difficulty in testing the implementation of this change to show that it works. The only way would be:
Bug Fixes
BUGFIX: Using metaproperty date.date in a condition causes exception Minimal example: create a workflow as follows.
Save, then click the “run now” button on the user event. You will get the following exception:
Failed to fetch data: Caught unexpected exception while running workflow: No enum constant com.springsolutions.biskit.core.search.PseudoDateProperty.date
Edit: same issue occurs with metaproperty date.time.
BUGFIX: Edit workflow shows IllegalArgumentException it tries to show a drop-down with a list of abstract biskits Test3/0201 (11.5.32#6476cc90)
Steps:
In the Bakery > Create a BiskitDef “Test3942c” and marked it as Abstract
Set the ‘Shareable Table’ field to it’s default value “Shareable with sub-types”
Save
In Admin > Workflow Manager > Create a Define Subroutine action, and assign an input as a list of biskits of the type just created; error message “Failed to load data: IllegalArgumentException during GetBiskitsRequest: org.hibernate.hql.internal.ast.QuerySyntaxException: Shard956.test3942c is not mapped [from Shard956.test3942c]: request=GetBiskitsRequest[biskitType=null, pk=0, typeMap={Shard956.test3942c=null}]” is prompted.
BUGFIX: Calculation to display ‘Cancelling Booking’ pop-up window is wrong
US3 / 0401 (12.0.0#eb082ce7)
Go to Trials > Subjects by Study Select ‘Test 3820’ > Amy Devito Select Visit 1 > Show in Calendar Edit/View appointment > Drag the scroll bar down to the bottom of the window Click Cancel Appointment button; noticed that ‘Cancelling Booking’ pop-up is displayed at the topmost part of the screen and user is not able to select reason for cancelling the booking. See attached screenshot.
*Issue encountered using Firefox browser.
Testing Note ————
This issue is described as a CTMS thing, but it’s not CTMS at all. You must start with a booking for a resource that has cancellation reasons enabled. Then when you view the booking pop-up, and you click the “Cancel booking” button, it will display a cancellation reasons pop-up. That second pop-up should display close to the button you clicked to make it appear. However, it instead appears close to the status displayed in the booking pop-up.
If there are many custom properties on the booking, and you scroll the booking pop-up to the bottom, then it is possible the status will be off screen. Well, that’s where the cancellation reasons pop-up is being displayed next to.
This has been changed so that the cancellation reasons popup displays next to the “Cancel booking” button. This should be much more sensible and expected. For testing, the one thing to watch out for is whether there are any other ways of showing the cancellation reason popup other than clicking the “Cancel booking” button. I don’t think there are, so I don’t think there are any gotchas in there. But it’s worth bearing in mind just in case.
BUGFIX: Syntax colouring not displayed in pop-up text editor Test3/0102 (12.0.0#eb082ce7)
Setup:
Steps:
BUGFIX: Failed to load bookings: ShallowException error is prompted in Calendar page Test1/1101 (11.0.9.1#575f74b5) Login as eillen.vispo (admin)
Steps to replicate:
Other Testing:
Testing note ————
It doesn’t make much sense to deny READ access on a booking and to allow EXISTS access to bookings. The reason is that if you deny READ access, then the booking is useless because you can’t display it on a calendar since you don’t know the resource or the date/time. However, if you set up permissions that way, then when displaying bookings in the calendar, you get a ShallowException as described above.
BUGFIX: Clicking ‘x’ button of Change Password pop-out window does not let another user to login Related to Issue 4256
Test3/0104 (12.0.4#e959ab28)
Pre-requisite: Password reset of user ‘mary’ is set to ‘Password must be reset on next login’
Steps:
BUGFIX: Password change Apply button must be disabled until user was able to enter data in the required fields Test3/0104 (12.0.4#e959ab28)
Pre-requisite: Password reset of user ‘mary’ is set to ‘Password must be reset on next login’
Steps:
Testing note ————
I find this issue description confusing. The words in the detail description don’t seem to be true to me - or possibly they were true when it was written, but I don’t think they are now. I’ve changed the order of doing things in the server so that it now checks you’ve entered the right password before it checks the legality of the new password so that it will complain about the old password being wrong before it will complain about the new password not meeting the password requirements.
However, the one-line issue description says something quite different. It looks like a different issue altogether. It’s asking for the Apply button to be greyed out until you’ve entered something in the fields. I’ve changed the code so that it will grey out the apply button while the new password and the repeated new password are different. I think that’s as far as that can go.
BUGFIX: Unable to change password when user is forced to configure 2FA Test3/0104 (12.0.4#e959ab28)
Pre-requisite: Password reset of user ‘mary’ is set to ‘Password must be reset on next login’ 2FA Status = Force
Steps:
Testing note ————
Now, when both are required to be reset, then it forces you to do 2FA configuration first, and then once you’ve reset 2FA, it forces you to change your password.
BUGFIX: Unpinned pop-out window did not disappear after changing to a different page Test3/0104 (12.0.7#bdb0baac)
Steps:
BUGFIX: Downgrade script for GTD is not working Test3/0104 (12.0.8#8789d9a3)
Steps:
Note from Paul: The GTD module includes an HTML file that describes its contents. When unloading the module, the HTML file is interpretted as if it contains SQL commands, and this is what breaks things.
BUGFIX: Cannot set SMTP system settings to use email addresses with angle brackets like “name” Go to system settings, and the email tab. Enable email, if not already enabled, and add an entry for SMTP From and SMTP Reply To that is of the form: “name” email@address
For example, it might be: “demo calpendo” calpendo.demo@calpendo.com
or “My Calpendo” something@somewhere.com
Then save system settings. What you should see is that anything between < and > is removed.
This is happening because the properties smtpFrom and smtpReplyTo were marked as not being allowed to contain any HTML, and this was causing the bits between < and > to be removed.
Version 12.0.9 modifies two things:
This was discussed here: https://exprodo.slack.com/archives/C06GACZHVB8/p1753799596073039
BUGFIX: No warning given when editing a mapped integer that is being used as a bitset and add out-of-range values When a mapped integer is used as a bitset, only numbers 0-31 are allowed. But when adding a value to a mapped integer, there is no warning about this, and higher values than 31 are added without issues; they then cause problems in the bitset (32 is identified with 0 and so on).
It would be good if the system could, on edit of a mapped integer, check if that mapped integer is being used as a bitset, and issue a warning to the user to not add integers over 31.
Testing Note ————
Note that there was previously a warning when you edited a PropertyDef, for example when you make it a BitSet and assign it an incompatible MappedInt. But there was no check when you edit a MappedInt that is already marked as being used as a BitSet.
This new issue relates specifically to editing a MappedInt that is already being used for a bit set property.
BUGFIX: CTMS: Add task to visit and you get an exception This bug was observed in 12.1, and is easy to replicate there:
On adding the task, you get an exception.
This bug shows up in 12.1, and not in 12.0.8. It is possible this will only show up in 12.1, although there is the potential for it to happen in 12.0. The drop-down filtering support added to 12.1 makes some things happen in a different order, and that triggers this bug. But it is strictly speaking a bug in 12.0 (and 11.5 and 11.0 and 10.2), but will be harder and perhaps impossible to occur there.
This is being fixed in 12.0 in case it makes a difference there.
It’s a CTMS error, so it should be tested in 12.0. However, if I’m right and you can’t replicate it easily in 12.0, then don’t spend much time trying to replicate it.
BUGFIX: Database upgrade script for Step.writeable2 ran even when not needed If you downgrade from 12.0.8 to 12.0.7 and then upgrade to 12.0.8 again, there would be an attempt to create the property definition for Step.writeable2 again.
This would create an error in system events and in the tomcat log.
BUGFIX: Downgrading Enterprise and re-upgrading fails on some upgrade scripts Any database upgrade scripts that create biskit definitions, property definitions or java enum definitions would have a problem on downgrade followed by upgrade on an Enterprise system.
When an upgrade must add a new PropertyDef, it takes some special steps that affect only Enterprise systems:
When there’s a downgrade, then the relevant biskit definitions etc would be deleted. However, the records to say what had been imported or exported were not changed.
Consequently, if we then upgrade again, there’s a problem because it tries to insertupgrades with 12.0.8 have been modified.
the import or export records again, and it fails because they already exist.
This means a downgrade of Enterprise and then an upgrade can cause an error at boot time.
This has been addressed in two ways:
An upgrade that auto-inserts these import/export entries will now delete anything referencing the same things before doing the insert. In the normal course of events, this won’t change anything. If there has been a downgrade that did not delete the relevant entries then it will automatically replace them.
There’s now support for automatic deletion of the import/export entries when a database is downgrade. This requires a special line in the downgrade script.
Note that this has NOT been applied retroactively, so only those downgrades to 12.0.8 or later will benefit from this. It means that one should avoid downgrading an Enterprise system to anything earlier than 12.0.8.
BUGFIX: FileType is not auto-exported correctly on Enterprise systems Version 12.0.8 introduced the ability to support restricting which file types can be uploaded. Previously, you could limit an individual attachment property to accept files with a particular file suffix. But 12.0.8 allowed global settings of which file types exist, and what sort of content that contain so that we could, for example, prevent a Windows .exe file being uploaded with a .txt extension.
To support this, a new biskit called “FileType” was created. These specify the name, MIME type and file extensions for each file type, and also a flag to indicate whether we should allow such files to be uploaded.
However, in an Enterprise system, the automatic records that are created to show the import or export of the FileType properties were recorded incorrectly, so that it would appear they were not imported or exported.
BUGFIX: PropertyDef for ProcessDef.priority was not created in 12.0.8 12.0.8 introduced a new property on ProcessDef called “priority” as part of issue 4424 (“Add priority to ProcessDef so when multiple apply, we can pick one”).
However, the database upgrade script that added this PropertyDef had a bug that prevented the PropertyDef from being created.
BUGFIX: Auto-generated SQL for permissions does not specify new “root” value Permissions can now indicate whether they are a “root” permission. This means only root can modify them. This was introduced for issues 4230 and 4241.
When generating SQL for existing permissions on the #generate page, the SQL did not include specifying the column value for the “root” property.
BUGFIX: System settings are auto-modified every day If you look in the audit log history of the system settings (biskit type “Calpendo Config”), you will see that it is automatically modified every day by Calpendo itself. This is because it keeps track of some things and stores them in the system settings because it needs them at boot time.
This covers three items:
So these things are automatically changed all the time, and this causes noise in the audit log because it’s hard to see what configuration changes have been made when the audit log is flooded by these automatic changes.
Another problem is that if you leave open a Calpendo on the system settings page for a day, then it’s guaranteed that it will have been modified in the meantime, so you won’t be able to save any changes you make.
Worse than that, if the automated update happens after your browser’s Calpendo session starts, then you won’t be able to edit the system settings either because of the automated update.
To address this, we now have a new biskit type called “Boot Settings”.
This contains all the automated information, and it also records some information about the version of Java, the Calpendo version, and the underlying operating system name, version and architecture.
The other information it stores is:
You can search for Boot Settings in the general search page in the usual way, but you cannot modify any of its properties, nor can you create new instances.
Testing note
BUGFIX: Cannot execute schema update when it includes “;” on a line by itself When you click “Update DB Schema” in the bakery, sometimes it will generate an update script that has a semi-colon on a line by itself.
If this happens, then asking it to then apply the script to update the schema gives an error like this:
Database schema update failed: Failed to run update: ;;...
caused by 'org.hibernate.exception.SQLGrammarException: could not execute statement'...
caused by 'could not execute statement'...
caused by '(conn=2040) Query was empty'BUGFIX: CTMS module creates ctms_subjects table with wrong character set for the biskit type We now control the character set used for columns that store a biskit type, but this was not being done for the ctms_subjects table
BUGFIX: Log in with 2FA and remember-this-device active duplicates session When you log in, a session is created for you and stored in the database.
With 2FA enabled, the session is initially marked as invalid. Only after you provide a valid 2FA code is the session marked valid so that it can be used.
If you have “remember-this-device” enabled, and you log in as somebody for whom there is an active “remember-this-device” in play, then the 2FA validation part is done automatically.
However, instead of modifying the existing session to mark it as valid, it creates a second session which is marked as valid and with a new (integer) id number, but with the exact same value for all other properties (such as the string session id).
BUGFIX: Audit log only showed times to the minute, whereas seconds available The audit log only displays the minute when something happened, but it’s often useful to know to the second. This information was already available in the database, but just not displayed.
BUGFIX: Borrowing an Enterprise login session with 2FA doesn’t work Suppose you start off logged out of all shards in an Enterprise system with 2FA enabled, and then log in to one shard and complete the 2FA check.
Then in another tab of the same browser, you go to another shard.
Then this should cause your login session from the first shard to be “borrowed” by the second shard. This means that the second shard will ask other shards if this user has a current session, and if so, they are allowed to log in without providing their login details.
At the same time, the original shard’s session should become marked as “borrowed” (there is a property on Session for that).
However, while the originally session was marked as having had its 2FA check validated, the borrowed copy on the second shard did not mark the session as valid, and so the login did not work.
Consequently, the user was shown a message suggesting the server was down (which it wasn’t).
Further, the original session, which should be marked as borrowed, did not have its “borrowed” property set to true.
BUGFIX: Enterprise FederationImport’s targetUrl shows hyperlink with unformatted ID In an Enterprise system, if you look at a FederationImport instance, then it shows a targetUrl as a link to the biskit being imported.
Click on it, and it does show the biskit being imported, with the URL including the id number for this biskit. However, the id number is an unformatted number.
This is similar to issues 4302 and 4344 which were the same kind of thing, but for ProcessDef instances.
BUGFIX: Optimise execution of reports with date conditions Some tables end up with a lot of data. For example, system events, audit log, bookings, templates and (if you collect actual usage from a PC-based instrument) remote user logs.
Let’s take remote user logs as an example. Given a database with 4 million rows, a report with conditions like this takes several seconds to run:
(created earlier than now plus 14 day to the minute)
and
(created later than now minus 4 day to the minute)whereas the report is almost instantaneous if you change the conditions to this:
(created earlier than now plus 14 day to the second)
and
(created later than now minus 4 day to the second)The problem is that way we handle the rounding of the dates to the nearest minute to do the query is relatively slow.
When we have report conditions that are rounded to-the-second, then we don’t do any rounding, and so the result was already fast.
But rounding like to-the-minute or to-the-day is much slower.
We now include two optimisations. One is a general one for any date-time relations where there’s a fixed value to compare against (even one calculated as an offset from “now” like above), and the other is in the particular case, as in the above example, when you are searching for something based on when it was created.
The general optimisation is that the rounding we do is slow, and so we avoid that by calculating an exact time to compare against, and use that in the same way that we do to-the-second searches.
The second optimisation has some limitations. We can only use it when:
Then in that case, we convert the date/time limit we would normally use, and generate a primary key value from that. Then look for biskits by their primary key value instead.
BUGFIX: Auto-logout on MySQL systems whenever exception reports (aka RecordedEmail cannot be created) In systems using MariaDB, we create a stored function in the database called formatpk and another called parsepk.
These are generally only useful when somebody is manually interrogating the database, as they convert an id number to and from its formatted form.
Calpendo almost doesn’t use it itself.
However, there was one dependency, which was in RecordedEmail. It has a formulaic property “triggerURL” which is a URL to the workflow action that sent the email, if it was sent by a workflow. The URL is calculated in a way that used the formatpk stored function.
Now MySQL is more fussy about some things with stored functions than MariaDB, and it can mean that formatpk and parsepk do not exist in a MySQL Calpendo database.
What this would mean is that on a MySQL database, if the user did anything which triggered an exception, and the exception triggered an email being sent to report the error, then the RecordedEmail could not be saved, and this ended up with the browser showing a message to say the server was down.
(Note that system settings has an option on the General tab “Submit crash reports” to control whether an email is sent back to Exprodo Software when there is an execption as described above).
BUGFIX: Forced password change pop-up gives exception when you enter new password If you are forced to change your password when you log in, then on typing anything at all into the new password field, it would generate an exception that would pop-up.
BUGFIX: First-time 2FA configuration fails forcing you to configure twice Suppose you create a new local user, and set their 2FA status to FORCE.
Then log in as that user and it takes you through the 2FA configuration.
However, after configuring 2FA, it then makes you go through the 2FA configuration a second time.
BUGFIX: FileType was missing text/plain for *.md Markdown files If you try to upload a Markdown file - this is a text file with a “.md” file extension, then you would be refused the ability to upload it for security reasons as an unknown type.
This is a command type that should be allowed.
The fix for this is to create an instance of the “FileType” biskit - this is new in 12.0 - that defines the different types of files, and indicates which ones are allowed to be uploaded.
So you can test this in two ways:
Search for “FileType”. You should find in 12.0.8 an instance with the name “Markdown” and MIME type “text/markdown”. But that wasn’t sufficient when I tried to upload a *.md file. In 12.0.9 you should see an additional item with name “Markdown”, with this one having MIME type “text/plain”, file extension .md and uploadable set to “Upload of this file type is allowed”
Try uploading a file that contains some markdown and has a name ending “.md”. It should fail to upload in 12.0.8, and work in 12.0.9. Any plain text content can be used as a markdown file content. So a file containing just the word “Hello” would suffice, for example.
BUGFIX: Change message on skip-2FA token signature failure Sometimes, a cookie indicates that the 2FA token should not be checked when logging in. For this to be valid, it has to be signed, and it must have been signed by the current system.
When this fails for some reason, then the message was a little opaque in its meaning than it should have been.
It did say: Received 2FA skip token with wrong signature
it now says: Ignoring 2FA skip token. Either it is malformed, or signed by a different Calpendo
Security Fixes
SECURITY FIX: Change password pop-out window is not prompted and incorrect username is displayed on the right-top corner of the screen US8/0502 (12.0.9#c066d557)
Steps:
Bug Fixes
BUGFIX: ‘Change Password’ window is displayed twice upon login Test3/0104 (12.0.9#c066d557)
Pre-requisite: Password reset setting of ‘userb’ and ‘userc’ should be set to ‘Password must be reset on next login’
Steps:
Testing note ————
This issue appears to be stale. I don’t think the description is true, and this probably means it was fixed earlier.
Modified steps to demonstrate the reported issue.
BUGFIX: Layout: top level heading doesn’t save tags Create a layout with text in the Heading field in the top level of the layout. add tags to alter the text (eg
) Save - the tags disappear. (nb the tags save perfectly fine in any child headings)
BUGFIX: IllegalArgumentException is prompted when selecting an input of ‘List’ or ‘Set’ of Biskit upon creating a workflow This is related to issue 3971
Test3/0104 (12.0.9#c066d557)
Steps:
*The same error is encountered when selecting an input ‘Set of Biskits of Test 3971’
BUGFIX: Failed to create booking: ShallowException is prompted upon creating a booking Related to issue 4174
US8/0502 (12.0.9#c066d557)
Pre-requisite: A permission to hide booking with ‘Room’ resource type should be setup (352 Test 4174 Permission)
Steps:
BUGFIX: CSV Import will not load csv file - wants GZIP format
When using the CSV file import page, it gives an error message:
Failed to fetch data: Could not read uploaded file: Not in GZIP format
Issue: 4566
BUGFIX: Database upgrade in 12.0.9 sometimes fails on process_defs table Some upgrades to 12.0 fail with a SQL error about a problem with the process_defs table having a row that is too large. Specifically, with this error:
Row size too large (> 8126). Changing some columns to TEXT or BLOB or using ROW_FORMAT=DYNAMIC or ROW_FORMAT=COMPRESSED may helpIt’s not clear exactly when this happens and when it doesn’t, but the fix is straightforward: we added lots of varchar(255) columns to the process_defs table, and for some databases (possibly MySQL?) it triggers an error. The fix for this is to make the columns be of type “longtext” instead. This allows more text to be stored, but unlike a varchar, it doesn’t contribute to the row size limit.
BUGFIX: Password rating pop-up remains when password field loses focus When you enter some text into a password field, it will show the password rating pop-up. This is what tells you how good your password is, and what you need to do to it to make it acceptable.
When you use the tab key to move to another field, or click elsewhere, and the password rating popup would remain on screen - possibly hiding the thing that now had focus. It should disappear instead.
BUGFIX: Press escape when password rating pop-up shown and it hides too much Suppose you have a list of users, and you click on a user. Then you see a pop-up with the user in it. Now edit the user, and go to the password field and type something. A pop-up appears which tells you about how good or bad the password is.
Now if you press the escape key, then the password rating pop-up would disappear, as it should. However, the user pop-up would also disappear, which it should not.
BUGFIX: Upgrade to 12.0 fails on MySQL with error “Illegal mix of collations” Full error message is: Illegal mix of collations (utf8mb4_0900_ai_ci,IMPLICIT) and (utf8mb4_general_ci,IMPLICIT) for operation ‘=’
It’s not clear whether this is a failure that will always occur, but it related to upgrades in 12.0.2 and 12.0.3. That means that upgrading from 11.5 or earlier might cause this when running on MySQL.
This relates to temporary database tables that used just for the duration of the upgrade.
Optimisations
Optimisation: Bookings calendar loads bookings more slowly than it could (when there are many bookings)
Security Fixes
SECURITY FIX: Change password pop-out window is not prompted and incorrect username is displayed on the right-top corner of the screen US8/0502 (12.0.9#c066d557)
Steps:
Bug Fixes
BUGFIX: Cannot save any report that uses a referenced-by condition
BUGFIX: CTMS reception page shows an error whenever trying to use it
BUGFIX: CSV Import will not load csv file - wants GZIP format
When using the CSV file import page, it gives an error message:
Failed to fetch data: Could not read uploaded file: Not in GZIP format
Issue: 4566
BUGFIX: CTMS subjects are not directly creatable, but create button enabled
BUGFIX: Referenced-by condition report gives error about unknown props
Suppose you search for any biskit of type X, with a referenced-by condition such as:
Referenced by Y.x with z = nullThen when you run the report, it will complain about “Y.x” and “z” being unknown properties.
Issue: 4572
BUGFIX: Export of a workflow to SQL with a CreateVariables action fails
BUGFIX: Bookings calendar is missing data from some properties
BUGFIX: Report specifying exact start and finish date/time finds nothing
BUGFIX: Rounding dates up did not round to the last possible moment
BUGFIX: Rounding dates down to the month or year gave the wrong date
BUGFIX: Forgotten password page confused by auto-completing code and password If your login fails and you are using a local authentication method, then you are shown a “forgotten password” link. This lets you enter your email address, and it will send an email with a code and a link to the forgotten password page.
You copy and paste the code, and enter your new password twice, and then
your password will be reset.
On at least Microsoft Edge, the forgotten password page auto-completes
the form so that the code is filled in with your login name and the
password is pre-populated as well.
Neither of these should ever be filled in automatically like this.BUGFIX: All-day calendar bookings with custom HTML code shows HTML code instead of formatted content
BUGFIX: All-day bookings are displayed as though not all-day in the calendar
BUGFIX: FileType is marked as not enumerable, which is wrong
BUGFIX: Exprodo Authentication Method fails to verify 2FA with 520 status code
BUGFIX: CTMS errors in a project are display with HTML escape codes
Changes
Add a custom string PropertyDef to Config and you can’t save system settings
Modify ICS attachments to be more likely to be handled as an invitation
Bug Fixes
BUGFIX: Some alert messages displayed contain HTML source code tags displayed as if plain text
BUGFIX: CTMS error when click on any appointment which has events in reception view
BUGFIX: Removal of Config.icon failed in 11.0.0 upgrade
BUGFIX: Generate system event whenever relative time events are refreshed
BUGFIX: Templates display at the wrong time on the background of the bookings calendar
BUGFIX: Create copy of workflow with CreateVariables & original loses variables
BUGFIX: Export to SQL of workflow containing SortWorkflowAction fails
BUGFIX: Bookings calendar fails to load with error about shallow property Resource.aferFinishedChangeOption
BUGFIX: Timezone info was set in the wrong place in ICS files
BUGFIX: Add a prefix to the summary on each ICS VEVENT of the resource name
The SUMMARY on a VEVENT in an ICS is generated from information on a booking, but this does not include the resource name.
The enhancement is that the summary should add a prefix using the resource name to make it easier to see what resource this is for in a calendar importing the ICS file.
The resource name will not be present if permissions are set to hide it.
Issue: 4623
BUGFIX: ICS files specify a non-standard “Mailto:” for organiser/sentBy
BUGFIX: A booking owner without an email address generates non-standard ics
BUGFIX: Cannot import SQL for workflow generated from 12.0.11 (sibling_order)
BUGFIX: Calendar bookings coloured by Project.colour show as black
If you configure calendar bookings to be coloured by using their Project colour, then the bookings would display as black on the calendar regardless of the actual project colour.
Issue: 4627
Changes
Currently, the length of an appointment is determined by the time and gaps between all events created for that appointment.
When defining a visit, you can now schedule a task which will create an event that is not time critical for the appointment length.
For example, a blood lab event to process blood taken, which needs to be scheduled at some point after the ‘blood taking’ event but which could be done after the subject had left the premises.
Issue: 4644
Version 12.0 now has support for pop-ups that can be pinned so that you can keep it present while you then go to another page, so you can see things side-by-side.
If you find a biskit via standard search and click on the list item to show detail, it opens in a separate pinnable pop-up.
However, if you are viewing a biskit from a menu custom tree such as “CTMS→Trials→Subjects by Study” then this option isn’t available.
So there’s now a button ‘Open in Popup’ which allows you to view the current biskit in a pinnable pop-up. This is available on all biskits when viewing its detail, but not already in a popup.
Issue: 4645
A booking that is tentative or requested has a button to allow it to be confirmed.
This means “promoting” the status to requested or approved.
Sometimes, the confirm button is disabled. This will happen if there are any unsaved changes because the confirmation process is quite complicated, and so we avoid trying to save changes at the same time as the confirmation.
There is now a tooltip you can see when you hover your mouse over the Confirm button that will tell you why it is disabled, when it is disabled, and will tell you what it will do when it is enabled.
Issue: 4646
There are now system events that can optionally be created whenever somebody verifies their 2FA details.
To turn this on, go to #verboseSearch and search for something whose name contains “TwoFactorAuthenticationHandler”, and enable it.
From that point on, running 2FA checks will generate system events saying what happened in the checks.
Issue: 4656
There is a problem with sometimes being unable to use the bakery to update the database schema or reload the database configuration due to running out of database connections.
The bakery System Info tool now includes more information about the database connections than it did before.
Database connections are obtained in “pools”, and whenever the schema is updated or the database configuration is reloaded, then a new pool is created so that two pools are in use, before this then drops down to one by the old pool being closed (if reloading the database configuration) or the new one is closed (after updating the database schema).
So there should always be either one or two pools in use.
The UI now shows for both the current main pool and all pools, the number of connections that are idle, busy, or are unclosed orphans as well as the total number.
It also shows the number of connection pools.
Further, this information is now recorded in the database along with the “System Status” biskit which records information about memory usage every 10 minutes.
It is also written out every time somebody asks the bakery tool to refresh the system status information.
Issue: 4657
Optimisations
Bug Fixes
When dragging an appointment in the calendar, it shows an exception saying “Shallow property”
Issue: 4603
Suppose there is something in place which stops a person from being enrolled into a project. For example, a task has an ancillary resource set on it, and the resource in question requires that all projects making bookings for it are authorised, but this resource is not specified in the project’s “resource settings”.
Then trying to enroll a person into the project won’t work, and it displays a message (correctly) saying the person could not be enrolled into the project.
However, a subject was still created, and it was one without any appointments.
The subject should not have been created at all in this kind of situation.
Issue: 4604
When editing a subject, if you change the date of an appointment, Calpendo will show all the consequential changes to other events and appointments.
If you then cancel those changes, and modify the time, but not the date of the appointment, then it only shows you changes it proposes to make to events in the current appointment and it does not cascade those changes to other appointments as it should.
Issue: 4605
When the Confirm button on an appointment is disabled, while looking at a Subject, if you click on another appointment and back to the original appointment, then the Confirm button would be enabled again, even when it should not be.
Issue: 4606
When viewing a subject, the tree of appointments and events still show as tentative even after they have been confirmed, until subject is closed and reopened.
Issue: 4607
When an appointment with at least one event is approved using the Confirm button, then the events are all confirmed (assuming nothing stops that), but the appointment is not confirmed.
Viewing the log detail available in the UI, it says something about the appointment having the old version of an event.
Issue: 4608
When an appointment is confirmed, but one or more of the appointments or events cannot be approved or made requested, then the information popup telling you what happened doesn’t tell you enough to really understand.
For example, something might go wrong with one of the events, which would in turn mean that the appointment would not be confirmed. But the message would tell you how many bookings had been confirmed without any indication that some were not, or that it had approved only events.
Issue: 4609
When your mouse hovers over a booking in the calendar, a tooltip is generated and displayed.
Sometimes, the tooltip is sometimes generated in the browser, and sometimes generated by asking the server to generate it.
If the tooltip definition references a non-existent property (for example you set it up and then removed the property it references), then whenever the server generates a tooltip, it generates an exception.
Issue: 4635
Suppose a user exists in the system, and then a new required property is added to the definition of a user.
Then editing this user is not allowed unless the required property is provided.
This meant that if this user logged in and tried to configure 2FA, they would not be able to do so.
This is now allowed, regardless of whether there are any required properties that have not been set on the user object.
Issue: 4640
Version 12.0 added a property “valid” on a session, which records whether a user that has authenticated still needs to do 2FA validation.
When a user does NOT have 2FA, then as soon as they identify themselves and provide the right password, they should get a session marked with valid=true.
When an 11.0 system is upgraded to 12.0, then a new “valid” column is added to the sessions table. However, no value is provided for “valid”, so it is treated as invalid.
A consequence of this is that a user whose session was working just before a 12.0 upgrade would find that they have to log in again immediately after the upgrade. This is because their session is marked as not valid.
Issue: 4642
Depending on Calpendo configuration, when fetching bookings for the calendar, there could be an exception message saying “ShallowException”, with no bookings displayed at all.
Issue: 4643
Changes
Can’t create new instances of FileType
The properties name, mimeType and fileExtensionsCSV on FileType were all marked as not editable in the bakery.
This meant that if you tried to create a new instance, then you couldn’t enter values for those properties, and if you edit an existing instance, those properties couldn’t be modified.
Issue: 4581,4676
Modify ICS attachments to be more likely to be handled as an invitation
Log files overwrote the previous one when restarted
Add new version of workflow function createCalendarInvite
Add support for Export button to export in JSON as well as SQL
Bug Fixes
BUGFIX: Timezone info was set in the wrong place in ICS files
BUGFIX: Add a prefix to the summary on each ICS VEVENT of the resource name
The SUMMARY on a VEVENT in an ICS is generated from information on a booking, but this does not include the resource name.
The enhancement is that the summary should add a prefix using the resource name to make it easier to see what resource this is for in a calendar importing the ICS file.
The resource name will not be present if permissions are set to hide it.
Issue: 4623
BUGFIX: ICS files specify a non-standard “Mailto:” for organiser/sentBy
BUGFIX: A booking owner without an email address generates non-standard ics
BUGFIX: Upgrade from 10.1 to 11.0 can convert bookings that were never repeats into custom repeat booking
When upgrading from 10.1 to 11.0, the biggest change is to the way repeat bookings and templates are handled. In 11.0, there is a separate table that stores the repeat information, whereas in 10.1, the repeat info is stored directly in the same row that stores each booking or template.
Due to the nature of how repeats work in 10.1, when a repeat instance goes into the past, a non-repeat booking is created to represent the state of that repeat as it was at that time.
However, this process loses information about what type of repeat it was.
For this reason, the upgrade will convert some non-repeating bookings to repeating bookings with a special repeat type of “Custom”.
However, if you use the booking search page to create a copy of a booking that is one of this non-repeating bookings that records what a repeat looked like when it went into the past, and then upgrade to 11.0, then the new booking would also be converted to a custom repeat.
Issue: 4670
BUGFIX: Cannot export to SQL a workflow with a Timed workflow event that repeats
BUGFIX: Cannot create a virgin system (connection to database is lost)
BUGFIX: 2FA verification system events logged as errors instead of info
Changes
Allow Enterprise history page to show history from other shards
Add licence information to Enterprise Tools “System Status” tab
Enterprise should allow bakery search for properties to include references in all shards
Add workflow function ‘exists’ to check existence of a Biskit
Add workflow function ‘getName’ to return the name of a Biskit
This is a workflow function with two versions: one that takes a biskit, and the other takes a BiskitDef and a numeric ID.
The function calculates the name of the biskit.
This uses the format or name property as specified in the biskit’s BiskitDef.
Issue: 4686
Bug Fixes
If you log in to an Enterprise system, then on most biskits that you view, there will be a “Share” button that lets you share that biskit with another shard, see whether it is shared, or unshare it.
Once you see the button, if you log out and log back in, then the button no longer displays. A browser refresh is required before the button appears again.
An earlier fix for this issue (in 11.5.7) addressed this problem for the “Export” button (which produces SQL) and “Shareability” button, but not the “Share” button.
Issue: 3741
This issue was originally fixed in 12.0.13, but only for users authenticated using an Exprodo authentication method.
This now works for all users.
Issue: 4640
If you have a dynamic BiskitDef in an Enterprise system, then it will have a type like ‘ShardN.Foo’ where N is the shard number and Foo is whatever name you give it.
Then, in the bakery if you like ‘Create Copy’, it would automatically choose a type for the copy as ‘ShardN.Foo2’, which is okay, and a table name of ‘shardN_shardN.foo’ and an ID column name of shardN_shardN.foo_id
Both the table name and ID column name should not have a period in them because it is illegal.
Also, the duplication of “shardN” should not happen.
Issue: 4682
Changes
This is a change in support of a fix for issue 4680, which relates to scrollbars not appearing when some actions become too tall for the vertical screen space available for them.
In the case of Import Workflow Action, it displays two list of things where both can grow.
So that they can both display scrollbars at an appropriate time, they have been switched so that instead of appearing in two caption panels, they instead appear in tabs.
Issue: 4702
When creating a condition on a string-valued property, where you can choose a relation like “equals” or “does not equal”, there are now new options for “Equals ignore case” and “Does not equal ignore case”.
The query API also offers an option for these new relations so you can have something like this:
https://some.calpendo.com/api/q/Calpendo.CalpendoUser/email/EQUALS_IGNORE_CASE/thing@place.comhttps://some.calpendo.com/api/q/Calpendo.CalpendoUser/email/DOES_NOT_EQUAL_IGNORE_CASE/thing@place.com
Issue: 4725
Bug Fixes
BUGFIX: Workflow debugger shows “now” date conditions incorrectly
BUGFIX: Booking display flags don’t appear when conditions rely on properties not otherwise needed for calendar display
Bookings display flags can specify conditions which must be true for the flag to appear on the calendar.
However, if you chose conditions that referenced booking properties that were not otherwise used to display the calendar, then the flags did not appear.
Issue: 4662
BUGFIX: “On Error = Continue” in workflow can cause actions to run twice
Workflow actions have an “On Error” option on the “Control” tab that lets you choose what should happen when there is an error running the action.
The default is that the workflow would be stopped at that point, but there is an option of “Continue”, which can occasionally be useful.
However, using this could sometimes cause child actions of the action that has the error to be run twice.
Issue: 4663
BUGFIX: “On Error = Continue” in workflow does not allow sibling actions to run
When an action is configured with its “On Error” set to “Continue”, then its sibling actions would not run, even though they should do.
Issue: 4664
BUGFIX: Transition of near/far term templates wrong when using “Months” units
When you specify the time when a template transitions as a number of months, rather than days or weeks, then the calculation of when this should happen is based on one-twelfth of a year.
That means if you specify one month as the period, then the transition actually happens 730 hours from now, or 30.4 days.
Depending on the number of days in the current month, this can happen before or after you would be expecting.
This has now been fixed so that a month before gives the same day-of-month on the relevant required month.
Note that if this puts it on a month when the same day-of-month does not exist, then the actual date will be found by rolling forward the required number of days.
For example, 1 month before March 31 2026 would be February 31, 2026. But that is 3 days beyond the end of February, so it is converted to March 3, 2026.
There is now a warning issued when a template specifies a number of months to be used for the transition between the phases of a template.
It is recommended to use a number of days or weeks instead of months so that the behaviour is perfectly clear.
Issue: 4669
BUGFIX: Search and While conditions do not acquire a scroll bar
All actions have conditions that control whether the action runs.
However, the Search and While actions both have an additional conditions entry.
This is used to identify what to search for and whether to continue a “while” loop.
If these conditions were to grow to be bigger than the screen space available, then they would not display a scrollbar making some of the conditions unreadable without making the window bigger or reducing the font size.
They now both display a scrollbar when required.
Issue: 4678
BUGFIX: Allow non-repeat bookings to be deleted from booking search
A recent change (issue 3922) prevented any bookings from being deleted from the booking search page. This was because doing so can cause real problems to repeat bookings.
Instead of deleting, bookings were cancelled.
This has now been changed so that repeat bookings are cancelled, but the option to properly delete non-repeat bookings is reinstated.
Issue: 4679
BUGFIX: Some workflow actions have content that can grow arbitrarily large without ever displaying a scrollbar
BUGFIX: Scrollbar in layouts does not allow one to reach the last properly
The layout editor doesn’t quite show all the things it should. The bottom of some of the panes are cut off, so that even when there’s a scrollbar, it doesn’t let you scroll all the way to the bottom.
Issue: 4681
BUGFIX: User needs permission to read child of one-to-many biskit property in order to edit parent
If you have a parent-children relationship (a one-to-many), and a user has permission to edit the parent, but does not have READ permission on the children, then the user is not able to modify the parent.
Issue: 4688
BUGFIX: Number of bookings rule doesn’t trigger past repeatable expansion date
BUGFIX: Edit Resources To Display (when creating Bookmarks) behaves in a non-intuitive manner
When editing the resources to display on the bookings calendar, if you use filtering by any of the options on the left (eg location and type), then Shift-click to select a range of resources can produce a counter-intuitive result, where more resources are selected than are displayed.
Issue: 4693
BUGFIX: iCal feed makes all-day bookings appear to be one day longer than they should
BUGFIX: Gaps form in repeat bookings over time when using a booking subtype
If you create a repeat booking for a resource that uses a subtype of Booking, rather than Booking itself, then over time the repeat sequence does not generate all the instances it should.
When creating a repeat booking, it looks okay.
The problem occurs later as, with an extra one-day gap appearing every day.
Issue: 4701
BUGFIX: Newly created tables need a second Update DB Schema to complete
When you create a new BiskitDef, it can sometimes require two schema updates to get it into the right state rather than doing it in a single update.
Issue: 4703
BUGFIX: Timeout and localhost ignored when sending email via smtps
When you set the “security type” to “SSL/TLS” on the email settings in system settings, then the settings for “localhost”, “timeout” and “connectiontimeout” are not set.
Issue: 4704
BUGFIX: Cancel repeat bookings ALL-ITEMS from booking search cancels only one
If you check one or more repeat bookings in a booking search and click the delete button, then it will cancel the selected bookings rather than actually deleting them.
It will ask you to choose whether you want to cancel THIS-ITEM, THIS-AND-LATER or ALL-ITEMS.
If you choose THIS-AND-LATER or ALL-ITEMS, then it should cancel not only the instance(s) you select, but also all subsequent items in the same repeat sequence.
However, it only did the equivalent of THIS-ITEM and did not cancel any others.
Issue: 4706
BUGFIX: A workflow that sends an email with no content or attachments fails
If you set up a workflow to send an email with a subject but no plain text body and no HTML body, and also to have no attachments, then on trying to send the email, you get an exception and the email is not sent.
Issue: 4708
BUGFIX: Delay workflow actions fail to run
A delay workflow action is a mechanism for choosing a pause within a workflow.
This was no longer working, and would instead always fail when it tried to run.
Issue: 4710
BUGFIX: Linked bookings can clash with each other and so can’t move with parent booking
BUGFIX: Relative links in a page banner are removed
BUGFIX: Group report cannot successfully group RecordedEmail by “To” address
Suppose you have many outgoing emails that go to the same email addresses, and you then search for RecordedEmail to see what outgoing emails you had.
If you use a group report and add a column for the “To” addresses, then even though you may have multiple emails going to the same addresses, the report will not group them together.
Issue: 4715
BUGFIX: Search doesn’t always add forced date bounds on searches when required
BUGFIX: Relogin breaks ability to view biskit with collection of children
BUGFIX: Number of Bookings rule does not prevent bookings that repeat forever
When a booking repeats, and the repeat has no end date, then it repeats forever.
This effectively creates and infinite number of bookings.
This is incompatible with a Number Of Bookings rule because that rule applies a limit to the number of bookings; whatever the limit set in the rule, something that repeats forever will go beyond it.
Therefore, if a Number of Bookings rule applies to any booking that repeats forever, then that booking will now be prevented, regardless of the number of bookings the rule allows.
Issue: 4723
BUGFIX: Number Of Bookings rule does not count repeats beyond expansion range
When using a Number of Bookings rule, it expands repeats from now until the expansion limit, and uses that for the count of the number of bookings it finds.
However, if you make a booking beyond the expansion limit, then the rule should count repeat bookings further into the future.
So it now adds the number of expansion days to the date of the booking you are creating or updating to decide how far to expand bookings when checking a Number Of Bookings rule.
Issue: 4724
BUGFIX: Biskit query API case sensitivity differs from the UI when using string property conditions
If you go to a user search page and add a condition to search for users with a particular email address, then this search will be case sensitive.
However, if you generate the equivalent using the Biskit query API, then this is case insensitive (apart from an Enterprise system, where it is case sensitive).
This inconsistency has been fixed so that it is always case sensitive.
If you want a case insensitive match, then there is a new case insensitive relation for that.
Issue: 4726
BUGFIX: View Enterprise calendar with bookings from multiple shards with different Booking BiskitDefs gives exception
When an Enterprise bookings calendar displays bookings for resources from different shards, and where the bookings have different properties defined on them, then the calendar shows an exception when it fetches bookings.
Issue: 4727
BUGFIX: Biskit layout heading cannot include relative links or links without protocol
BUGFIX: htmlClean workflow function did not allow relative links or links without protocol
BUGFIX: The bakery allows BiskitDef with an illegal table or id column name
In the bakery, if you create a new BiskitDef, and then before saving, you insert a period into the table name and the ID column name, the save will still go ahead.
However, this will not work and so should be prevented.
We now require the table name and column name to start with a letter or underscore, and contain only letters, digits or underscores.
Issue: 4730
BUGFIX: Upgrade GWT from 2.12.2 to 2.13.0
Changes
This allows us to set up a licence for a Calpendo Starter which extends the normal limit of custom properties from 20 to whatever number we want on a case-by-case basis.
Issue: 4740
Bug Fixes
BUGFIX: Project dropdown omits projects which the booking owner owns
When system settings sets “owner defaults to project owner” to no, a user cannot edit a booking that is owned by somebody else and change the project to a project owned by that user, even if the logged-in user is a user on the existing and desired project.
Issue: 4733
BUGFIX: Bakery reload DB config and update DB schema fail with all 20/20 connections in use
This is a continuation of attempts to solve issues around an inability to update the DB schema or reload the DB config in the bakery due to insufficient connections being available.
The problem is not reliably provoked, which makes fixing it difficult.
Issue: 4738
BUGFIX: System Status biskit properties given incorrect labels
Labels for properties allPoolsNumberOfConnections, numberOfConnectionPools and allPoolsNumberOfBusyConnections were mixed up.
Issue: 4739
BUGFIX: Loading CTMS module gives bakery validation error with on CTMS.Person.bmi
Bug Fixes
This is a problem with the new property ExprodoLicence.maxNumberOfCustomProperties that was introduced in 12.0.17.
Issue: 4745
Changes