Profisee is a Master Data Management tool which enables organisations to build a trusted foundation of data. It is growing in popularity largely thanks to its ease of use, fast time-to-value deployment and cost effectiveness.
Profisee continues to deliver improvements to the software through each release, and has maintained its position as Challenger for the second year in a row on Gartner’s 2020 Magic Quadrant for MDM solutions.
With each release of Profisee, we update this page so you can:
- Get a breakdown of the latest features and functionalities added
- See how Profisee has progressed
- Decide what can be implemented for your organisation
Profisee release 2020 R1 – June 2020
The mid-2020 release from Profisee sees the solution make the leap into the Platform as a Service (PaaS) arena with its new containerisation deployment option.
This significant shift enables companies to easily and flexibly deploy Profisee MDM on cloud and private platforms with minimal application configuration and the ability to scale up and down as may be required.
It’s going to be hard to say though, which enhancement is going to have the biggest benefit, as there are other major feature enhancements being released at the same time.
For the time being, the MSI option will continue to be available alongside container deployment. The same application code is used regardless of the chosen deployment option; select whichever form factor best suits your needs.
- Profisee’s container is available via the Azure Container Registry but you will need to contact Profisee support to get the necessary access token and licence file. You will then be able to deploy to Azure using Azure Kubernetes Service (AKS). Refer to the new documentation Deploying the Platform in Azure for details. Note: Amazon certification is coming.
- MSI installations work the same way as before with the exception of including additional steps primarily to support the increased flexibility with regards to authentication and logging options.
Multiple authentication providers are now supported (including simultaneously, if required). This includes 3rd party and cloud-native options based on the Open Authorization 2.0 (OAUTH2) standard, Identity Server, and Open ID Connect (OIDC). Such as:
- Azure AD
- Okta, etc.
Note: Windows Active Directory is still supported for on-prem solutions.
Logging has been updated to support new trends and patterns in containerised architectures. Rather than logging to the Windows Event Log, Profisee will now multicast log entries to file as well as centralised logging in the Profisee database.
Further, logging levels (Verbose, Warning, Error, etc.) are now configurable for troubleshooting purposes and can be searched across nodes in a cluster.
Profisee is leveraging Serilog to enable these new logging features, which is also compatible with other commercial logging platforms should customers wish to centralise their logging and diagnostics in the future.
Matching strategies can now be “turned on”. What it will do is then catch up on any new/changed members since the last time it was on. Then, going forward, as new members arrive or changes are made, the matching index will be kept up to date in real time.
You can use this simply to keep the matching index up to date, but optionally:
- Automatically Match: new/changed unmatched members are evaluated live for matching.
- Automatically Survive: new/changed match groups will immediately have survivorship applied.
This is a pretty big change. And it should remove the need for micro-batching or SDK customisation for near real-time implementations, whilst achieving real-time.
In the future, we expect to see support for automated unmatching and rematching. This has been a popular request among customers.
For hybrid implementations – continuous matching during the day, with overnight batches – the strategy can be paused while you run a batch mode operation and then resumed afterwards.
Survivorship has been made significantly more powerful. For a start, it is now more surgical (able to leverage logging if you have it turned on) in the way that it detects and is applied to deltas.
But further to that you can now filter to specific data sets for which you wish to apply survivorship logic, and then customise that logic for the specific characteristics of each data set. For this purpose, re-using attributes is now possible across multiple survivorship sets.
You can now explicitly choose whether nulls are applied per attribute in an attribute set.
The biggest change, however, is in the flexibility and power now available for configuration:
- Full filter dialogue functionality – now available for group members, masters, and harmonisation. Previously, you could only apply a single condition in each of these places (note: the Filters tab replaces the Value Condition criteria in previous versions).
- Full sort dialogue functionality – you can now choose from, not just the attributes on your entity, but relevant attributes from related domain-based entities as well. Sorting is possible for each attribute set on multiple attributes, descending or ascending, as required.
This added flexibility and power should allow survivorship to be more targeted, potentially reducing processing time. But the biggest impact will be on natively supporting more real-world complex scenarios without the need for additional strategies or external logic.
Workflow has been foundationally reworked. More of the configuration and deployment steps have been automated, making it simpler to register and configure workflows.
You still have to use visual studio, but now you can just import/register your workflows from the VS project file. Once deployed, they can then be exported and stored/imported somewhere else just like other Profisee artifacts.
Forms have been rewritten to address underlying issues as well as giving them a number of improvements:
- Better real-estate usage and nice new animations
- Attribute descriptions can be defined in the model and these will show as tooltips in forms
- More flexible script actions (able to reference attributes that aren’t on the form)
- Missing or unavailable (ie, due to security) attributes are hidden rather than making the form invalid
- Error messages have been made more descriptive
There are a couple of feature improvements not yet covered elsewhere.
- Reversing transactions – the logged changes to each record could previously be viewed by a data steward but there was no way to “undo” certain actions. The way this will work in Profisee going forward is that a steward with sufficient permissions can request to reverse a change. It won’t strictly undo the original change.
Instead, it will record a reversing transaction – much like accounting systems – that will set the values back to the way they were before. Effectively this will cancel out the change in question but still accurately track all changes.
- Re-assigning tasks – sometimes it becomes necessary to change tasks or their assignments as situations change. However, there wasn’t a way to manage tasks like this in the past. Version 2020R1 introduces a new task management interface in FastApp Portal where steward tasks may be reassigned or even cancelled if necessary.
Profisee release 2019 R1.2 – March 2020
To start off the new year, Profisee has provided updates to version 2019 R1 with several quality of life updates and bug fixes to improve the end user experience.
Staging – fixes to UI and performance issues
With several staging bugs reported by users in the previous version R1.1, Profisee made sure to listen and respond to its user community base with bug fixes. As a result, the following staging related issues were corrected in this latest patch.
- Missing UI – for staging tables that haven’t run in the last 25 days, it was noted that the UI disappeared during processing, and not before the process has completed.
- Endless processing – for situations where processing a staging table with the “Clear contents after a successful run option” was selected by the user and if the entity selected with its name containing a space, the staging table processing would never complete successfully.
Improvements to the FastApps Studio Client memory consumption
If you had previously used the 32-bit version of the FastApps studio client, you may have encountered performance issues while running larger models.
If you are like me, and constantly making changes to the models, it is common knowledge that running larger models will always consume more memory. As a result, both the 32-bit and 64-bit version of the FastApps Studio Client in this patch has received substantial performance gains.
In addition, for those users frequently using the administration tabs of FastApps Studio, the slowdown or lag faced while working with the tabs has now been rectified as well.
Unlocking the platform when accessing the same server – fixing problems of long running queries
There will always be a potential resource locking issue when you have a single platform/server shared among multiple users accessing it at the same time.
In the previous version of the platform, if a long query was being run, there were persistent connectivity issues encountered by users with regards to opening new tabs or even trying to connect to the server until the query has finished. Thankfully, this is no longer an issue in this patch. Regardless of the query runs, users can all access the server and platform as intended.
Improvements to the performance of Permissions Calculations – cutting the processing times down
In the scenario where you are making changes to a user or team, you may have noticed the effective permission calculation processing times takes longer than you would like.
In response, Profisee has implemented changes to the permissions logic and resulted in greatly reducing processing times when creating and configuring users/teams.
Profisee release 2019 R1.1 – November 2019
Improved data quality rule / Staging table management experience and performance
Initial portal load times have been reduced by only loading the rules necessary instead of all rules. Further, the rules are now serviced via cache rather than going back to the database.
Administrators will appreciate the new quality of life features for the management of very large numbers of Data Quality Rules or Staging Tables:
- List pagination reducing the volume of data retrieved during rule administration
- Sorting, making rules easier to manage and administer.
Rule cache management has been optimised to reduce cache refresh delays.
Staging redesigned to process through platform API – coupled with general API improved performance
The staging process now routes through the platform API layer rather than separately coded stored procedures. This change reduces complexity, improves reliability and consistency, whilst further positioning Profisee for future architectural changes.
With the staging improvements come the following features:
- Automatic batching – replaces the previous recommendation to manually configure batch loads to avoid loading very large data sets in a single batch.
- Staging parallelism – a new system setting to configure the number of parallel API requests when publishing.
- Enforce constraints – it is now possible to apply constraint rules in the staging process
- Reduced contention – performance of staging batches has been improved by deferring subsequent processes until the full staging process has completed.
Also noteworthy, is that the ability to bulk create staging tables (originally for Merge) has been extended to Delete tables, and further, the Clear action tables have been removed this release after being deprecated in the previous release when the ability to apply nulls was added to Merge tables.
Matching index size optimisations – reduce memory consumption during ongoing operation.
Once built, the match index uses approximately 30% less memory. This is particularly useful when multiple indexes are kept in memory, such as an additional Look Up Before Create index, but it should be noted that the initial build process will still consume the same amount of peak memory.
Data quality rule execution on published matching results
Data quality rules will now execute immediately on any records affected by the results of a matching process. This ensures consistency with other records across the entity.
General stability improvements
For example, a timeout condition was removed when turning on Logging for large entities.
Profisee release 2019 R1 – September 2019
Release overview: This is a major version update of the Profisee platform.
Data quality rules
- Expression language
Expressions in Rule Assignments
- Constraint rules
The expression language is available! Excel-like in nature, we can now reference other attributes, apply functions, do calculations, and leverage varied combinations thereof with logical, numeric, and string operators.
Currently this is available in the What column of data quality rule definitions but will be made available more widely in coming releases.
Data quality rules can now be toggled to apply as constraints. In constraint mode they will act to prevent member creation or update operations where the rule would be violated. This can currently be bypassed by the staging process, but future updates will see constraints applied consistently.
It is now possible to set a default value at an attribute level. This can be done either in Advanced Modelling or Adaptive Modelling, and is a great way to pre-populate entry forms in the Portal.
AD group support
Users can now be managed as individual accounts or via AD groups.
When a new user logs in to Profisee they will inherit their permissions from any AD groups of which they are a member. An individual’s effective permissions will be the additive permissions of the individual, Profisee Team, and AD Group.
- Usability and layout
- Improved Matching Stewardship Control
Steward Match Group
- Quick update
Vast improvements have been made in the usability of the Portal. The layout of each page is now customisable beyond the initial templates provided: frames can be split, merged, and resized as required.
Dead space has been removed by tightening-up white space around the frames, while better use of headers and footers allows more of the frame to be dedicated to content. Forms open within their host frame rather than taking up the whole page, and in cases where there is sufficient screen real-estate the form will slide out and only cover 50% of the frame.
The former case makes it possible to keep other relevant information on screen, and as a bonus, in the latter case, the form even becomes a contextual information panel allowing you to click through members in the 50% of the frame that remains uncovered while having the form refresh itself as you go. When a frame isn’t large enough for its content it will now be scroll-able both horizontally and vertically whilst staying within its frame. Infinite scrolling loads members as you go rather than requiring pagination.
Common tasks such as Analytics and Work are now easily accessible from a tray at the top the menu without taking up space as separate menu items. In case you have a large number of menu items to manage they can now be managed logically in folders. And when you just need a little bit of extra space the menu itself can be collapsed to an icon view.
Profisee is working towards feature parity for stewardship in the Portal and the next two features are a couple of big steps in that direction.
Firstly, Quick Update allows the bulk changing of records by first selecting the members to update and then setting the desired value for each attribute to be updated.
Secondly, the Matching Stewardship Control has some extra features and an additional mode. The existing Group mode allowed stewards to approve or reject members, perform manual survivorship, and merge duplicate members via SAIM. Now it also includes the Match Detail and View Similarity features. The additional mode is Membership mode which allows the viewing of members from multiple groups. They can be pinned and compared, or even shifted between groups. Presentation Views can be used to control the attributes desired to be available.
Unstructured data in the form of documents, images, PDFs, etc. can now be attached to members within a Profisee entity. This functionality has been implemented as a microservice; representing the first of many such planned features as the Profisee Platform architecture shifts towards a fully PaaS offering.
- Bulk staging table creation
- New options to ignore invalid records, and add domain values
- Audit changes as processor
There have been a few welcome changes to the staging process and the level of control available for the processing of data. By popular demand the user account logged as having processed a staging batch will now be the service account (usually, but could be an individual) who triggered the process, rather than the account that initially created the staging table.
In order to provide greater control, we now have the ability to decide whether a staging table will ignore invalid records (ie, process the remaining valid records) or stop the batch as a whole. Further, at an attribute level we can choose to allow new domain members into reference set, or otherwise continue to reject potentially bad data.
Lastly, a quality of life feature, we can now mass produce staging tables. This works by first selecting a list of entities for which we wish to create staging tables, and then choosing the shared features (ie, prefix, suffix, staging options).