Skip to main content

Release 3.2

· 5 min read
Dirk Lerner
Dirk Lerner
Creator of ExMeX Framework and Co-Author of the book Data Engine Thinking

We are thrilled to announce the release of ExMeX Core Framework 3.2, packed with powerful new features and performance enhancements designed to elevate your data management experience. This version delivers key improvements in performance, security, and functionality, ensuring your workflows are more efficient and secure than ever.

The focus of this release is on optimizing data processes, streamlining workflows, and providing advanced tools for data management, archiving, and compliance.

Upgrading to version 3.2 is straightforward, following the principles of Semantic Versioning. With our commitment to backward compatibility in minor releases, you can confidently upgrade without fear of breaking existing functionality.

Enhanced Metadata Installation​

In addition to minor improvements, there is now an option to delete all data models in a specified section at once. The MetadataInstall.bat script includes a new parameter, --metadata-bulk-del, to facilitate this. The goal is to achieve significantly better performance when installing metadata. For example, to reinstall only the metadata in the "All" section without compiling modules, you can use the following command:

MetadataInstall.bat --no-compile-lps --metadata-bulk-del All

For detailed guidance, refer to Metadata Install Tool (Batch File).

Enhanced Archiving Framework​

Effective data archiving is essential for maintaining system performance while controlling storage costs. The Archiving Framework in ExMeX 3.2 allows for seamless archiving and deletion of outdated data, based on configurable object and retention policies.

With this powerful tool, you can securely store historical data while ensuring it remains accessible when needed. You also gain the ability to automate data purging once retention periods expire, keeping your system optimized and compliant.

For detailed guidance, refer to our Archiving Framework documentation.

Improved Hash Management: Surrogate Key, HashDiff, and HashCDC​

In this release, we’ve added more flexibility to how hash values are generated for your data solutions. This includes updates to Surrogate Keys, HashDiff, and HashCDC. However, when altering existing hash values, proceed with caution to avoid unintended consequences in your current setup.

Checksum (HashDiff & HashCDC)​

With the IsUtf8Checksum framework variable enabled, checksums will now be generated from an NVARCHAR(MAX) string (UTF-8 encoded). If needed, you can exclude specific checksums and generate them using a VARCHAR(1024) string by applying the VarcharChecksum column stereotype.

This applies to:

  • Hub-Satellite HashDiff (3)
  • Link-Satellite HashDiff (5)
  • CDC-ODLB-Stage (16)
  • CDC-HiZ-Stage (17)
  • One-to-One Data Copy (19)

Surrogate Key​

Similarly, when IsUtf8HashKey is set to yes, hash values for surrogate keys will also be generated from an NVARCHAR(MAX) string (UTF-8). If necessary, specific values can be excluded by using the VarcharHashKey column stereotype.

This applies to:

  • Hub (1)
  • Link (2)
  • Hub-Satellite HashDiff (3)
  • Hub-Satellite Insert Only (4)
  • Link-Satellite HashDiff (5)
  • Link-Satellite EndDating OneToMany (6)
  • Hub-Satellite (7)
  • Link-Satellite (8)
  • Hub-Satellite Bitemporal (9)
  • Link-Satellite Bitemporal (11)
  • Generic (12)
  • PIT (13)
  • Bridge (14)

New Framework Plugins​

PowerDesigner - ExMeX Plugin (Metadata Export)​

Version 1.10.0 of the PowerDesigner ExMeX plugin introduces several notable improvements:

  • New Feature: Export Replica Mapping Information (Ticket 0000654)
  • Improvements: Code structure enhancements for better performance and maintainability
  • Added Support: UTF encoding compatibility

These updates simplify the metadata export process, making it easier to integrate and use within your existing workflows.

Core Framework Enhancements​

The ExMeX Core Framework has received key updates in version 3.2, including improved compatibility with UTF-8 encoding. You can now create hash values for checksums and hash keys using NVARCHAR() strings, ensuring seamless integration with modern encoding standards.

Patches Included in This Release​

  • Patch 669
  • Patch 668
  • Patch 664
  • Patch 661

Resolved Tickets​

This release also addresses several critical issues and improvements based on user feedback:

  • 0000649: Support for NVARCHAR hashes (Checksum/HashDiff)
  • 0000659: Support for NVARCHAR hashes (Checksum/HashCDC)
  • 0000660: Support for NVARCHAR hashes (SurrogateKey)
  • 0000668: Resolved sync issues in Load Window
  • 0000664: Improved handling of bitemporal data with multiple records
  • 0000661: Optimized restart times for various modules
  • 0000647 & 0000648: Enhanced error detection in data item mapping (Link/Satellite)
  • 0000630: Extended filter functionality for more efficient data processing
  • 0000403: Added comment parameters to the filter USP
  • 0000665: Enabled SSIS package execution with environment variables
  • 0000655: Schema distinction in RowsRead function
  • 0000656: Fixed issues with long-running compiler
  • 0000657: Improved template handling for pre/post-body scenarios
  • 0000658: Enhanced SQL Agent job status checks
  • 0000650: Parallel execution support for all modules
  • 0000666: Resolved blocking issue with sys.dm_db_partition_stats
  • 0000601: Removal of obsolete Stage DDL
  • 0000612: Automated deletion of outdated files during installation
  • 0000669: Corrected typo in DimDate
  • 0000663: Enhanced archiving for logging tables

Upgrading to Version 3.2​

Before upgrading, we strongly recommend backing up your database and the TEDAMOH ExMeX framework folder to prevent data loss. Follow all upgrade instructions closely to ensure a smooth process.

For detailed steps, refer to our Upgrade Guide.

Important Global Variable Settings​

During the upgrade, review and adjust the following global variables in your configuration file:

  • @IsUtf8Checksum (Line 58)
  • @IsUtf8HashKey (Line 60)
  • @RetentionPeriod (Line 152)
  • @ArchivingRetentionPeriod (Line 153)

These variables are located in etc/config/ExMeX Set Config Variable.orig.sql. Be sure to merge them with your local configuration file etc/config/ExMeX Set Config Variable.sql as needed.

New Installation​

For new installations, please follow the comprehensive instructions in our Install Framework Guide.