Algorithm: Date Shift (KBA7105)
KBA
KBA# 7105This is an Out-of-the-Box algorithm built on the New Algorithm Framework. It updates the functionality of the old Date Shift algorithms. These algorithms can be extended with the Date and the Dependent Date Shift algorithm Frameworks.
Applicable Delphix Versions
- Click here to view the versions of the Delphix engine to which this article applies
-
Major Release All Sub Releases Date Release Jan 25, 2024 19.0.0.0 Dec 20, 2023 | Jan 10, 2024 18.0.0.0 | 18.0.0.1 Nov 21, 2023 17.0.0.0 Oct 18, 2023 16.0.0.0 Sep 21, 2023 15.0.0.0 Aug 24, 2023 14.0.0.0 Jul 24, 2023 13.0.0.0 Jun 21, 2023 12.0.0.0 May 25, 2023 11.0.0.0 Apr 13, 2023 10.0.0.0 | 10.0.0.1 Mar 13, 2023 | Mar 20, 2023 9.0.0.0 | 9.0.0.1 Feb 13, 2023 8.0.0.0 Jan 12, 2023 7.0.0.0 Releases Prior to 2023 Major Release All Sub Releases 6.0 6.0.15.0, 6.0.16.0, 6.0.17.0, 6.0.17.1, 6.0.17.2
At a Glance
Versions | Applicable Delphix Masking versions: From 6.0.7.0. | |||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Released | This algorithm was released in version 6.0.7.0 and will replace the old Date Shift algorithms (KBA1812). It has been created using the Date (Shift) Framework. | |||||||||||||||||||||
Characteristics |
1 Referential Integrity - The masked value will be the same between tables, databases, and jobs. |
|||||||||||||||||||||
FIXED | Fixed adds x days to the date and the date rolls so only the day of the month will change.
|
|||||||||||||||||||||
DISCRETE | Discrete masks all dates, with the same year-month combination, to the same day.
|
|||||||||||||||||||||
VARIABLE | Variable returns a random date within the same month-year as the input date.
|
|||||||||||||||||||||
Data Type | Valid and tested Data Types:
|
|||||||||||||||||||||
Encoding | ASCII Numeric | |||||||||||||||||||||
Strength | Very fast | |||||||||||||||||||||
Customization | These algorithms can't be modified. The framework enables the creation of new customizable algorithms. | |||||||||||||||||||||
For information about investigating errors and the logs:
|
Details about the algorithms
One of each comes pre-configured with the masking engine. These algorithms cannot be modified (Date Shift framework can be used to create similar algorithms).
Memory Requirements
These algorithms have a very small memory footprint.
Masked Value
The built-in algorithms only mask the Day part of the date (including dates with time). The year, month, hour, min, et cetera are left as is.
Date Formats
A date format must be specified. This field appears when you select the DATE SHIFT algorithm from the Masking Algorithm dropdown list.
The default format is 'yyyy-MM-dd' and the list contains some common formats.
You can set the format to any format allowed in JAVA by navigation to the Rulset, editing the column specifying a date algorithm, and then selecting or entering the date format. The information button next to the date format field provides more information on specifying date formats.
Examples of some common formats are (note single-quotes around text):
Tip: 'M' and 'd' for single-digit dates. 'MM' for '01, 02, ..., 12'.
yyyyMd yyyyMMdd yyyy-MM-dd yyyy-MM-dd HH:mm:ss yyyy-MM-dd'T'HH:mm:ss'Z' yyyy-MM-dd'T'HH:mm:ss.SSS'Z'
Special characters and invalid date strings
Special characters and invalid date strings will be passed through unmasked. NULL and blank string will be passed through unmasked and all other cases will be marked as Nonconforming Data (and passed through).
Casting to string
You may need to CAST a date to a STRING for unsupported datatypes. Look at the value and CAST as required and set Date Formats accordingly.
For example, if a date is stored in an unsupported data type and the SELECT returns '20210626', then cast the value to a STRING and set the Date Format to 'yyyyMMdd'
Examples
Masking Examples
Below are examples of the three algorithms: Discrete, Fixed, and Variable.
- ID reference number.
- Orig - The Original date.
- DS_Discrete - The masked value using Discrete.
- DS_Fixed- The masked value using Fixed.
- DS_Variable - The masked value using Variable.
With date only (format: 'yyyy-MM-dd') +----+------------+-------------+------------+-------------+ | id | orig | DS_Discrete | DS_Fixed | DS_Variable | +----+------------+-------------+------------+-------------+ | 1 | NULL | NULL | NULL | NULL | << Passthrough values | 2 | | | | | +----+------------+-------------+------------+-------------+ | 3 | 1977-02-01 | 1977-02-06 | 1977-02-06 | 1977-02-03 | << Note: id 3 same as 4 | 4 | 1977-02-01 | 1977-02-06 | 1977-02-06 | 1977-02-10 | Variable has different value. +----+------------+-------------+------------+-------------+ | 3 | 1977-02-01 | 1977-02-06 | 1977-02-06 | 1977-02-10 | << Different year | 5 | 1978-02-01 | 1978-02-20 | 1978-02-06 | 1978-02-04 | +----+------------+-------------+------------+-------------+ | 3 | 1977-02-01 | 1977-02-06 | 1977-02-06 | 1977-02-10 | << Different month | 6 | 1977-03-01 | 1977-03-13 | 1977-03-06 | 1977-03-06 | +----+------------+-------------+------------+-------------+ | 3 | 1977-02-01 | 1977-02-06 | 1977-02-06 | 1977-02-10 | << Different day | 7 | 1977-02-02 | 1977-02-06 | 1977-02-07 | 1977-02-12 | +----+------------+-------------+------------+-------------+ | 8 | 1978-04-25 | 1978-04-06 | 1978-04-30 | 1978-04-01 | << Month rollover | 9 | 1978-04-26 | 1978-04-06 | 1978-04-01 | 1978-04-28 | | 10 | 1978-04-27 | 1978-04-06 | 1978-04-02 | 1978-04-01 | | 11 | 1978-04-28 | 1978-04-06 | 1978-04-03 | 1978-04-29 | | 12 | 1978-04-29 | 1978-04-06 | 1978-04-04 | 1978-04-07 | | 13 | 1978-04-30 | 1978-04-06 | 1978-04-05 | 1978-04-07 | | 14 | 1978-05-01 | 1978-05-24 | 1978-05-06 | 1978-05-03 | | 15 | 1978-05-02 | 1978-05-24 | 1978-05-07 | 1978-05-06 | +----+------------+-------------+------------+-------------+ Note: DateShift Fixed and Variable will never mask to the same as the input. With date only (format: 'yyyy-M-d') +----+------------+-------------+------------+-------------+ | id | orig | DS_Discrete | DS_Fixed | DS_Variable | +----+------------+-------------+------------+-------------+ | 15 | 1978-05-02 | 1978-5-24 | 1978-5-7 | 1978-5-6 | +----+------------+-------------+------------+-------------+ | 16 | 1978-12-1 | 1978-12-23 | 1978-12-6 | 1978-12-27 | | 17 | 1978-5-12 | 1978-5-24 | 1978-5-7 | 1978-5-6 | +----+------------+-------------+------------+-------------+ With time: (format: 'yyyy-MM-dd HH:mm') +----+------------------+------------------+------------------+------------------+ | ID | Orig | DS_Discrete | DS_Fixed | DS_Variable | +----+------------------+------------------+------------------+------------------+ | 1 | 1978-04-30 14:34 | 1978-04-06 14:34 | 1978-04-05 14:34 | 1978-04-08 14:34 | +----+------------------+------------------+------------------+------------------+
Nonconforming Data Examples
Below are some examples of Nonconforming Data when masking a date.
Date format set to 'yyyy-MM-dd'.
+----+------------------+------------------+-----------------------+ | id | orig | Nonconforming | Comment | +----+------------------+------------------+-----------------------+ | 16 | 1978-12-1 | NNNNPNNPN | Incorrect Date format | | 17 | 1978-5-12 | NNNNPNPNN | Incorrect Date format | | 18 | 1978-04-02 14:34 | NNNNPNNPNNZNNPNN | Incorrect Date format | | 19 | 1978-31-31 | NNNNPNNPNN | Invalid date | | 20 | FOO BAR | LLLZLLL | Not a date | +----+------------------+------------------+-----------------------+
Related Articles
The following articles may provide more information or related information to this article: