Skip to main content
Delphix

Algorithm: Date Shift (KBA7105)

 

 

KBA

KBA# 7105

This is an Out-of-the-Box algorithm built on the New Algorithm Framework. It updates the functionality of the old Date Shift algorithms. These algorithms can be extended with the Date and the Dependent Date Shift algorithm Frameworks.

Applicable Delphix Versions

Click here to view the versions of the Delphix engine to which this article applies
Major Release All Sub Releases
Date Release
Jan 25, 2024 19.0.0.0
Dec 20, 2023 | Jan 10, 2024 18.0.0.0 | 18.0.0.1
Nov 21, 2023 17.0.0.0
Oct 18, 2023 16.0.0.0
Sep 21, 2023 15.0.0.0
Aug 24, 2023 14.0.0.0
Jul 24, 2023 13.0.0.0
Jun 21, 2023 12.0.0.0
May 25, 2023 11.0.0.0
Apr 13, 2023 10.0.0.0 | 10.0.0.1
Mar 13, 2023 | Mar 20, 2023 9.0.0.0 | 9.0.0.1
Feb 13, 2023 8.0.0.0
Jan 12, 2023 7.0.0.0
Releases Prior to 2023
Major Release All Sub Releases
6.0

6.0.15.0, 6.0.16.0, 6.0.17.0, 6.0.17.1, 6.0.17.2

At a Glance

Versions Applicable Delphix Masking versions: From 6.0.7.0.
Released This algorithm was released in version 6.0.7.0 and will replace the old Date Shift algorithms (KBA1812). It has been created using the Date (Shift) Framework. 
Characteristics
Date Shift Framework RI1 1:12 NSV3
Fixed Date Shift (dlpx-core) Yes Yes Yes
Discrete Bespoke Plugin (dlpx-core) Yes No No
Variable Bespoke Plugin (dlpx-core) No No Yes

1 Referential Integrity - The masked value will be the same between tables, databases, and jobs.
2 1:1 Mapping - The masked value will be mapped uniquely to the input value.
3 Never mask to Same Value - Two of the DateShift algorithms are designed so that the masked value will never be the same as the Input value.

  FIXED Fixed adds x days to the date and the date rolls so only the day of the month will change.
  • This algorithm is deterministic:
  • Will not mask to the same value.
  • Will not produce duplicates.
  DISCRETE Discrete masks all dates, with the same year-month combination, to the same day.
  • A new day is returned for each year-month combination.
  • This algorithm will mask a date to the same value 1 in 30 times. 
  • Will produce duplicates.
  VARIABLE Variable returns a random date within the same month-year as the input date. 
  • This algorithm is stochastic.
  • Will not mask to the same value.
  • Might produce duplicates.
Data Type Valid and tested Data Types: 
  • Date, Datetime, Int, Char, VarChar, nChar, nVarChar (and similar). 
Encoding ASCII Numeric
Strength Very fast
Customization These algorithms can't be modified. The framework enables the creation of new customizable algorithms.  
  For information about investigating errors and the logs:


Other sources: 

Details about the algorithms

One of each comes pre-configured with the masking engine. These algorithms cannot be modified (Date Shift framework can be used to create similar algorithms). 

Memory Requirements 

These algorithms have a very small memory footprint. 

Masked Value

The built-in algorithms only mask the Day part of the date (including dates with time). The year, month, hour, min, et cetera are left as is.  

Note

Note:

To mask other parts of the date (or time) is easy. You can create a new algorithm using one of the OOTB Frameworks.

 

Date Formats 

A date format must be specified. This field appears when you select the DATE SHIFT algorithm from the Masking Algorithm dropdown list.

The default format is 'yyyy-MM-dd' and the list contains some common formats. You can also type a date format desired if the date format is not listed. 

Examples of some common formats are (note single-quotes around text):

  • Note: 'M' and 'd' for single-digit dates. 'MM' for '01, 02, ..., 12'.
yyyyMd
yyyyMMdd
yyyy-MM-dd
yyyy-MM-dd HH:mm:ss
yyyy-MM-dd'T'HH:mm:ss'Z'
yyyy-MM-dd'T'HH:mm:ss.SSS'Z'

Special characters and invalid date strings

Special characters and invalid date strings will be passed through unmasked. NULL and blank string will be passed through unmasked and all other cases will be marked as Nonconforming Data (and passed through). 

Casting to string

You may need to CAST a date to a STRING for unsupported datatypes. Look at the value and CAST as required and set Date Formats accordingly. 

For example, if a date is stored in an unsupported data type and the SELECT returns '20210626', then cast the value to a STRING and set the Date Format to 'yyyyMMdd'

Examples 

Masking Examples

Below are examples of the three algorithms: Discrete, Fixed, and Variable. 

  • ID reference number.
  • Orig - The Original date.
  • DS_Discrete - The masked value using Discrete
  • DS_Fixed- The masked value using Fixed
  • DS_Variable - The masked value using Variable
With date only (format: 'yyyy-MM-dd')
+----+------------+-------------+------------+-------------+
| id | orig       | DS_Discrete | DS_Fixed   | DS_Variable |
+----+------------+-------------+------------+-------------+
| 1  | NULL       | NULL        | NULL       | NULL        | << Passthrough values
| 2  |            |             |            |             |
+----+------------+-------------+------------+-------------+
| 3  | 1977-02-01 | 1977-02-06  | 1977-02-06 | 1977-02-03  | << Note: id 3 same as 4
| 4  | 1977-02-01 | 1977-02-06  | 1977-02-06 | 1977-02-10  |    Variable has different value.
+----+------------+-------------+------------+-------------+
| 3  | 1977-02-01 | 1977-02-06  | 1977-02-06 | 1977-02-10  | << Different year
| 5  | 1978-02-01 | 1978-02-20  | 1978-02-06 | 1978-02-04  |
+----+------------+-------------+------------+-------------+
| 3  | 1977-02-01 | 1977-02-06  | 1977-02-06 | 1977-02-10  | << Different month
| 6  | 1977-03-01 | 1977-03-13  | 1977-03-06 | 1977-03-06  |
+----+------------+-------------+------------+-------------+
| 3  | 1977-02-01 | 1977-02-06  | 1977-02-06 | 1977-02-10  | << Different day
| 7  | 1977-02-02 | 1977-02-06  | 1977-02-07 | 1977-02-12  |
+----+------------+-------------+------------+-------------+
| 8  | 1978-04-25 | 1978-04-06  | 1978-04-30 | 1978-04-01  | << Month rollover
| 9  | 1978-04-26 | 1978-04-06  | 1978-04-01 | 1978-04-28  |
| 10 | 1978-04-27 | 1978-04-06  | 1978-04-02 | 1978-04-01  |
| 11 | 1978-04-28 | 1978-04-06  | 1978-04-03 | 1978-04-29  |
| 12 | 1978-04-29 | 1978-04-06  | 1978-04-04 | 1978-04-07  |
| 13 | 1978-04-30 | 1978-04-06  | 1978-04-05 | 1978-04-07  |
| 14 | 1978-05-01 | 1978-05-24  | 1978-05-06 | 1978-05-03  |
| 15 | 1978-05-02 | 1978-05-24  | 1978-05-07 | 1978-05-06  |
+----+------------+-------------+------------+-------------+
Note: DateShift Fixed and Variable will never mask to the same as the input.


With date only (format: 'yyyy-M-d')

+----+------------+-------------+------------+-------------+
| id | orig       | DS_Discrete | DS_Fixed   | DS_Variable |
+----+------------+-------------+------------+-------------+
| 15 | 1978-05-02 | 1978-5-24   | 1978-5-7   | 1978-5-6    |
+----+------------+-------------+------------+-------------+
| 16 | 1978-12-1  | 1978-12-23  | 1978-12-6  | 1978-12-27  |
| 17 | 1978-5-12  | 1978-5-24   | 1978-5-7   | 1978-5-6    |
+----+------------+-------------+------------+-------------+


With time: (format: 'yyyy-MM-dd HH:mm')
+----+------------------+------------------+------------------+------------------+
| ID |       Orig       |    DS_Discrete   |      DS_Fixed    |   DS_Variable    |
+----+------------------+------------------+------------------+------------------+
| 1  | 1978-04-30 14:34 | 1978-04-06 14:34 | 1978-04-05 14:34 | 1978-04-08 14:34 |
+----+------------------+------------------+------------------+------------------+

Nonconforming Data Examples

Below are some examples of Nonconforming Data when masking a date.

Date format set to 'yyyy-MM-dd'.

+----+------------------+------------------+-----------------------+
| id | orig             | Nonconforming    | Comment               |
+----+------------------+------------------+-----------------------+
| 16 | 1978-12-1        | NNNNPNNPN        | Incorrect Date format |
| 17 | 1978-5-12        | NNNNPNPNN        | Incorrect Date format |
| 18 | 1978-04-02 14:34 | NNNNPNNPNNZNNPNN | Incorrect Date format |
| 19 | 1978-31-31       | NNNNPNNPNN       | Invalid date          |
| 20 | FOO BAR          | LLLZLLL          | Not a date            |
+----+------------------+------------------+-----------------------+

 

 


Related Articles

The following articles may provide more information or related information to this article: