Skip to main content
Delphix

Masking and Transformation Order (DB) (KBA6208)

 

 

KBA

KBA# 6208

 

Applicable Delphix Versions

Click here to view the versions of the Delphix engine to which this article applies
Major Release All Sub Releases
6.0 6.0.0.0, 6.0.1.0, 6.0.1.1, 6.0.2.0, 6.0.2.1, 6.0.3.0, 6.0.3.1, 6.0.4.0, 6.0.4.1, 6.0.4.2, 6.0.5.0, 6.0.6.0, 6.0.6.1, 6.0.7.0, 6.0.8.0, 6.0.8.1, 6.0.9.0, 6.0.10.0, 6.0.10.1, 6.0.11.0, 6.0.12.0, 6.0.12.1, 6.0.13.0, 6.0.13.1, 6.0.14.0, 6.0.15.0, 6.0.16.0, 6.0.17.0, 6.0.17.2

5.3

5.3.0.0, 5.3.0.1, 5.3.0.2, 5.3.0.3, 5.3.1.0, 5.3.1.1, 5.3.1.2, 5.3.2.0, 5.3.3.0, 5.3.3.1, 5.3.4.0, 5.3.5.0, 5.3.6.0, 5.3.7.0, 5.3.7.1, 5.3.8.0, 5.3.8.1, 5.3.9.0

5.2

5.2.2.0, 5.2.2.1, 5.2.3.0, 5.2.4.0, 5.2.5.0, 5.2.5.1, 5.2.6.0, 5.2.6.1

5.1

5.1.0.0, 5.1.1.0, 5.1.2.0, 5.1.3.0, 5.1.4.0, 5.1.5.0, 5.1.5.1, 5.1.6.0, 5.1.7.0, 5.1.8.0, 5.1.8.1, 5.1.9.0, 5.1.10.0

5.0

5.0.1.0, 5.0.1.1, 5.0.2.0, 5.0.2.1, 5.0.2.2, 5.0.2.3, 5.0.3.0, 5.0.3.1, 5.0.4.0, 5.0.4.1, 5.0.5.0, 5.0.5.1, 5.0.5.2, 5.0.5.3, 5.0.5.4

At a Glance

Summary: This page describes all transformation auxiliary jobs in a Database Masking Job and the execution sequence of these auxiliary jobs.
Versions: This KBA is applicable for all Database Masking jobs - for Oracle and SQL Server this applies:
  • Oracle - all versions up to 6.0.10.
  • SQL Server - all versions up to 6.0.11.
Auxiliary Jobs: The Pre Auxiliary jobs are: 
  1. CREATE IDENTITY TRANSFORMATION (optional)
  2. DROP INDEX TRANSFORMATION
  3. PRESCRIPT TRANSFORMATION
         a. Custom Scripts
         b. Triggers
         c. Constraints


Masking Job: Mask Tables


The Post Auxiliary jobs are: 
  1. POSTSCRIPT TRANSFORMATION
         a. Custom Scripts
         b. Triggers
         c. Constraints
  2. CREATE INDEX TRANSFORMATION
  3. DROP IDENTITY TRANSFORMATION (optional)
Note: The steps in Pre- and PostScript Transformation are executed in parallel. This means that Triggers and Constraints can finish before Custom Scripts. If these features are used together, this needs to be considered to ensure correct execution. 
Masking Logs:  Logs from these transformations are only written when the transformation has finished. If any of these steps are taking a longer time, this will not be visible in the logs until the transformation is done.
More info: Troubleshooting:

 

This Knowledge Article describes the job execution and the order of auxiliary jobs.

Note

Notes:

  • Only the masking operations are tracked in the Job Monitor.
  • The Auxiliary job might take a long time to complete (neither the progress nor any errors are indicated in the Job Monitor).

 

Transformation Order

 

KBA-Masking Transformations Pre Post.png

 

Pre Processing
1. CREATE IDENTITY TRANSFORMATION
2. DROP INDEX TRANSFORMATION
3. PRESCRIPT TRANSFORMATION
    a. Custom Scripts
    b. Disable Triggers
    c. Disable Constraints

Masking Transformation
1. INNER_JOB
    a. Mask Stream S0
        - Table 1, Table 2, ...
    b. Mask Stream S1
        - Table x, Table x+1, ...
    n. Mask Stream Sn
        - Table y, Table y+1, ...
    
Post Processing
1. POSTSCRIPT TRANSFORMATION
    a. Custom Scripts
    b. Enable Triggers
    c. Enable Constrains
2. CREATE INDEX TRANSFORMATION
3. DROP IDENTITY TRANSFORMATION

 

Main Masking Auxilary Job

Sub: INNER_JOB

The transformation INNER_JOB manages streams and the execution of each Masking transformation. 

Note

Note:

Details about which stream is used and the table/file order are unknown and can be different between executions. 

 

Auxiliary Jobs

Listed below are short descriptions of each auxiliary job.

Note

Notes:

  • These steps can take a long time.
  • These steps will neither show the Progress Bar nor any errors in the Job Monitor.

 

CREATE IDENTITY TRANSFORMATION

If a table does not have a Primary Key (PK), or a Logical Key has not been defined, the masking engine will create an Identity Column called 'DMS_ROW_ID' together with an Index called 'i_DMS_ROW_ID'. 

This feature is available on some database connectors. 

Additional note: Some databases (i.e. Oracle and DB2 (optional)) have an identifier that can be used as the Unique Row Identifier (URI). The identifier in Oracle is called ROWID and is technically the address to the record. If this identifier exists, it is best to set this column as the Logical Key in the Rule Set.

DROP INDEX TRANSFORMATION

If selected, and the feature is available, this transformation drops indexes on masked columns. 

Drop Indexes feature availability is dependent on the database connector type. It is executed through ALTER TABLE statements. 

The feature is a tickbox in the Job configuration popup. 

PRESCRIPT TRANSFORMATION

This transformation executes three SQL statements in the following order: 

  1. Custom (Pre) Script
  2. Disable Triggers 
  3. Disable Constrains

Disable Triggers/Constraints are dependent on the database connector type and executed through ALTER TABLE statements. 

Custom Pre-Script is a Transaction SQL script added in the Job configuration popup and the Disable Triggers and Disable Constraints features are enabled through tickboxes in the Job configuration popup. 

Note

Note:

Some of the Constraints might be implemented through an Index. To disable those, the index needs to be dropped. 

 

POSTSCRIPT TRANSFORMATION

This transformation executes three SQL statements in the following order: 

  1. Custom (Post) Script
  2. Enable Triggers
  3. Enable Constrains

CREATE INDEX TRANSFORMATION

If Drop Indexes is selected, any dropped indexes are recreated in this transformation. 

DROP IDENTITY TRANSFORMATION

If created in the first transformation, this transformation will drop the index 'i_DMS_ROW_ID' and the Identity column 'DMS_ROW_ID'.

 

 


Related Articles

The following articles may provide more information or related information to this article: