Skip to main content
Delphix

Troubleshooting Job Fetch in Masked Provisioning (KBA4266)

 

 

KBA

KBA#
4266

At a Glance  

Versions Applicable to all Continuous Compliance Engine (aka Masking) versions from 5.x forward (verified on 20.0).
Description The Continuous Data Engine (aka Virtualization) uses a process called fetch to get masking jobs from the specified Masking Engine. These masking jobs are used to provision a Masked vDB. 

Note: Masked Provisioning is only available on In-Place (IP) jobs. On-The-Fly (OTF) is not supported. 
Issue 1 - Connect Connection Issue - Fail to fetch Masking Jobs
  Solution 1. Make sure the Continuous Data Engine is correctly configured and connects to the Continuous Compliance Engine.

2. See the "Configuration" section below.
Issue 2 - User Connection Issue - Username or Password error
  Solution 1. Use the CLI to test the connection using fetch.

2. If there is an error, use the CLI to change the maskingjob serviceconfig and set username or password.
Issue 3 - No jobs Fetch Issue - Fetch works but no masking jobs
  Solution 1 1. Make sure that the Masking Jobs are configured as Multi Tenant and In-Place. 
    (UI: Overview > Job Configuration).

2. Use the CLI to test the connection using fetch.

3. Use the CLI and use ls to list fetched jobs.
  Solution 2 1. Make sure all masking jobs fit in the fetch page size (see "Page Size Too Small" below).

2. Use the CLI to test the connection using fetch.

3. Use the CLI and use ls to list fetched jobs.
Issue 4 - Adv Connector Job Issue - Masked provisioning cannot be run against an "Advanced Connector" 
  Solution 1. Change the Masking Connector to a Basic Connector.

2. If Custom JDBC is needed you might need to script the execution of the masking job.
Issue 5 - HTTPS HTTPS Connection issue - "Could not get a response from..."
  Solution If custom HTTPS certificates are used and it has not been configured on the Continuous Data Engine you need to follow the instructions in the KBA:
Comments Special notes:
  • For a job to be used in the Continuous Data Engine, the Masking Job needs to be configured as Multi Tenant or it won't be fetched. Only Multi Tenant and In-Place jobs are used on the Continuous Data Engine.
     
  • An Advanced Connector on the Masking Connector cannot be used with Masked Provisioning. 
     
  • Only one Masking Job can be assigned to one VDB. If multiple masking jobs need to be executed in order to mask a database (VDB) this needs to be scripted using the API. 
     
  • The port number used is configurable and should be the same as used when accessing the Continuous Compliance Engine UI. 
Version Updates Version 5.2 removed the REMOTEDMS flag. 

Version 5.3.3 changed to the default port for HTTP to 80 and HTTPS to 443. 

Version 6.1.0.0 Page Size set to 500.

Applicable Delphix Versions

Click here to view the versions of the Delphix engine to which this article applies
Date Release
Feb 21, 2024 20.0.0.0
Jan 25, 2024 19.0.0.0
Dec 20, 2023 | Jan 10, 2024 18.0.0.0 | 18.0.0.1
Nov 21, 2023 17.0.0.0
Oct 18, 2023 16.0.0.0
Sep 21, 2023 15.0.0.0
Aug 24, 2023 14.0.0.0
Jul 24, 2023 13.0.0.0
Jun 21, 2023 12.0.0.0
May 25, 2023 11.0.0.0
Apr 13, 2023 10.0.0.0 | 10.0.0.1
Mar 13, 2023 | Mar 20, 2023 9.0.0.0 | 9.0.0.1
Feb 13, 2023 8.0.0.0
Jan 12, 2023 7.0.0.0
Releases Prior to 2023
Major Release All Sub Releases
6.0

6.0.7.0, 6.0.8.0, 6.0.8.1, 6.0.9.0, 6.0.10.0, 6.0.10.1, 6.0.11.0, 6.0.12.0, 6.0.12.1, 6.0.13.0, 6.0.13.1, 6.0.14.0, 6.0.15.0, 6.0.16.0, 6.0.17.0, 6.0.17.1, 6.0.17.2

Issues and Solutions

Below are some examples of common errors and examples of the Continuous Data Engine configuration used to integrate with the Continuous Compliance Engine.

Connection Issues

Error examples: 

  • Invalid Username or Password
  • Could not get a response from the local Delphix Masking Engine instance.

Invalid Username or Password

CLI Command
My_VE.cli maskingjob> fetch
My_VE.cli maskingjob fetch *> commit
    Dispatched job JOB-5
    MASKINGJOB_FETCH job started for "unknown".
    Delphix Masking Engine returned an invalid response "{"errorMessage":"Invalid username or password"}".
  Action: Make sure the Delphix Masking Engine instance is online and available.
Continuous Data Engine Error Message
Description: Fetching all Masking Jobs from "localhost".
Fail message: Delphix Masking Engine returned an invalid response "{"errorMessage":"Invalid username or password"}".
Resolution

Use the CLI to change the Username / Password configuration. 

 

Incorrect Port Number

CLI Command
My_VE.cli maskingjob> fetch
My_VE.clicode5350gold.dcenter.delphix.com maskingjob fetch *> commit
    Dispatched job JOB-8
    MASKINGJOB_FETCH job started for "unknown".
    Could not get a response from the local Delphix Masking Engine instance.
  Action: Make sure the Delphix Masking Engine instance is online and available.
Continuous Data Engine Error Message
Description: Fetching all Masking Jobs from "localhost".
Fail message: Could not get a response from the local Delphix Masking Engine instance.
Resolution

Use the CLI and correct the server or port details. 

HTTPS and Certificate Errors

If the job fails with a generic error, it can also be an issue with certificates:

Error
Fetching all Masking Jobs from "XYZ". 
Could not get a response from the local Delphix Masking Engine instance. 
Resolution

Please see this KBA for details on how to resolve certificate issues:

No Masking Jobs after Fetch

Masked Provisioning Cannot Be Run Against an Advanced Connector 

This error is thrown on the Continuous Data Engine when a masking job with an Advanced Connector is executed. 

For more information see the section Related Articles below. 

VE error message
Running Masking Job ApplicationId "app" with MaskingJobId "1".
Masked provisioning cannot be run against an advanced connector.
Select a basic connector (i.e.: a connector that does not use a JDBC string to connect to the database).
Resolution 

It is not possible to use an Advanced Connector during Masked Provisioning. There are two solutions: 

  1. Change the Masking Connector to a Basic Connector.
  2. For any JDBC Properties, use the Properties Feature.

If this is not possible, you might need to use a custom ad hoc script with Masked Provisioning.

 

 

No Masking Jobs after Fetch

Multi-Tenant and In-Place

For a job to be used in the Continuous Data Engine, the Masking Job needs to be configured as Multi Tenant and In-Place.

Resolution 
  • Go to the Masking Engine, 
  • Check the Job Configuration, check Multi Tenant, and set the job to In-Place.

Page Size Too Small

If there are Masking Jobs still missing, it may be that the page size configuration on the Continuous Compliance Engine is too small. 

Resolution
 

The size was set to 500 in 6.1.0.0 (tracked in DLPX-59757). 
 

If still an issue contact Support if assistance is needed. 
 

CLI Guide

Configuration 

Open the maskingjob menu

In the CLI type maskingjob:

My_VE.cli > maskingjob

List Masking Jobs and Commands

[Optional] 

This will list all Masking Jobs and Commands available at this level. ls can be used anywhere. As this is the first time, there are no Masking Jobs listed. 

My_VE.cli maskingjob> ls
Children
serviceconfig

Operations
fetch
getLocalServiceStatus

Open the serviceconfig Menu

Type serviceconfig , and then type ls to see the the current connection details:

  • Currently localhost on port 80 is used (default on 5.3.4.0). 
My_VE.cli maskingjob> serviceconfig
My_VE.cli maskingjob serviceconfig> ls
Objects
NAME                       SERVER     PORT  USERNAME       CREDENTIALS  SCHEME
`MASKING_SERVICE_CONFIG-1  localhost  80    admin          { ... }      HTTP

Select `MASKING_SERVICE_CONFIG-1

To change connection details type select `MASKING_SERVICE_CONFIG-1   , ls will list fields and the current configuration:

My_VE.cli maskingjob serviceconfig> select `MASKING_SERVICE_CONFIG-1
My_VE.cli maskingjob serviceconfig '`MASKING_SERVICE_CONFIG-1'> ls
Properties
    type: MaskingServiceConfig
    name: (unset)
    credentials:
        type: PasswordCredential
        password: ********
    port: 80
    reference: MASKING_SERVICE_CONFIG-1
    scheme: HTTP
    server: localhost
    username: admin

Operations
update

Update to set a value

Use update to set a specific value. Please note that the password will be in clear text while entering.

My_VE.cli maskingjob serviceconfig '`MASKING_SERVICE_CONFIG-1'> update
My_VE.cli maskingjob serviceconfig '`MASKING_SERVICE_CONFIG-1' update *> set credentials.password=ABCDE
My_VE.cli maskingjob serviceconfig '`MASKING_SERVICE_CONFIG-1' update *> commit

Troubleshooting

Below are two tests that can be done using the CLI to test that the connection works as expected.

Manual Fetch

The masking jobs can be fetched manually using the CLI:

  • On the Continuous Data Engine, JOB-999 was dispatched and successful. 
My_VE.cli > maskingjob
My_VE.cli maskingjob > fetch
My_VE.cli maskingjob fetch *> commit
    Dispatched job JOB-999
    MASKINGJOB_FETCH job started for "unknown".
    MASKINGJOB_FETCH job for "unknown" completed successfully.

List fetched jobs

To list that the Masking Jobs were retrieved (this is after the masking job has been assigned to a Dataset on the Continuous Data Engine): 

  • One Masking Job was fetched.
    • Name: vDB_Mask
    • Job ID: 1
    • Associated Container: rh74db03
My_VE.cli maskingjob > ls
Objects
NAME      ASSOCIATEDCONTAINER  MASKINGJOBID
vDB_Mask  rh74db03             4

Children
serviceconfig

Operations
fetch
getLocalServiceStatus
Assignment in the Continuous Data Engine

This shows the job being assigned in the Continuous Data Engine. To assign or change the assignment click on the Configuration tab, select Masking, and then click the pen to edit. 

VE UI - Masking Job Configuration.png

 

Related Articles

The following articles may provide more information or related information to this article: