Troubleshooting Masking Job Fetch on the Virtualization Engine (KBA4266)

At a Glance

Versions:	Applicable Delphix Masking versions: 5.x, 6.0
Description:	The process the Virtualization Engine uses to fetch masking jobs from the specified Masking Engine. These masking jobs are used to provision a Masked vDB. Note: that Masked Provisioning is only available on In-Place jobs. On-The-Fly is not supported.
Issue 1 + Solution:	Connection Issue - Fail to fetch Masking Jobs Make sure the Virtualization Engine is correctly configured and connects to the Masking Engine. See section 'Configuration' below.
Issue 2 + Solution:	Connection Issue - Username or Password error Use the CLI to test the connection using 'fetch'. If error, use the CLI to change the 'maskingjob serviceconfig' and set username or password.
Issue 3 + Solution:	Fetch Issue - Fetch works but masking job missing (1) Make sure that the Masking Jobs are configured as 'Multi Tenant' (UI: Overview > Job Configuration). Use the CLI to test the connection using 'fetch'. Use the CLI and use 'ls' to list fetched jobs.
Issue 4 + Solution:	Fetch Issue - Fetch works but masking job missing (2) Make sure all masking jobs fit in the fetch page size (see 'Page Size too small' below). Use the CLI to test the connection using 'fetch'. Use the CLI and use 'ls' to list fetched jobs.
Issue 5 + Solution:	Job Issue - Masked provisioning cannot be run against an "Advanced Connector" Change the Masking Connector to a "Basic Connector". If Custom JDBC is needed you might need to script the execution of the masking job.
Comments:	Special notes: For a job to be used in the Virtualization Engine, the Masking Job needs to be configured as 'Multi Tenant' or it won't be fetched. Only 'Multi Tenant' jobs are used on the Virtualization Engine. 'Advanced Connector' on the Masking Connector cannot be used with Masked Provisioning. Only one Masking Job can be assigned to one vDB. If multiple masking jobs need to be executed in order to mask a database (vDB) this needs to be scripted using the API. The port number used is configurable and should be the same as used when accessing the Masking UI.
Version Updates:	In version 5.2 the REMOTEDMS flag was removed. In version 5.3.3 changed to the default port for HTTP to 80 and HTTPS to 443.

Fetching Masking Jobs and Issues

Below are some examples of common errors and examples of VE configuring to integrate with ME.

Errors Example

Error examples:

"Invalid Username or Password"
"Could not get a response from the local Delphix Masking Engine instance."

Error - Invalid Username or Password

CLI command

My_VE.cli maskingjob> fetch
My_VE.cli maskingjob fetch *> commit
    Dispatched job JOB-5
    MASKINGJOB_FETCH job started for "unknown".
    Delphix Masking Engine returned an invalid response "{"errorMessage":"Invalid username or password"}".
  Action: Make sure the Delphix Masking Engine instance is online and available.

VE error message

Description: Fetching all Masking Jobs from "localhost".
Fail message: Delphix Masking Engine returned an invalid response "{"errorMessage":"Invalid username or password"}".

Resolution

Use the CLI to change the Username / Password configuration.

Error - Incorrect Port Number

CLI command

My_VE.cli maskingjob> fetch
My_VE.clicode5350gold.dcenter.delphix.com maskingjob fetch *> commit
    Dispatched job JOB-8
    MASKINGJOB_FETCH job started for "unknown".
    Could not get a response from the local Delphix Masking Engine instance.
  Action: Make sure the Delphix Masking Engine instance is online and available.

VE error message

Description: Fetching all Masking Jobs from "localhost".
Fail message: Could not get a response from the local Delphix Masking Engine instance.

Resolution

Use the CLI and correct the server or port details.

Error - "Masked provisioning cannot be run against an Advanced Connector"

This error is thrown on the Virtualization Engine when a masking job with an Advanced Connector is executed.

For more information see section Related Articles below.

VE error message

Running Masking Job ApplicationId "app" with MaskingJobId "1".
Masked provisioning cannot be run against an advanced connector.
Select a basic connector (i.e.: a connector that does not use a JDBC string to connect to the database).

Resolution

It is not possible to use an Advanced Connector in a Masked Provisioning. There are two solutions:

Change the Masking Connector to a Basic Connector.
If this is not possible, you might need to use a custom hock script with the Masked Provisioning.

Masking Jobs are missing after fetch

Masking Job not configured as Multi-Tenant

For a job to be used in the Virtualization Engine, the Masking Job needs to be configured as 'Multi Tenant'.

Go to the Masking Engine and check the Job Configuration and tick the 'Multi Tenant' flag.

Page Size too small

If there are missing Masking Jobs on the Masking Engine, it can be that the page size configuration on the Masking Engine is too small.

Please contact Support if assistance is needed.

CLI Guide

Configuration

Open the maskingjob menu

In the CLI select 'maskingjob':

My_VE.cli > maskingjob

List Masking jobs and Commands

[Optional]

This will list all Masking Jobs and Commands available at this level. 'ls' can be used anywhere. As this is the first time, there are no Masking Jobs listed.

My_VE.cli maskingjob> ls
Children
serviceconfig

Operations
fetch
getLocalServiceStatus

Open the serviceconfig menu

Open the 'serviceconfig' and issue 'ls' will get the current connection details:

Currently 'localhost' on port '80' is used (default on 5.3.4.0).

My_VE.cli maskingjob> serviceconfig
My_VE.cli maskingjob serviceconfig> ls
Objects
NAME                       SERVER     PORT  USERNAME       CREDENTIALS  SCHEME
`MASKING_SERVICE_CONFIG-1  localhost  80    admin          { ... }      HTTP

Select `MASKING_SERVICE_CONFIG-1

To change connection details select '`MASKING_SERVICE_CONFIG-1' - 'ls' will list fields and current configuration:

My_VE.cli maskingjob serviceconfig> select `MASKING_SERVICE_CONFIG-1
My_VE.cli maskingjob serviceconfig '`MASKING_SERVICE_CONFIG-1'> ls
Properties
    type: MaskingServiceConfig
    name: (unset)
    credentials:
        type: PasswordCredential
        password: ********
    port: 80
    reference: MASKING_SERVICE_CONFIG-1
    scheme: HTTP
    server: localhost
    username: admin

Operations
update

Update to set a value

Use update to set a specific value - note that password will be in clear text when entering.

My_VE.cli maskingjob serviceconfig '`MASKING_SERVICE_CONFIG-1'> update
My_VE.cli maskingjob serviceconfig '`MASKING_SERVICE_CONFIG-1' update *> set credentials.password=ABCDE
My_VE.cli maskingjob serviceconfig '`MASKING_SERVICE_CONFIG-1' update *> commit

Troubleshooting

Below are two tests that can be done using the CLI to test that the connection works are expected.

Manual Fetch

The masking jobs can be fetched manually using the CLI:

On the Virtualization Engine, JOB-999 was dispatched and successful.

My_VE.cli > maskingjob
My_VE.cli maskingjob > fetch
My_VE.cli maskingjob fetch *> commit
    Dispatched job JOB-999
    MASKINGJOB_FETCH job started for "unknown".
    MASKINGJOB_FETCH job for "unknown" completed successfully.

List fetched jobs

To list that the Masking Jobs were retrieved (this is after the masking job has been assigned to a Dataset on the VE):

One Masking Job was fetched.
- Name: 'vDB_Mask'
- Job ID: 1
- Associated Container: rh74db03

My_VE.cli maskingjob > ls
Objects
NAME      ASSOCIATEDCONTAINER  MASKINGJOBID
vDB_Mask  rh74db03             4

Children
serviceconfig

Operations
fetch
getLocalServiceStatus

Assignment in VE

This shows the job being assigned in the VE. To assign or change the assignment click on the tab Configuration, Masking and then the pen to edit.

VE UI - Masking Job Configuration.png