Troubleshooting Job Fetch in Masked Provisioning (KBA4266)
KBA
KBA#At a Glance
Versions | Applicable to all Continuous Compliance Engine (aka Masking) versions from 5.x forward (verified on 20.0). | |
---|---|---|
Description | The Continuous Data Engine (aka Virtualization) uses a process called fetch to get masking jobs from the specified Masking Engine. These masking jobs are used to provision a Masked vDB. Note: Masked Provisioning is only available on In-Place (IP) jobs. On-The-Fly (OTF) is not supported. |
|
Issue 1 - Connect | Connection Issue - Fail to fetch Masking Jobs | |
Solution | 1. Make sure the Continuous Data Engine is correctly configured and connects to the Continuous Compliance Engine. 2. See the "Configuration" section below. |
|
Issue 2 - User | Connection Issue - Username or Password error | |
Solution | 1. Use the CLI to test the connection using fetch .2. If there is an error, use the CLI to change the maskingjob serviceconfig and set username or password. |
|
Issue 3 - No jobs | Fetch Issue - Fetch works but no masking jobs | |
Solution 1 | 1. Make sure that the Masking Jobs are configured as Multi Tenant and In-Place. (UI: Overview > Job Configuration). 2. Use the CLI to test the connection using fetch .3. Use the CLI and use ls to list fetched jobs. |
|
Solution 2 | 1. Make sure all masking jobs fit in the fetch page size (see "Page Size Too Small" below). 2. Use the CLI to test the connection using fetch .3. Use the CLI and use ls to list fetched jobs. |
|
Issue 4 - Adv Connector | Job Issue - Masked provisioning cannot be run against an "Advanced Connector" | |
Solution | 1. Change the Masking Connector to a Basic Connector. 2. If Custom JDBC is needed you might need to script the execution of the masking job. |
|
Issue 5 - HTTPS | HTTPS Connection issue - "Could not get a response from..." | |
Solution | If custom HTTPS certificates are used and it has not been configured on the Continuous Data Engine you need to follow the instructions in the KBA: | |
Comments | Special notes:
|
|
Version Updates | Version 5.2 removed the REMOTEDMS flag. Version 5.3.3 changed to the default port for HTTP to 80 and HTTPS to 443. Version 6.1.0.0 Page Size set to 500. |
Applicable Delphix Versions
- Click here to view the versions of the Delphix engine to which this article applies
-
Date Release Feb 21, 2024 20.0.0.0 Jan 25, 2024 19.0.0.0 Dec 20, 2023 | Jan 10, 2024 18.0.0.0 | 18.0.0.1 Nov 21, 2023 17.0.0.0 Oct 18, 2023 16.0.0.0 Sep 21, 2023 15.0.0.0 Aug 24, 2023 14.0.0.0 Jul 24, 2023 13.0.0.0 Jun 21, 2023 12.0.0.0 May 25, 2023 11.0.0.0 Apr 13, 2023 10.0.0.0 | 10.0.0.1 Mar 13, 2023 | Mar 20, 2023 9.0.0.0 | 9.0.0.1 Feb 13, 2023 8.0.0.0 Jan 12, 2023 7.0.0.0 Releases Prior to 2023 Major Release All Sub Releases 6.0 6.0.7.0, 6.0.8.0, 6.0.8.1, 6.0.9.0, 6.0.10.0, 6.0.10.1, 6.0.11.0, 6.0.12.0, 6.0.12.1, 6.0.13.0, 6.0.13.1, 6.0.14.0, 6.0.15.0, 6.0.16.0, 6.0.17.0, 6.0.17.1, 6.0.17.2
Issues and Solutions
Below are some examples of common errors and examples of the Continuous Data Engine configuration used to integrate with the Continuous Compliance Engine.
Connection Issues
Error examples:
Invalid Username or Password
Could not get a response from the local Delphix Masking Engine instance.
Invalid Username or Password
CLI Command
My_VE.cli maskingjob> fetch My_VE.cli maskingjob fetch *> commit Dispatched job JOB-5 MASKINGJOB_FETCH job started for "unknown". Delphix Masking Engine returned an invalid response "{"errorMessage":"Invalid username or password"}". Action: Make sure the Delphix Masking Engine instance is online and available.
Continuous Data Engine Error Message
Description: Fetching all Masking Jobs from "localhost". Fail message: Delphix Masking Engine returned an invalid response "{"errorMessage":"Invalid username or password"}".
Resolution
Use the CLI to change the Username / Password configuration.
Incorrect Port Number
CLI Command
My_VE.cli maskingjob> fetch My_VE.clicode5350gold.dcenter.delphix.com maskingjob fetch *> commit Dispatched job JOB-8 MASKINGJOB_FETCH job started for "unknown". Could not get a response from the local Delphix Masking Engine instance. Action: Make sure the Delphix Masking Engine instance is online and available.
Continuous Data Engine Error Message
Description: Fetching all Masking Jobs from "localhost". Fail message: Could not get a response from the local Delphix Masking Engine instance.
Resolution
Use the CLI and correct the server or port details.
HTTPS and Certificate Errors
If the job fails with a generic error, it can also be an issue with certificates:
Error
Fetching all Masking Jobs from "XYZ".
Could not get a response from the local Delphix Masking Engine instance.
Resolution
Please see this KBA for details on how to resolve certificate issues:
No Masking Jobs after Fetch
Masked Provisioning Cannot Be Run Against an Advanced Connector
This error is thrown on the Continuous Data Engine when a masking job with an Advanced Connector is executed.
For more information see the section Related Articles below.
VE error message
Running Masking Job ApplicationId "app" with MaskingJobId "1". Masked provisioning cannot be run against an advanced connector. Select a basic connector (i.e.: a connector that does not use a JDBC string to connect to the database).
Resolution
It is not possible to use an Advanced Connector during Masked Provisioning. There are two solutions:
- Change the Masking Connector to a Basic Connector.
- For any JDBC Properties, use the Properties Feature.
If this is not possible, you might need to use a custom ad hoc script with Masked Provisioning.
No Masking Jobs after Fetch
Multi-Tenant and In-Place
For a job to be used in the Continuous Data Engine, the Masking Job needs to be configured as Multi Tenant and In-Place.
Resolution
- Go to the Masking Engine,
- Check the Job Configuration, check Multi Tenant, and set the job to In-Place.
Page Size Too Small
If there are Masking Jobs still missing, it may be that the page size configuration on the Continuous Compliance Engine is too small.
Resolution
The size was set to 500 in 6.1.0.0 (tracked in DLPX-59757).
If still an issue contact Support if assistance is needed.
CLI Guide
Configuration
Open the maskingjob menu
In the CLI type maskingjob
:
My_VE.cli > maskingjob
List Masking Jobs and Commands
[Optional]
This will list all Masking Jobs and Commands available at this level. ls
can be used anywhere. As this is the first time, there are no Masking Jobs listed.
My_VE.cli maskingjob> ls Children serviceconfig Operations fetch getLocalServiceStatus
Open the serviceconfig Menu
Type serviceconfig
, and then type ls
to see the the current connection details:
- Currently
localhost
on port80
is used (default on 5.3.4.0).
My_VE.cli maskingjob> serviceconfig My_VE.cli maskingjob serviceconfig> ls Objects NAME SERVER PORT USERNAME CREDENTIALS SCHEME `MASKING_SERVICE_CONFIG-1 localhost 80 admin { ... } HTTP
Select `MASKING_SERVICE_CONFIG-1
To change connection details type select
`MASKING_SERVICE_CONFIG-1
, ls
will list fields and the current configuration:
My_VE.cli maskingjob serviceconfig> select `MASKING_SERVICE_CONFIG-1 My_VE.cli maskingjob serviceconfig '`MASKING_SERVICE_CONFIG-1'> ls Properties type: MaskingServiceConfig name: (unset) credentials: type: PasswordCredential password: ******** port: 80 reference: MASKING_SERVICE_CONFIG-1 scheme: HTTP server: localhost username: admin Operations update
Update to set a value
Use update to set a specific value. Please note that the password will be in clear text while entering.
My_VE.cli maskingjob serviceconfig '`MASKING_SERVICE_CONFIG-1'> update My_VE.cli maskingjob serviceconfig '`MASKING_SERVICE_CONFIG-1' update *> set credentials.password=ABCDE My_VE.cli maskingjob serviceconfig '`MASKING_SERVICE_CONFIG-1' update *> commit
Troubleshooting
Below are two tests that can be done using the CLI to test that the connection works as expected.
Manual Fetch
The masking jobs can be fetched manually using the CLI:
- On the Continuous Data Engine, JOB-999 was dispatched and successful.
My_VE.cli > maskingjob My_VE.cli maskingjob > fetch My_VE.cli maskingjob fetch *> commit Dispatched job JOB-999 MASKINGJOB_FETCH job started for "unknown". MASKINGJOB_FETCH job for "unknown" completed successfully.
List fetched jobs
To list that the Masking Jobs were retrieved (this is after the masking job has been assigned to a Dataset on the Continuous Data Engine):
- One Masking Job was fetched.
- Name:
vDB_Mask
- Job ID:
1
- Associated Container:
rh74db03
- Name:
My_VE.cli maskingjob > ls Objects NAME ASSOCIATEDCONTAINER MASKINGJOBID vDB_Mask rh74db03 4 Children serviceconfig Operations fetch getLocalServiceStatus
Assignment in the Continuous Data Engine
This shows the job being assigned in the Continuous Data Engine. To assign or change the assignment click on the Configuration tab, select Masking, and then click the pen to edit.
Related Articles
The following articles may provide more information or related information to this article:
- KBA: Troubleshooting and Accessing Masking Logs (KBA7988)
- Docs: Continuous Compliance Engine Creating Masking Job
- Docs: Continuous Compliance Engine Provision Masked vDBs
- Docs: Continuous Data Engine Provisioning Virtual Databases