Skip to main content
Delphix

Sync Environments with Async Export and Async Import (KBA9955)

 

 

KBA

KBA# 9955

Applicable Delphix Versions

Click here to view the versions of the Delphix engine to which this article applies
Date Release
Feb 13, 2023 8.0.0.0
Jan 12, 2023 7.0.0.0
Releases Prior to 2023
Major Release All Sub Releases
6.0 6.0.14.0, 6.0.15.0, 6.0.16.0, 6.0.17.0, 6.0.17.1, 6.0.17.2

Async Export/Import of Environments using the Masking API

This technical note focuses on using the Masking API (async export & async import) to sync environments between masking engines.

screenshot1.jpg
 

Note

Note:

Environment Sync APIs are the preferred way of handling environment export/import versus  XML-based transfer.
Please refer to the Delphix Masking documentation (maskingdocs.delphix.com) for  further information on the terms and functionalities used in this document.

https://maskingdocs.delphix.com/Managing_Multiple_Engines_for_Masking/Masking_API_Call_Concepts/

 

 

Prerequisites

You have source and target masking engines already configured, and you want to export the objects within the environment from the source engine and import them to a target engine. An environment must previously exist on the target engine to receive the environment objects being exported. Please refer to the documentation for further information on Managing Environments.

To export a source environment

  1. Exporting the Environment from the Source Masking Engine using Async Export.
    1. Review the Env details of the environment to be exported. The simplest way to find this information is to go to the Source Masking Engine GUI Environments page as can be seen below:

EnvListing.jpg

From the list, take a note of the Environment ID you want to export.

  1. Log in into the Source engine via the Masking API Client (and Authorize). For information on logging into the Masking API Client, see Logging into and Connecting to the Masking API Client (KBA5754).
  2. The following steps are to be executed while logged into or connected to the Source Engine with the Masking API Client.
  3. Expand the sync section (by clicking on sync) and then expand  GET /syncable-objects.
    Screenshot 2020-07-15 at 17.10.52.png
  4. From the object_type list, select Environment, as below,  then click on Execrequest.jpg

Screenshot 2020-07-15 at 14.23.16.png

A successful 'get' will return a Response Body similar to:
envlist1.jpg
envlist2.jpg

For this example, you are interested in the environment with the objectIdentifier id of 22.

Copy or take a note of the string required for the next step:

Note: The export call in the next step takes a JSON object that is a list of the objects to export.  While you only need a single object for the task at hand, it is written to handle multiple objects if needed.  In JSON, lists are denoted with square brackets ([ and ])  so you will see that the desired object above is wrapped in square brackets in the export call below because the export call is expecting a list.

envobjectid.png

  1. Export the environment using POST /export async  from under sync.
    1. You will see the format of the expected input under the Parameters block, in the Model Example Value provided in the lower part of the screenshot below.
      async_exp1.jpg
       
    2. Click on the block with the Example value (the block with the yellow background). This will automatically copy the Input format into the Parameter body (the block that contains the string (required).
    3. Add the 'object identifier' string copied earlier (from 1.e.), into the Parameter Value field, overwriting any text string in between '[' and ']' , then click Execrequest.jpg   as seen below:

      async_exp2.jpg

      A successful execution of the export request  should return a Response Code of 200 and a Response Body similar-looking to the screenshot below, indicating that an async task has been started.

image.png
The duration of the async export task is dependent on the number of environments and its objects that are being exported.

  1. To get the current status of the async task, take a note of the asyncTaskId as above. Navigate to  asyct1.jpg
    asynct2.jpg
    asynct3.jpg
  2. Enter the async task id you noted earlier and click Execute API Request.
    The status of the Async Export task should be returned, with a 'SUCCEEDED' status as below:

42.jpg

  1. Download the export into a file that can be imported into the Target masking engine. For this you need to take note of the export reference above, which is the value returned as reference. In this example it is: EXPORT-ZXhwb3J0X2RvY3VtZW50X0tzRjdjZ3VRLmpzb24=
    This value would be the same value as that returned as 'reference' when 'Post export-async' was initiated.
    1. Navigate to:
      filedownload.jpg
    2. Provide input parameter expected in the fileDownloadID field:
      filedownload2.jpg

      Paste or copy in the 'EXPORT-xxxxxxx' reference noted earlier, as the Value for the fileDownloadId, and click on Execrequest.jpg
      filedownload4.jpg
    3. Clicking on the Execute should return a 'success' response code of 200 and a Response body similar to:
      responsebody1.jpg
       
    4. Click on the active link/string that starts with Download, a file will get downloaded to your desktop with the a filename similar to the following:
      fileexample.jpg
      You can rename this file if you wish.

Note: A video demonstrating Async export of an Environment is provided below:


To Import the Environment on the Target Masking Engine

Complete the following procedure to import the Environment into the Target Masking Engine using the Masking Client API:

  1. On the Target masking engine, create a new environment into which the exported environment definitions will be imported.
    (An existing environment can also be used).

    Note: When performing an environment sync as described here, you must first find or create a destination environment on the target Engine.  The import process will populate objects into the selected environment rather than automatically recreating the old environment name.

    Addenv.jpg

    Make a note of the environment ID of the environment that you will be using to import into.
     
  2. Log in to the Target masking engine using the Masking API client and Authorize.
  3. Navigate to sync then 
    importasync1.jpg
    improtasync2.jpg
     
  4. Input or paste in the export file name generated via Async export from the source or master masking engine using Choose File.
  5. Decide on desired value for force-overwrite: true or false.
  6. Add the environment Id of the environment (existing or pre-created) that you are importing into.
  7. Source-environment_id is required if there are ON-THE-FLY masking jobs in the environment being imported.  If that is the case, another environment needs to be pre-created on the target engine prior to the import.

    From the docs:
    docs.jpg

    Example:

    creenshot10.jpg
     
  8. Click Execute API Request to start the Async import. Take a note of the asyncTaskId for the Async task initiated for this import, as seen in this example of Response Body returned:

    newasyncimpshot.jpg

    Dependent on the size of the file and number of objects being imported, this import may take a few minutes. Check the status of the async import task using  the asyncTaskid returned by the import, and

    asynct2.jpg.

    A successful and completed import returns a similar response to the one below with status SUCCEEDED.

    async2.jpg

    Note: The following is a video demonstrating Async import of an Environment:

 

How to handle errors during environment import

During the environment import, it is possible that you receive an algorithm-related error as seen below.
Screenshot 2020-07-15 at 15.01.26.png

In this case one of the following extra actions is required before attempting another import of the environment:

  • A Sync or Async export of the referenced algorithm and its related domain from the Source masking engine and import into the Target engine. 
  • If on the subsequent import, further algorithm-related errors get reported suggesting 'Try syncing the original object',  then do an Async export of the  Global Object from the source masking engine and async import it into the Target Engine. There is an example video of how to do this in the Global Object: Async Export (From Source Masking Engine) and Async Import (into Target engine) section.
Sync export of the referenced algorithm and related domain from the Source engine and import into the Target engine
  1. Return to the source Masking Engine and export the domain that is related to the algorithm referenced in the error.  The domain Sync export will also include the masking key for the referenced algorithm.
    1. Use Screenshot 2020-07-15 at 17.50.15.png object_type=DOMAIN, to get the object Id/information to use for exporting the domain:
      Screenshot 2020-07-15 at 15.16.34.png

      In this example, you are interested in the domain linked with the algorithm in the error, domain 'DOB':  
            {
            "objectIdentifier": {
              "id": "DOB"
            },
            "objectType": "DOMAIN",
            "revisionHash": "6dc0a42038c40842d80ab0ccd5ce64a2d7d5f926"
          }
          
    2. Provide this information in the Parameter body value field for Sync - Export Masking Object as seen in the image below.
      Screenshot 2020-07-15 at 15.19.43.png
       
    3. Click on  Execrequest.jpg.

      If successful (Response Code 200), the exportResponseMetadata for the domain will be found under Response Body as seen below:
      image (1).png
       
    4. Copy the information in the Response Body, this will be used to import the domain into the target masking engine.
       
  2. Go to the target engine.
    1. On the target engine, navigate to and click Post - Import Masking Objects.
    2. Paste in the copied domain-related information into the Parameter Value field.
    3. Click Execute API Request. A successful domain import will return a  response body as per example below.
      Screenshot 2020-07-15 at 15.25.55.png

      Async export and async import of a domain can also be done for a domain if preferred.

 

Global Object: Async Export (From Source Masking Engine) and Async Import (into Target engine)

If there are too many errors related to other algorithms then it maybe more time-efficient to do a Sync export and import of the Global_Object. As per the Continuous Compliance Engine documentation:
"When a user requests to export GLOBAL_OBJECT, every syncable algorithm, profile set, profile expression and domain on the engine will be exported as the bundle". 

The following video demonstrates async export of a Global Object from a source masking engine.

The following video demonstrates async import of a Global Object to a target masking engine.

Checking Masking Debug Logs

The following video demonstrates the occurrence of an import error and how to check masking debug logs when an Async Import fails , here:

Related Articles

The following articles may provide more information or related information to this article: