Skip to main content
Delphix

Using Curl to Collect Masking Job Reports (KBA1400)

 

Issue

Audit procedures within some customer environments require collecting detailed reports about masking jobs. In environments with excessive (>100000) numbers of tables such as SAP/PeopleSoft collecting these reports is simply not feasible.

Example of Monitor Page in the Masking UI

Resolution

The solution proposed in this document is to use curl from the command line. Since the result of the curl query exposes data in json format, this method is in many situations more convenient than the standard PDF report, because the json formatted data can be queried and parsed. 

Commands to collect the Report 

To collect the report use the three curl statements below. Please ensure to:

  1. Change YOURSERVER to your masking engine 
  2. Change PASSWORD to the delphix_admin (or other user - if so please also change user)
  3. Change the JOBNM to the Job Number of the job you want the report from

curl can be run on any system where curl binaries are installed. The sole requirement is that the client system has HTTP access to the engine.

curl commands
# 1. Initiate the cookie jar
curl --cookie-jar cjar --output /dev/null http://YOURSERVER:8282/dmsuite/login.do

# 2. Login  
curl --cookie cjar --cookie-jar cjar --data 'userName=delphix_admin' --data 'password=PASSWORD' --output /dev/null http://YOURSERVER:8282/dmsuite/login.do

# 3. Get the data - please change the jobNm to the job data you want to collect the info from
curl --cookie cjar  http://YOURSERVER:8282/dmsuite/monitorJobsDBCompleted.do?jobNm=JOBNM | grep "{ID"

Example 

Example of an Execution

To help highlight what to expect from the data, the example chosen below has been collected from a job run against a minimal environment.

curl example using engine on DC
$ # 1. Initiate the cookie jar
$ curl --cookie-jar cjar --output /dev/null http://ak50build.dcenter.delphix.com:8282/dmsuite/login.do
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  4349  100  4349    0     0   5550      0 --:--:-- --:--:-- --:--:--  5554


$ # 2. Login - please change userName and password accordingly
$ curl --cookie cjar --cookie-jar cjar --data 'userName=delphix_admin' --data 'password=Delphix_123' --output /dev/null http://ak50build.dcenter.delphix.com:8282/dmsuite/login.do
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    43    0     0  100    43      0    108 --:--:-- --:--:-- --:--:--   108


$ # 3. Get the data - please change the jobNm to the job data you want to collect the info from
$ curl --cookie cjar  http://ak50build.dcenter.delphix.com:8282/dmsuite/monitorJobsDBCompleted.do?jobNm=25 | grep "{ID"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 20075    0 20075    0     0  27488      0 --:--:-- --:--:-- --:--:-- 27500
                                 {ID :'0',Name :'DEPT',DisplayName :'DEPT',NameLink :'',Progress :'100',StatusIcon :'status-icon-success.png',Status :'Success',Time :'0d 0h 0m',RowsPerMin :'240',TransformationCode :'25_113_DEPT.txt',TransformationReport :'2016/07/01 02:08:29 - DEPT - Loading transformation from XML file [/var/delphix/dmsuite//output/Env/DMSApplicator/RuleSet/25/KETTLE_MASK_XML_25_DEPT_113.xml]<br/>2016/07/01 02:08:30 - DEPT - Dispatching started for transformation [KETTLE_MASK_XML_25_DEPT_113]<br/>2016/07/01 02:08:30 - Table input.0 - Finished reading query, closing connection.<br/>2016/07/01 02:08:30 - Table input.0 - Finished processing (I=4, O=0, R=0, W=4, U=0, E=0)<br/>2016/07/01 02:08:30 - Select values.0 - Finished processing (I=0, O=0, R=4, W=4, U=0, E=0)<br/>2016/07/01 02:08:31 - Get All Lookups Values.0 - Finished processing (I=0, O=0, R=4, W=4, U=0, E=0)<br/>2016/07/01 02:08:31 - Col0_dataFromXML.0 - Finished processing (I=5002, O=0, R=0, W=5002, U=0, E=0)<br/>2016/07/01 02:08:31 - Col0_decryptData.0 - Finished processing (I=0, O=0, R=5002, W=5002, U=0, E=0)<br/>2016/07/01 02:08:31 - Col0_lookUp.0 - Finished processing (I=0, O=0, R=5006, W=4, U=0, E=0)<br/>2016/07/01 02:08:31 - SelectValues_MetaData.0 - Finished processing (I=0, O=0, R=4, W=4, U=0, E=0)<br/>2016/07/01 02:08:31 - String Cut.0 - Finished processing (I=0, O=0, R=4, W=4, U=0, E=0)<br/>2016/07/01 02:08:31 - Update.0 - Finished processing (I=4, O=0, R=4, W=4, U=4, E=0)<br/>',ReportHeading :'Success Report',RowsMasked :'4',RowsRemaining :'0'}

 

The result can be piped to a file and used as required. 

Example of Collected Data

The salient key-value pairs in this particular (and small) masking job are:

json Key Value
ID :'0'
ID 0
Name :'DEPT'
Name DEPT
Progress :'100'
Progress 100
Status :'Success'
Status Success
Time :'0d 0h 0m'
Time 0d 0h 0m 
RowsPerMin :'240'
RowsPerMin 240
ReportHeading :'Success Report'
ReportHeading Success Report
 RowsMasked :'4' RowsMasked 4
 RowsRemaining :'0' RowsRemaining 0