Best Practice: Mitigating the Storage Footprint of Masked or Encrypted VDBs (KBA1728)

Last updated
Save as PDF
Share
1. Share
2. Tweet
3. Share

KBA

KBA# 1728

Summary

This article describes the storage utilization implications of using TDE (encryption) or masked data with VDBs. In general, both masking and TDE are wholly unrelated topics. However, they have similar implications for storage. As such, these topics and the best practices for mitigating the storage footprint are described together in this article.

Applicable Delphix Versions

This article applies to the following versions of the Delphix Engine:

All versions

Storage Implications

Under normal usage scenarios, the Delphix appliance will ingest (Link) a dSource object by making a true 1:1 copy of all the data stored on a live database or file system. Child VDBs provisioned from that dSource each have an effective storage footprint of near zero. All data blocks at the time of VDB provisioning are shared with the dSource. This is how the Delphix appliance is able to provision many copies of a large database with minimal storage impact.

The storage consumed by any VDB is therefore attributed to the delta from the time of provisioning.

Any operation that performs changes to the VDB (INSERT or UPDATE statements being easy examples) will cause data blocks to change away from the parent dSource and consume storage. Masking and encryption operations are two common actions that can cause a large number of data block changes. The effects of this cannot be nullified, but they can be mitigated and managed.

Masking VDBs

Masking generally does not effect 100% change on the storage blocks. The effect of masking is highly variable. If we pretend that a specific masking job causes 50% of all the data blocks to change (in reality this could be much less or much more), then attempting to use the Delphix appliance to provision three masked VDBs would work like this:

The original 1 TiB production dSource is linked and all its data ingested to the Delphix File System.
A user then creates three VDBs from this dSource. At this stage, each VDB has a storage footprint close to zero.
Post provisioning, a masking job is run using a product like Delphix Masking against each VDB. The specifics of this job cause 50% of the data blocks to change.
As a result, the final usable VDBs each consume 500 GiB and the total storage footprint is 1.5 TiB. Not to mention the time taken and compute resources required to run masking jobs on each VDB.

Re-encrypting VDBs with TDE

TDE as-is, does not cause data blocks to change. However, re-encrypting a database using a new key will cause a 100% change if all table spaces are marked for encryption. Similar to masking, this may be required for compliance or testing reasons. Attempting to use the Delphix appliance to provision three VDBs with re-encryption would work like this:

The original 1 TiB production dSource is linked and all its data ingested to the Delphix File System.
A user then creates three VDBs from this dSource. At this stage, each VDB has a storage footprint close to zero.
Post provisioning, each VDB is re-encrypted with a new key. This causes 100% of the data blocks to change.
As a result, the final usable VDBs each consume 1 TiB and total storage footprint is 3 TiB. Not to mention the time taken and compute resources required to re-encrypt each VDB.

TDE storage impact_64c.png

Best Practices

As mentioned, operations like masking or encryption absolutely will have a storage impact. This is unavoidable. However, the effects can be mitigated in use cases where a dSource has multiple child VDBs. These measures focus on reducing redundant steps that cause data change.

Masking VDBs

Grouping a dSource and a VDB into a logical dSource unit will allow for a streamlined masking approach. This grouping is not part of the Delphix Data Management product, but merely a conceptual process. The same three masked VDBs with a footprint of 1.5 TiB can be achieved by using four VDBs and a much reduced storage cost.

The original 1 TiB production dSource is linked and all its data ingested to the Delphix File System.
A user then creates a VDB to use as a pair with the dSource. In this example they call it VDB M (M for Masked). At this stage it has nearly no storage footprint.
Post provisioning, a masking job is run using a product like Delphix Masking against VDB M. The specifics of this job cause 50% of the data blocks to change.
The three desired VDBs are then provisioned as child objects from VDB M. The parent VDB is already masked. Therefore, these VDBs are ready for use.
As a result, only one operation that caused delta was performed and the three desired VDBs now exist with a storage footprint of 500 GiB that is attributed entirely to the masked data on the parent VDB M.

Masking storage impact_2_64c.png

Re-encrypting VDBs with TDE

Grouping a dSource and a VDB into a logical dSource unit will allow for a streamlined TDE implementation. This grouping is not part of the Delphix Data Management product, but merely a conceptual process. The same three re-encrypted VDBs with a footprint of 3 TiB can be achieved by using four VDBs and a much reduced storage cost.

The original 1 TiB production dSource is linked and all its data ingested to the Delphix File System.
A user then creates a VDB to use as a pair with the dSource. In this example they call it VDB E (E for Encrypted). At this stage it has nearly no storage footprint.
Post provisioning, VDB E is re-encrypted with a new key. This causes 100% of the data blocks to change.
The three desired VDBs are then provisioned as child objects from VDB E. The parent VDB is already re-encrypted with a new key that differs from the production dSource. Therefore, these VDBs are ready for use.
As a result, only one operation that caused delta was performed and the three desired VDBs now exist with a storage footprint of 1 TiB that is attributed entirely to the re-encrypted data on the parent VDB E.

TDE storage impact_2_64c.png