ASE ValidatedSync or SnapSync May Fail with ASE Error 5055 or 5056 (KBA1816)

Issue

After an ASE database has had its log space shrunk (reference the ASE "ALTER DATABASE LOG OFF" syntax), Delphix may fail to ingest transaction logs during SnapSync or ValidatedSync. These processes may fail with ASE error 5055 or 5056:

ALTER DATABASE dx77TtHZEwNdpnO1V0t3L_spsrdb OFF dxfnf17Lzr86vu9vv9_srdb_HOLE FROM 42311680 TO 43335680

ASE message 5088 : ALTER DATABASE has finished in database 'dx77TtHZEwNdpnO1V0t3L_spsrdb'. This command processed 0 allocation units of 4000 requested.

ASE ERROR: 5055 : You cannot remove 1024000 log pages (2000 MB) from the database 'dx77TtHZEwNdpnO1V0t3L_spsrdb' because the amount of free log space remaining (148992 pages, 291 MB) would fall below last chance threshold (441504 pages, 863 MB).

Applicable Delphix Versions

All Versions

Resolution

To resolve this issue, take a full database dump using the ASE "SHRINK_LOG" clause:

1> DUMP DATABASE sales_db to "/home/sybase/dump/sales_db.full.13:43" with SHRINK_LOG
2> go
Backup Server: 4.172.1.4: The value of 'allocated pages threshold' has been set to 40%.
Backup Server session id is: 63. Use this value when executing the 'sp_volchanged' system stored procedure after fulfilling any volume change request from the Backup Server.
Backup Server: 4.41.1.1: Creating new disk file /home/sybase/dump/sales_db.full.13:43.
Backup Server: 6.28.1.1: Dumpfile name 'ales_db190110B2E7' section number 1 mounted on disk file '/home/sybase/dump/sales_db.full.13:43'
Backup Server: 4.166.1.1: Using dbiosize of 131072 bytes for device /home/sybase/dump/sales_db.full.13:43.
Backup Server: 4.165.1.1: Using iocount of 2 for device /home/sybase/dump/sales_db.full.13:43.
Backup Server: 4.166.1.2: Using zonesize of 262144 bytes for device /home/sybase/dump/sales_db.full.13:43.
Backup Server: 4.166.1.3: Using blocksize of 65536 bytes for device /home/sybase/dump/sales_db.full.13:43.
Backup Server: 4.165.1.2: Using numzones of 3 for device /home/sybase/dump/sales_db.full.13:43.
Backup Server: 4.165.1.3: Using archcnt of 1 for device /home/sybase/dump/sales_db.full.13:43.
Backup Server: 4.165.1.4: Using dbdevcnt of 2 for device /home/sybase/dump/sales_db.full.13:43.
Backup Server: 4.166.1.4: Using pagesize of 4096 bytes for device /home/sybase/dump/sales_db.full.13:43.
Backup Server: 4.188.1.1: Database sales_db: 748 kilobytes (24%) DUMPED.
Backup Server: 4.188.1.1: Database sales_db: 4994 kilobytes (100%) DUMPED.
Backup Server: 3.43.1.1: Dump phase number 1 completed.
Backup Server: 3.43.1.1: Dump phase number 2 completed.
Backup Server: 3.43.1.1: Dump phase number 3 completed.
The WITH SHRINK_LOG option has broken the dump sequence. When restoring the sequence, this database dump must be loaded prior to loading the transaction log that follows it.
Backup Server: 4.188.1.1: Database sales_db: 5004 kilobytes (100%) DUMPED.
Backup Server: 3.42.1.1: DUMP is complete (database sales_db).

NOTE: The "DUMP DATABASE" command automatically removes any hole at the end of the database if the database is not in a dump sequence (that is, when you are forced to run dump database because dump transaction is not allowed, when, for example, any minimally logged command is performed). The with "SHRINK_LOG" clause removes holes at the end of the database, regardless of whether the database is in a dump sequence or not. 

Troubleshooting

Running the following query shows there is a hole at the end of the database:

1> select * from sysusages where dbid=db_id("sales_db")
2> go
 dbid   segmap      lstart     size       vstart      location unreservedpgs crdate                          vdevno      
 ------ ----------- ---------- ---------- ----------- -------- ------------- ------------------------------- ----------- 
      9           3          0       2560           0        0           901             Jan 11 2019 11:29AM          19 
      9           4       2560       2560           0        0          2541             Jan 11 2019 11:29AM          20 
      9           0       5120       2560        5120        9          2550             Jan 11 2019 11:31AM          -9 

Running the same query after the "DUMP DATABASE ... with SHRINK_LOG" completes shows the hole has been removed:

1> select * from sysusages where dbid=db_id("sales_db")
2> go
 dbid   segmap      lstart     size       vstart      location unreservedpgs crdate                          vdevno      
 ------ ----------- ---------- ---------- ----------- -------- ------------- ------------------------------- ----------- 
      9           3          0       2560           0        0           901             Jan 11 2019 11:29AM          19 
      9           4       2560       2560           0        0          2541             Jan 11 2019 11:29AM          20