Skip to main content
Delphix

Adding or Refreshing a Unix/Linux Environment Fails With Error Code exception.ccc.writefile.failed or exception.ccc.connect.failed (KBA8569)

 

KBA

KBA# 8569

 

Issue

Adding or refreshing a Unix/Linux environment fails with Error Code: exception.ccc.writefile.failed or exception.ccc.connect.failed.

Please refer to the following possible examples:

  • Command output net.schmizz.sshj.xfer.scp.SCPException: Received unknown response code:

clipboard_eef47f3e0748865081716b022208da3c2.png

  • MTU issues:

clipboard_e4d8799f1692a00b7f53a0baa40fe4833.png

  • The following error is more generic and it may be caused by various issues:

clipboard_e14f2ebc437d0e7e8ad3571834255aa73.png

Prerequisites

  • Delphix engine
  • Unix/Linux host environment

Applicable Delphix Versions

Click here to view the versions of the Delphix engine to which this article applies
Major Release All Sub Releases
6.0 6.0.0.0, 6.0.1.0, 6.0.1.1, 6.0.2.0, 6.0.2.1, 6.0.3.0, 6.0.3.1, 6.0.4.0, 6.0.4.1, 6.0.4.2, 6.0.5.0, 6.0.6.0, 6.0.6.1, 6.0.7.0, 6.0.8.0, 6.0.8.1, 6.0.9.0, 6.0.10.0, 6.0.10.1, 6.0.11.0

5.3

5.3.0.0, 5.3.0.1, 5.3.0.2, 5.3.0.3, 5.3.1.0, 5.3.1.1, 5.3.1.2, 5.3.2.0, 5.3.3.0, 5.3.3.1, 5.3.4.0, 5.3.5.0, 5.3.6.0, 5.3.7.0, 5.3.7.1, 5.3.8.0, 5.3.8.1, 5.3.9.0

5.2

5.2.2.0, 5.2.2.1, 5.2.3.0, 5.2.4.0, 5.2.5.0, 5.2.5.1, 5.2.6.0, 5.2.6.1

5.1

5.1.0.0, 5.1.1.0, 5.1.2.0, 5.1.3.0, 5.1.4.0, 5.1.5.0, 5.1.5.1, 5.1.6.0, 5.1.7.0, 5.1.8.0, 5.1.8.1, 5.1.9.0, 5.1.10.0

5.0

5.0.1.0, 5.0.1.1, 5.0.2.0, 5.0.2.1, 5.0.2.2, 5.0.2.3, 5.0.3.0, 5.0.3.1, 5.0.4.0, 5.0.4.1, 5.0.5.0, 5.0.5.1, 5.0.5.2, 5.0.5.3, 5.0.5.4

Resolution

Please refer to the troubleshooting section to determine the possible issue and relative corrective actions.

Troubleshooting

Running scp -vvv [file] user@host:/home/user where the file is of a few MBs from another host should be able to provide more insight on the issue.

Below are a few examples of what to expect based on different possible issues:

Failed via ulimit -f (ulimit -f 10)

debug1: Sending command: scp -v -t /home/oracle3
debug2: channel 0: request exec confirm 1
debug3: send packet: type 98
debug2: channel_input_open_confirmation: channel 0: callback done
debug2: channel 0: open confirm rwindow 0 rmax 32768
debug2: channel 0: rcvd adjust 2097152
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 0
debug2: exec request accepted on channel 0
Sending file modes: C0640 85540044 4229c416-f8e3-70e6-0619-2e6ccd0de36f-20150915-18-30-22.tar.gz
debug2: channel 0: rcvd ext data 83
Sink: C0640 85540044 4229c416-f8e3-70e6-0619-2e6ccd0de36f-20150915-18-30-22.tar.gz
debug2: channel 0: written 83 to efd 8
4229c416-f8e3-70e6-0619-2e6ccd0de36f-20150915-18-30-22.tar.gz                                                                                                                                                                                                                              0%    0     0.0KB/s   --:-- ETAdebug3: receive packet: type 98
debug1: client_input_channel_req: channel 0 rtype exit-signal reply 0
debug3: receive packet: type 98
debug1: client_input_channel_req: channel 0 rtype eow@openssh.com reply 0
debug2: channel 0: rcvd eow
debug2: channel 0: chan_shutdown_read (i0 o0 sock -1 wfd 4 efd 8 [write])
debug2: channel 0: input open -> closed
debug3: receive packet: type 96
debug2: channel 0: rcvd eof
debug2: channel 0: output open -> drain
debug2: channel 0: obuf empty
debug2: channel 0: chan_shutdown_write (i3 o1 sock -1 wfd 7 efd 8 [write])
debug2: channel 0: output drain -> closed
debug3: receive packet: type 97
debug2: channel 0: rcvd close
debug3: channel 0: will not send data after close
debug2: channel 0: almost dead
debug2: channel 0: gc: notify user
debug2: channel 0: gc: user detached
debug2: channel 0: send close
debug3: send packet: type 97
debug2: channel 0: is dead
debug2: channel 0: garbage collecting
debug1: channel 0: free: client-session, nchannels 1
debug3: channel 0: status: The following connections are open:
  #0 client-session (t4 r0 i3/1900544 o3/0 e[write]/0 fd -1/-1/8 sock -1 cc -1)
debug3: send packet: type 1
debug1: fd 0 clearing O_NONBLOCK
debug3: fd 1 is not O_NONBLOCK
Transferred: sent 199312, received 2640 bytes, in 1.2 seconds
Bytes per second: sent 165274.7, received 2189.2
debug1: Exit status -1
lost connection

Login as the OS environment user and check that the ulimit -f is set as to unlimited:

[delphix@environment ~]$ ulimit -f
unlimited

If not, please contact your sysadmin to change it.

Failing because of an "echo" in .bashrc (.bashrc with interactive commands in it):

debug1: Sending command: scp -v -t /home/oracle3
debug2: channel 0: request exec confirm 1
debug3: send packet: type 98
debug2: channel_input_open_confirmation: channel 0: callback done
debug2: channel 0: open confirm rwindow 0 rmax 32768
debug2: channel 0: rcvd adjust 2097152
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 0
debug2: exec request accepted on channel 0
This line will make SCP fail <---- this is output from the echo command
debug2: channel 0: read<=0 rfd 4 len 0
debug2: channel 0: read failed
debug2: channel 0: chan_shutdown_read (i0 o0 sock -1 wfd 4 efd 8 [write])
debug2: channel 0: input open -> drain
debug2: channel 0: ibuf empty
debug2: channel 0: send eof
debug3: send packet: type 96
debug2: channel 0: input drain -> closed
debug2: channel 0: write failed
debug2: channel 0: chan_shutdown_write (i3 o0 sock -1 wfd 7 efd 8 [write])
debug2: channel 0: send eow
debug3: send packet: type 98
debug2: channel 0: output open -> closed
slatini:Downloads slatini$ debug3: receive packet: type 96
debug2: channel 0: rcvd eof
debug3: receive packet: type 98
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug3: receive packet: type 97
debug2: channel 0: rcvd close
debug3: channel 0: will not send data after close
debug2: channel 0: almost dead
debug2: channel 0: gc: notify user
debug2: channel 0: gc: user detached
debug2: channel 0: send close
debug3: send packet: type 97
debug2: channel 0: is dead
debug2: channel 0: garbage collecting
debug1: channel 0: free: client-session, nchannels 1
debug3: channel 0: status: The following connections are open:
  #0 client-session (t4 r0 i3/0 o3/0 e[write]/0 fd -1/-1/8 sock -1 cc -1)
debug3: send packet: type 1
debug1: fd 0 clearing O_NONBLOCK
debug3: fd 1 is not O_NONBLOCK
Transferred: sent 2440, received 2504 bytes, in 0.6 seconds
Bytes per second: sent 3856.4, received 3957.6
debug1: Exit status 0

Please review your .bashrc file and remove/comment out any interactive commands or change the script to detect the shell type (external link with example: https://unix.stackexchange.com/questions/154395/running-scp-when-bashrc-of-remote-machine-includes-source-command).

MTU mismatch

It is possible that the following returns a diverse error from the one reported in the KB. In fact, the connection may hang for a long time in such cases:

debug1: Sending command: scp -v -t /home/oracle3
debug2: channel 0: request exec confirm 1
debug3: send packet: type 98
debug2: channel_input_open_confirmation: channel 0: callback done
debug2: channel 0: open confirm rwindow 0 rmax 32768
debug2: channel 0: rcvd adjust 2097152
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 0
debug2: exec request accepted on channel 0
Sending file modes: C0664 103023513 jdk.tar.gz
debug2: channel 0: rcvd ext data 33
Sink: C0664 103023513 jdk.tar.gz
debug2: channel 0: written 33 to efd 6
4229c416-f8e3-70e6-0619-2e6ccd0de36f-20150915-18-30-22.tar.gz    0%    0     0.0KB/s   --:-- ETA
ebug3: send packet: type 1
packet_write_wait: Connection to 10.10.10.173 port 22: Broken pipe
lost connection

Our documentation provide a few examples on how to test the MTU from your environment to the Delphix engine here: https://docs.delphix.com/docs/best-practices/best-practices-for-network-configuration
Please refer to section 2.f Jumbo frames check via ping.
Please notice that the test is not limited to Jumbo frames but also standard MTU of 1500 bytes can be checked with the same method.

MTU can be determined via "ifconfig -a" in most Unix/Linux environments. For example:

[delphix@environment ~]$ ifconfig -a
ens192: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500

The Delphix engine's MTU can be viewed under the Network section in the Server Setup page, by logging in as a sysadmin user.
For example:

clipboard_ef6d88d949aa7927f7e2dd70fe5503d70.png

Working example:

debug1: Sending command: scp -v -t /home/oracle3
debug2: channel 0: request exec confirm 1
debug3: send packet: type 98
debug2: channel_input_open_confirmation: channel 0: callback done
debug2: channel 0: open confirm rwindow 0 rmax 32768
debug2: channel 0: rcvd adjust 2097152
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 0
debug2: exec request accepted on channel 0
Sending file modes: C0640 85540044 4229c416-f8e3-70e6-0619-2e6ccd0de36f-20150915-18-30-22.tar.gz
debug2: channel 0: rcvd ext data 83
Sink: C0640 85540044 4229c416-f8e3-70e6-0619-2e6ccd0de36f-20150915-18-30-22.tar.gz
debug2: channel 0: written 83 to efd 8
4229c416-f8e3-70e6-0619-2e6ccd0de36f-20150915-18-30-22.tar.gz                                                                                                                                                                                                                              0%    0     0.0KB/s   --:-- ETAdebug2: channel 0: rcvd adjust 98381
4229c416-f8e3-70e6-0619-2e6ccd0de36f-20150915-18-30-22.tar.gz                                                                                                                                                                                                                              2% 2208KB   2.2MB/s   00:36 ETAdebug2: channel 0: rcvd adjust 131072
<snip>
4229c416-f8e3-70e6-0619-2e6ccd0de36f-20150915-18-30-22.tar.gz                                                                                                                                                                                                                            100%   82MB   4.9MB/s   00:16
debug2: channel 0: read<=0 rfd 4 len 0
debug2: channel 0: read failed
debug2: channel 0: chan_shutdown_read (i0 o0 sock -1 wfd 4 efd 8 [write])
debug2: channel 0: input open -> drain
debug2: channel 0: ibuf empty
debug2: channel 0: send eof
debug3: send packet: type 96
debug2: channel 0: input drain -> closed
debug3: receive packet: type 96
debug2: channel 0: rcvd eof
debug2: channel 0: output open -> drain
debug2: channel 0: obuf empty
debug2: channel 0: chan_shutdown_write (i3 o1 sock -1 wfd 7 efd 8 [write])
debug2: channel 0: output drain -> closed
debug3: receive packet: type 98
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug3: receive packet: type 97
debug2: channel 0: rcvd close
debug3: channel 0: will not send data after close
debug2: channel 0: almost dead
debug2: channel 0: gc: notify user
debug2: channel 0: gc: user detached
debug2: channel 0: send close
debug3: send packet: type 97
debug2: channel 0: is dead
debug2: channel 0: garbage collecting
debug1: channel 0: free: client-session, nchannels 1
debug3: channel 0: status: The following connections are open:
  #0 client-session (t4 r0 i3/0 o3/0 e[write]/0 fd -1/-1/8 sock -1 cc -1)
debug3: send packet: type 1
debug1: fd 0 clearing O_NONBLOCK
debug3: fd 1 is not O_NONBLOCK
Transferred: sent 85601512, received 15880 bytes, in 17.6 seconds
Bytes per second: sent 4875603.8, received 904.5
debug1: Exit status 0

 

 

Related Articles

The following articles may provide more information or related information to this article: