Skip to main content
Delphix

How to restore 32-bit glibc in Oracle Exadata configurations

Issue

Currently, Delphix requires 32-bit version of glibc to be installed on Linux source Environments. As a result of periodic updates from Oracle for Exadata source Environments, the 32-bit libraries may be removed therefore affecting Delphix operations. This document details methodology for restoration of these libraries.

Although Delphix provides details on restoring required libraries for toolkit operations, there is no warranty expressed or implied for altering Exadata configurations. Oracle Technical Support should be consulted for any questions or concerns beyond the detail provided in this document, or for concerns regarding warranty coverage, etc. Delphix assumes no liability for violation of contract terms or conditions with any third party in offering restoration of libraries to the Exadata operating environment.

Troubleshooting

In instances where the 32-bit library dependency issue is encountered, the following events may be observed.

Hostchecker

error "/lib/ld-linux.so.2: bad ELF interpreter

or

During Environment refresh, Delphix UI may return:

Error: Missing 32 bit glibc libraries on host <hostname>
Action: Install the missing 32 bit glibc libraries on host <hostname>

 

Resolution

Assumptions:

1. OS installed is Oracle Linux Server release 6.7


Procedure:
 

1. Login to one of your database nodes in Exadata as root


2. Install wget rpm needed to download the public repository repo file (since it does not come by default in Exadata in the rpms installed)

To do so, we need to download an ISO from the proper OEL version we have installed and these are the steps followed:


2a) Create folder to mount the ISO

[root@dm01db08 yum.repos.d]# mkdir -p /var/OSimage/OEL6.7

 

2b) Mount the ISO to the folder previously created

[root@dm01db08 yum.repos.d]# mount -o loop,ro /software/LINUX/OEL/6.7/V77197-01.iso /var/OSimage/OEL6.7
[root@dm01db08 yum.repos.d]#

 

2c) Create OEL6.7 repo file in /etc/yum.repos.d to specify that the ISO will be a local repository

[root@dm01db08 yum.repos.d]# pwd
/etc/yum.repos.d


[root@dm01db08 yum.repos.d]# cat OEL6.repo
[OEL6.7]
name=Oracle Linux 6.7 x86_64
baseurl=file:///var/OSimage/OEL6.7
gpgcheck=0
enabled=1


[root@dm01db08 yum.repos.d]#

 

2d) Install wget rpm in the system

[root@dm01db08 yum.repos.d]# yum install wget
Setting up Install Process
OEL6.7 | 3.7 kB 00:00 ...
OEL6.7/primary_db | 3.0 MB 00:00 ...
Resolving Dependencies
--> Running transaction check
---> Package wget.x86_64 0:1.12-5.el6_6.1 will be installed
--> Finished Dependency Resolution
 
Dependencies Resolved
 
=================================================================================================================================================================================================
Package Arch Version Repository Size
=================================================================================================================================================================================================
Installing:
wget x86_64 1.12-5.el6_6.1 OEL6.7 483 k
 
Transaction Summary
=================================================================================================================================================================================================
Install 1 Package(s)
 
Total download size: 483 k
Installed size: 1.8 M
Is this ok [y/N]: y
Downloading Packages:
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Warning: RPMDB altered outside of yum.
Installing : wget-1.12-5.el6_6.1.x86_64 1/1
Verifying : wget-1.12-5.el6_6.1.x86_64 1/1
 
Installed:
wget.x86_64 0:1.12-5.el6_6.1
 
Complete!
[root@dm01db08 yum.repos.d]#


3. Once wget is installed, configure wget to use the proxy to be able to download the yum repo config file from the Oracle Public Yum repository

[root@dm01db08 etc]# pwd
/etc


[root@dm01db08 etc]# cat wgetrc |tail -5
# Force the default remote server encoding
#remoteencoding = UTF-8
#Enable Proxy
http_proxy=http://<proxy>:80

[root@dm01db08 etc]#


4. Download yum repo config file from the Oracle Public Yum repository (make sure you are positioned in /etc/yum.repos.d)

[root@dm01db08 yum.repos.d]# pwd
/etc/yum.repos.d
[root@dm01db08 yum.repos.d]#
[root@dm01db08 yum.repos.d]# wget http://public-yum.oracle.com/public-yum-ol6.repo
--2016-06-15 16:00:59-- http://public-yum.oracle.com/public-yum-ol6.repo
Resolving <proxy>... x.x.x.x
Connecting to <proxy>|x.x.x.x|:80... connected.
Proxy request sent, awaiting response... 200 OK
Length: 7303 (7.1K) [text/plain]
Saving to: “public-yum-ol6.repo”

100%[=======================================================================================================================================================>] 7,303 --.-K/s in 0.02s

2016-06-15 16:00:59 (325 KB/s) - “public-yum-ol6.repo” saved [7303/7303]

[root@dm01db08 yum.repos.d]#
[root@dm01db08 yum.repos.d]# ls -lrth
total 20K
-r--r----- 1 root root 896 Feb 8 00:31 Exadata-computenode.repo.sample
-rw-r--r-- 1 root root 7.2K May 31 15:58 public-yum-ol6.repo
-rw-r--r-- 1 root root 291 Jun 4 20:16 Exadata-computenode.repo
-rw-r--r-- 1 root root 94 Jun 15 15:31 OEL6.repo
[root@dm01db08 yum.repos.d]#


5. Edit public-yum-ol6.repo to configure the first repository listed to point to the latest 32 bits repository (baseurl parameter)

[ol6_latest]
name=Oracle Linux $releasever Latest ($basearch)
baseurl=http://yum.oracle.com/repo/OracleLinux/OL6/latest/i386/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=1



6. Edit /etc/yum.conf to add the GE Proxy to be able to communicate with Public Oracle Yum repository for OEL 6.7 (last 2 lines of output below added to file)

[root@dm01db08 ~]# cat /etc/yum.conf
# generated by Exadata -
[main]
cachedir=/var/cache/yum
keepcache=0
debuglevel=2
logfile=/var/log/yum.log
distroverpkg=redhat-release
tolerant=1
exactarch=1
obsoletes=1
gpgcheck=1
plugins=1

# Note: yum-RHN-plugin doesn't honor this.
metadata_expire=1h

# Default.
# installonly_limit = 3

# PUT YOUR REPOS HERE OR IN separate files named file.repo
# in /etc/yum.repos.d
# Never update kernel and kernel-uek
installonlypkgs=kernel kernel-smp kernel-bigmem kernel-enterprise kernel-debug kernel-unsupported kernel-uek
exclude=up2date
enableProxy=1
Proxy=http://<proxy>:80



7. Validate that you can see the public repository you configured (ol6_latest) from Oracle public site

[root@dm01db08 yum.repos.d]# yum repolist
OEL6.7 | 3.7 kB 00:00 ...
ol6_UEK_latest | 1.2 kB 00:00
ol6_UEK_latest/primary | 26 MB 00:08
ol6_UEK_latest 554/554
ol6_latest | 1.4 kB 00:00
ol6_latest/primary | 53 MB 00:19
ol6_latest 28786/28786
repo id repo name status
OEL6.7 Oracle Linux 6.7 x86_64 3,800
ol6_UEK_latest Latest Unbreakable Enterprise Kernel for Oracle Linux 6Server (x86_64) 471+83
ol6_latest Oracle Linux 6Server Latest (x86_64) 22,072+6,714
repolist: 26,343

[root@dm01db08 yum.repos.d]#


8. Search the glibc 32bits rpm that you is needed by Delphix, and once you locate it, look for the subversion needed.

[root@dm01db08 yum.repos.d]# yum list |grep glibc
glibc.x86_64                              2.12-1.166.el6_7.7          installed
glibc-common.x86_64                       2.12-1.166.el6_7.7          installed
glibc-devel.x86_64                        2.12-1.166.el6_7.7          installed
glibc-headers.x86_64                      2.12-1.166.el6_7.7          installed
compat-glibc.i686                         1:2.5-46.2.0.1              ol6_latest
compat-glibc.x86_64                       1:2.5-46.2.0.1              OEL6.7
compat-glibc-headers.i686                 1:2.5-46.2.0.1              ol6_latest
compat-glibc-headers.x86_64               1:2.5-46.2.0.1              OEL6.7
glibc.i686                                2.12-1.192.el6              ol6_latest
glibc-common.i686                         2.12-1.192.el6              ol6_latest
glibc-devel.i686                          2.12-1.192.el6              ol6_latest
glibc-headers.i686                        2.12-1.192.el6              ol6_latest
glibc-static.i686                         2.12-1.192.el6              ol6_latest
glibc-utils.x86_64                        2.12-1.166.el6              OEL6.7
glibc-utils.i686                          2.12-1.192.el6              ol6_latest


[root@dm01db08 yum.repos.d]# yum --showduplicates list glibc.i686
Available Packages
glibc.i686                                                                           2.12-1.7.el6                                                                                      ol6_latest
glibc.i686                                                                           2.12-1.7.el6_0.3                                                                                  ol6_latest
glibc.i686                                                                           2.12-1.7.el6_0.4                                                                                  ol6_latest
glibc.i686                                                                           2.12-1.7.el6_0.5                                                                                  ol6_latest
glibc.i686                                                                           2.12-1.25.el6                                                                                     ol6_latest
glibc.i686                                                                           2.12-1.25.el6_1.3                                                                                 ol6_latest
glibc.i686                                                                           2.12-1.47.el6                                                                                     ol6_latest
glibc.i686                                                                           2.12-1.47.el6_2.5                                                                                 ol6_latest
glibc.i686                                                                           2.12-1.47.el6_2.9                                                                                 ol6_latest
glibc.i686                                                                           2.12-1.47.el6_2.12                                                                                ol6_latest
glibc.i686                                                                           2.12-1.47.0.2.el6_2.12                                                                            ol6_latest
glibc.i686                                                                           2.12-1.80.el6                                                                                     ol6_latest
glibc.i686                                                                           2.12-1.80.el6_3.3                                                                                 ol6_latest
glibc.i686                                                                           2.12-1.80.el6_3.4                                                                                 ol6_latest
glibc.i686                                                                           2.12-1.80.el6_3.5                                                                                 ol6_latest
glibc.i686                                                                           2.12-1.80.el6_3.6                                                                                 ol6_latest
glibc.i686                                                                           2.12-1.80.el6_3.7                                                                                 ol6_latest
glibc.i686                                                                           2.12-1.107.el6                                                                                    ol6_latest
glibc.i686                                                                           2.12-1.107.el6_4.2                                                                                ol6_latest
glibc.i686                                                                           2.12-1.107.el6_4.4                                                                                ol6_latest
glibc.i686                                                                           2.12-1.107.el6_4.5                                                                                ol6_latest
glibc.i686                                                                           2.12-1.132.el6                                                                                    ol6_latest
glibc.i686                                                                           2.12-1.132.el6_5.1                                                                                ol6_latest
glibc.i686                                                                           2.12-1.132.el6_5.2                                                                                ol6_latest
glibc.i686                                                                           2.12-1.132.el6_5.3                                                                                ol6_latest
glibc.i686                                                                           2.12-1.132.el6_5.4                                                                                ol6_latest
glibc.i686                                                                           2.12-1.149.el6                                                                                    ol6_latest
glibc.i686                                                                           2.12-1.149.el6_6.4                                                                                ol6_latest
glibc.i686                                                                           2.12-1.149.el6_6.5                                                                                ol6_latest
glibc.i686                                                                           2.12-1.149.el6_6.7                                                                                ol6_latest
glibc.i686                                                                           2.12-1.149.el6_6.9                                                                                ol6_latest
glibc.i686                                                                           2.12-1.166.el6                                                                                    OEL6.7
glibc.i686                                                                           2.12-1.166.el6                                                                                    ol6_latest
glibc.i686                                                                           2.12-1.166.el6_7.1                                                                                ol6_latest
glibc.i686                                                                           2.12-1.166.el6_7.3                                                                                ol6_latest
glibc.i686                                                                           2.12-1.166.el6_7.7                                                                                ol6_latest
glibc.i686                                                                           2.12-1.192.el6                                                                                    ol6_latest


9. Install proper glibc.i686 rpm indicated in previous list (glibc.i686)

[root@dm01db08 yum.repos.d]# yum install glibc-2.12-1.166.el6_7.7
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package glibc.i686 0:2.12-1.166.el6_7.7 will be installed
--> Processing Dependency: libfreebl3.so for package: glibc-2.12-1.166.el6_7.7.i686
--> Processing Dependency: libfreebl3.so(NSSRAWHASH_3.12.3) for package: glibc-2.12-1.166.el6_7.7.i686
--> Running transaction check
---> Package nss-softokn-freebl.i686 0:3.14.3-23.el6_7 will be installed
--> Finished Dependency Resolution
 
Dependencies Resolved
 
=================================================================================================================================================================================================
Package                                             Arch                                  Version                                             Repository                                   Size
=================================================================================================================================================================================================
Installing:
glibc                                               i686                                  2.12-1.166.el6_7.7                                  ol6_latest                                  4.3 M
Installing for dependencies:
nss-softokn-freebl                                  i686                                  3.14.3-23.el6_7                                     ol6_latest                                  156 k
 
Transaction Summary
=================================================================================================================================================================================================
Install       2 Package(s)
 
Total download size: 4.5 M
Installed size: 14 M
Is this ok [y/N]: y
Downloading Packages:
(1/2): glibc-2.12-1.166.el6_7.7.i686.rpm                                                                                                                                  | 4.3 MB     00:01
(2/2): nss-softokn-freebl-3.14.3-23.el6_7.i686.rpm                                                                                                                        | 156 kB     00:00
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                            913 kB/s | 4.5 MB     00:05
warning: rpmts_HdrFromFdno: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
Importing GPG key 0xEC551F03:
Userid : Oracle OSS group (Open Source Software group) <build@oss.oracle.com>
Package: 6:oraclelinux-release-6Server-7.0.6.x86_64 (@exadata_generated_040616190730)
From   : /etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
Is this ok [y/N]: yy
Is this ok [y/N]: y
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : glibc-2.12-1.166.el6_7.7.i686                                                                                                                                                 1/2
  Installing : nss-softokn-freebl-3.14.3-23.el6_7.i686                                                                                                                                       2/2
  Verifying  : nss-softokn-freebl-3.14.3-23.el6_7.i686                                                                                                                                       1/2
  Verifying  : glibc-2.12-1.166.el6_7.7.i686                                                                                                                                                 2/2
 
Installed:
  glibc.i686 0:2.12-1.166.el6_7.7
 
Dependency Installed:
  nss-softokn-freebl.i686 0:3.14.3-23.el6_7
 
Complete!


10. Copy the /etc/yum.conf and /etc/yum.repos.d/public-yum-ol6.repo to all the other DB nodes in the same Exadata and install the rpm in each node.

[root@dm01db07 ~]# yum install glibc-2.12-1.166.el6_7.7
Setting up Install Process
ol6_UEK_latest | 1.2 kB 00:00
ol6_UEK_latest/primary | 26 MB 00:08
ol6_UEK_latest 554/554
ol6_latest | 1.4 kB 00:00
ol6_latest/primary | 53 MB 00:13
ol6_latest 
...