Symptoms
Summary: program fails with error: "unable to initialize mutex: Function not implemented"
A program (
rpm
,
OpenLDAP's
slapd
) aborts with an error message like:
rpmdb: unable to initialize mutex: Function not implemented
...
error: db4 error(38) from dbenv->open: Function not implemented
The exact db4 function in which the problem occurs may vary, but the
error code 38 and the message
unable to initialize mutex: Function not implemented
are constant.
The problem we are facing (and its variations, depending on the
workarounds that one can try) is quite accurately described in
http://www.openldap.org/lists/openldap-software/200603/msg00199.html
Occurrences
Reproduceable on any Xen
DomU running RHEL4-derived systems (
CentOS4, SL4, SLC4).
Observations
On the site-BDII, the problem was fixed by using the "ldbm" backend
and avoiding Berkeley DB4 completely, see
IssueBdiiNotStartingOnXenDomU
We cannot do this for Phedex, since it uses a CMS-built version of
apt+rpm+db4, so we cannot rebuild that with different flags.
Solution or Workaround
I wasn't able to find much information by googling: all the related
pages date back to 2005/2006 (when
CentOS4/RHEL4 was current), and the
links to software packages or patches are often broken. Some suggest
fixing the problem by recompiling the libc with a special gcc option;
finally I found the exact patch here:
http://lists.centos.org/pipermail/centos-devel/2005-December/000175.html
Rebuilding the libc with the patch applied solved the issue,
apparently. (Patch and new glibc RPM attached.) Caveat: installation
of the glibc RPM directly from the DomU command-line prompt led to a
failed upgrade and an unusable system. I had to:
- stop the DomU
- mount its root filesystem in the Dom0:
-
chroot
into it (bind-mount /dev
and /proc
for RPM to work properly):
mount /dev/vm-root-filesys /mnt
mount --bind /dev /mnt/dev
mount --bind /proc /mnt/proc
chroot /mnt rpm -Uvh glibc-2.3.4-2.41.xenu.rpm
umount /mnt/proc
umount /mnt/dev
umount /mnt
Note that upgrading the glibc RPM can lead to problems when bulk
upgrades of the system RPMs are performed (
yum update
), as that can
replace the DomU-specific
glibc
with a new version. A workaround
to this is described in:
http://marc.info/?l=xen-users&m=111761896306738&w=2
Also note that the same issue is known to the gLite/WLCG upstream, and tracked in
bug #42475 at CERN Savannah
where other workarounds are suggested, none of which worked here.
--
RiccardoMurri - 05 Nov 2008