libreoffice-online/loolwsd-systemplate-setup
Ashod Nakashian d6259d6a54 wsd: support parallel systemplate setup
When tests are run in parallel, they will all
compete to update and set up the systemplate
directory, which has a handful of files that
need to be up-to-date. This is a source of errors.

Normally, these files are linked (hard- or soft-
link, whichever succeeds). With linking, we
only need to worry about the initial setup,
as the files will never be out-of-date from
then on. However, when linking fails, we need
to copy the files, and update them (by copying
over fresh versions of the files, if necessary)
every time a new kit is forked. Copying over
is tricky, as it's not atomic. To make it
atomic, we copy the files to the destination
directory under a temporary (random) name,
and then rename to the final name (which is
atomic, including replacing the target file,
if it exists).

No such race exists in production, where there
is (or should be) but one instance of loolwsd
(which does the initial setup) and forkit
(which updates systemplate before forking
new kit instances).
This is an issue with parallel tests only.

Change-Id: I6ba1514d00a84da7397d28efeb6378619711d52f
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/97785
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
2020-07-07 19:05:23 +02:00

93 lines
2.6 KiB
Bash
Executable file

#!/bin/bash
test $# -eq 2 || { echo "Usage: $0 <chroot template directory for system libs to create> <LO installation directory>"; exit 1; }
# No provision for spaces or other weird characters in pathnames. So sue me.
CHROOT=$1
INSTDIR=$2
test -d "$INSTDIR" || { echo "No such directory: $INSTDIR"; exit 1; }
mkdir -p $CHROOT || exit 1
CHROOT=`cd $CHROOT && /bin/pwd`
INSTDIR=`cd $INSTDIR && /bin/pwd`
cd / || exit 1
(
# Produce a list of file names, one per line, that will be copied
# into the template tree of system files for the chroot jails.
# First essential files and shared objects
find etc/ld.so.* \
lib/ld-* lib64/ld-* \
lib/libnss_* lib64/libnss_* lib/*/libnss_* \
lib/libresolv* lib64/libresolv* lib/*/libresolv* \
var/cache/fontconfig \
etc/fonts \
usr/lib/locale/en_US.utf8 \
usr/lib/locale/C.UTF-8 \
usr/lib/locale/locale_archive \
usr/lib/*/nss/*.so \
usr/lib/*/libsqlite* \
usr/share/zoneinfo/* \
usr/share/liblangtag \
usr/share/hyphen \
-type f 2>/dev/null
find etc/fonts \
lib/ld-* lib64/ld-* \
lib/libnss_* lib64/libnss_* lib/*/libnss_* \
lib/libresolv* lib64/libresolv* lib/*/libresolv* \
usr/lib/*/libsqlite* \
-type l 2>/dev/null
# Go through the LO shared objects and check what system libraries
# they link to.
find $INSTDIR -name 'xpdfimport' |
while read file; do
ldd $file 2>/dev/null
done |
grep -v dynamic | cut -d " " -f 3 | grep -E '^(/lib|/usr)' | sort -u | sed -e 's,^/,,'
) |
# Can't use -l because then symlinks won't be handled well enough.
# This will now copy the file a symlink points to, but whatever.
cpio -p -d -L $CHROOT
# Remove the dynamic files, which are linked by loolwsd.
rm -f $CHROOT/etc/{hosts,nsswitch.conf,resolv.conf,passwd,group,host.conf,timezone,localtime}
mkdir -p $CHROOT/lo
mkdir -p $CHROOT/dev
mkdir -p $CHROOT/tmp/dev
# /usr/share/fonts needs to be taken care of separately because the
# directory time stamps must be preserved for fontconfig to trust
# its cache.
cd $CHROOT || exit 1
mkdir -p usr/share || exit 1
cp -r -p -L /usr/share/fonts usr/share
if [ -h usr/share/fonts/ghostscript ]; then
mkdir usr/share/ghostscript || exit 1
cp -r -p -L /usr/share/ghostscript/fonts usr/share/ghostscript
fi
# Remove obsolete & unused bitmap fonts
find usr/share -name '*.pcf' | xargs rm -f
find usr/share -name '*.pcf.gz' | xargs rm -f
# Debugging only hackery to avoid confusion.
if test "z$ENABLE_DEBUG" != "z" -a "z$HOME" != "z"; then
echo "Copying development users's fonts into systemplate"
mkdir -p $CHROOT/$HOME
test -d $HOME/.fonts && cp -r -p -L $HOME/.fonts $CHROOT/$HOME
fi
exit 0