strepsirrhini/libreoffice-online
3
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
libreoffice-online/common/JailUtil.hpp

73 lines
2.2 KiB
C++
Raw Normal View History

wsd: faster jail setup via bind-mount loolmount now works and supports mounting and unmounting, plus numerous improvements, refactoring, logging, etc.. When enabled, binding improves the jail setup time by anywhere from 2x to orders of magnitude (in docker, f.e.). A new config entry mount_jail_tree controls whether mounting is used or the old method of linking/copying of jail contents. It is set to true by default and falls back to linking/copying. A test mount is done when the setting is enabled, and if mounting fails, it's disabled to avoid noise. Temporarily disabled for unit-tests until we can cleanup lingering mounts after Jenkins aborts our build job. In a future patch we will have mount/jail cleanup as part of make. The network/system files in /etc that need frequent refreshing are now updated in systemplate to make their most recent version available in the jails. These files can change during the course of loolwsd lifetime, and are unlikely to be updated in systemplate after installation at all. We link to them in the systemplate/etc directory, and if that fails, we copy them before forking each kit instance to have the latest. This reworks the approach used to bind-mount the jails and the templates such that the total is now down to only three mounts: systemplate, lo, tmp. As now systemplate and lotemplate are shared, they must be mounted as readonly, this means that user/ must now be moved into tmp/user/ which is writable. The mount-points must be recursive, because we mount lo/ within the mount-point of systemplate (which is the root of the jail). But because we (re)bind recursively, and because both systemplate and lotemplate are mounted for each jails, we need to make them unbindable, so they wouldn't multiply the mount-points for each jails (an explosive growth!) Contrarywise, we don't want the mount-points to be shared, because we don't expect to add/remove mounts after a jail is created. The random temp directory is now created and set correctly, plus many logging and other improvements. Change-Id: Iae3fda5e876cf47d2cae6669a87b5b826a8748df Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92829 Tested-by: Jenkins Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com> Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
2020-04-09 08:02:58 -05:00
/* -*- Mode: C++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4; fill-column: 100 -*- */
/*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
#pragma once
#include <string>
#include <Poco/File.h>
#include <Poco/Path.h>
namespace JailUtil
{
/// Bind mount a jail directory.
bool bind(const std::string& source, const std::string& target);
/// Remount a bound mount point as readonly.
bool remountReadonly(const std::string& source, const std::string& target);
/// Unmount a bind-mounted jail directory.
bool unmount(const std::string& target);
wsd: move jail setup to the script to support readonly systemplate We now gracefully fallback to copying when/if systemplate is readonly. The bulk of the change is to support proper cleanup in both cases. First, we had to move as much of the jail bootstrapping into the loolwsd-systemplate-setup script, so systemplate will be as complete as possible before it is locked down. Next, we needed to update the jail with graceful fallback to linking/copying upon failure. For that, the jail setup logic in Kit.cpp has been reworked to support not just update failures, but also more comprehensive mounting failures as well. Finally, jail cleanup now is seamless. To support proper cleanup when we had mounting enabled but had to fallback, we mark jails that aren't mounted so we can 'rm -rf' the contents safely and without fear or causing undue damage (as unlikely as that is, technically we wouldn't want to rm systemplate files, if mounting read-only had failed). There are a few minor refactorings of JailUtil to make it cleaner and more robust. Change-Id: Iac34869cb84f45acf64fbbc46d46898367b496d2 Reviewed-on: https://gerrit.libreoffice.org/c/online/+/101260 Tested-by: Jenkins Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com> Reviewed-by: Andras Timar <andras.timar@collabora.com>
2020-08-23 11:11:23 -05:00
/// Marks a jail as having been copied instead of mounted.
void markJailCopied(const std::string& root);
wsd: faster jail setup via bind-mount loolmount now works and supports mounting and unmounting, plus numerous improvements, refactoring, logging, etc.. When enabled, binding improves the jail setup time by anywhere from 2x to orders of magnitude (in docker, f.e.). A new config entry mount_jail_tree controls whether mounting is used or the old method of linking/copying of jail contents. It is set to true by default and falls back to linking/copying. A test mount is done when the setting is enabled, and if mounting fails, it's disabled to avoid noise. Temporarily disabled for unit-tests until we can cleanup lingering mounts after Jenkins aborts our build job. In a future patch we will have mount/jail cleanup as part of make. The network/system files in /etc that need frequent refreshing are now updated in systemplate to make their most recent version available in the jails. These files can change during the course of loolwsd lifetime, and are unlikely to be updated in systemplate after installation at all. We link to them in the systemplate/etc directory, and if that fails, we copy them before forking each kit instance to have the latest. This reworks the approach used to bind-mount the jails and the templates such that the total is now down to only three mounts: systemplate, lo, tmp. As now systemplate and lotemplate are shared, they must be mounted as readonly, this means that user/ must now be moved into tmp/user/ which is writable. The mount-points must be recursive, because we mount lo/ within the mount-point of systemplate (which is the root of the jail). But because we (re)bind recursively, and because both systemplate and lotemplate are mounted for each jails, we need to make them unbindable, so they wouldn't multiply the mount-points for each jails (an explosive growth!) Contrarywise, we don't want the mount-points to be shared, because we don't expect to add/remove mounts after a jail is created. The random temp directory is now created and set correctly, plus many logging and other improvements. Change-Id: Iae3fda5e876cf47d2cae6669a87b5b826a8748df Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92829 Tested-by: Jenkins Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com> Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
2020-04-09 08:02:58 -05:00
wsd: move jail setup to the script to support readonly systemplate We now gracefully fallback to copying when/if systemplate is readonly. The bulk of the change is to support proper cleanup in both cases. First, we had to move as much of the jail bootstrapping into the loolwsd-systemplate-setup script, so systemplate will be as complete as possible before it is locked down. Next, we needed to update the jail with graceful fallback to linking/copying upon failure. For that, the jail setup logic in Kit.cpp has been reworked to support not just update failures, but also more comprehensive mounting failures as well. Finally, jail cleanup now is seamless. To support proper cleanup when we had mounting enabled but had to fallback, we mark jails that aren't mounted so we can 'rm -rf' the contents safely and without fear or causing undue damage (as unlikely as that is, technically we wouldn't want to rm systemplate files, if mounting read-only had failed). There are a few minor refactorings of JailUtil to make it cleaner and more robust. Change-Id: Iac34869cb84f45acf64fbbc46d46898367b496d2 Reviewed-on: https://gerrit.libreoffice.org/c/online/+/101260 Tested-by: Jenkins Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com> Reviewed-by: Andras Timar <andras.timar@collabora.com>
2020-08-23 11:11:23 -05:00
/// Returns true iff the jail in question was copied and not mounted.
bool isJailCopied(const std::string& root);
/// Remove the jail directory and all its contents.
void removeJail(const std::string& root);
wsd: faster jail setup via bind-mount loolmount now works and supports mounting and unmounting, plus numerous improvements, refactoring, logging, etc.. When enabled, binding improves the jail setup time by anywhere from 2x to orders of magnitude (in docker, f.e.). A new config entry mount_jail_tree controls whether mounting is used or the old method of linking/copying of jail contents. It is set to true by default and falls back to linking/copying. A test mount is done when the setting is enabled, and if mounting fails, it's disabled to avoid noise. Temporarily disabled for unit-tests until we can cleanup lingering mounts after Jenkins aborts our build job. In a future patch we will have mount/jail cleanup as part of make. The network/system files in /etc that need frequent refreshing are now updated in systemplate to make their most recent version available in the jails. These files can change during the course of loolwsd lifetime, and are unlikely to be updated in systemplate after installation at all. We link to them in the systemplate/etc directory, and if that fails, we copy them before forking each kit instance to have the latest. This reworks the approach used to bind-mount the jails and the templates such that the total is now down to only three mounts: systemplate, lo, tmp. As now systemplate and lotemplate are shared, they must be mounted as readonly, this means that user/ must now be moved into tmp/user/ which is writable. The mount-points must be recursive, because we mount lo/ within the mount-point of systemplate (which is the root of the jail). But because we (re)bind recursively, and because both systemplate and lotemplate are mounted for each jails, we need to make them unbindable, so they wouldn't multiply the mount-points for each jails (an explosive growth!) Contrarywise, we don't want the mount-points to be shared, because we don't expect to add/remove mounts after a jail is created. The random temp directory is now created and set correctly, plus many logging and other improvements. Change-Id: Iae3fda5e876cf47d2cae6669a87b5b826a8748df Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92829 Tested-by: Jenkins Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com> Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
2020-04-09 08:02:58 -05:00
/// Remove all jails.
void cleanupJails(const std::string& jailRoot);
/// Setup the jails.
void setupJails(bool bindMount, const std::string& jailRoot, const std::string& sysTemplate);
/// Setup /dev/random and /dev/urandom in the given jail path.
void setupJailDevNodes(const std::string& root);
wsd: move jail setup to the script to support readonly systemplate We now gracefully fallback to copying when/if systemplate is readonly. The bulk of the change is to support proper cleanup in both cases. First, we had to move as much of the jail bootstrapping into the loolwsd-systemplate-setup script, so systemplate will be as complete as possible before it is locked down. Next, we needed to update the jail with graceful fallback to linking/copying upon failure. For that, the jail setup logic in Kit.cpp has been reworked to support not just update failures, but also more comprehensive mounting failures as well. Finally, jail cleanup now is seamless. To support proper cleanup when we had mounting enabled but had to fallback, we mark jails that aren't mounted so we can 'rm -rf' the contents safely and without fear or causing undue damage (as unlikely as that is, technically we wouldn't want to rm systemplate files, if mounting read-only had failed). There are a few minor refactorings of JailUtil to make it cleaner and more robust. Change-Id: Iac34869cb84f45acf64fbbc46d46898367b496d2 Reviewed-on: https://gerrit.libreoffice.org/c/online/+/101260 Tested-by: Jenkins Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com> Reviewed-by: Andras Timar <andras.timar@collabora.com>
2020-08-23 11:11:23 -05:00
/// Enable bind-mounting in this process.
void enableBindMounting();
/// Disable bind-mounting in this process.
void disableBindMounting();
/// Returns true iff bind-mounting is enabled in this process.
bool isBindMountingEnabled();
wsd: faster jail setup via bind-mount loolmount now works and supports mounting and unmounting, plus numerous improvements, refactoring, logging, etc.. When enabled, binding improves the jail setup time by anywhere from 2x to orders of magnitude (in docker, f.e.). A new config entry mount_jail_tree controls whether mounting is used or the old method of linking/copying of jail contents. It is set to true by default and falls back to linking/copying. A test mount is done when the setting is enabled, and if mounting fails, it's disabled to avoid noise. Temporarily disabled for unit-tests until we can cleanup lingering mounts after Jenkins aborts our build job. In a future patch we will have mount/jail cleanup as part of make. The network/system files in /etc that need frequent refreshing are now updated in systemplate to make their most recent version available in the jails. These files can change during the course of loolwsd lifetime, and are unlikely to be updated in systemplate after installation at all. We link to them in the systemplate/etc directory, and if that fails, we copy them before forking each kit instance to have the latest. This reworks the approach used to bind-mount the jails and the templates such that the total is now down to only three mounts: systemplate, lo, tmp. As now systemplate and lotemplate are shared, they must be mounted as readonly, this means that user/ must now be moved into tmp/user/ which is writable. The mount-points must be recursive, because we mount lo/ within the mount-point of systemplate (which is the root of the jail). But because we (re)bind recursively, and because both systemplate and lotemplate are mounted for each jails, we need to make them unbindable, so they wouldn't multiply the mount-points for each jails (an explosive growth!) Contrarywise, we don't want the mount-points to be shared, because we don't expect to add/remove mounts after a jail is created. The random temp directory is now created and set correctly, plus many logging and other improvements. Change-Id: Iae3fda5e876cf47d2cae6669a87b5b826a8748df Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92829 Tested-by: Jenkins Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com> Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
2020-04-09 08:02:58 -05:00
namespace SysTemplate
{
/// Setup links for /dev/random and /dev/urandom in systemplate.
void setupRandomDeviceLinks(const std::string& root);
wsd: move jail setup to the script to support readonly systemplate We now gracefully fallback to copying when/if systemplate is readonly. The bulk of the change is to support proper cleanup in both cases. First, we had to move as much of the jail bootstrapping into the loolwsd-systemplate-setup script, so systemplate will be as complete as possible before it is locked down. Next, we needed to update the jail with graceful fallback to linking/copying upon failure. For that, the jail setup logic in Kit.cpp has been reworked to support not just update failures, but also more comprehensive mounting failures as well. Finally, jail cleanup now is seamless. To support proper cleanup when we had mounting enabled but had to fallback, we mark jails that aren't mounted so we can 'rm -rf' the contents safely and without fear or causing undue damage (as unlikely as that is, technically we wouldn't want to rm systemplate files, if mounting read-only had failed). There are a few minor refactorings of JailUtil to make it cleaner and more robust. Change-Id: Iac34869cb84f45acf64fbbc46d46898367b496d2 Reviewed-on: https://gerrit.libreoffice.org/c/online/+/101260 Tested-by: Jenkins Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com> Reviewed-by: Andras Timar <andras.timar@collabora.com>
2020-08-23 11:11:23 -05:00
/// Setup the dynamic files within the sysTemplate by either
wsd: faster jail setup via bind-mount loolmount now works and supports mounting and unmounting, plus numerous improvements, refactoring, logging, etc.. When enabled, binding improves the jail setup time by anywhere from 2x to orders of magnitude (in docker, f.e.). A new config entry mount_jail_tree controls whether mounting is used or the old method of linking/copying of jail contents. It is set to true by default and falls back to linking/copying. A test mount is done when the setting is enabled, and if mounting fails, it's disabled to avoid noise. Temporarily disabled for unit-tests until we can cleanup lingering mounts after Jenkins aborts our build job. In a future patch we will have mount/jail cleanup as part of make. The network/system files in /etc that need frequent refreshing are now updated in systemplate to make their most recent version available in the jails. These files can change during the course of loolwsd lifetime, and are unlikely to be updated in systemplate after installation at all. We link to them in the systemplate/etc directory, and if that fails, we copy them before forking each kit instance to have the latest. This reworks the approach used to bind-mount the jails and the templates such that the total is now down to only three mounts: systemplate, lo, tmp. As now systemplate and lotemplate are shared, they must be mounted as readonly, this means that user/ must now be moved into tmp/user/ which is writable. The mount-points must be recursive, because we mount lo/ within the mount-point of systemplate (which is the root of the jail). But because we (re)bind recursively, and because both systemplate and lotemplate are mounted for each jails, we need to make them unbindable, so they wouldn't multiply the mount-points for each jails (an explosive growth!) Contrarywise, we don't want the mount-points to be shared, because we don't expect to add/remove mounts after a jail is created. The random temp directory is now created and set correctly, plus many logging and other improvements. Change-Id: Iae3fda5e876cf47d2cae6669a87b5b826a8748df Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92829 Tested-by: Jenkins Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com> Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
2020-04-09 08:02:58 -05:00
/// copying or linking. See updateJail_DynamicFilesInSysTemplate.
wsd: move jail setup to the script to support readonly systemplate We now gracefully fallback to copying when/if systemplate is readonly. The bulk of the change is to support proper cleanup in both cases. First, we had to move as much of the jail bootstrapping into the loolwsd-systemplate-setup script, so systemplate will be as complete as possible before it is locked down. Next, we needed to update the jail with graceful fallback to linking/copying upon failure. For that, the jail setup logic in Kit.cpp has been reworked to support not just update failures, but also more comprehensive mounting failures as well. Finally, jail cleanup now is seamless. To support proper cleanup when we had mounting enabled but had to fallback, we mark jails that aren't mounted so we can 'rm -rf' the contents safely and without fear or causing undue damage (as unlikely as that is, technically we wouldn't want to rm systemplate files, if mounting read-only had failed). There are a few minor refactorings of JailUtil to make it cleaner and more robust. Change-Id: Iac34869cb84f45acf64fbbc46d46898367b496d2 Reviewed-on: https://gerrit.libreoffice.org/c/online/+/101260 Tested-by: Jenkins Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com> Reviewed-by: Andras Timar <andras.timar@collabora.com>
2020-08-23 11:11:23 -05:00
/// If the dynamic files need updating and systemplate is read-only,
/// this will fail and mark files for copying.
wsd: faster jail setup via bind-mount loolmount now works and supports mounting and unmounting, plus numerous improvements, refactoring, logging, etc.. When enabled, binding improves the jail setup time by anywhere from 2x to orders of magnitude (in docker, f.e.). A new config entry mount_jail_tree controls whether mounting is used or the old method of linking/copying of jail contents. It is set to true by default and falls back to linking/copying. A test mount is done when the setting is enabled, and if mounting fails, it's disabled to avoid noise. Temporarily disabled for unit-tests until we can cleanup lingering mounts after Jenkins aborts our build job. In a future patch we will have mount/jail cleanup as part of make. The network/system files in /etc that need frequent refreshing are now updated in systemplate to make their most recent version available in the jails. These files can change during the course of loolwsd lifetime, and are unlikely to be updated in systemplate after installation at all. We link to them in the systemplate/etc directory, and if that fails, we copy them before forking each kit instance to have the latest. This reworks the approach used to bind-mount the jails and the templates such that the total is now down to only three mounts: systemplate, lo, tmp. As now systemplate and lotemplate are shared, they must be mounted as readonly, this means that user/ must now be moved into tmp/user/ which is writable. The mount-points must be recursive, because we mount lo/ within the mount-point of systemplate (which is the root of the jail). But because we (re)bind recursively, and because both systemplate and lotemplate are mounted for each jails, we need to make them unbindable, so they wouldn't multiply the mount-points for each jails (an explosive growth!) Contrarywise, we don't want the mount-points to be shared, because we don't expect to add/remove mounts after a jail is created. The random temp directory is now created and set correctly, plus many logging and other improvements. Change-Id: Iae3fda5e876cf47d2cae6669a87b5b826a8748df Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92829 Tested-by: Jenkins Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com> Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
2020-04-09 08:02:58 -05:00
void setupDynamicFiles(const std::string& sysTemplate);
/// Update the dynamic files within the sysTemplate before each child fork.
wsd: support read-only systemplate For various reasons, systemplate may be read-only or under a different owner and therefore impossible to update the dynamic files in it. To support such a scenario, we first link the eight dynamic files in /etc when creating systemplate. If this fails, we copy the files. When creating jails, we always check that all the dynamic files are up-to-date. If they are, nothing further is necessary and we bind-mount, if enabled and possible. However, if the dynamic files are not up-to-date, we disable bind-mounting and force linking the files in the jails. Failing that, we copy them, which is not ideal, but allows us to ensure the dynamic files are up-to-date as we copy them too. Ideally, the dynamic files in question would be hard-link (or at least soft-linked) in systemplate at creation. From then on we would bind-mount the jails and everything would work perfectly and no files would need updating. This patch is fallback for when this scheme fails, which should be exceedingly rare anyway, but which still ensures correct operation. Change-Id: I09c6f057c49396579aaddb1b8bf4af0930dd4247 Reviewed-on: https://gerrit.libreoffice.org/c/online/+/100834 Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com> Tested-by: Jenkins Reviewed-by: Andras Timar <andras.timar@collabora.com>
2020-08-16 21:46:33 -05:00
/// Returns false on failure.
bool updateDynamicFiles(const std::string& sysTemplate);
wsd: faster jail setup via bind-mount loolmount now works and supports mounting and unmounting, plus numerous improvements, refactoring, logging, etc.. When enabled, binding improves the jail setup time by anywhere from 2x to orders of magnitude (in docker, f.e.). A new config entry mount_jail_tree controls whether mounting is used or the old method of linking/copying of jail contents. It is set to true by default and falls back to linking/copying. A test mount is done when the setting is enabled, and if mounting fails, it's disabled to avoid noise. Temporarily disabled for unit-tests until we can cleanup lingering mounts after Jenkins aborts our build job. In a future patch we will have mount/jail cleanup as part of make. The network/system files in /etc that need frequent refreshing are now updated in systemplate to make their most recent version available in the jails. These files can change during the course of loolwsd lifetime, and are unlikely to be updated in systemplate after installation at all. We link to them in the systemplate/etc directory, and if that fails, we copy them before forking each kit instance to have the latest. This reworks the approach used to bind-mount the jails and the templates such that the total is now down to only three mounts: systemplate, lo, tmp. As now systemplate and lotemplate are shared, they must be mounted as readonly, this means that user/ must now be moved into tmp/user/ which is writable. The mount-points must be recursive, because we mount lo/ within the mount-point of systemplate (which is the root of the jail). But because we (re)bind recursively, and because both systemplate and lotemplate are mounted for each jails, we need to make them unbindable, so they wouldn't multiply the mount-points for each jails (an explosive growth!) Contrarywise, we don't want the mount-points to be shared, because we don't expect to add/remove mounts after a jail is created. The random temp directory is now created and set correctly, plus many logging and other improvements. Change-Id: Iae3fda5e876cf47d2cae6669a87b5b826a8748df Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92829 Tested-by: Jenkins Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com> Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
2020-04-09 08:02:58 -05:00
} // namespace SysTemplate
} // end namespace JailUtil
/* vim:set shiftwidth=4 softtabstop=4 expandtab: */
Reference in a new issue Copy permalink