libreoffice-online/common/FileUtil.hpp
Ashod Nakashian 5c9988f2e3 wsd: faster jail setup via bind-mount
loolmount now works and supports mounting and
unmounting, plus numerous improvements,
refactoring, logging, etc..  When enabled,
binding improves the jail setup time by anywhere
from 2x to orders of magnitude (in docker, f.e.).

A new config entry mount_jail_tree controls
whether mounting is used or the old method of
linking/copying of jail contents. It is set to
true by default and falls back to linking/copying.
A test mount is done when the setting is enabled,
and if mounting fails, it's disabled to avoid noise.

Temporarily disabled for unit-tests until we can
cleanup lingering mounts after Jenkins aborts our
build job. In a future patch we will have mount/jail
cleanup as part of make.

The network/system files in /etc that need frequent
refreshing are now updated in systemplate to make
their most recent version available in the jails.
These files can change during the course of loolwsd
lifetime, and are unlikely to be updated in
systemplate after installation at all. We link to
them in the systemplate/etc directory, and if that
fails, we copy them before forking each kit
instance to have the latest.

This reworks the approach used to bind-mount the
jails and the templates such that the total is
now down to only three mounts: systemplate, lo, tmp.

As now systemplate and lotemplate are shared, they
must be mounted as readonly, this means that user/
must now be moved into tmp/user/ which is writable.

The mount-points must be recursive, because we mount
lo/ within the mount-point of systemplate (which is
the root of the jail). But because we (re)bind
recursively, and because both systemplate and
lotemplate are mounted for each jails, we need to
make them unbindable, so they wouldn't multiply the
mount-points for each jails (an explosive growth!)
Contrarywise, we don't want the mount-points to
be shared, because we don't expect to add/remove
mounts after a jail is created.

The random temp directory is now created and set
correctly, plus many logging and other improvements.

Change-Id: Iae3fda5e876cf47d2cae6669a87b5b826a8748df
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92829
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
2020-07-01 05:42:43 +02:00

127 lines
5.3 KiB
C++

/* -*- Mode: C++; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4; fill-column: 100 -*- */
/*
* This file is part of the LibreOffice project.
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
#pragma once
#include <cerrno>
#include <string>
#include <sys/stat.h>
#include <Poco/Path.h>
namespace FileUtil
{
/// Used for anonymizing URLs
void setUrlAnonymization(bool anonymize, const std::uint64_t salt);
/// Anonymize the basename of filenames, preserving the path and extension.
std::string anonymizeUrl(const std::string& url);
/// Anonymize user names and IDs.
/// Will use the Obfuscated User ID if one is provided via WOPI.
std::string anonymizeUsername(const std::string& username);
/// Create a secure, random directory path.
std::string createRandomDir(const std::string& path);
// Save data to a file (overwriting an existing file if necessary) with checks for errors. Write
// to a temporary file in the same directory that is then atomically renamed to the desired name
// if everything goes well. In case of any error, both the destination file (if it already
// exists) and the temporary file (if was created, or existed already) are removed. Return true
// if everything succeeded.
bool saveDataToFileSafely(const std::string& fileName, const char* data, size_t size);
// We work around some of the mess of using the same sources both on the server side and in unit
// tests with conditional compilation based on BUILDING_TESTS.
// Add the file system that 'path' is located on to a list of file systems that are periodically
// checked for available space. The list is initially empty.
void registerFileSystemForDiskSpaceChecks(const std::string& path);
// Perform the check. If the free space on any of the registered file systems is below 5%, call
// 'alertAllUsers("internal", "diskfull")'. The check will be made no more often than once a
// minute if cacheLastCheck is set to true.
std::string checkDiskSpaceOnRegisteredFileSystems(const bool cacheLastCheck = true);
// Check disk space on a specific file system, the one where 'path' is located. This does not
// add that file system to the list used by 'registerFileSystemForDiskSpaceChecks'. If the free
// space on the file system is below 5%, return false, otherwise true. Note that this function
// does not call 'alertAllUsers'.
bool checkDiskSpace(const std::string& path);
/// Safely remove a file or directory.
/// Suppresses exception when the file is already removed.
/// This can happen when there is a race (unavoidable) or when
/// we don't care to check before we remove (when no race exists).
void removeFile(const std::string& path, const bool recursive = false);
inline void removeFile(const Poco::Path& path, const bool recursive = false)
{
removeFile(path.toString(), recursive);
}
/// Returns true iff the directory is empty (or doesn't exist).
bool isEmptyDirectory(const char* path);
inline bool isEmptyDirectory(const std::string& path) { return isEmptyDirectory(path.c_str()); }
/// Copy a file from @fromPath to @toPath, throws on failure.
void copyFileTo(const std::string &fromPath, const std::string &toPath);
/// Make a temp copy of a file, and prepend it with a prefix.
std::string getTempFilePath(const std::string& srcDir, const std::string& srcFilename,
const std::string& dstFilenamePrefix);
/// Make a temp copy of a file.
/// Primarily used by tests to avoid tainting the originals.
/// srcDir shouldn't end with '/' and srcFilename shouldn't contain '/'.
/// Returns the created file path.
inline std::string getTempFilePath(const std::string& srcDir, const std::string& srcFilename)
{
return getTempFilePath(srcDir, srcFilename, std::string());
}
/// Link source to target, and copy if linking fails.
bool linkOrCopyFile(const char* source, const char* target);
/// File/Directory stat helper.
class Stat
{
public:
/// Stat the given path. Symbolic links are stats when @link is true.
Stat(const std::string& file, bool link = false)
: _path(file)
, _res(link ? lstat(file.c_str(), &_sb) : stat(file.c_str(), &_sb))
, _errno(errno)
{
}
bool good() const { return _res == 0; }
bool bad() const { return !good(); }
bool erno() const { return _errno; }
const struct ::stat& sb() const { return _sb; }
const std::string path() const { return _path; }
bool isDirectory() const { return S_ISDIR(_sb.st_mode); }
bool isFile() const { return S_ISREG(_sb.st_mode); }
bool isLink() const { return S_ISLNK(_sb.st_mode); }
/// Returns true iff the path exists, regarlesss of access permission.
bool exists() const { return good() || (_errno != ENOENT && _errno != ENOTDIR); }
private:
const std::string _path;
struct ::stat _sb;
const int _res;
const int _errno;
};
} // end namespace FileUtil
/* vim:set shiftwidth=4 softtabstop=4 expandtab: */