Leaving behind jails with bind-mount
entries makes build-workspace removal
complicated, and jenkins builds start failing.
The cleanup stage is integrated in Makefiles
and should be transparent.
In the event that manual cleanup is necessary,
'loolwsd --cleanup' can be invoked.
Change-Id: Ia4b99b0c66e56dfa2d50e79b0ba98f714cf32886
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/97470
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
loolmount now works and supports mounting and
unmounting, plus numerous improvements,
refactoring, logging, etc.. When enabled,
binding improves the jail setup time by anywhere
from 2x to orders of magnitude (in docker, f.e.).
A new config entry mount_jail_tree controls
whether mounting is used or the old method of
linking/copying of jail contents. It is set to
true by default and falls back to linking/copying.
A test mount is done when the setting is enabled,
and if mounting fails, it's disabled to avoid noise.
Temporarily disabled for unit-tests until we can
cleanup lingering mounts after Jenkins aborts our
build job. In a future patch we will have mount/jail
cleanup as part of make.
The network/system files in /etc that need frequent
refreshing are now updated in systemplate to make
their most recent version available in the jails.
These files can change during the course of loolwsd
lifetime, and are unlikely to be updated in
systemplate after installation at all. We link to
them in the systemplate/etc directory, and if that
fails, we copy them before forking each kit
instance to have the latest.
This reworks the approach used to bind-mount the
jails and the templates such that the total is
now down to only three mounts: systemplate, lo, tmp.
As now systemplate and lotemplate are shared, they
must be mounted as readonly, this means that user/
must now be moved into tmp/user/ which is writable.
The mount-points must be recursive, because we mount
lo/ within the mount-point of systemplate (which is
the root of the jail). But because we (re)bind
recursively, and because both systemplate and
lotemplate are mounted for each jails, we need to
make them unbindable, so they wouldn't multiply the
mount-points for each jails (an explosive growth!)
Contrarywise, we don't want the mount-points to
be shared, because we don't expect to add/remove
mounts after a jail is created.
The random temp directory is now created and set
correctly, plus many logging and other improvements.
Change-Id: Iae3fda5e876cf47d2cae6669a87b5b826a8748df
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92829
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Seems to not cause any serious regressions in the iOS app or in "make
run", but of course I am not able to run a comprehensive check of all
functionality.
Change-Id: I44a0e8d60bdbc0a885db88475961575c5e95ce88
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/93037
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
Share it with various other places requiring similar data.
Change-Id: I873f56798f5a34dcf7440456bd649b68f6d3df98
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/94069
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Also adds ServiceRoot handling for clipboard.
Change-Id: I7bc6591130fcc7d693e59ab8561fb9e99f4e93d5
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/93578
Tested-by: Michael Meeks <michael.meeks@collabora.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
It will give an Independence (at least) to the
gradle build system to package the product
Change-Id: I127c2f921b506ec280a244d609707f3480e0f92e
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92719
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Andras Timar <andras.timar@collabora.com>
Very useful with vim plugin YouCompleteMe
with libclang C-family Semantic Completion
Change-Id: I9dfa9b16c5eb11c410d26e21086283c142a57228
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/91860
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Henry Castro <hcastro@collabora.com>
The switch away from LOOLWebSocket and the use of a websocket
for talking to forkit removes the need for the pipe code.
Change-Id: Ifb0c6c88681289e7a1709d9bc3281532935c7be4
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92033
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Sometimes it is very useful to have one "lokit" process,
to focus on a 100% reproducible bug, and not worry
that server pre-spawn several processes.
Change-Id: I414a8145b53a0601a282cba9c245833f5d07f404
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89999
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Henry Castro <hcastro@collabora.com>
fix runing the server with --disable-setcap, very useful
to debug when attaching kit process
Change-Id: I5ded5a1aa1924a9325ae76be46b32c0020d8ce35
WARNING: This is just for development and debugging purposes
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89970
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Henry Castro <hcastro@collabora.com>
The bulk of this commit just changes std::vector<std::string> to
StringVector when we deal with tokens from a websocket message.
The less boring part of it is the new StringVector class, which is a
wrapper around std::vector<std::string>, and provides the same API,
except that operator[] returns a string, not a string&, and this allows
returning an empty string in case that prevents reading past the end of
the underlying array.
This means in case client code forgets to check size() before invoking
operator[], we don't crash. (See the ~3 previous commits which fixed
such crashes.)
Later the ctor could be changed to take a single underlying string to
avoid lots of tiny allocations, that's not yet done in this commit.
Change-Id: I8a6082143a8ac0b65824f574b32104d7889c184f
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89687
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
- target ClientSession::_handleInput(), since crashing there would bring
down the whole loolwsd (not just a kit process), and it deals with
input from untrusted users (browsers)
- add a --enable-fuzzers configure switch to build with
-fsanitize=fuzzer (compared to normal sanitizers build, this is the only
special flag needed)
- configuring other sanitizers is not done automatically, either use
--with-sanitizer=... or the environment variables from LODE's sanitizer
config
- run the actual fuzzer like this:
./clientsession_fuzzer -max_len=16384 fuzzer/data/
- note that at least openSUSE Leap 15.1 sadly ships with a clang with
libfuzzer static libs removed from the package, so you need a
self-built clang to run the fuzzer (either manual build or one from
LODE)
- <https://chromium.googlesource.com/chromium/src/testing/libfuzzer/+/refs/heads/master/efficient_fuzzing.md#execution-speed>
suggests that "You should aim for at least 1,000 exec/s from your fuzz
target locally" (i.e. one run should not take more than 1 ms), so try
this minimal approach first. The alternative would be to start from the
existing loolwsd_fuzzer binary, then step by step cut it down to not
fork(), not do any network traffic, etc -- till it's fast enough that
the fuzzer can find interesting input
- the various configurations start to be really complex (the matrix is
just very large), so try to use Util::isFuzzing() for fuzzer-specific
changes (this is what core.git does as well), and only resort to ifdefs
for the Util::isFuzzing() itself
Change-Id: I72dc1193b34c93eacb5d8e39cef42387d42bd72f
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89226
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
So we can see how stable the cypress tests are. Now other unit
tests under tests folder are failing randomly, so it's hard to
monitor the cypress tests.
Enable one unstable unit test, which was disabled for the same
reason.
Change-Id: Ib1646de1647c9a2bdb769bbb6b92a5794dd7a598
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/88559
Tested-by: Tamás Zolnai <tamas.zolnai@collabora.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
Removing the cypress_test subdir from top level
folder made the packaging process to fail.
So better to use a flag to enable cypress tests.
Change-Id: Iead4b7cbbea5c6aaba18c0b85f23d67a4fbe920b
rm -rf of the assets was newly introduced, done after copying the
templates to their place. So missing templates => crash; fixed.
Change-Id: I53d243b4f44c2c38843336e38e603ae0bd4b3c89
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/86449
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
Tested-by: Jan Holesovsky <kendy@collabora.com>
It was a source of confusion that these bundled headers had to be
synced with core all the time, and when it did not happen, the builds
broke.
Change-Id: Id9d0a8a496b293d20c39e855176a00bcb85df640
Reviewed-on: https://gerrit.libreoffice.org/84358
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Andras Timar <andras.timar@collabora.com>
It was not very useful to let this setting configurable by the user.
On the other hand, old path in config file caused issues after
upgrade. It is better to decide the location of LOKit core
during compilation. From now on the --with-lo-path configure
option is compulsory.
Change-Id: Icdcbc21bde5dad329fdb6e30ed17efde6b0e73de
Reviewed-on: https://gerrit.libreoffice.org/79943
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Andras Timar <andras.timar@collabora.com>
Tool to automatically reload used .css and .js files.
To make it work symlinks are created instead of a copy
if browsersync is enabled.
1. install:
npm install -g browser-sync
2. use configure option: --enable-browsersync
3. remove loleaflet/dist directory
4. run server:
LOOL_SERVE_FROM_FS=1 make run -j10
5. run browsersync:
browser-sync start --config browsersync-config.js
or
make sync-writer
Change-Id: Iebee1cc0b9a03bc866954ff33e3cb8a10b48e0af
Instead of the one that takes a string literal consisting of a single
character.
Also remove the clang-tidy target, 'run-clang-tidy' does a better job of
doing the same today.
Change-Id: I58f55a0e64af42694d55f9588f25618164275852
This is needed so we can use this inside ChildSession.
Change-Id: I88f2cc767412fd52dbb242938f0f9897d4277639
Reviewed-on: https://gerrit.libreoffice.org/63836
Reviewed-by: Tomaž Vajngerl <quikee@gmail.com>
Tested-by: Tomaž Vajngerl <quikee@gmail.com>
The idea is that on a Linux box you have a tree of online that you
configure with --enable-iosapp. Then running 'make' there will only
create the stuff in loleaflet/dist. That loleaflet/dist can then be
copied to the Mac where you build the iOS app.
(To me, this approach seemed for now simpler than to get all the
PKG_CONFIG etc stuff working that running configure normally requires,
and run all the node, npm, and associated crack, on a Mac.)
Change-Id: Id2e495d0521922d0666fdab5fdcb5fcd460136f1
Make prints the sudo setcap commands it executes, no need to
additionally echo "Set required capabilities".
Especially as "set" can be misunderstood as being in the imperative
mood, telling the developer to do something, instead of being past
tense telling what was done on the previous line.
Change-Id: I1138fa28a4d08b57d55552d0e012dcfd778b423c
Possibilities are endless. With a simple /etc/pam.d/loolwsd config below,
the user which runs loolwsd ('lool' in production environment) can login
to admin console with normal linux password.
auth required pam_unix.so
account required pam_unix.so
Change-Id: I354a7e9b4705e8fe346d17d6b6041d1406198b37
Reviewed-on: https://gerrit.libreoffice.org/48307
Reviewed-by: Andras Timar <andras.timar@collabora.com>
Tested-by: Andras Timar <andras.timar@collabora.com>
So that we can use same arguments for both. `etags` binary is also
supplied on some systems by emacs-common package which doesn't
understand ctag like arguments. Best to use one program for all.
Change-Id: I22b57a43f7688c96c9e138e5783ee4157a34a172
To be able to set the support key directly from the command line, and to show
the option, etc.
Change-Id: Iac93bc47a6f4b9d5a5ad0ac8b06bda978e01b760
Reviewed-on: https://gerrit.libreoffice.org/43098
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Michael Meeks <michael.meeks@collabora.com>
Also run the clang static analyzer on net/ by excluding test/, and not
explicitly listing all non-test directories.
Change-Id: Iac203f1cf9161da1c51501de8b0c3cc9dc6e2027
A normal usage to set the admin password would be like :
loolconfig set-admin-password --config-file ./loolwsd.xml
Other command line options can also be given, like --pwd-salt-length,
--pwd-hash-length, --pwd-iterations.
Change-Id: I2a6f8d25e068b53a3f945426f0779c8410b2c8ba
When callgrinding, we really don't want to measure the overhead of the logging
- disable it as much as reasonably possible.
And with 'make run' don't pollute the screen, because the instructions what to
do run away before one manages to read them :-)
Change-Id: I1f72e4a926b3b3c209f31ea257d77234fac255e0
Without this, online doesn't link, atleast for me. The problem seems to
be because in presence of target specific _LDFLAGS, AM_LDFLAGS is
ignored and it is only the latter where we are specifying ZLIB_LIBS.
Further, since the LDFLAGS values are same for all targets, we can
simplify here and rather use AM_LDFLAGS and remove all target specific
LDFLAGS.
Change-Id: I2f1c30c7f34b32c638a4ac60fab709b32f1e6c9a
Reviewed-on: https://gerrit.libreoffice.org/35959
Reviewed-by: pranavk <pranavk@collabora.co.uk>
Tested-by: pranavk <pranavk@collabora.co.uk>
The new test sends data of of 1 byte to
N bytes and expects the exact same data
returned in response.
This tests both buffering sizes and
websocket frames.
Change-Id: Ic6232b4e899d82d90a0ee7c96e4852ffaaf8e958
Reviewed-on: https://gerrit.libreoffice.org/34441
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
To perform one fuzzing iteration, use something like:
./loolwsd_fuzzer --config-file=loolwsd.xml --o:lo_template_path="/opt/libreoffice/instdir" --o:storage.filesystem[@allow]=true --fuzz=/tmp/looltrace
Change-Id: I27210d55a65f75e7d51e05c2f5f999acb758c4b1
These executables are binaries with Linux capabilities, in which case
LD_LIBRARY_PATH is ignored. As a workaround, we can use rpath to support
the alternative Poco library path of a "classic" snap package.
(cherry-picked from commit 74bc911efbf4882f389a49c8e627e7e8057d4a4b)
Change-Id: I23c1d58cce64875abb9629943bfffd7ea633b2bc
Reviewed-on: https://gerrit.libreoffice.org/33705
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
Tested-by: Jan Holesovsky <kendy@collabora.com>
To avoid degrading performance for everyone
because of a single slow/bad connection, we
send data to clients each in its own thread.
Change-Id: I6f980c25a404c4d05bcdb1979849ea3d2776c7b9
Reviewed-on: https://gerrit.libreoffice.org/31984
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Top level make would now build loleaflet too if necessary.
Respect value of ENABLE_DEBUG and if present add debug-info to
generated js bundles too.
Add new rule, clean-local in loleaflet.
Change-Id: I3a9b727824f2877fc72719a8bfdeebed5448cb7d
This adds SenderQueue and a wrapper of messages to
send back to clients.
Currently no threading takes place, but the messages
are pumped through the queue nonetheless.
Change-Id: Id9997539c0a2a351cbf406f649c268dd3643e88e
Reviewed-on: https://gerrit.libreoffice.org/31883
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
We need a fast and good (high avalanche properties)
hash function for the png caching to avoid collissions
(even in the very limited samples we have, since tiles
are likely to have patters, such as all 0's and all 1's
etc.).
Bob Jenkins's public domain SpookyV2 is used here.
It has great avalanache properties and is fast at
~3-bytes / cycle for large messages.
Only trailing whitespace was removed from original
sources and 4 tabs converted to spaces.
Change-Id: Ife57237321625c836d85c894d939fd04a8f577bb
Reviewed-on: https://gerrit.libreoffice.org/31292
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
