The next version of Collabora Online will be version 2021.
Development of version 2021 will be carried on master branch
for the time being.
The corresponding core branch is distro/collabora/co-2021
Signed-off-by: Andras Timar <andras.timar@collabora.com>
Change-Id: I9c97aaac711c9e1f1e48ed25066b169ea7e26e84
A broken loolwsd may be lurking around, which will
then get used during the next `make` invocation to
cleanup the jails. If it fails, it will break the
build altogether. This is not ideal. In this case,
we delete the loolwsd binary so that we force
building it anew. And in any case it was useless
for make, if it failed to do the only thing that
make needs it for: cleaning up the jails.
The new loolwsd will subsequently get used to
clean any left over jails before running the
tests, so that should be fine.
Change-Id: I76c16b5fc7c6f08308c9fb2e619228f8e0266b74
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
This implements HTTP/1.1 per RFC 7230, partially.
Unit-tests are provided with documentation on usage.
This is desgined to serve as the http implementation
throughout loolwsd, for both synchronous and
asynchronous requests.
Change-Id: Iaf1b8c5fcb8cec032445e27c9f70d2fb807aa4dc
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
This reverts commit f6bf6f49ed. I did not
consider that this makes it harder to do a local developer setup with
reverse proxy + make run; also that if you don't read the doc to set up
a reverse proxy, then the default now serves http content and advertises
https, which is not helping sysadmins.
This needs more thinking how to improve one scenario without hurting
others.
Signed-off-by: Miklos Vajna <vmiklos@collabora.com>
Change-Id: Ic789faa2dc5bb19a79d651dc81d4eaaf0b48607a
Move the connect function into the NetUtil
translation unit to aid using it for the
upcoming async socket logic.
The NetUtil should also come in handy for
the miscellaneous network helpers we have.
Change-Id: I2ee0c6e3e1769fd87572d7407d3b4979b59ffe6a
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
The intention is to have defaults which are close to how people
typically use Online in production.
However, keep using ssl for 'make run', so that the https environment in
the browser is unchanged.
Signed-off-by: Miklos Vajna <vmiklos@collabora.com>
Change-Id: I7fd725a83b0e9ca1012f2c0e0c3bf038e5fa0059
In some cases we cannot do a fast bind-mount of the files we want
in our jail since we don't have cap_sys_admin for loolmount inside
eg. docker.
Thus we need to fallback to hard-linking, however various security
systems namespace parts of our tree, such that link() fails with
EXDEV even across the (apparently) same file-system.
As such we need to assemble a copy of what we want to hard-link
close to our jails. However, this needs to be owned by root / the
system to avoid having writable files shared between jails. Hence
we need cap_chown in addition to cap_fowner, to get ownership right
and then hard-link.
Change-Id: Iba0ef46ddbc1c03f3dc7177bc1ec1755624135db
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
This moves the test log macros into a new home,
test/testlog.hpp, to avoid cycling dependencies.
Change-Id: Iacb80e813a64ff830fa18f63ec4de2535ee702b7
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
For large transfers eg. image previews, particularly with SSL's
protocol limit of 16k byte blocks, we see lots of inefficiency
repeatedly copying a 20Mb image and shuffling it down a
std::vector as we write data out.
Change-Id: I620568cad2e6f41684c35289b0ee77cf7f59c077
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
This adds the infrastructure to be able to pass the info which elements
like the statusbar / ruler / sidebar are supposed to be shown or hidden
on startup of the editor.
Change-Id: I188264dec6961074444934ff5fd7088e23b170d4
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/103169
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Mert Tumer <mert.tumer@collabora.com>
We now gracefully fallback to copying when/if systemplate
is readonly.
The bulk of the change is to support proper cleanup in
both cases.
First, we had to move as much of the jail bootstrapping
into the loolwsd-systemplate-setup script, so systemplate
will be as complete as possible before it is locked down.
Next, we needed to update the jail with graceful fallback
to linking/copying upon failure. For that, the jail setup
logic in Kit.cpp has been reworked to support not just
update failures, but also more comprehensive mounting
failures as well.
Finally, jail cleanup now is seamless. To support proper
cleanup when we had mounting enabled but had to fallback,
we mark jails that aren't mounted so we can 'rm -rf' the
contents safely and without fear or causing undue damage
(as unlikely as that is, technically we wouldn't want to
rm systemplate files, if mounting read-only had failed).
There are a few minor refactorings of JailUtil to make
it cleaner and more robust.
Change-Id: Iac34869cb84f45acf64fbbc46d46898367b496d2
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/101260
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Andras Timar <andras.timar@collabora.com>
libfuzzer only produces fuzzer binaries, not a loolwsd binary, so don't
expect it in the libfuzzer case.
Change-Id: Ib818667031665aa60a447ddd5edd3a09bca76e18
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/98910
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
Since this file is only created manually,
it shouldn't be removed automatically.
Change-Id: I8d26b7bfc7f7cd899318b2edd3e5ef9bd462cc99
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/98184
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
A small re-factoring to help planned re-plumbing of the iOS app.
Change-Id: I21f09216a7c5adf965179765a75f5a0d521cd7f3
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/97771
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
Leaving behind jails with bind-mount
entries makes build-workspace removal
complicated, and jenkins builds start failing.
The cleanup stage is integrated in Makefiles
and should be transparent.
In the event that manual cleanup is necessary,
'loolwsd --cleanup' can be invoked.
Change-Id: Ia4b99b0c66e56dfa2d50e79b0ba98f714cf32886
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/97470
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
loolmount now works and supports mounting and
unmounting, plus numerous improvements,
refactoring, logging, etc.. When enabled,
binding improves the jail setup time by anywhere
from 2x to orders of magnitude (in docker, f.e.).
A new config entry mount_jail_tree controls
whether mounting is used or the old method of
linking/copying of jail contents. It is set to
true by default and falls back to linking/copying.
A test mount is done when the setting is enabled,
and if mounting fails, it's disabled to avoid noise.
Temporarily disabled for unit-tests until we can
cleanup lingering mounts after Jenkins aborts our
build job. In a future patch we will have mount/jail
cleanup as part of make.
The network/system files in /etc that need frequent
refreshing are now updated in systemplate to make
their most recent version available in the jails.
These files can change during the course of loolwsd
lifetime, and are unlikely to be updated in
systemplate after installation at all. We link to
them in the systemplate/etc directory, and if that
fails, we copy them before forking each kit
instance to have the latest.
This reworks the approach used to bind-mount the
jails and the templates such that the total is
now down to only three mounts: systemplate, lo, tmp.
As now systemplate and lotemplate are shared, they
must be mounted as readonly, this means that user/
must now be moved into tmp/user/ which is writable.
The mount-points must be recursive, because we mount
lo/ within the mount-point of systemplate (which is
the root of the jail). But because we (re)bind
recursively, and because both systemplate and
lotemplate are mounted for each jails, we need to
make them unbindable, so they wouldn't multiply the
mount-points for each jails (an explosive growth!)
Contrarywise, we don't want the mount-points to
be shared, because we don't expect to add/remove
mounts after a jail is created.
The random temp directory is now created and set
correctly, plus many logging and other improvements.
Change-Id: Iae3fda5e876cf47d2cae6669a87b5b826a8748df
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92829
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Seems to not cause any serious regressions in the iOS app or in "make
run", but of course I am not able to run a comprehensive check of all
functionality.
Change-Id: I44a0e8d60bdbc0a885db88475961575c5e95ce88
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/93037
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
Share it with various other places requiring similar data.
Change-Id: I873f56798f5a34dcf7440456bd649b68f6d3df98
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/94069
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Also adds ServiceRoot handling for clipboard.
Change-Id: I7bc6591130fcc7d693e59ab8561fb9e99f4e93d5
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/93578
Tested-by: Michael Meeks <michael.meeks@collabora.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
It will give an Independence (at least) to the
gradle build system to package the product
Change-Id: I127c2f921b506ec280a244d609707f3480e0f92e
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92719
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Andras Timar <andras.timar@collabora.com>
Very useful with vim plugin YouCompleteMe
with libclang C-family Semantic Completion
Change-Id: I9dfa9b16c5eb11c410d26e21086283c142a57228
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/91860
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Henry Castro <hcastro@collabora.com>
The switch away from LOOLWebSocket and the use of a websocket
for talking to forkit removes the need for the pipe code.
Change-Id: Ifb0c6c88681289e7a1709d9bc3281532935c7be4
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92033
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Sometimes it is very useful to have one "lokit" process,
to focus on a 100% reproducible bug, and not worry
that server pre-spawn several processes.
Change-Id: I414a8145b53a0601a282cba9c245833f5d07f404
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89999
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Henry Castro <hcastro@collabora.com>
fix runing the server with --disable-setcap, very useful
to debug when attaching kit process
Change-Id: I5ded5a1aa1924a9325ae76be46b32c0020d8ce35
WARNING: This is just for development and debugging purposes
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89970
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Henry Castro <hcastro@collabora.com>
The bulk of this commit just changes std::vector<std::string> to
StringVector when we deal with tokens from a websocket message.
The less boring part of it is the new StringVector class, which is a
wrapper around std::vector<std::string>, and provides the same API,
except that operator[] returns a string, not a string&, and this allows
returning an empty string in case that prevents reading past the end of
the underlying array.
This means in case client code forgets to check size() before invoking
operator[], we don't crash. (See the ~3 previous commits which fixed
such crashes.)
Later the ctor could be changed to take a single underlying string to
avoid lots of tiny allocations, that's not yet done in this commit.
Change-Id: I8a6082143a8ac0b65824f574b32104d7889c184f
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89687
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
- target ClientSession::_handleInput(), since crashing there would bring
down the whole loolwsd (not just a kit process), and it deals with
input from untrusted users (browsers)
- add a --enable-fuzzers configure switch to build with
-fsanitize=fuzzer (compared to normal sanitizers build, this is the only
special flag needed)
- configuring other sanitizers is not done automatically, either use
--with-sanitizer=... or the environment variables from LODE's sanitizer
config
- run the actual fuzzer like this:
./clientsession_fuzzer -max_len=16384 fuzzer/data/
- note that at least openSUSE Leap 15.1 sadly ships with a clang with
libfuzzer static libs removed from the package, so you need a
self-built clang to run the fuzzer (either manual build or one from
LODE)
- <https://chromium.googlesource.com/chromium/src/testing/libfuzzer/+/refs/heads/master/efficient_fuzzing.md#execution-speed>
suggests that "You should aim for at least 1,000 exec/s from your fuzz
target locally" (i.e. one run should not take more than 1 ms), so try
this minimal approach first. The alternative would be to start from the
existing loolwsd_fuzzer binary, then step by step cut it down to not
fork(), not do any network traffic, etc -- till it's fast enough that
the fuzzer can find interesting input
- the various configurations start to be really complex (the matrix is
just very large), so try to use Util::isFuzzing() for fuzzer-specific
changes (this is what core.git does as well), and only resort to ifdefs
for the Util::isFuzzing() itself
Change-Id: I72dc1193b34c93eacb5d8e39cef42387d42bd72f
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89226
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
So we can see how stable the cypress tests are. Now other unit
tests under tests folder are failing randomly, so it's hard to
monitor the cypress tests.
Enable one unstable unit test, which was disabled for the same
reason.
Change-Id: Ib1646de1647c9a2bdb769bbb6b92a5794dd7a598
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/88559
Tested-by: Tamás Zolnai <tamas.zolnai@collabora.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
Removing the cypress_test subdir from top level
folder made the packaging process to fail.
So better to use a flag to enable cypress tests.
Change-Id: Iead4b7cbbea5c6aaba18c0b85f23d67a4fbe920b
rm -rf of the assets was newly introduced, done after copying the
templates to their place. So missing templates => crash; fixed.
Change-Id: I53d243b4f44c2c38843336e38e603ae0bd4b3c89
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/86449
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
Tested-by: Jan Holesovsky <kendy@collabora.com>
It was a source of confusion that these bundled headers had to be
synced with core all the time, and when it did not happen, the builds
broke.
Change-Id: Id9d0a8a496b293d20c39e855176a00bcb85df640
Reviewed-on: https://gerrit.libreoffice.org/84358
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Andras Timar <andras.timar@collabora.com>
