Commit graph

147 commits

Author SHA1 Message Date
Ashod Nakashian
71a9d21d3e make: don't clean compile_commands.json
Since this file is only created manually,
it shouldn't be removed automatically.

Change-Id: I8d26b7bfc7f7cd899318b2edd3e5ef9bd462cc99
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/98184
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
2020-07-06 13:47:27 +02:00
Yunusemre Şentürk
187aefb626 Fix lool user control issue on packaging
Change-Id: I1e75279f955a74da5f167dff4648d142af51ab1b
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/98192
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Andras Timar <andras.timar@collabora.com>
2020-07-06 10:41:14 +02:00
Tor Lillqvist
3edc4d2201 Move the setupKitEnvironment() function into a file of its own
A small re-factoring to help planned re-plumbing of the iOS app.

Change-Id: I21f09216a7c5adf965179765a75f5a0d521cd7f3
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/97771
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
2020-07-03 08:40:49 +02:00
Ashod Nakashian
9041f14f00 make: cleanup before removing loolwsd binary
Change-Id: I28dbb165a716646ce5e423e28980ba41ca81d18f
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/97782
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
2020-07-03 02:03:39 +02:00
Ashod Nakashian
a062581be6 make: improve cleanup dependency graph
Change-Id: I8a7edd3b49a272cb7bd8bff4d91b189a5856c5c8
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/97647
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Tested-by: Jenkins
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
2020-07-02 02:32:43 +02:00
Ashod Nakashian
9a427524d1 wsd: support --cleanup in loolwsd
Leaving behind jails with bind-mount
entries makes build-workspace removal
complicated, and jenkins builds start failing.

The cleanup stage is integrated in Makefiles
and should be transparent.

In the event that manual cleanup is necessary,
'loolwsd --cleanup' can be invoked.

Change-Id: Ia4b99b0c66e56dfa2d50e79b0ba98f714cf32886
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/97470
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
2020-07-01 06:14:39 +02:00
Ashod Nakashian
5c9988f2e3 wsd: faster jail setup via bind-mount
loolmount now works and supports mounting and
unmounting, plus numerous improvements,
refactoring, logging, etc..  When enabled,
binding improves the jail setup time by anywhere
from 2x to orders of magnitude (in docker, f.e.).

A new config entry mount_jail_tree controls
whether mounting is used or the old method of
linking/copying of jail contents. It is set to
true by default and falls back to linking/copying.
A test mount is done when the setting is enabled,
and if mounting fails, it's disabled to avoid noise.

Temporarily disabled for unit-tests until we can
cleanup lingering mounts after Jenkins aborts our
build job. In a future patch we will have mount/jail
cleanup as part of make.

The network/system files in /etc that need frequent
refreshing are now updated in systemplate to make
their most recent version available in the jails.
These files can change during the course of loolwsd
lifetime, and are unlikely to be updated in
systemplate after installation at all. We link to
them in the systemplate/etc directory, and if that
fails, we copy them before forking each kit
instance to have the latest.

This reworks the approach used to bind-mount the
jails and the templates such that the total is
now down to only three mounts: systemplate, lo, tmp.

As now systemplate and lotemplate are shared, they
must be mounted as readonly, this means that user/
must now be moved into tmp/user/ which is writable.

The mount-points must be recursive, because we mount
lo/ within the mount-point of systemplate (which is
the root of the jail). But because we (re)bind
recursively, and because both systemplate and
lotemplate are mounted for each jails, we need to
make them unbindable, so they wouldn't multiply the
mount-points for each jails (an explosive growth!)
Contrarywise, we don't want the mount-points to
be shared, because we don't expect to add/remove
mounts after a jail is created.

The random temp directory is now created and set
correctly, plus many logging and other improvements.

Change-Id: Iae3fda5e876cf47d2cae6669a87b5b826a8748df
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92829
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
2020-07-01 05:42:43 +02:00
Tor Lillqvist
7f25109f72 tdf#128502: Chunk of work to enable "multi-tasking" in the iOS app
Seems to not cause any serious regressions in the iOS app or in "make
run", but of course I am not able to run a comprehensive check of all
functionality.

Change-Id: I44a0e8d60bdbc0a885db88475961575c5e95ce88
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/93037
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
2020-06-26 13:09:51 +02:00
Ashod Nakashian
68bdbcf798 wsd: remove unused QueueHandler
Change-Id: Iec80d2afb1762175088b92a6e7140b0ca863a483
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/95339
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Tested-by: Jenkins
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
2020-06-02 17:26:08 +02:00
Michael Meeks
dec683218a Proxy: move RequestDetails to its own header.
Share it with various other places requiring similar data.

Change-Id: I873f56798f5a34dcf7440456bd649b68f6d3df98
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/94069
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-05-12 20:30:17 +02:00
Michael Meeks
18c4301a1f Proxy: re-factor proxy handling into ServerURL and cleanup copy/paste.
Also adds ServiceRoot handling for clipboard.

Change-Id: I7bc6591130fcc7d693e59ab8561fb9e99f4e93d5
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/93578
Tested-by: Michael Meeks <michael.meeks@collabora.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-05-06 23:12:12 +02:00
Michael Meeks
bf0662bda1 Proxy websocket prototype.
Try to read/write avoiding a websocket.

Change-Id: I382039fa88f1030a63df1e47f687df2ee5a6055b
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92805
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
2020-04-24 13:09:09 +02:00
Andras Timar
25bc0a1088 Proof: add loolwsd-generate-proof-key helper script
Change-Id: Ibbd99b6431b1a2992c520d3fad5f52d0770905f6
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92788
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Andras Timar <andras.timar@collabora.com>
2020-04-23 19:36:06 +02:00
Henry Castro
a2410c599c android: convert remaining rules to gradle tasks
It will give an Independence (at least) to the
gradle build system to package the product

Change-Id: I127c2f921b506ec280a244d609707f3480e0f92e
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92719
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Andras Timar <andras.timar@collabora.com>
2020-04-23 09:09:11 +02:00
Henry Castro
5bc712c917 loolwsd: generate compile_commands.js
Very useful with vim plugin YouCompleteMe
with libclang C-family Semantic Completion

Change-Id: I9dfa9b16c5eb11c410d26e21086283c142a57228
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/91860
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Henry Castro <hcastro@collabora.com>
2020-04-17 14:41:32 +02:00
Michael Meeks
a7dc2d1467 kill IoUtil - obsolete & unused.
The switch away from LOOLWebSocket and the use of a websocket
for talking to forkit removes the need for the pipe code.

Change-Id: Ifb0c6c88681289e7a1709d9bc3281532935c7be4
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92033
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-04-10 16:38:25 +02:00
Miklos Vajna
a4e0a00bfe Add an initial libfuzzer based fuzzer for the admin console
Run the actual fuzzer like this:

./admin_fuzzer -max_len=16384 fuzzer/admin-data/

Change-Id: I5891df8033ff1837afce86775ee62447587f2f20
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/91504
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
2020-04-02 12:11:24 +02:00
Andras Timar
370392d71e add missing test/lokassert.hpp to dist tarball
Change-Id: I13bc2926cad6483b1cef7ddd6547d45b8705b185
2020-03-16 12:37:43 +01:00
Henry Castro
e99f0da28d wsd: makefile: allow to set GDB Front Ends
Change-Id: I5b667938b25d8fe63d6c378cb383e357a071e134
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90022
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Henry Castro <hcastro@collabora.com>
2020-03-06 14:18:05 +01:00
Henry Castro
7ba4e9dd23 wsd: debug: run the server with a single "lokit" process
Sometimes it is very useful to have one "lokit" process,
to focus on a 100% reproducible bug, and not worry
that server pre-spawn several processes.

Change-Id: I414a8145b53a0601a282cba9c245833f5d07f404
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89999
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Henry Castro <hcastro@collabora.com>
2020-03-06 13:37:53 +01:00
Henry Castro
8a8eb43375 wsd: makefile: run server with disabled capabilities
fix runing the server with --disable-setcap, very useful
to debug when attaching kit process

Change-Id: I5ded5a1aa1924a9325ae76be46b32c0020d8ce35
WARNING: This is just for development and debugging purposes
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89970
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Henry Castro <hcastro@collabora.com>
2020-03-05 13:07:25 +01:00
Miklos Vajna
b8bd1990aa Rework LOOLProtocol::tokenize() to return a StringVector object
The bulk of this commit just changes std::vector<std::string> to
StringVector when we deal with tokens from a websocket message.

The less boring part of it is the new StringVector class, which is a
wrapper around std::vector<std::string>, and provides the same API,
except that operator[] returns a string, not a string&, and this allows
returning an empty string in case that prevents reading past the end of
the underlying array.

This means in case client code forgets to check size() before invoking
operator[], we don't crash. (See the ~3 previous commits which fixed
such crashes.)

Later the ctor could be changed to take a single underlying string to
avoid lots of tiny allocations, that's not yet done in this commit.

Change-Id: I8a6082143a8ac0b65824f574b32104d7889c184f
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89687
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
2020-02-28 16:07:56 +01:00
Miklos Vajna
57a35bb96c Add an initial libfuzzer based fuzzer
- target ClientSession::_handleInput(), since crashing there would bring
  down the whole loolwsd (not just a kit process), and it deals with
  input from untrusted users (browsers)

- add a --enable-fuzzers configure switch to build with
  -fsanitize=fuzzer (compared to normal sanitizers build, this is the only
  special flag needed)

- configuring other sanitizers is not done automatically, either use
  --with-sanitizer=... or the environment variables from LODE's sanitizer
  config

- run the actual fuzzer like this:

  ./clientsession_fuzzer -max_len=16384 fuzzer/data/

- note that at least openSUSE Leap 15.1 sadly ships with a clang with
  libfuzzer static libs removed from the package, so you need a
  self-built clang to run the fuzzer (either manual build or one from
  LODE)

- <https://chromium.googlesource.com/chromium/src/testing/libfuzzer/+/refs/heads/master/efficient_fuzzing.md#execution-speed>
  suggests that "You should aim for at least 1,000 exec/s from your fuzz
  target locally" (i.e. one run should not take more than 1 ms), so try
  this minimal approach first. The alternative would be to start from the
  existing loolwsd_fuzzer binary, then step by step cut it down to not
  fork(), not do any network traffic, etc -- till it's fast enough that
  the fuzzer can find interesting input

- the various configurations start to be really complex (the matrix is
  just very large), so try to use Util::isFuzzing() for fuzzer-specific
  changes (this is what core.git does as well), and only resort to ifdefs
  for the Util::isFuzzing() itself

Change-Id: I72dc1193b34c93eacb5d8e39cef42387d42bd72f
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89226
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-02-22 12:18:22 +01:00
Tamás Zolnai
6a0d4103f2 Revert "tests: Change the order of cypress-test and other tests."
Now other unit test seems to be more stable.

This reverts commit 4fc7bc76db.
2020-02-19 11:06:51 +01:00
Tamás Zolnai
4fc7bc76db tests: Change the order of cypress-test and other tests.
So we can see how stable the cypress tests are. Now other unit
tests under tests folder are failing randomly, so it's hard to
monitor the cypress tests.

Enable one unstable unit test, which was disabled for the same
reason.

Change-Id: Ib1646de1647c9a2bdb769bbb6b92a5794dd7a598
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/88559
Tested-by: Tamás Zolnai <tamas.zolnai@collabora.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
2020-02-12 22:46:34 +01:00
Tamás Zolnai
0145c03ed6 cypress: Introduce an --enable-cypress config option.
Removing the cypress_test subdir from top level
folder made the packaging process to fail.
So better to use a flag to enable cypress tests.

Change-Id: Iead4b7cbbea5c6aaba18c0b85f23d67a4fbe920b
2020-01-23 14:55:43 +01:00
Tamás Zolnai
423f2eb0db cypress: Don't run cypress test for top level make check for now.
Change-Id: I06c724e82b3717e36a7a4550118baba3aac1a7f6
(cherry picked from commit b48ac5933c4b8671e8ba68c9f9d50216ff4dd620)
2020-01-21 16:13:01 +01:00
Tamás Zolnai
524bbb3398 cypress: Initial integration of cypress test framework.
Change-Id: Ibf300b33d32ce3e7f1affbaf0a59364ddb618b56
2020-01-21 16:13:01 +01:00
Jan Holesovsky
0cfd90d829 android: Fix creating new file from a template.
rm -rf of the assets was newly introduced, done after copying the
templates to their place.  So missing templates => crash; fixed.

Change-Id: I53d243b4f44c2c38843336e38e603ae0bd4b3c89
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/86449
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
Tested-by: Jan Holesovsky <kendy@collabora.com>
2020-01-08 18:29:15 +01:00
Andras Timar
b83610704b remove bundled LOKit headers
It was a source of confusion that these bundled headers had to be
synced with core all the time, and when it did not happen, the builds
broke.

Change-Id: Id9d0a8a496b293d20c39e855176a00bcb85df640
Reviewed-on: https://gerrit.libreoffice.org/84358
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Andras Timar <andras.timar@collabora.com>
2019-12-06 13:40:09 +01:00
Andras Timar
da56f1fc73 deb: trigger systemplate update after apt operations (e.g. installation of a new font, etc.)
Change-Id: Ia356249598d2f451d135985511b0fce71335a28d
2019-11-26 22:56:58 +01:00
Gabriel Masei
2164f5207c Add REST endpoint for admin metrics.
Change-Id: I701485631931334d27594c4907cb770f9888e5bf
Reviewed-on: https://gerrit.libreoffice.org/82492
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Michael Meeks <michael.meeks@collabora.com>
2019-11-25 13:06:01 +01:00
Andras Timar
bbaf010f4f add wsd/ProofKey.hpp to dist tarball
Change-Id: I99a018ad2194f124df9e7a373f9268ba33b0a5e1
2019-11-21 21:39:01 +01:00
Mike Kaganski
a986aabeb1 Initial implementation of proof-key
Change-Id: I7ab79218ca2af268dd4573cb64c6353dc71b5f03
Reviewed-on: https://gerrit.libreoffice.org/82232
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
Tested-by: Mike Kaganski <mike.kaganski@collabora.com>
2019-11-21 12:56:53 +01:00
Andras Timar
03ee39a082 add kit/Watermark.hpp to dist tarball
Change-Id: I40eda4254657f0c92fa9bc5038416d2d3359ba6b
Reviewed-on: https://gerrit.libreoffice.org/81097
Reviewed-by: Andras Timar <andras.timar@collabora.com>
Tested-by: Andras Timar <andras.timar@collabora.com>
2019-10-18 21:52:53 +02:00
Andras Timar
db176ee45b Remove lo_template_path option
It was not very useful to let this setting configurable by the user.
On the other hand, old path in config file caused issues after
upgrade. It is better to decide the location of LOKit core
during compilation. From now on the --with-lo-path configure
option is compulsory.

Change-Id: Icdcbc21bde5dad329fdb6e30ed17efde6b0e73de
Reviewed-on: https://gerrit.libreoffice.org/79943
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Andras Timar <andras.timar@collabora.com>
2019-10-08 09:36:44 +02:00
Andras Timar
99f76c7b2b add init script for RHEL6/CentOS6 2019-09-12 18:33:22 +02:00
Michael Meeks
13469e8b00 clipboard - set it with a post now.
Change-Id: I6b9234988185984335408da502f479ccda8a95a7
2019-08-05 21:18:44 -04:00
Michael Meeks
ba4907b791 clipboard: factor out parsing, and add checks in unit test.
Change-Id: I8c23b10769293b07cc94fdc0d6110e9d540a16b9
2019-08-05 21:18:20 -04:00
Jan Holesovsky
f77c8cd967 android: make clean should clean the assets too.
Change-Id: I1cb2ddb1a7ed71234a5ffc9bdf3631d701df6e5c
2019-06-20 14:14:00 +02:00
Miklos Vajna
8f6d45889d Revert "Allow 'make run' after 'make build-nocheck'"
This reverts commit 1e84ea37c7. This is no
longer needed, at least not for the sanitizers.

Change-Id: I70774525ce322870d0c6d625f33e6fd2476d9b5e
2019-05-29 08:52:27 +02:00
Miklos Vajna
1e84ea37c7 Allow 'make run' after 'make build-nocheck'
This is useful for the sanitizers build, which can build the code, but
not yet the tests.

Change-Id: I02a0493dd18cf68f68572cd08af22d0f7805d97a
2019-05-23 16:30:44 +02:00
Samuel Mehrbrodt
ef862cd054 Fix copy/paste error
Change-Id: I073feea761579ac9f196c430b8a6745811ee31fd
Reviewed-on: https://gerrit.libreoffice.org/70417
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Michael Meeks <michael.meeks@collabora.com>
2019-04-08 15:54:53 +02:00
Jan Holesovsky
14c646067a Revert "configure: add libraries from Boost Locale to the linker"
This reverts commit 21315768c0.
2019-03-15 17:27:40 +01:00
Henry Castro
21315768c0 configure: add libraries from Boost Locale to the linker
Change-Id: Icd56a9d18cd42c0300430a5ba1ef4e866b93ccdb
2019-03-05 17:42:07 -04:00
Szymon Kłos
8a61a53027 Integrate browser-sync
Tool to automatically reload used .css and .js files.
To make it work symlinks are created instead of a copy
if browsersync is enabled.

1. install:
npm install -g browser-sync

2. use configure option: --enable-browsersync

3. remove loleaflet/dist directory

4. run server:
LOOL_SERVE_FROM_FS=1 make run -j10

5. run browsersync:
browser-sync start --config browsersync-config.js

or

make sync-writer

Change-Id: Iebee1cc0b9a03bc866954ff33e3cb8a10b48e0af
2019-02-21 09:23:47 +01:00
Tor Lillqvist
8f71365f0f Remove unnecessary leftovers after Michael's removal of the on-disk tile cache
Change-Id: I435679b48f90d2580bb9c5c86a26c9a1d43c5b59
Reviewed-on: https://gerrit.libreoffice.org/67885
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Michael Meeks <michael.meeks@collabora.com>
2019-02-15 21:36:16 +01:00
Jan Holesovsky
e79f5f7ec6 android: Create the html, js and css and copy it to the right place.
Change-Id: I3efbba0ae369387796e2d08b3003e4f469610dda
2019-02-13 13:22:25 +01:00
Miklos Vajna
d11056deaa test: use the more effective overload accepting a character for find_last_of()
Instead of the one that takes a string literal consisting of a single
character.

Also remove the clang-tidy target, 'run-clang-tidy' does a better job of
doing the same today.

Change-Id: I58f55a0e64af42694d55f9588f25618164275852
2019-01-23 09:06:04 +01:00
Tomaž Vajngerl
73c8fa9d09 Extract Authorization out of Auth and put it into common
This is needed so we can use this inside ChildSession.

Change-Id: I88f2cc767412fd52dbb242938f0f9897d4277639
Reviewed-on: https://gerrit.libreoffice.org/63836
Reviewed-by: Tomaž Vajngerl <quikee@gmail.com>
Tested-by: Tomaž Vajngerl <quikee@gmail.com>
2018-11-29 21:47:13 +01:00