We output the informative name we give ourselves to the thread anyway
which is much more useful.
Change-Id: I74721cc0014fa657adfb6ecea05bcd7f846421a4
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/98477
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
When tests are run in parallel, they will all
compete to update and set up the systemplate
directory, which has a handful of files that
need to be up-to-date. This is a source of errors.
Normally, these files are linked (hard- or soft-
link, whichever succeeds). With linking, we
only need to worry about the initial setup,
as the files will never be out-of-date from
then on. However, when linking fails, we need
to copy the files, and update them (by copying
over fresh versions of the files, if necessary)
every time a new kit is forked. Copying over
is tricky, as it's not atomic. To make it
atomic, we copy the files to the destination
directory under a temporary (random) name,
and then rename to the final name (which is
atomic, including replacing the target file,
if it exists).
No such race exists in production, where there
is (or should be) but one instance of loolwsd
(which does the initial setup) and forkit
(which updates systemplate before forking
new kit instances).
This is an issue with parallel tests only.
Change-Id: I6ba1514d00a84da7397d28efeb6378619711d52f
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/97785
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
The new utility is safer and more readable.
Change-Id: I3a86675378d458cb004e5534dbf2b401936d0e57
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/98183
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
String comparisons are costly and much
less readable. Now we have a fast switch
with LOK_CALLBACK enum values.
Change-Id: Icc24b91b174cd9bbb7e0d64039df080c0a4338f2
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/96375
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
This:
while (*pos++);
leaves pos pointing to the char *after* the first null char. Thus we
lost the timestamp, thread name, and message log level.
The log output ended up loooking like:
Mobile-0x42a805To JS: loolserver { "Version": "master..", "Hash": "1e3b28b0", "Protocol": "0.1", "Id": "e42d1a33" }| CODocument.mm:115
with no space between the thread id and the message, while what we
expect is:
Mobile-0x42be9e 2020-07-02 18:59:30.490298 [ main ] INF To JS: loolserver { "Version": "master..", "Hash": "1e3b28b0", "Protocol": "0.1", "Id": "46ebf726" }| CODocument.mm:115
As such it is not necessary to show "Mobile" (the name of the
process's main program) in the log messages. There is just one
process. Will remove that in a later commit.
Change-Id: I55c4a82f2b34e3b9e70e86cc7af8ea42a3108695
This is to defend the sneaking of extra http-headers
in the access_header URI param that was recently fixed.
Change-Id: Ic28cf58854847ac278bed8043f398b107f7992b3
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/96862
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Authorization class now handles the parsing and creation
of its instances, which makes it centralized.
We also avoid repeatedly constructing Authorization objects
in ClientSession and instead do it once at construction
and cache it.
A bunch of new unit-tests added.
Change-Id: I9b5939be51a5957214d07ed8f1096efd179686c6
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/96825
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
loolmount now works and supports mounting and
unmounting, plus numerous improvements,
refactoring, logging, etc.. When enabled,
binding improves the jail setup time by anywhere
from 2x to orders of magnitude (in docker, f.e.).
A new config entry mount_jail_tree controls
whether mounting is used or the old method of
linking/copying of jail contents. It is set to
true by default and falls back to linking/copying.
A test mount is done when the setting is enabled,
and if mounting fails, it's disabled to avoid noise.
Temporarily disabled for unit-tests until we can
cleanup lingering mounts after Jenkins aborts our
build job. In a future patch we will have mount/jail
cleanup as part of make.
The network/system files in /etc that need frequent
refreshing are now updated in systemplate to make
their most recent version available in the jails.
These files can change during the course of loolwsd
lifetime, and are unlikely to be updated in
systemplate after installation at all. We link to
them in the systemplate/etc directory, and if that
fails, we copy them before forking each kit
instance to have the latest.
This reworks the approach used to bind-mount the
jails and the templates such that the total is
now down to only three mounts: systemplate, lo, tmp.
As now systemplate and lotemplate are shared, they
must be mounted as readonly, this means that user/
must now be moved into tmp/user/ which is writable.
The mount-points must be recursive, because we mount
lo/ within the mount-point of systemplate (which is
the root of the jail). But because we (re)bind
recursively, and because both systemplate and
lotemplate are mounted for each jails, we need to
make them unbindable, so they wouldn't multiply the
mount-points for each jails (an explosive growth!)
Contrarywise, we don't want the mount-points to
be shared, because we don't expect to add/remove
mounts after a jail is created.
The random temp directory is now created and set
correctly, plus many logging and other improvements.
Change-Id: Iae3fda5e876cf47d2cae6669a87b5b826a8748df
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92829
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Seems to not cause any serious regressions in the iOS app or in "make
run", but of course I am not able to run a comprehensive check of all
functionality.
Change-Id: I44a0e8d60bdbc0a885db88475961575c5e95ce88
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/93037
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
The access_header can contain a lot of nonsense, like whitespace around
or additional \n's or \r's. We used to sanitize that, but then
regressed in e95413d151 where the
"tokenize by any of \n\r" was by mistake replaced with "tokenize by
string '\n\r'".
Unfortunately the unit test didn't uncover that, and the further
refactorings of the related code have hidden that even more.
Change-Id: Ie2bf950d0426292770b599e40ee2401101162ff2
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/96638
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Andras Timar <andras.timar@collabora.com>
The json generated by boost::property_tree::write_json() has no
white-spaces around { , : } json-characters. We use write_json()
extensively in core.git. Without this patch we will need work-arounds
like inserting spaces in the json strings thus generated to work with
online's Message::jsonString()
Change-Id: I0f0631088f4a8b727301bde449884e03163093f0
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/96383
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
The tokenizer(s) are more generic than the protocol
logic, and are used from contexts that don't involve
the protocol as such.
Change-Id: Ie8c256bf11a91e466bff794021f41603c9596a7f
When parsing, we virtually always know the name
of the token we are parsing at compile time.
Taking advantage of that means we also know
its size at compile time, and can optimize
std::string allocation, size counting and
the implementation of getTokenInteger.
Change-Id: I502a643c14cace7dd755df565b3b5c445688faad
StringVector is heavily used for tokenization
and benefits from inlining of small functions.
Also, cat doesn't need to be slower than necessary.
Change-Id: I4ab2ff1b1f1a81092049d2cde64b6df10b34b5f7
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/95287
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Jenkins
More readable and typically more efficient.
Change-Id: I9bd5bfc91f4ac255bb8ae0987708fb8b56b398f8
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/95285
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
We now don't need to call snprintf (which is best
for signal-safetly), and are much faster thanks
to a custom replacement.
Change-Id: Iae5861e42e8e335967499f93b71b39b0b4b09bf6
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/94146
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Tested-by: Jenkins
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
This patch allows the lok core to know about the device form facor of
the client requesting the creation of a new view, immediately instead
of a later time.
When a request for a new view is sent a 'deviceFormFactor' parameter
is appended to the message.
This parameter can have one of the following values: 'desktop',
'tablet','mobile' and is forwarded to the lok core.
Change-Id: I21739ddb8c43c960164b3c625e4cf0a80f4616a4
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92691
Tested-by: Marco Cecchetti <marco.cecchetti@collabora.com>
Reviewed-by: Marco Cecchetti <marco.cecchetti@collabora.com>
Sometimes kit process goes into a heavy processing state (or even hangs)
and is not able to report its memory usage. Thus we can't implement cleanup
of problematic kit processes based on memory information reported by kit.
By moving memory reporting to admin module we avoid this problem.
Change-Id: Icf274e3a3a97b33623a93f9d2dc1e640ad9b7d99
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92752
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
But it is insecure, so warn about that.
Change-Id: I151be64f53521e217a5498c0531c9ef2ff8db818
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92822
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
Getting rid of Util::alertAllUsers() calls in arbitrary places is good
because the semi-obscure way in which it works in the multi-process
web-based Online is potentially hard to fit together with the desire
to make the single-process mobile app able to handle several open
documents.
Change-Id: I6055a993ee31941e3592508aac5f0edf6497a836
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92571
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
Which is good, Util::alertAllUsers() is enough of a complicated mess.
Change-Id: Ibac302ea8a7506baa992d71e3891b5764b6ed279
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92569
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
LibreOffice core uses that, too, and we support an even more
restricted set of compilers.
Change-Id: I0d0e2c8608e323eb5ef0f35ee8c46d02ab49a745
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92467
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
Also add the missing newlines before the opening braces, the surrounding
code has that style.
Change-Id: I23bd26ba6d2446858ae3213212e2813a38fd1d46
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92146
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
The switch away from LOOLWebSocket and the use of a websocket
for talking to forkit removes the need for the pipe code.
Change-Id: Ifb0c6c88681289e7a1709d9bc3281532935c7be4
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92033
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
1) Don't actually kill anything with the kill command, otherwise kill(0,
SIGKILL) will kill the fuzzer itself.
2) Don't require a valid signature when authenticating with JWT, since
the private key is generated on each process startup.
3) Log when the JWT would be invalid due to an expired timestamp.
Change-Id: I0da285617e27910329c0e7ed80a6d02e86344ccf
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/91737
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
So that it is easy to find it in the log.
Change-Id: I1b367db9e97d7215c46e824998e99048531acb07
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/91413
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
LOOLWebSocket is used only for tests and a few tools anyway.
Change-Id: I18e04efc2fb3a4cc075f7cefa4ebd0d10fbdb3df
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90974
Tested-by: Michael Meeks <michael.meeks@collabora.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
This sometimes causes unit-integration to fail and now is consistent
with other member functions of Session that handle missing protocol.
Change-Id: I43c7fcae964cfcb5911ff57d63bd4cb569e6b97c
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90732
Tested-by: Michael Meeks <michael.meeks@collabora.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
It took both an std::string and a length. Take a char* and a length
instead.
Change-Id: Id37dfa67fe1baae09b69819680848a0a8a1d80ed
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90552
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
Essentially we want to be able to separate low-level socket code
for eg. TCP vs. UDS, from Protocol handling: eg. WebSocketHandler
and client sessions themselves which handle and send messages
which now implement the simple MessageHandlerInterface.
Some helpful renaming too:
s/SocketHandlerInterface/ProtocolHandlerInterface/
Change-Id: I58092b5e0b5792fda47498fb2c875851eada461d
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90138
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Allows comparing tokens with C strings without a heap allocation. Do the
same when comparing two tokens from two different StringVectors.
And use it at all places where operator ==() has an argument, which is a
StringVector::operator []() result.
Change-Id: Id36eff96767ab99b235ecbd12fb14446a3efa869
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90201
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
WebSocketHandler handles this complexity for us now, and for the
forseeable future. Simplify to prepare for larger re-factor.
Change-Id: I73b919885adc358cb6502a13492cdac85c34459c
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90059
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Debian 8 has gcc-4.9, which emits -Werror in case a parameter and a
member function has the same name. Given that we also use -Werror
unconditionally, this breaks the build. Newer gcc/clang versions relaxed
this warning, so this was not a problem in those cases.
Change-Id: I7ad09370d96aa7384b2c117dd8de421644898b50
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89785
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
This is meant to reduce lots of small allocations and instead have
pointers into the single string for the various tokens instead.
This has a few requirements, though:
1) It's no longer OK to modify the tokens, changing their length would
invalidate the start/length of other tokens. Rework
DocumentBroker::load() to avoid such mutation.
2) The iterators no longer expose zero-terminated strings, so
Poco::cat() doesn't work anymore: add an own cat() instead and use that
in e.g. ChildSession. The own cat() has the benefit that it won't read
past the end of the array if the begin index is out of bounds to add
more safety.
(This nicely works towards killing Poco usage in general.)
3) If zero-terminated strings for all individual tokens is needed, a
copy has to be made, as done in spawnProcess().
(For all of these requirements, the build fails if there are problems.)
Change-Id: Iea40e4400e630b2d669f5c72aea85cb40edf9a2c
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89711
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
The bulk of this commit just changes std::vector<std::string> to
StringVector when we deal with tokens from a websocket message.
The less boring part of it is the new StringVector class, which is a
wrapper around std::vector<std::string>, and provides the same API,
except that operator[] returns a string, not a string&, and this allows
returning an empty string in case that prevents reading past the end of
the underlying array.
This means in case client code forgets to check size() before invoking
operator[], we don't crash. (See the ~3 previous commits which fixed
such crashes.)
Later the ctor could be changed to take a single underlying string to
avoid lots of tiny allocations, that's not yet done in this commit.
Change-Id: I8a6082143a8ac0b65824f574b32104d7889c184f
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89687
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
Termination flag is a very harsh way of exiting.
It works in most cases, but not when we have a
modified document. What happens is the following:
Unit-test flags for termination.
During session cleanup we have to save the modified doc.
Because save is in progress we don't 'disconnect' the view.
This leaves the view in loaded state until saving is done.
But because of the termination flag we don't wait for saving.
DocBroker sends 'exit' to child to forcefully exit.
This causes at least one assertion due to active LOKWindows (Sidebar).
Instead of the above, we flag for graceful shutdown from unittests,
and after we wait to cleanup all DocBrokers, we flag for termination.
This way, we get clean shutdown and all assertions/validations
pass, while we guarantee never to deadlock the unittests,
in case we end up waiting forever for shutdown to complete.
Change-Id: I7fc34137ea373e329795b1ed0090261c085e955a
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89308
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
- target ClientSession::_handleInput(), since crashing there would bring
down the whole loolwsd (not just a kit process), and it deals with
input from untrusted users (browsers)
- add a --enable-fuzzers configure switch to build with
-fsanitize=fuzzer (compared to normal sanitizers build, this is the only
special flag needed)
- configuring other sanitizers is not done automatically, either use
--with-sanitizer=... or the environment variables from LODE's sanitizer
config
- run the actual fuzzer like this:
./clientsession_fuzzer -max_len=16384 fuzzer/data/
- note that at least openSUSE Leap 15.1 sadly ships with a clang with
libfuzzer static libs removed from the package, so you need a
self-built clang to run the fuzzer (either manual build or one from
LODE)
- <https://chromium.googlesource.com/chromium/src/testing/libfuzzer/+/refs/heads/master/efficient_fuzzing.md#execution-speed>
suggests that "You should aim for at least 1,000 exec/s from your fuzz
target locally" (i.e. one run should not take more than 1 ms), so try
this minimal approach first. The alternative would be to start from the
existing loolwsd_fuzzer binary, then step by step cut it down to not
fork(), not do any network traffic, etc -- till it's fast enough that
the fuzzer can find interesting input
- the various configurations start to be really complex (the matrix is
just very large), so try to use Util::isFuzzing() for fuzzer-specific
changes (this is what core.git does as well), and only resort to ifdefs
for the Util::isFuzzing() itself
Change-Id: I72dc1193b34c93eacb5d8e39cef42387d42bd72f
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89226
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
==13901==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x000000904678 bp 0x7ffdb9e21580 sp 0x7ffdb9e21340 T0)
==13901==The signal is caused by a READ memory access.
==13901==Hint: address points to the zero page.
#0 0x904677 in LOOLProtocol::tokenize[abi:cxx11](char const*, unsigned long, char) common/Protocol.hpp:113:40
#1 0x898c52 in LOOLProtocol::tokenize(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, char) common/Protocol.hpp:141:16
#2 0x18dc2d9 in LOOLProtocol::ParseVersion(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) common/Protocol.cpp:35:51
#3 0x1148824 in ClientSession::_handleInput(char const*, int) wsd/ClientSession.cpp:358:64
#4 0x18efcb8 in Session::handleMessage(bool, WSOpCode, std::vector<char, std::allocator<char> >&) common/Session.cpp:232:13
Next commit will add the actual simple fuzzer that found this.
Change-Id: I8623b4451a57390f6f84c11084c5a1120a11fcc5
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89225
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
After fixing this single assertion failure, the
./loolwsd_fuzzer --config-file=loolwsd.xml --o:storage.filesystem[@allow]=true --o:logging.level=fatal
invocation works. (It does not really fuzz anything, but it's a single
unpriviliged process at least.)
Change-Id: I45f877e5eb023e3ddfc96a7373c6300e4bb77962
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89115
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
It can happen that a test succeeds, but right after that, the timeout
thread still fails the test:
09:21:10.411840 [ loolwsd ] INF exitTest: 1. Flagging for termination.| common/Unit.cpp:201
09:21:10.412016 [ loolwsd ] ERR Timed out waiting for unit test to complete| common/Unit.cpp:212
09:21:10.412060 [ loolwsd ] INF exitTest: 2. Flagging for termination.| common/Unit.cpp:201
(1 is TestResult::Ok, 2 is TestResult::TimedOut.)
Fix these cases by just ignoring the timeout invocation, we'll exit soon
anyway.
With this, finally a sanitizers-enabled 'make check' passes agains a
sanitizers-enabled core.git.
Change-Id: Id52c4b6f88a795d4ff2af7929a71d5eb83c56773
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/88999
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
This dates back to 2016 and our pre non-blocking and pre-unipoll state.
It is no longer necessary - a single thread reads all data from the
socket and feeds events into the Kit process; much cleaner.
Change-Id: I46ad6806a1e0cdbb0e5cf4ea5d3e5e5078d3391a
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/88741
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
When trying to open a link normally from help->Online help
nothing happens but the popup is closed.
When trying to open a like forcefully in new tab
from help->online help it crashes the server.
Change-Id: I7e0944ebe521002625a84e155e379ed7e25d2309
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/85466
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
ignoring the segv can lead to not making progress, while churning debug.
Change-Id: I97af266cec3feefe2dcbd9adb8dbf4b13a4d69bd
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/87002
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Use a fully reliable uniqueness check, rather than a hash, and get
simpler ids as a bonus. Fetch view data from the session itself
rather than passing it in too.
Change-Id: Ibcd625156b5a98eb280e35d6537b5c8c026d0197
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/86150
Reviewed-by: Mert Tümer <mert.tumer@collabora.com>
Tested-by: Mert Tümer <mert.tumer@collabora.com>
It is painful to check and search manually the PID to attach the LOKit
process when exists several pre-spawned waiting to load a document.
This patch helps to attach the debugger when the LOKit process is about
to load a document then send the "signal SIGUSR1" to resume it.
Change-Id: I3b15bd522c6ef3ef57dc3453b457dcf91f2661b9
Reviewed-on: https://gerrit.libreoffice.org/85430
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Henry Castro <hcastro@collabora.com>
Reviewed-by: Henry Castro <hcastro@collabora.com>
This is the cleanest way to achieve the goal
of immediately exiting a child. This is used
for cleaning up kit instances when closing
docs, as well as in unit-tests.
Change-Id: I76870234b130a508044044b102419646abe81ac8
Reviewed-on: https://gerrit.libreoffice.org/83699
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
malloc is not signal safe, and must not be called
from signal-safe functions. If malloc itself signals,
calling it in the signal handler can deadlock.
Luckily, we only needed malloc for getting the
backtrace strings. Now we just write directly to
stderr, which is faster, cleaner, and safer.
Change-Id: I54093f45e05f2a0fd3c5cde0cc2104ffe6d81d2a
Reviewed-on: https://gerrit.libreoffice.org/83151
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
There are a few things acceptable/safe in a
signal handler, and taking locks is not one of them.
This replaces the logic with a simple counter that
serves the purpose just as well.
If we get a double signal, we log and ignore.
Change-Id: If589c18492468c120d00c213805467bcbba05d27
Reviewed-on: https://gerrit.libreoffice.org/83150
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
removed use of Poco::StringTokenizer from the common directory
used LOOLProtocol::tokenize and std::vecor<std::string>
regex is used in Authorization.cpp due to limitation of toeknize mathod
regular expression helps to keep the original intention of the code
Change-Id: Ic87597d8b30cb385000f983389a57dc5d2533d98
Reviewed-on: https://gerrit.libreoffice.org/82575
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
Tested-by: Jan Holesovsky <kendy@collabora.com>
Adds possibility to tokenize using a regex easily.
Change-Id: Ie327d4faabec330c76d4cadb1d14bbe1527d332f
Reviewed-on: https://gerrit.libreoffice.org/82333
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
Tested-by: Jan Holesovsky <kendy@collabora.com>
Particularly those used on Android...
Change-Id: I47bf9692f5e99ba30140d698558472201168a198
Reviewed-on: https://gerrit.libreoffice.org/82302
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
Tested-by: Jan Holesovsky <kendy@collabora.com>
This introduces basic C++17 support, because the functionality needed
here is easy to implement using std::filesystem.
Adds also the necessary checks to ./configure. The code still uses POCO
when C++17 is not available in the compiler.
Change-Id: I03353834d10201bf0a13ea72715560b9b9b16265
Reviewed-on: https://gerrit.libreoffice.org/82294
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
Tested-by: Jan Holesovsky <kendy@collabora.com>
The following flags are affected:
ShutdownRequestFlag
TerminationFlag
DumpGlobalState
Since it's common to grep for all places
that set or reset these global flags, it
makes more sense to have explicit functions
for each operation. Now we have set and reset
accessors where appropriate and get is reserved
for read-only access.
This changes the getters to only return
the boolean value of these flags rather than
a reference to the atomic object, now that
they are read-only.
Also, a few Mobile-specific cases were folded
either with other Mobile-specific sections, or
they were now identical to the non-Mobile case
and therefore deduplicated, making the code
cleaner and more readable.
Change-Id: Icc852aa43e86695d4e7d5962040a9b5086d9d08c
Reviewed-on: https://gerrit.libreoffice.org/81978
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Using double caused all sorts of rounding issues,
especially with random unit-test failures.
Luckily, we don't need doubles and can do everything
with integers.
Also added a new function to print time_point as
iso8601 string, for logging and convenience.
Change-Id: I1c2040c02d1143282dbde0dadef32613b77c330d
Reviewed-on: https://gerrit.libreoffice.org/81578
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Sometimes core renderes with sub-pixel differences
(the crosshair at the corners of the Writer pages
show line anti-aliasing differences). This causes
failure of the tests that count the tile deduplication.
We now tolerate when we get an unchanged tile twice,
assuming it was due to such a rendering difference,
but we re-trigger another change and this time we
don't expect any extra tiles, no more than two
variations of the anti-aliased crosshair was
observed.
We also move some duplicate code into utility
functions to improve readability and reuse.
Reviewed-on: https://gerrit.libreoffice.org/81196
Reviewed-by: Andras Timar <andras.timar@collabora.com>
Tested-by: Andras Timar <andras.timar@collabora.com>
(cherry picked from commit 9603597fd1aaecb27893792cfd2d243e450b58b8)
Change-Id: I1a66732dd3443bfbd770d8dc65721571dfa08615
Reviewed-on: https://gerrit.libreoffice.org/81572
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
The initial child spawning takes significantly longer
than subsequent ones (for obvious reasons) and this
lead to unit-tests being sensitive to the timeout
we use for child spawning. Too short, and we
spawn more than we want on startup, too long
and crash-recovery tests fail (we don't
recover fast enough, as we wait too long before
spawning new children).
Dynamically setting the timeout allows us to give
longer timeout at startup, and reduce it afterwards.
Reviewed-on: https://gerrit.libreoffice.org/81194
Reviewed-by: Andras Timar <andras.timar@collabora.com>
Tested-by: Andras Timar <andras.timar@collabora.com>
(cherry picked from commit 32fa1d95fc2ec65866d0cb47d619885182db7040)
Change-Id: I8423f5c6619e57030ab43d519aaa41d8712c36d3
Reviewed-on: https://gerrit.libreoffice.org/81570
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Always log when we set the termination flag
so we can trace how (and implicitly why) we
terminated. In practice trace logging is not
enabled, so a key event such as termination
should be logged at info level.
Reviewed-on: https://gerrit.libreoffice.org/80324
Reviewed-by: Andras Timar <andras.timar@collabora.com>
Tested-by: Andras Timar <andras.timar@collabora.com>
(cherry picked from commit 3f8d516e1399df687c2935fc551b171cbd850b7b)
Change-Id: Id6615181c81ea56777f44b551b39925065b0e578
Reviewed-on: https://gerrit.libreoffice.org/81562
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Michael Meeks <michael.meeks@collabora.com>
We need to give the child more time than we
poll, so we would get a chance to register
prespawned children before we account for
any missing ones. Also, allowing only 5
seconds risks spawning too many children
on a slow, or somewhat loaded system.
This raises the child timeout to twice
the poll timeout, which is 5 seconds,
allowing for 10 seconds for spawning.
Change-Id: I4cc9e2fc656268fa82a935a5cb1626540bd49980
Reviewed-on: https://gerrit.libreoffice.org/80322
Reviewed-by: Andras Timar <andras.timar@collabora.com>
Tested-by: Andras Timar <andras.timar@collabora.com>
(cherry picked from commit afb96fcbad4e36ef4eb07bf6d0b67329b714858e)
Reviewed-on: https://gerrit.libreoffice.org/81560
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Michael Meeks <michael.meeks@collabora.com>
With password-protected files, the first loading attempt
always fails due to missing password. At that point the
client is notified of the missing password and the user
is prompted. The second attempt includes a (hopefully)
correct password and the document loading commences.
Due to the fact that an exception is raised when
the loading fails, this left the loading latch
triggered, which blocked subsequent attempts.
Change-Id: I7cc257a36eb1cc080f460aac8cdb7030783a5914
Util added getHttpTime
WhiteBoxTests added test for getHttpTime
Change-Id: Ifb6a3fb2dc9b059b925e7b881362b72759a8b56b
Reviewed-on: https://gerrit.libreoffice.org/79754
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Michael Meeks <michael.meeks@collabora.com>
As the comment details, this avoids having C++ objects
in the same frame as setjmp, which may reset their
contents without the dtor getting called.
Change-Id: I851ae8bffb4356d465a25dfc815a1fecb489fa30
Reviewed-on: https://gerrit.libreoffice.org/79338
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
The thread name helps one track threads in logs.
When renaming threads it's important to log the process
and previous thread name (if any), so grepping is more
fruitful and tracking is easier.
Change-Id: I47a948d77629b387cc1e9fd58fdd88e1ae1168df
Reviewed-on: https://gerrit.libreoffice.org/79327
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Avoids a warning when compiling for iOS: format specifies type
'unsigned long' but the argument has type 'uint64_t' (aka 'unsigned
long long').
Change-Id: I8b5205dd0c3a8ae2f531f1647b3e3bac27ea6065
Reviewed-on: https://gerrit.libreoffice.org/78985
Reviewed-by: Tor Lillqvist <tml@collabora.com>
Tested-by: Tor Lillqvist <tml@collabora.com>
We never exit an app process intentionally, so Log::isShutdownCalled()
can always return false.
Change-Id: I6b3cce85ccac5e3ffbb7b29ca552cb9e4441df2a
Reviewed-on: https://gerrit.libreoffice.org/78979
Reviewed-by: Tor Lillqvist <tml@collabora.com>
Tested-by: Tor Lillqvist <tml@collabora.com>
net/Socket.hpp:405:9: runtime error: member call on address 0x6070007a2210 which does not point to an object of type 'Poco::Logger'
0x6070007a2210: note: object has invalid vptr
05 00 80 0e 7c 01 80 6f 3c 7f 00 00 c8 b8 0f 01 3c 7f 00 00 00 00 00 00 be be be be 38 22 7a 00
^~~~~~~~~~~~~~~~~~~~~~~
invalid vptr
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior net/Socket.hpp:405:9 in
Which is use-after-free: once Poco::Logger::shutdown() is called, we
need to tolerate LOG_DBG() and other similar calls, even if they don't
go anywhere.
Change-Id: Ic76433743177dd2b604ff34e340309c506d83350
It is std::chrono::system_clock that has to_time_t.
std::chrono::steady_clock does not have to_time_t.
std::chrono::high_resolution_clock is either the same as system_clock
(in libstdc++, on Linux) or steady_clock (libc++, on iOS).
(This change does not fix the actual bugs in the code, just makes it
compile for iOS. The new ISO8601 fractional time code is not unit
tested at the moment. The testTime() function is not part of the test
suite in WhiteBoxTests.cpp. If it is made part of it, it reveals
problems in the code (and/or in the unit test code).)
Change-Id: Id33342bc8b26465632f3d21d6ec2f3c975ae3681
Reviewed-on: https://gerrit.libreoffice.org/78550
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Michael Meeks <michael.meeks@collabora.com>
Added functions to get file timestamp and to convert
chrono timestamp in ISO8601 fraction format and some
test cases.
Change-Id: I58961a31f7262b367cff9f33cffdec7571a2f8f7
Better hashing algorithm based on FNV-1a.
Adds support for salting the hash, and
for providing salt via configuration.
More unit-tests added, and better formatting.
Change-Id: I2be42675d0cdbaa73c3d7faed99e07631a9c20fc
Reviewed-on: https://gerrit.libreoffice.org/70034
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Reviewed-on: https://gerrit.libreoffice.org/71091
unit-copy-paste with ubsan failed with:
common/Util.hpp:751:43: runtime error: null pointer passed as argument 2, which is declared to never be null
/usr/include/string.h:43:28: note: nonnull attribute specified here
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior common/Util.hpp:751:43 in
Change-Id: I71ae9b43a63f979c300d704419afc9a14cd303cd
Otherwise both loolwsd and unit-copy-paste.so would have a
SigHandlerTrap:
==26186==ERROR: AddressSanitizer: odr-violation (0x000002090ae0):
[1] size=40 'SigHandlerTrap' ../common/SigUtil.cpp:76:12
[2] size=40 'SigHandlerTrap' common/SigUtil.cpp:76:12
These globals were registered at these points:
[1]:
#0 0x5f9a28 in __asan_register_globals.part.13 /home/vmiklos/git/libreoffice/lode/packages/llvm-472c6ef8b0f53061b049039f9775ab127beafbe4.src/compiler-rt/lib/asan/asan_globals.cc:365
#1 0x7f8f537f1d8b in asan.module_ctor (/home/vmiklos/git/libreoffice/online-san/test/../test/.libs/unit-copy-paste.so+0x60ad8b)
[2]:
#0 0x5f9a28 in __asan_register_globals.part.13 /home/vmiklos/git/libreoffice/lode/packages/llvm-472c6ef8b0f53061b049039f9775ab127beafbe4.src/compiler-rt/lib/asan/asan_globals.cc:365
#1 0xe2bcfe in asan.module_ctor (/home/vmiklos/git/libreoffice/online-san/loolwsd+0xe2bcfe)
Change-Id: I54d5f6d4298848cacd437d302cff0e8c5003fb8c
Otherwise both loolwsd and unit-copy-paste.so would have a
ShutdownRequestFlag:
==13663==ERROR: AddressSanitizer: odr-violation (0x00000208f860):
[1] size=1 'ShutdownRequestFlag' ../common/SigUtil.cpp:60:19
[2] size=1 'ShutdownRequestFlag' common/SigUtil.cpp:60:19
These globals were registered at these points:
[1]:
#0 0x5f9a18 in __asan_register_globals.part.13 /home/vmiklos/git/libreoffice/lode/packages/llvm-472c6ef8b0f53061b049039f9775ab127beafbe4.src/compiler-rt/lib/asan/asan_globals.cc:365
#1 0x7f9b903f1d0b in asan.module_ctor (/home/vmiklos/git/libreoffice/online-san/test/../test/.libs/unit-copy-paste.so+0x60ad0b)
[2]:
#0 0x5f9a18 in __asan_register_globals.part.13 /home/vmiklos/git/libreoffice/lode/packages/llvm-472c6ef8b0f53061b049039f9775ab127beafbe4.src/compiler-rt/lib/asan/asan_globals.cc:365
#1 0xe2b9fe in asan.module_ctor (/home/vmiklos/git/libreoffice/online-san/loolwsd+0xe2b9fe)
Change-Id: I247760325f804813249e814dbb4576493619dee7
Otherwise both loolwsd and unit-copy-paste.so would have a
DumpGlobalState:
==5783==ERROR: AddressSanitizer: odr-violation (0x00000208f7a0):
[1] size=1 'DumpGlobalState' ../common/SigUtil.cpp:49:19
[2] size=1 'DumpGlobalState' common/SigUtil.cpp:49:19
These globals were registered at these points:
[1]:
#0 0x5f9a08 in __asan_register_globals.part.13 /home/vmiklos/git/libreoffice/lode/packages/llvm-472c6ef8b0f53061b049039f9775ab127beafbe4.src/compiler-rt/lib/asan/asan_globals.cc:365
#1 0x7f5c5edf1c9b in asan.module_ctor (/home/vmiklos/git/libreoffice/online-san/test/../test/.libs/unit-copy-paste.so+0x60ac9b)
[2]:
#0 0x5f9a08 in __asan_register_globals.part.13 /home/vmiklos/git/libreoffice/lode/packages/llvm-472c6ef8b0f53061b049039f9775ab127beafbe4.src/compiler-rt/lib/asan/asan_globals.cc:365
#1 0xe2b98e in asan.module_ctor (/home/vmiklos/git/libreoffice/online-san/loolwsd+0xe2b98e)
Change-Id: I4b7b0238eb9b38a30875e8788c1dcb27f1d1643f
Otherwise both loolwsd and unit-copy-paste.so would have a
TerminationFlag:
==11732==ERROR: AddressSanitizer: odr-violation (0x00000208f4a0):
[1] size=1 'TerminationFlag' ../common/SigUtil.cpp:41:19
[2] size=1 'TerminationFlag' common/SigUtil.cpp:41:19
These globals were registered at these points:
[1]:
#0 0x5f9988 in __asan_register_globals.part.13 /home/vmiklos/git/libreoffice/lode/packages/llvm-472c6ef8b0f53061b049039f9775ab127beafbe4.src/compiler-rt/lib/asan/asan_globals.cc:365
#1 0x7f5df9cf18cb in asan.module_ctor (/home/vmiklos/git/libreoffice/online-san/test/../test/.libs/unit-copy-paste.so+0x60a8cb)
[2]:
#0 0x5f9988 in __asan_register_globals.part.13 /home/vmiklos/git/libreoffice/lode/packages/llvm-472c6ef8b0f53061b049039f9775ab127beafbe4.src/compiler-rt/lib/asan/asan_globals.cc:365
#1 0xe2b4fe in asan.module_ctor (/home/vmiklos/git/libreoffice/online-san/loolwsd+0xe2b4fe)
Change-Id: Ic620b143ecb77699f40676ff39d0fa7abceb34d5
re-factor ClientSession state to be a simpler state machine.
Have a nice disconnect / disconnected handshake on view close.
Change-Id: Ie933cc5c7dfab46c66f4d38a4d75c459aa1cff87
