This is meant to reduce lots of small allocations and instead have
pointers into the single string for the various tokens instead.
This has a few requirements, though:
1) It's no longer OK to modify the tokens, changing their length would
invalidate the start/length of other tokens. Rework
DocumentBroker::load() to avoid such mutation.
2) The iterators no longer expose zero-terminated strings, so
Poco::cat() doesn't work anymore: add an own cat() instead and use that
in e.g. ChildSession. The own cat() has the benefit that it won't read
past the end of the array if the begin index is out of bounds to add
more safety.
(This nicely works towards killing Poco usage in general.)
3) If zero-terminated strings for all individual tokens is needed, a
copy has to be made, as done in spawnProcess().
(For all of these requirements, the build fails if there are problems.)
Change-Id: Iea40e4400e630b2d669f5c72aea85cb40edf9a2c
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89711
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
The bulk of this commit just changes std::vector<std::string> to
StringVector when we deal with tokens from a websocket message.
The less boring part of it is the new StringVector class, which is a
wrapper around std::vector<std::string>, and provides the same API,
except that operator[] returns a string, not a string&, and this allows
returning an empty string in case that prevents reading past the end of
the underlying array.
This means in case client code forgets to check size() before invoking
operator[], we don't crash. (See the ~3 previous commits which fixed
such crashes.)
Later the ctor could be changed to take a single underlying string to
avoid lots of tiny allocations, that's not yet done in this commit.
Change-Id: I8a6082143a8ac0b65824f574b32104d7889c184f
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89687
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
- target ClientSession::_handleInput(), since crashing there would bring
down the whole loolwsd (not just a kit process), and it deals with
input from untrusted users (browsers)
- add a --enable-fuzzers configure switch to build with
-fsanitize=fuzzer (compared to normal sanitizers build, this is the only
special flag needed)
- configuring other sanitizers is not done automatically, either use
--with-sanitizer=... or the environment variables from LODE's sanitizer
config
- run the actual fuzzer like this:
./clientsession_fuzzer -max_len=16384 fuzzer/data/
- note that at least openSUSE Leap 15.1 sadly ships with a clang with
libfuzzer static libs removed from the package, so you need a
self-built clang to run the fuzzer (either manual build or one from
LODE)
- <https://chromium.googlesource.com/chromium/src/testing/libfuzzer/+/refs/heads/master/efficient_fuzzing.md#execution-speed>
suggests that "You should aim for at least 1,000 exec/s from your fuzz
target locally" (i.e. one run should not take more than 1 ms), so try
this minimal approach first. The alternative would be to start from the
existing loolwsd_fuzzer binary, then step by step cut it down to not
fork(), not do any network traffic, etc -- till it's fast enough that
the fuzzer can find interesting input
- the various configurations start to be really complex (the matrix is
just very large), so try to use Util::isFuzzing() for fuzzer-specific
changes (this is what core.git does as well), and only resort to ifdefs
for the Util::isFuzzing() itself
Change-Id: I72dc1193b34c93eacb5d8e39cef42387d42bd72f
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89226
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Using double caused all sorts of rounding issues,
especially with random unit-test failures.
Luckily, we don't need doubles and can do everything
with integers.
Also added a new function to print time_point as
iso8601 string, for logging and convenience.
Change-Id: I1c2040c02d1143282dbde0dadef32613b77c330d
Reviewed-on: https://gerrit.libreoffice.org/81578
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Util added getHttpTime
WhiteBoxTests added test for getHttpTime
Change-Id: Ifb6a3fb2dc9b059b925e7b881362b72759a8b56b
Reviewed-on: https://gerrit.libreoffice.org/79754
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Michael Meeks <michael.meeks@collabora.com>
The thread name helps one track threads in logs.
When renaming threads it's important to log the process
and previous thread name (if any), so grepping is more
fruitful and tracking is easier.
Change-Id: I47a948d77629b387cc1e9fd58fdd88e1ae1168df
Reviewed-on: https://gerrit.libreoffice.org/79327
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
It is std::chrono::system_clock that has to_time_t.
std::chrono::steady_clock does not have to_time_t.
std::chrono::high_resolution_clock is either the same as system_clock
(in libstdc++, on Linux) or steady_clock (libc++, on iOS).
(This change does not fix the actual bugs in the code, just makes it
compile for iOS. The new ISO8601 fractional time code is not unit
tested at the moment. The testTime() function is not part of the test
suite in WhiteBoxTests.cpp. If it is made part of it, it reveals
problems in the code (and/or in the unit test code).)
Change-Id: Id33342bc8b26465632f3d21d6ec2f3c975ae3681
Reviewed-on: https://gerrit.libreoffice.org/78550
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Michael Meeks <michael.meeks@collabora.com>
Added functions to get file timestamp and to convert
chrono timestamp in ISO8601 fraction format and some
test cases.
Change-Id: I58961a31f7262b367cff9f33cffdec7571a2f8f7
Better hashing algorithm based on FNV-1a.
Adds support for salting the hash, and
for providing salt via configuration.
More unit-tests added, and better formatting.
Change-Id: I2be42675d0cdbaa73c3d7faed99e07631a9c20fc
Reviewed-on: https://gerrit.libreoffice.org/70034
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Reviewed-on: https://gerrit.libreoffice.org/71091
Added a function to Util to get current time in HTTP
format using std::chrono.
Change-Id: I9e7a732f585c1758c9348c450a01713a66f1e7b7
Reviewed-on: https://gerrit.libreoffice.org/72585
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Michael Meeks <michael.meeks@collabora.com>
The former is the standard C++ approach
and is reportedly faster than __thread
(at least with gcc).
Change-Id: Ibdefd32172774a280637f73dd062282b7bf62025
Reviewed-on: https://gerrit.libreoffice.org/71019
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Also fix unexpected concatenation error in Poco::URI::encode generating
eg. authorid=localhost0 xauthorid=localhost0localhost0 in the output.
Change-Id: I560e47e31884eeb1c662f468436ed7541cfb082d
Also support anonymization of downloadas documents
and renaming of documents.
Reviewed-on: https://gerrit.libreoffice.org/57541
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
Tested-by: Jan Holesovsky <kendy@collabora.com>
(cherry picked from commit 78248a542c9ca31bf9ad4cad9b55d78690384395)
Change-Id: I81a80e6290217659987d73f625e5f0fb81cb7ef2
This is important for when we abort with some explanation.
Often said explanation doesn't show up anywhere to be useful.
Also, issue fatal logs for abnormal exist and use SFL to log errno.
Reviewed-on: https://gerrit.libreoffice.org/57540
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
Tested-by: Jan Holesovsky <kendy@collabora.com>
(cherry picked from commit ad7964393eadb68873b820e0a620fb40f1e1b06a)
Change-Id: Ic67064ef40ef6e93d26e5847ecd32bdd49c3cc8b
The async-signal-safe functions to get thread-id
and thread-name, which cache the results, are
faster, cleaner, and signal-safe. No reason why
we shouldn't always use them.
Especially since it appears the logic was
inverted in Log::prefix, such that the signal
un-safe calls were made during signal-handling,
and the safe ones were called otherwise!
Instead of passing the signal-safe flag to
Log::prefix, we pass the buffer size, for
improved security.
Furthermore, reduce header dependencies
and reduce clutter.
Change-Id: I697689b2f0a290b6d8cce4babc3ac1e576141da6
The idea is that it would work sufficiently identically, so that even
people without a Mac and without an iOS device could participate in
development of the non-iOS-specific bits, like the JavaScript, or the
online MOBILEAPP-specific plumbing. Which would be great.
No, this doesn't do anything sane yet. It does compile the same online
C++ files as the iOS app, though. (Some minor tweaks were needed in a
couple of them to silence gcc warnings.)
There is a plain Makefile, but I should change to using autofoo, too.
Eventually, this will need to be built in a separate tree from a
normal online, just like when using the --enable-iosapp configure
switch. (But for now, doesn't matter.)
Change-Id: I13e4d921acb99d802d2f9da4b0df4a237ca60ad6
The app is unimaginatively called "Mobile" for now.
Runs but crashes pretty quickly after loading the document by the LO
core. Will need some heavy changes to get a ClientSession object
created in there, too, to handle the (emulated) WebSocket messages
from the JavaScript. It would then handle some of these messages
itself, and forwards some to the ChildSession, which in this case is
in the same process. Now the messsages from the JavaScript go to a
ChildSession, which is wrong. As the assertion says, "Tile traffic
should go through the DocumentBroker-LoKit WS"
Re-think Linux vs mobile ifdefs a bit. Use #ifdef __linux only to
surround code that actually is Linux-specific. Use #ifdef MOBILEAPP
for code that is for a mobile version (with no separste wsd, forkit,
and kit processes, and with no WebSocket protocol used).
Bypass UnitFoo for mobile. Possibly we do want the UnitFoo stuff after
all on mobile, to run in some special testing mode? Hard to say, let's
skipt it for now.
And improve the logging support in unit-tests to
help troubleshoot issues faster and more accurately.
Also makes the code more readable (hopefully).
Change-Id: I4f8aafb5245e2f774b03231591a74544f9ec84aa
Reviewed-on: https://gerrit.libreoffice.org/48645
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
eventually the log file descriptor will be closed and unhandled exception it will throw it
"terminate called after throwing an instance of 'Poco::WriteFileException'
what(): Cannot write file"
Change-Id: I1d6ae3a4d4d4910f2ed2cdc80b162c27f93d55d9
Reviewed-on: https://gerrit.libreoffice.org/49055
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Michael Meeks <michael.meeks@collabora.com>
* logs helpful messages for various error corner-cases.
* optimized file descriptor closing for large fd counts.
Change-Id: I8cba9ecb3d71ddc6e22e20d89368d8c6b9b5097f
Start killing documents when memory usage goes above threshold.
Also make it possible to close documents from admin instance.
In DocumentBroker::closeDocument, just set the _stop flag and wake
up the polling thread which will terminate the children, instead of
manually terminating the children.
Change-Id: Ie70e05b3fb6ea816a87b6dcfaed92cdddb94aa90
userextrainfo is a json array that contains
extra user-specific links.
Currently 'avatar' is assumed to hold the
image url for the user's avatar.
'mail' and other links can also be added.
Change-Id: I37c4c68bfa0b7ee659e017b4867dcb8cf5c2ca2f
Reviewed-on: https://gerrit.libreoffice.org/38120
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
The new password hash property is called secure_password in the config
file. `loolconfig` tool should be used to set the password hash in
appropriate format with desired salt length, password length, number of
iterations in PBKDF2.
To be backward compatible, plain-text password for admin-console in
config file is still accepted in case secure_password property is
missing from the config file.
Change-Id: If229999dac62856e368555c0242c4aa6f8061fba
Jail paths are now generate from a PRNG
instead of using the PID of the kit process.
The PRN is converted to base-64 and used
as the directory name where a given
kit is jailed.
Change-Id: I8e4bc35d9ccdfdae0e542ab707c417cd29ad52f3
Reviewed-on: https://gerrit.libreoffice.org/37372
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
There is no need to use `ps` here as reading
directly is trivial and has far less overhead.
Change-Id: I27d0432c1f3a9d35763d67fc445d8bd828f1b27e
Reviewed-on: https://gerrit.libreoffice.org/34052
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Each Kit process now reports its own PSS,
which is much more accurate as they share
a significant ratio of their pages with
one another.
Admin tracks the PSS values of the Kits
and reports to the console.
Change-Id: Ifa66d17749c224f0dc211db80c44f7c913f2d6c4
Reviewed-on: https://gerrit.libreoffice.org/33864
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
