Commit graph

514 commits

Author SHA1 Message Date
Ashod Nakashian
8e784f2bdd wsd: tokenization and hexify utils
Change-Id: I3a8eb39092ad5ed1a8589c05a893fd0ce6445e8b
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/96827
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
2020-06-23 06:38:34 +02:00
Jan Holesovsky
2c246eed85 Sanitize the access_header.
The access_header can contain a lot of nonsense, like whitespace around
or additional \n's or \r's.  We used to sanitize that, but then
regressed in e95413d151 where the
"tokenize by any of \n\r" was by mistake replaced with "tokenize by
string '\n\r'".

Unfortunately the unit test didn't uncover that, and the further
refactorings of the related code have hidden that even more.

Change-Id: Ie2bf950d0426292770b599e40ee2401101162ff2
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/96638
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Andras Timar <andras.timar@collabora.com>
2020-06-19 11:48:44 +02:00
Dennis Francis
bc7a0b2626 jsonString: allow json messages without white-spaces after '{'
The json generated by boost::property_tree::write_json() has no
white-spaces around { , : } json-characters. We use write_json()
extensively in core.git. Without this patch we will need work-arounds
like inserting spaces in the json strings thus generated to work with
online's Message::jsonString()

Change-Id: I0f0631088f4a8b727301bde449884e03163093f0
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/96383
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
2020-06-16 08:02:27 +02:00
Gabriel Masei
ac17984226 admin: cleanup resource consuming kits
Change-Id: Ifafbadc61b788adc90c03fb92e0231f9e599c360
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/95794
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-06-09 11:25:44 +02:00
Ashod Nakashian
4a8937d0d1 wsd: performance improvements
Change-Id: I137dc67b4231df1cd23a9dce72e6b12dc1bf364e
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/95343
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
2020-06-02 20:01:39 +02:00
Ashod Nakashian
223a1d08f0 wsd: performance-unnecessary-value-param
Change-Id: I1eb092c676da8600e0f8ed70cbc7e1f37fdd5a02
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/95338
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Tested-by: Jenkins
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
2020-06-02 20:01:29 +02:00
Ashod Nakashian
fae093d89d wsd: std::move rather than copy
Change-Id: I7c4eea8aa5ab57444aa395f727e4ce6ac713b665
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/95337
Tested-by: Jenkins
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
2020-06-02 20:01:16 +02:00
Ashod Nakashian
d2d0492245 wsd: move LOOLProtocol::tokenize to Util::tokenize
The tokenizer(s) are more generic than the protocol
logic, and are used from contexts that don't involve
the protocol as such.

Change-Id: Ie8c256bf11a91e466bff794021f41603c9596a7f
2020-06-02 18:03:36 +01:00
Ashod Nakashian
d89f509609 wsd: overload getTokenInteger for string-literals
When parsing, we virtually always know the name
of the token we are parsing at compile time.
Taking advantage of that means we also know
its size at compile time, and can optimize
std::string allocation, size counting and
the implementation of getTokenInteger.

Change-Id: I502a643c14cace7dd755df565b3b5c445688faad
2020-06-02 18:00:20 +01:00
Ashod Nakashian
21ae15a49b wsd: optimized tokenization
Change-Id: I79b5117b89c982c15cee5c1e230a44785059affd
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/95336
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
2020-06-02 17:25:19 +02:00
Ashod Nakashian
784b7dc39d wsd: optimize StringVector
StringVector is heavily used for tokenization
and benefits from inlining of small functions.

Also, cat doesn't need to be slower than necessary.

Change-Id: I4ab2ff1b1f1a81092049d2cde64b6df10b34b5f7
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/95287
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Jenkins
2020-06-02 01:39:37 +02:00
Ashod Nakashian
224ef08c7f wsd: single-char string literals -> char
More readable and typically more efficient.

Change-Id: I9bd5bfc91f4ac255bb8ae0987708fb8b56b398f8
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/95285
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
2020-06-02 01:31:26 +02:00
Michael Meeks
87d039e00a IO stats - handle convert-to case, and initiailize all temporaries.
Change-Id: I16c60c89745a7cc2f369a010c77cffbb08759f7b
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/94775
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-05-25 12:37:40 +02:00
Ashod Nakashian
287b70a504 wsd: logging cleanup and improvements
We now don't need to call snprintf (which is best
for signal-safetly), and are much faster thanks
to a custom replacement.

Change-Id: Iae5861e42e8e335967499f93b71b39b0b4b09bf6
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/94146
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Tested-by: Jenkins
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
2020-05-22 00:40:37 +02:00
Pranam Lashkari
8aa9b37b86 removed tokenize method with regex
Added new tokenize method with const char* delimiter

Change-Id: Id1c4e89e5418d66aaf348ff4d8c3855f80fb4656
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/83574
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-05-04 15:11:26 +02:00
Michael Meeks
92eff552a5 Lower convert-to process priorities by default.
Interactive / editing processes should take precedence over batch
thumbnailing processes to keep responsiveness good.

Change-Id: Ib100409e312cb2ca545586a734711a31a92f110c
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/93323
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-05-02 22:03:36 +02:00
Pranam Lashkari
8b349716eb killpoco: removed Poco::Process completely
Change-Id: Iba67abf9342c11517c69c1d94903bf4752aa87d6
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/83770
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
2020-04-29 14:27:54 +02:00
Marco Cecchetti
255d24ba8f forward the device form factor of the client on a new view request
This patch allows the lok core to know about the device form facor of
the client requesting the creation of a new view, immediately instead
of a later time.
When a request for a new view is sent a 'deviceFormFactor' parameter
is appended to the message.
This parameter can have one of the following values: 'desktop',
'tablet','mobile' and is forwarded to the lok core.

Change-Id: I21739ddb8c43c960164b3c625e4cf0a80f4616a4
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92691
Tested-by: Marco Cecchetti <marco.cecchetti@collabora.com>
Reviewed-by: Marco Cecchetti <marco.cecchetti@collabora.com>
2020-04-29 13:18:49 +02:00
Andras Timar
33000dfded typo fixes in comments and code
Change-Id: I6f31e050aab701e31064e1abc9429dce0a50c279
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92953
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Andras Timar <andras.timar@collabora.com>
2020-04-26 23:19:52 +02:00
Gabriel Masei
6b486f229a wsd: admin: move kit memory reporting from kit to admin module
Sometimes kit process goes into a heavy processing state (or even hangs)
and is not able to report its memory usage. Thus we can't implement cleanup
of problematic kit processes based on memory information reported by kit.
By moving memory reporting to admin module we avoid this problem.

Change-Id: Icf274e3a3a97b33623a93f9d2dc1e640ad9b7d99
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92752
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-04-25 09:11:35 +02:00
Jan Holesovsky
eb88cb485c Allow running the production build under a non-lool user when necessary.
But it is insecure, so warn about that.

Change-Id: I151be64f53521e217a5498c0531c9ef2ff8db818
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92822
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
2020-04-24 20:58:43 +02:00
Tor Lillqvist
fdf5687d7c Bin some dead code, this gets rid of a use of Util::alertAllUsers(), yay
Getting rid of Util::alertAllUsers() calls in arbitrary places is good
because the semi-obscure way in which it works in the multi-process
web-based Online is potentially hard to fit together with the desire
to make the single-process mobile app able to handle several open
documents.

Change-Id: I6055a993ee31941e3592508aac5f0edf6497a836
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92571
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
2020-04-20 16:38:40 +02:00
Tor Lillqvist
5fff61ef14 There doesn't seem to be any use of FileUtil::alertAllUsers()
Which is good, Util::alertAllUsers() is enough of a complicated mess.

Change-Id: Ibac302ea8a7506baa992d71e3891b5764b6ed279
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92569
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
2020-04-20 15:35:05 +02:00
Michael Meeks
3e4ac31c7c killpoco: remove WebSocket includes from a couple of places.
Change-Id: I06740cd978bec8e6a74beb8ed9ef8f4f970a2535
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92470
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-04-18 18:29:04 +02:00
Tor Lillqvist
378a2b5295 Bin a couple of unintentionally committed LOG_TRC() calls
Change-Id: Ieadc06a38d71aed723191f1738f8f39154978091
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92476
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
2020-04-18 16:35:15 +02:00
Tor Lillqvist
4eb598711c Use #pragma once
LibreOffice core uses that, too, and we support an even more
restricted set of compilers.

Change-Id: I0d0e2c8608e323eb5ef0f35ee8c46d02ab49a745
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92467
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
2020-04-18 15:00:18 +02:00
Michael Meeks
70201b7430 Log exception on recursive file removal failure.
Change-Id: Ie810f23af48877bbfe9dff7da8eee709186e108e
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92231
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-04-15 12:31:27 +02:00
Miklos Vajna
61790f6bb5 common: it's enough to look up PRETTY_NAME once
Also add the missing newlines before the opening braces, the surrounding
code has that style.

Change-Id: I23bd26ba6d2446858ae3213212e2813a38fd1d46
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92146
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
2020-04-14 13:41:55 +02:00
gokaysatir
9e0594ee2d add pretty name - Android patch
Change-Id: If0de49884954cde26bfbe8ba1dce8844af5b30bf
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/91831
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-04-11 13:43:31 +02:00
Michael Meeks
a7dc2d1467 kill IoUtil - obsolete & unused.
The switch away from LOOLWebSocket and the use of a websocket
for talking to forkit removes the need for the pipe code.

Change-Id: Ifb0c6c88681289e7a1709d9bc3281532935c7be4
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92033
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-04-10 16:38:25 +02:00
Michael Meeks
5710c86323 Poll - switch to ppoll for closer to microsecond accuracy.
Change-Id: Ib8a2bb6f60302df8631edadbbb8db626894c457c
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92000
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-04-10 10:06:23 +02:00
Gabriel Masei
0ac330f599 Add segmentation fault metric for Kit processes
Change-Id: Ifb0de004274213ef512f601e4419f98f456c7288
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/91857
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-04-08 15:06:39 +02:00
Gabriel Masei
70af76e28c Replaced pipe with websocket based on Unix socket in communication with ForKit
Change-Id: I80f1a4e84ca6820503966a8ee5d9958a150eac14
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/91585
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-04-07 15:04:47 +02:00
Miklos Vajna
42c8417e34 admin fuzzer: improve this further, so it can find interesting paths faster
1) Don't actually kill anything with the kill command, otherwise kill(0,
SIGKILL) will kill the fuzzer itself.

2) Don't require a valid signature when authenticating with JWT, since
the private key is generated on each process startup.

3) Log when the JWT would be invalid due to an expired timestamp.

Change-Id: I0da285617e27910329c0e7ed80a6d02e86344ccf
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/91737
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
2020-04-06 12:03:39 +02:00
gokaysatir
a486bad79b tdf#130568 - Add server os pretty name to help->about
Change-Id: Id6de533dfb8e34a05d348f8ae701bf3c524c9b95
2020-04-01 20:45:32 +01:00
Jan Holesovsky
120000347e Log every error we send to the JS.
So that it is easy to find it in the log.

Change-Id: I1b367db9e97d7215c46e824998e99048531acb07
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/91413
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
2020-03-31 20:15:25 +02:00
Michael Meeks
b30757417b jail removal - don't use flaky/experimental new C++ filesystem.
Simply not removing jails on some systems.

Change-Id: Idb1306fec83acebc61dec2e7a240ad89e7c91437
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/91067
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
2020-03-25 20:18:52 +01:00
Michael Meeks
f978982a7a cleanup: remove obsolete locking & dead code from LOOLWebSocket.
LOOLWebSocket is used only for tests and a few tools anyway.

Change-Id: I18e04efc2fb3a4cc075f7cefa4ebd0d10fbdb3df
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90974
Tested-by: Michael Meeks <michael.meeks@collabora.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-03-24 12:18:48 +01:00
Miklos Vajna
d6fe0979b7 common: handle missing protocol in Session::getIOStats()
This sometimes causes unit-integration to fail and now is consistent
with other member functions of Session that handle missing protocol.

Change-Id: I43c7fcae964cfcb5911ff57d63bd4cb569e6b97c
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90732
Tested-by: Michael Meeks <michael.meeks@collabora.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-03-19 11:50:54 +01:00
Corentin Noël
2edf0dcb3a Seccomp: Add AARCH64 support
Change-Id: I59e042b244711ef3780cee7946be78d31dca8d8d
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90717
Tested-by: Michael Meeks <michael.meeks@collabora.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-03-18 22:34:54 +01:00
Miklos Vajna
cd42700054 Rework the somewhat odd sendTextMessage() interface
It took both an std::string and a length. Take a char* and a length
instead.

Change-Id: Id37dfa67fe1baae09b69819680848a0a8a1d80ed
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90552
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
2020-03-16 11:55:19 +01:00
Ashod Nakashian
1339b36575 wsd: define Log::shutdown for Mobile to avoid special casing at call sites
Change-Id: I9125df562c96b3671cb47200a36aae145cf29259
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90423
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
2020-03-14 14:23:27 +01:00
Michael Meeks
e924625cc1 re-factor: Socket / WebSocketHandler.
Essentially we want to be able to separate low-level socket code
for eg. TCP vs. UDS, from Protocol handling: eg. WebSocketHandler
and client sessions themselves which handle and send messages
which now implement the simple MessageHandlerInterface.

Some helpful renaming too:

s/SocketHandlerInterface/ProtocolHandlerInterface/

Change-Id: I58092b5e0b5792fda47498fb2c875851eada461d
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90138
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-03-11 16:48:03 +01:00
Miklos Vajna
a7d3efdd4e Introduce StringVector::equals()
Allows comparing tokens with C strings without a heap allocation. Do the
same when comparing two tokens from two different StringVectors.

And use it at all places where operator ==() has an argument, which is a
StringVector::operator []() result.

Change-Id: Id36eff96767ab99b235ecbd12fb14446a3efa869
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90201
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
2020-03-09 09:46:33 +01:00
Michael Meeks
7eea5b85ab Remove unused filterHandleRequest unit test hook.
Change-Id: Id1ff6005e28bf02eb9cad3c1609f9a8079640689
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90155
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-03-07 14:26:42 +01:00
Michael Meeks
019f268ffd handleDisconnect - unused.
Change-Id: Ib305f762ae33424c8a598d5c31eeda31b9c87555
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90137
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-03-07 11:28:08 +01:00
Michael Meeks
f70e627795 WebSocket - simplify handleMessage for now.
WebSocketHandler handles this complexity for us now, and for the
forseeable future. Simplify to prepare for larger re-factor.

Change-Id: I73b919885adc358cb6502a13492cdac85c34459c
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90059
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-03-05 20:28:48 +01:00
Michael Meeks
17cbb2b7b9 Session: simplify shutdown and privatize some pieces.
Change-Id: Ifbb40f6a88590bd87dbfada73fa26eb0b61705fb
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90058
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-03-05 20:22:32 +01:00
Miklos Vajna
c99664b19c StringVector: fix build with gcc-4.9
Debian 8 has gcc-4.9, which emits -Werror in case a parameter and a
member function has the same name. Given that we also use -Werror
unconditionally, this breaks the build. Newer gcc/clang versions relaxed
this warning, so this was not a problem in those cases.

Change-Id: I7ad09370d96aa7384b2c117dd8de421644898b50
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89785
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
2020-03-02 16:25:05 +01:00
Miklos Vajna
547f9ea731 Rework StringVector to have a single underlying string
This is meant to reduce lots of small allocations and instead have
pointers into the single string for the various tokens instead.

This has a few requirements, though:

1) It's no longer OK to modify the tokens, changing their length would
invalidate the start/length of other tokens. Rework
DocumentBroker::load() to avoid such mutation.

2) The iterators no longer expose zero-terminated strings, so
Poco::cat() doesn't work anymore: add an own cat() instead and use that
in e.g. ChildSession. The own cat() has the benefit that it won't read
past the end of the array if the begin index is out of bounds to add
more safety.

(This nicely works towards killing Poco usage in general.)

3) If zero-terminated strings for all individual tokens is needed, a
copy has to be made, as done in spawnProcess().

(For all of these requirements, the build fails if there are problems.)

Change-Id: Iea40e4400e630b2d669f5c72aea85cb40edf9a2c
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89711
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
2020-02-28 18:31:37 +01:00
Miklos Vajna
b8bd1990aa Rework LOOLProtocol::tokenize() to return a StringVector object
The bulk of this commit just changes std::vector<std::string> to
StringVector when we deal with tokens from a websocket message.

The less boring part of it is the new StringVector class, which is a
wrapper around std::vector<std::string>, and provides the same API,
except that operator[] returns a string, not a string&, and this allows
returning an empty string in case that prevents reading past the end of
the underlying array.

This means in case client code forgets to check size() before invoking
operator[], we don't crash. (See the ~3 previous commits which fixed
such crashes.)

Later the ctor could be changed to take a single underlying string to
avoid lots of tiny allocations, that's not yet done in this commit.

Change-Id: I8a6082143a8ac0b65824f574b32104d7889c184f
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89687
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
2020-02-28 16:07:56 +01:00
Tor Lillqvist
f49aa533e0 Fix iOS build (and maybe Android, too)
Change-Id: Id6645ce7fe0f70d7e7cec0059ad25624d1057ad5
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89319
Tested-by: Tor Lillqvist <tml@collabora.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
2020-02-24 08:54:22 +01:00
Ashod Nakashian
18ab1ccaed wsd: cleanly shutdown when unittest times out
Termination flag is a very harsh way of exiting.
It works in most cases, but not when we have a
modified document. What happens is the following:

Unit-test flags for termination.
During session cleanup we have to save the modified doc.
Because save is in progress we don't 'disconnect' the view.
This leaves the view in loaded state until saving is done.
But because of the termination flag we don't wait for saving.
DocBroker sends 'exit' to child to forcefully exit.
This causes at least one assertion due to active LOKWindows (Sidebar).

Instead of the above, we flag for graceful shutdown from unittests,
and after we wait to cleanup all DocBrokers, we flag for termination.
This way, we get clean shutdown and all assertions/validations
pass, while we guarantee never to deadlock the unittests,
in case we end up waiting forever for shutdown to complete.

Change-Id: I7fc34137ea373e329795b1ed0090261c085e955a
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89308
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
2020-02-23 21:48:07 +01:00
Ashod Nakashian
ddd3108767 wsd: logging improvements and formatting
Some multi-line log entries were missing
because we didn't flush.

Change-Id: Iabf9e6f1ffee707ff51b831a12c7990f35bb7913
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89196
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
2020-02-23 21:46:55 +01:00
Miklos Vajna
57a35bb96c Add an initial libfuzzer based fuzzer
- target ClientSession::_handleInput(), since crashing there would bring
  down the whole loolwsd (not just a kit process), and it deals with
  input from untrusted users (browsers)

- add a --enable-fuzzers configure switch to build with
  -fsanitize=fuzzer (compared to normal sanitizers build, this is the only
  special flag needed)

- configuring other sanitizers is not done automatically, either use
  --with-sanitizer=... or the environment variables from LODE's sanitizer
  config

- run the actual fuzzer like this:

  ./clientsession_fuzzer -max_len=16384 fuzzer/data/

- note that at least openSUSE Leap 15.1 sadly ships with a clang with
  libfuzzer static libs removed from the package, so you need a
  self-built clang to run the fuzzer (either manual build or one from
  LODE)

- <https://chromium.googlesource.com/chromium/src/testing/libfuzzer/+/refs/heads/master/efficient_fuzzing.md#execution-speed>
  suggests that "You should aim for at least 1,000 exec/s from your fuzz
  target locally" (i.e. one run should not take more than 1 ms), so try
  this minimal approach first. The alternative would be to start from the
  existing loolwsd_fuzzer binary, then step by step cut it down to not
  fork(), not do any network traffic, etc -- till it's fast enough that
  the fuzzer can find interesting input

- the various configurations start to be really complex (the matrix is
  just very large), so try to use Util::isFuzzing() for fuzzer-specific
  changes (this is what core.git does as well), and only resort to ifdefs
  for the Util::isFuzzing() itself

Change-Id: I72dc1193b34c93eacb5d8e39cef42387d42bd72f
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89226
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-02-22 12:18:22 +01:00
Miklos Vajna
8d2a8da960 common: fix crash when the version string contains no dot character
==13901==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x000000904678 bp 0x7ffdb9e21580 sp 0x7ffdb9e21340 T0)
==13901==The signal is caused by a READ memory access.
==13901==Hint: address points to the zero page.
    #0 0x904677 in LOOLProtocol::tokenize[abi:cxx11](char const*, unsigned long, char) common/Protocol.hpp:113:40
    #1 0x898c52 in LOOLProtocol::tokenize(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, char) common/Protocol.hpp:141:16
    #2 0x18dc2d9 in LOOLProtocol::ParseVersion(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) common/Protocol.cpp:35:51
    #3 0x1148824 in ClientSession::_handleInput(char const*, int) wsd/ClientSession.cpp:358:64
    #4 0x18efcb8 in Session::handleMessage(bool, WSOpCode, std::vector<char, std::allocator<char> >&) common/Session.cpp:232:13

Next commit will add the actual simple fuzzer that found this.

Change-Id: I8623b4451a57390f6f84c11084c5a1120a11fcc5
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89225
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-02-22 12:18:11 +01:00
Miklos Vajna
d8e6fa411c fuzzer: fix this up so it works again
After fixing this single assertion failure, the

./loolwsd_fuzzer --config-file=loolwsd.xml --o:storage.filesystem[@allow]=true --o:logging.level=fatal

invocation works. (It does not really fuzz anything, but it's a single
unpriviliged process at least.)

Change-Id: I45f877e5eb023e3ddfc96a7373c6300e4bb77962
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89115
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
2020-02-20 18:03:10 +01:00
Miklos Vajna
c414b4a240 UnitBase::exitTest: make sure we flag for termination only once
It can happen that a test succeeds, but right after that, the timeout
thread still fails the test:

09:21:10.411840 [ loolwsd ] INF  exitTest: 1. Flagging for termination.| common/Unit.cpp:201
09:21:10.412016 [ loolwsd ] ERR  Timed out waiting for unit test to complete| common/Unit.cpp:212
09:21:10.412060 [ loolwsd ] INF  exitTest: 2. Flagging for termination.| common/Unit.cpp:201

(1 is TestResult::Ok, 2 is TestResult::TimedOut.)

Fix these cases by just ignoring the timeout invocation, we'll exit soon
anyway.

With this, finally a sanitizers-enabled 'make check' passes agains a
sanitizers-enabled core.git.

Change-Id: Id52c4b6f88a795d4ff2af7929a71d5eb83c56773
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/88999
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
2020-02-19 11:16:09 +01:00
Michael Meeks
701cb6a230 tdf#130673 - kill obsolete locking.
This dates back to 2016 and our pre non-blocking and pre-unipoll state.

It is no longer necessary - a single thread reads all data from the
socket and feeds events into the Kit process; much cleaner.

Change-Id: I46ad6806a1e0cdbb0e5cf4ea5d3e5e5078d3391a
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/88741
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-02-14 23:33:49 +01:00
Pranam Lashkari
a019c93d90 Resolved: Server crash when link opened in new tab
When trying to open a link normally from help->Online help
nothing happens but the popup is closed.

When trying to open a like forcefully in new tab
from help->online help it crashes the server.

Change-Id: I7e0944ebe521002625a84e155e379ed7e25d2309
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/85466
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-01-22 23:23:51 +01:00
Michael Meeks
9dfac6dab1 test: switch to parallel tests based on Unit framework.
Increase a few timeouts, bin old-style standalone unit tests,
fix a number of bugs.

Change-Id: Ia3d59466ecb9a9443807ba3445d04dd5f77e3dba
2020-01-21 15:07:54 +00:00
Michael Meeks
0599db3576 copyFile: de-poco-ize and handle EINTR and short writes.
Change-Id: I2046881c786a9f31f45c53f282de9ddd9a9cebcf
2020-01-21 15:07:54 +00:00
Michael Meeks
09bb16ad78 util: for process thread count - ignore '.' and '..' in /proc/self/tasks
Change-Id: Ieec6eaac475b4e318578cfc0d93c36e2395e6f19
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/87097
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-01-20 22:04:40 +01:00
Michael Meeks
e6a3364e93 Move file url anonymization down from LOOLWSD into FileUtil.
Change-Id: I415c73b10621d5c7c942367bbf38a3bbd9bf8f27
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/87024
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-01-18 20:05:35 +01:00
Michael Meeks
b72f37826e sighandler: break infinite loop with corrupted heap
ignoring the segv can lead to not making progress, while churning debug.

Change-Id: I97af266cec3feefe2dcbd9adb8dbf4b13a4d69bd
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/87002
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-01-18 17:23:02 +01:00
Michael Meeks
9cef0f385d watermarking: create SessionMap template to canonicalize views.
Use a fully reliable uniqueness check, rather than a hash, and get
simpler ids as a bonus. Fetch view data from the session itself
rather than passing it in too.

Change-Id: Ibcd625156b5a98eb280e35d6537b5c8c026d0197
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/86150
Reviewed-by: Mert Tümer <mert.tumer@collabora.com>
Tested-by: Mert Tümer <mert.tumer@collabora.com>
2020-01-06 13:52:29 +01:00
Henry Castro
9290981013 wsd: debug: add env variable "PAUSEFORDEBUGGER" to attach debugger
It is painful to check and search manually the PID to attach the LOKit
process when exists several pre-spawned waiting to load a document.

This patch helps to attach the debugger when the LOKit process is about
to load a document then send the "signal SIGUSR1" to resume it.

Change-Id: I3b15bd522c6ef3ef57dc3453b457dcf91f2661b9
Reviewed-on: https://gerrit.libreoffice.org/85430
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Henry Castro <hcastro@collabora.com>
Reviewed-by: Henry Castro <hcastro@collabora.com>
2019-12-20 19:57:47 +01:00
Michael Meeks
e3864a060e Dump DocumentBroker state more completely.
Change-Id: I3477fe70ba25d6e9a95c12f30138c3353994e384
2019-11-27 19:17:27 +00:00
Ashod Nakashian
a5946b4625 wsd: use SIGKILL to kill a child
This is the cleanest way to achieve the goal
of immediately exiting a child. This is used
for cleaning up kit instances when closing
docs, as well as in unit-tests.

Change-Id: I76870234b130a508044044b102419646abe81ac8
Reviewed-on: https://gerrit.libreoffice.org/83699
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2019-11-25 17:06:19 +01:00
Gabriel Masei
2164f5207c Add REST endpoint for admin metrics.
Change-Id: I701485631931334d27594c4907cb770f9888e5bf
Reviewed-on: https://gerrit.libreoffice.org/82492
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Michael Meeks <michael.meeks@collabora.com>
2019-11-25 13:06:01 +01:00
Corentin Noël
08aa7d9149 Seccomp: add ARM support
Change-Id: I6982c6e68da5e2211f9dad0eb88fccab27230ed9
Reviewed-on: https://gerrit.libreoffice.org/83511
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Michael Meeks <michael.meeks@collabora.com>
2019-11-22 18:49:00 +01:00
Miklos Vajna
204cff5c19 Avoid UB in Session::setHash()
Ubsan says that signed integer overflow is happening here:

    #0 0xec344a in Session::setHash(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) common/Session.cpp:267:17
    #1 0x918b54 in DocumentBroker::load(std::shared_ptr<ClientSession> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) wsd/DocumentBroker.cpp:682:18
...
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior common/Session.cpp:267:17 in

(While running unit_wopi_watermark.)

Change-Id: Idcb1692f6d5d0df8be35dc557d2d72a345acbcf4
2019-11-22 14:11:00 +01:00
Ashod Nakashian
876897ce96 wsd: kill with SIGABRT now that SIGTERM cleanly shutsdown
Change-Id: I198795be0cc399703f3207456ef0bfd1e3c3c844
Reviewed-on: https://gerrit.libreoffice.org/83359
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
Tested-by: Miklos Vajna <vmiklos@collabora.com>
2019-11-21 09:18:50 +01:00
Ashod Nakashian
3ebbc6213b wsd: avoid malloc in signal handler
malloc is not signal safe, and must not be called
from signal-safe functions. If malloc itself signals,
calling it in the signal handler can deadlock.

Luckily, we only needed malloc for getting the
backtrace strings. Now we just write directly to
stderr, which is faster, cleaner, and safer.

Change-Id: I54093f45e05f2a0fd3c5cde0cc2104ffe6d81d2a
Reviewed-on: https://gerrit.libreoffice.org/83151
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2019-11-19 16:13:33 +01:00
Ashod Nakashian
b74306acd5 wsd: replace mutex in signal handler
There are a few things acceptable/safe in a
signal handler, and taking locks is not one of them.

This replaces the logic with a simple counter that
serves the purpose just as well.

If we get a double signal, we log and ignore.

Change-Id: If589c18492468c120d00c213805467bcbba05d27
Reviewed-on: https://gerrit.libreoffice.org/83150
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2019-11-19 16:12:29 +01:00
Ashod Nakashian
4f15d1bd19 wsd: SIGTERM signals clean shutdown now
Change-Id: I8d5bf7232036bb103cd7433cd5b1c7b5021e06d4
Reviewed-on: https://gerrit.libreoffice.org/83052
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Michael Meeks <michael.meeks@collabora.com>
2019-11-18 06:48:52 +01:00
Ashod Nakashian
1ae9ce8ad2 test: thread-safe common shared test data
This protects against memory corruptions,
and a cascade of issues, such as the following:

 Attaching to process 56245
 [New LWP 56246]
 [New LWP 56252]
 [New LWP 56253]
 [New LWP 56254]
 [New LWP 56362]
 [New LWP 56364]
 [Thread debugging using libthread_db enabled]
 Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
 0x00007fda228f7d2d in __GI___pthread_timedjoin_ex (threadid=140574279595776, thread_return=0x0, abstime=0x0, block=<optimized out>) at pthread_join_common.c:89
 89	pthread_join_common.c: No such file or directory.

 Thread 7 (Thread 0x7fda197fa700 (LWP 56364)):
 #0  __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
 #1  0x00007fda228f9023 in __GI___pthread_mutex_lock (mutex=0x55e9937567e0 <SigHandlerTrap>) at ../nptl/pthread_mutex_lock.c:78
 #2  0x000055e9934cd4fe in __gthread_mutex_lock (__mutex=0x55e9937567e0 <SigHandlerTrap>) at /usr/include/x86_64-linux-gnu/c++/7/bits/gthr-default.h:748
 #3  std::mutex::lock (this=0x55e9937567e0 <SigHandlerTrap>) at /usr/include/c++/7/bits/std_mutex.h:103
 #4  std::unique_lock<std::mutex>::lock (this=<synthetic pointer>) at /usr/include/c++/7/bits/std_mutex.h:267
 #5  std::unique_lock<std::mutex>::unique_lock (__m=..., this=<synthetic pointer>) at /usr/include/c++/7/bits/std_mutex.h:197
 #6  SigUtil::handleFatalSignal (signal=11) at common/SigUtil.cpp:214
 #7  <signal handler called>
 #8  std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_capacity (this=<optimized out>, __capacity=0) at /usr/include/c++/7/bits/basic_string.h:200
 #9  std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string (__str=..., this=<optimized out>) at /usr/include/c++/7/bits/basic_string.h:542
 #10 std::_Construct<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > (
     __p=<optimized out>) at /usr/include/c++/7/bits/stl_construct.h:75
 #11 std::__uninitialized_copy<false>::__uninit_copy<std::move_iterator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*> (__result=0x7fd9f40017b0, __last=..., __first=...) at /usr/include/c++/7/bits/stl_uninitialized.h:83
 #12 std::uninitialized_copy<std::move_iterator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*> (__result=<optimized out>, __last=..., __first=...) at /usr/include/c++/7/bits/stl_uninitialized.h:134
 #13 std::__uninitialized_copy_a<std::move_iterator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > (__result=<optimized out>, __last=..., __first=...)
     at /usr/include/c++/7/bits/stl_uninitialized.h:289
 #14 std::__uninitialized_move_if_noexcept_a<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > (__alloc=..., __result=<optimized out>, __last=0x55e995662850,
     __first=<optimized out>) at /usr/include/c++/7/bits/stl_uninitialized.h:312
 #15 std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >::_M_realloc_insert<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&> (
     this=this@entry=0x55e9937566f0 <FileUtil::getTempFilePath(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::fileDeleter>,
     __position=<error: Cannot access memory at address 0x3735003238323a>, __args#0="/tmp/loadTorture_viewcursor.odp_725a6013_viewcursor.odp") at /usr/include/c++/7/bits/vector.tcc:424
 #16 0x000055e9934aa829 in std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >::push_back (__x="/tmp/loadTorture_viewcursor.odp_725a6013_viewcursor.odp",
     this=0x55e9937566f0 <FileUtil::getTempFilePath(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::fileDeleter>) at /usr/include/c++/7/bits/stl_vector.h:948
 #17 (anonymous namespace)::FileDeleter::registerForDeletion (file="/tmp/loadTorture_viewcursor.odp_725a6013_viewcursor.odp",
     this=0x55e9937566f0 <FileUtil::getTempFilePath(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::fileDeleter>) at common/FileUtil.cpp:69
 #18 FileUtil::getTempFilePath (srcDir=..., srcFilename=..., dstFilenamePrefix=...) at common/FileUtil.cpp:97
 #19 0x00007fda216b51c9 in helpers::getDocumentPathAndURL (docFilename="viewcursor.odp", documentPath="", documentURL="", prefix="loadTorture_viewcursor.odp_") at ./helpers.hpp:151
 #20 0x00007fda216ad007 in UnitLoadTorture::loadTorture (this=<optimized out>, testname="loadTorture_viewcursor.odp ", docName="viewcursor.odp", thread_count=thread_count@entry=3,
     max_jitter_ms=<optimized out>, max_jitter_ms@entry=75) at UnitLoadTorture.cpp:41
 #21 0x00007fda216ad7dd in UnitLoadTorture::<lambda()>::operator() (__closure=0x55e994dbc758) at UnitLoadTorture.cpp:186
 #22 std::__invoke_impl<void, UnitLoadTorture::testLoadTorture()::<lambda()> > (__f=...) at /usr/include/c++/7/bits/invoke.h:60
 #23 std::__invoke<UnitLoadTorture::testLoadTorture()::<lambda()> > (__fn=...) at /usr/include/c++/7/bits/invoke.h:95
 #24 std:🧵:_Invoker<std::tuple<UnitLoadTorture::testLoadTorture()::<lambda()> > >::_M_invoke<0> (this=0x55e994dbc758) at /usr/include/c++/7/thread:234
 #25 std:🧵:_Invoker<std::tuple<UnitLoadTorture::testLoadTorture()::<lambda()> > >::operator() (this=0x55e994dbc758) at /usr/include/c++/7/thread:243
 #26 std:🧵:_State_impl<std:🧵:_Invoker<std::tuple<UnitLoadTorture::testLoadTorture()::<lambda()> > > >::_M_run(void) (this=0x55e994dbc750) at /usr/include/c++/7/thread:186
 #27 0x00007fda22de366f in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
 #28 0x00007fda228f66db in start_thread (arg=0x7fda197fa700) at pthread_create.c:463
 #29 0x00007fda2261f88f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

 Thread 6 (Thread 0x7fd9fffff700 (LWP 56362)):
  #0  __lll_lock_wait_private () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:95
 #1  0x00007fda225952db in __GI___libc_malloc (bytes=bytes@entry=3635) at malloc.c:3063
 #2  0x00007fda2262f364 in __backtrace_symbols (array=array@entry=0x7fd9ffffc540, size=size@entry=26) at backtracesyms.c:69
 #3  0x000055e9934cc439 in SigUtil::dumpBacktrace () at common/SigUtil.cpp:253
 #4  0x000055e9934cd5ae in SigUtil::handleFatalSignal (signal=6) at common/SigUtil.cpp:236
 #5  <signal handler called>
 #6  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
 #7  0x00007fda2253e801 in __GI_abort () at abort.c:79
 #8  0x00007fda22587897 in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7fda226b4b9a "%s
") at ../sysdeps/posix/libc_fatal.c:181
 #9  0x00007fda2258e90a in malloc_printerr (str=str@entry=0x7fda226b2e0e "malloc(): memory corruption") at malloc.c:5350
 #10 0x00007fda22592994 in _int_malloc (av=av@entry=0x7fda00000020, bytes=bytes@entry=32) at malloc.c:3738
 #11 0x00007fda225952ed in __GI___libc_malloc (bytes=32) at malloc.c:3065
 #12 0x00007fda22db9258 in operator new(unsigned long) () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
 #13 0x00007fda244adaae in __gnu_cxx::new_allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::allocate (this=0x7fd9ffffe740, __n=1)
     at /usr/include/c++/5/ext/new_allocator.h:104
 #14 0x00007fda244ad300 in __gnu_cxx::__alloc_traits<std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >::allocate (__a=..., __n=1)
     at /usr/include/c++/5/ext/alloc_traits.h:182
 #15 0x00007fda244ac858 in std::_Vector_base<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >::_M_allocate (this=0x7fd9ffffe740, __n=1) at /usr/include/c++/5/bits/stl_vector.h:170
 #16 0x00007fda244ab7c7 in std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >::_M_insert_aux (this=0x7fd9ffffe740, __position=non-dereferenceable iterator for std::vector, __x="home") at /usr/include/c++/5/bits/vector.tcc:353
 #17 0x00007fda244aada2 in std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >::push_back (this=0x7fd9ffffe740, __x="home") at /usr/include/c++/5/bits/stl_vector.h:925
 #18 0x00007fda2350ddcb in Poco::Path::pushDirectory (this=0x7fd9ffffe6c0, dir="home") at src/Path.cpp:471
 #19 0x00007fda2350ea27 in Poco::Path::parseUnix (this=0x7fd9ffffe6c0, path="/home/ash/prj/lo/online/test/") at src/Path.cpp:671
 #20 0x00007fda2350cd6b in Poco::Path::assign (this=0x7fd9ffffe6c0, path="/home/ash/prj/lo/online/test/") at src/Path.cpp:182
 #21 0x00007fda2350c1be in Poco::Path::Path (this=0x7fd9ffffe6c0, path="/home/ash/prj/lo/online/test/") at src/Path.cpp:54
 #22 0x00007fda2350d33a in Poco::Path::makeAbsolute (this=0x7fd9ffffe8f0) at src/Path.cpp:318
 #23 0x00007fda216b52ca in helpers::getDocumentPathAndURL (docFilename="setclientpart.ods", documentPath="/tmp/loadTorture_setclientpart.ods_ae70d2e_setclientpart.ods", documentURL="",
     prefix="loadTorture_setclientpart.ods_") at ./helpers.hpp:153
 #24 0x00007fda216ad007 in UnitLoadTorture::loadTorture (this=<optimized out>, testname="loadTorture_setclientpart.ods ", docName="setclientpart.ods", thread_count=thread_count@entry=3,
     max_jitter_ms=<optimized out>, max_jitter_ms@entry=75) at UnitLoadTorture.cpp:41
 #25 0x00007fda216ad7dd in UnitLoadTorture::<lambda()>::operator() (__closure=0x55e994cffe18) at UnitLoadTorture.cpp:186
 #26 std::__invoke_impl<void, UnitLoadTorture::testLoadTorture()::<lambda()> > (__f=...) at /usr/include/c++/7/bits/invoke.h:60
 #27 std::__invoke<UnitLoadTorture::testLoadTorture()::<lambda()> > (__fn=...) at /usr/include/c++/7/bits/invoke.h:95
 #28 std:🧵:_Invoker<std::tuple<UnitLoadTorture::testLoadTorture()::<lambda()> > >::_M_invoke<0> (this=0x55e994cffe18) at /usr/include/c++/7/thread:234
 #29 std:🧵:_Invoker<std::tuple<UnitLoadTorture::testLoadTorture()::<lambda()> > >::operator() (this=0x55e994cffe18) at /usr/include/c++/7/thread:243
 #30 std:🧵:_State_impl<std:🧵:_Invoker<std::tuple<UnitLoadTorture::testLoadTorture()::<lambda()> > > >::_M_run(void) (this=0x55e994cffe10) at /usr/include/c++/7/thread:186
 #31 0x00007fda22de366f in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
 #32 0x00007fda228f66db in start_thread (arg=0x7fd9fffff700) at pthread_create.c:463
 #33 0x00007fda2261f88f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Change-Id: Ib60954e49e692082e686cb3fdc5fc9d5cfbc83c9
Reviewed-on: https://gerrit.libreoffice.org/83047
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Michael Meeks <michael.meeks@collabora.com>
2019-11-18 06:47:20 +01:00
Pranam Lashkari
e95413d151 killpoco: removed StringTokenizer from common directory
removed use of Poco::StringTokenizer from the common directory
used LOOLProtocol::tokenize and std::vecor<std::string>

regex is used in Authorization.cpp due to limitation of toeknize mathod
regular expression helps to keep the original intention of the code

Change-Id: Ic87597d8b30cb385000f983389a57dc5d2533d98
Reviewed-on: https://gerrit.libreoffice.org/82575
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
Tested-by: Jan Holesovsky <kendy@collabora.com>
2019-11-15 14:54:29 +01:00
Jan Holesovsky
f637f4a75a killpoco: Remove StringTokenizer from LOOLWSD.cpp.
Adds possibility to tokenize using a regex easily.

Change-Id: Ie327d4faabec330c76d4cadb1d14bbe1527d332f
Reviewed-on: https://gerrit.libreoffice.org/82333
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
Tested-by: Jan Holesovsky <kendy@collabora.com>
2019-11-08 22:24:08 +01:00
Jan Holesovsky
efa293290c killpoco: Remove trivial StringTokenizer uses from some files.
Particularly those used on Android...

Change-Id: I47bf9692f5e99ba30140d698558472201168a198
Reviewed-on: https://gerrit.libreoffice.org/82302
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
Tested-by: Jan Holesovsky <kendy@collabora.com>
2019-11-08 22:21:31 +01:00
Jan Holesovsky
d906c78198 killpoco: Remove POCO from Protocol.cpp + related StringTokenizer usage.
Change-Id: I9d93c03dde110a4e88f7ec86e5934916596a27c1
Reviewed-on: https://gerrit.libreoffice.org/82301
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
Tested-by: Jan Holesovsky <kendy@collabora.com>
2019-11-08 22:20:58 +01:00
Jan Holesovsky
07d5f8cf2a killpoco: Remove POCO usage from FileUtil.cpp when C++17 is available.
This introduces basic C++17 support, because the functionality needed
here is easy to implement using std::filesystem.

Adds also the necessary checks to ./configure.  The code still uses POCO
when C++17 is not available in the compiler.

Change-Id: I03353834d10201bf0a13ea72715560b9b9b16265
Reviewed-on: https://gerrit.libreoffice.org/82294
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
Tested-by: Jan Holesovsky <kendy@collabora.com>
2019-11-08 13:56:09 +01:00
Jan Holesovsky
189cd0e389 killpoco: Get rid of Poco::Process:id() usage.
Change-Id: If060767389f9fa57deba1ceefc872bac03763498
Reviewed-on: https://gerrit.libreoffice.org/82208
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
Tested-by: Jan Holesovsky <kendy@collabora.com>
2019-11-07 12:24:53 +01:00
Jan Holesovsky
5d0d3a4869 killpoco: Get rid of Poco/Thread.h from the rest of the Android-related files.
Change-Id: I724230a4428cab3cc26245ac4aa43a91af2e09ce
Reviewed-on: https://gerrit.libreoffice.org/82204
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
Tested-by: Jan Holesovsky <kendy@collabora.com>
2019-11-07 12:24:26 +01:00
Jan Holesovsky
f0ea9701d8 killpoco: Use std::thread instead of Poco::Thread.
Change-Id: I533429412f269cff8714407f3b99aef0ff133abf
Reviewed-on: https://gerrit.libreoffice.org/82199
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
Tested-by: Jan Holesovsky <kendy@collabora.com>
2019-11-07 12:24:01 +01:00
Jan Holesovsky
5b5e29b430 killpoco: Don't use POCO for app exit values.
Change-Id: I2948ac45a7b4243f7afde08d5245530fdbf9a070
Reviewed-on: https://gerrit.libreoffice.org/82125
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Michael Meeks <michael.meeks@collabora.com>
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
2019-11-07 12:23:47 +01:00
Ashod Nakashian
7a976488f0 wsd: cleanup the global flag accessors
The following flags are affected:
ShutdownRequestFlag
TerminationFlag
DumpGlobalState

Since it's common to grep for all places
that set or reset these global flags, it
makes more sense to have explicit functions
for each operation. Now we have set and reset
accessors where appropriate and get is reserved
for read-only access.

This changes the getters to only return
the boolean value of these flags rather than
a reference to the atomic object, now that
they are read-only.

Also, a few Mobile-specific cases were folded
either with other Mobile-specific sections, or
they were now identical to the non-Mobile case
and therefore deduplicated, making the code
cleaner and more readable.

Change-Id: Icc852aa43e86695d4e7d5962040a9b5086d9d08c
Reviewed-on: https://gerrit.libreoffice.org/81978
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2019-11-06 03:43:45 +01:00
Ashod Nakashian
d038ceb118 test: Improve iso8601ToTimestamp and tests
Using double caused all sorts of rounding issues,
especially with random unit-test failures.
Luckily, we don't need doubles and can do everything
with integers.

Also added a new function to print time_point as
iso8601 string, for logging and convenience.

Change-Id: I1c2040c02d1143282dbde0dadef32613b77c330d
Reviewed-on: https://gerrit.libreoffice.org/81578
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2019-10-29 02:39:34 +01:00
Ashod Nakashian
56b49b5000 wsd: warning logging on the console is now magenta for visibility
Change-Id: I8756a0c38d529495826131a3d2dcd1ab70d8c289
Reviewed-on: https://gerrit.libreoffice.org/81337
Reviewed-by: Andras Timar <andras.timar@collabora.com>
Tested-by: Andras Timar <andras.timar@collabora.com>
(cherry picked from commit 7c325c0fb32951dfc8786df98d72e72c92443088)
Reviewed-on: https://gerrit.libreoffice.org/81577
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2019-10-29 02:36:08 +01:00
Ashod Nakashian
d110f4cc58 test: improve TileCacheTests
Sometimes core renderes with sub-pixel differences
(the crosshair at the corners of the Writer pages
show line anti-aliasing differences). This causes
failure of the tests that count the tile deduplication.

We now tolerate when we get an unchanged tile twice,
assuming it was due to such a rendering difference,
but we re-trigger another change and this time we
don't expect any extra tiles, no more than two
variations of the anti-aliased crosshair was
observed.

We also move some duplicate code into utility
functions to improve readability and reuse.

Reviewed-on: https://gerrit.libreoffice.org/81196
Reviewed-by: Andras Timar <andras.timar@collabora.com>
Tested-by: Andras Timar <andras.timar@collabora.com>
(cherry picked from commit 9603597fd1aaecb27893792cfd2d243e450b58b8)

Change-Id: I1a66732dd3443bfbd770d8dc65721571dfa08615
Reviewed-on: https://gerrit.libreoffice.org/81572
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2019-10-29 02:32:05 +01:00
Ashod Nakashian
0b9bc00817 wsd: dynamic child timeout
The initial child spawning takes significantly longer
than subsequent ones (for obvious reasons) and this
lead to unit-tests being sensitive to the timeout
we use for child spawning. Too short, and we
spawn more than we want on startup, too long
and crash-recovery tests fail (we don't
recover fast enough, as we wait too long before
spawning new children).

Dynamically setting the timeout allows us to give
longer timeout at startup, and reduce it afterwards.

Reviewed-on: https://gerrit.libreoffice.org/81194
Reviewed-by: Andras Timar <andras.timar@collabora.com>
Tested-by: Andras Timar <andras.timar@collabora.com>
(cherry picked from commit 32fa1d95fc2ec65866d0cb47d619885182db7040)

Change-Id: I8423f5c6619e57030ab43d519aaa41d8712c36d3
Reviewed-on: https://gerrit.libreoffice.org/81570
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2019-10-29 02:30:07 +01:00
Ashod Nakashian
a4a4d65632 wsd: logging improvements
Always log when we set the termination flag
so we can trace how (and implicitly why) we
terminated. In practice trace logging is not
enabled, so a key event such as termination
should be logged at info level.

Reviewed-on: https://gerrit.libreoffice.org/80324
Reviewed-by: Andras Timar <andras.timar@collabora.com>
Tested-by: Andras Timar <andras.timar@collabora.com>
(cherry picked from commit 3f8d516e1399df687c2935fc551b171cbd850b7b)

Change-Id: Id6615181c81ea56777f44b551b39925065b0e578
Reviewed-on: https://gerrit.libreoffice.org/81562
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Michael Meeks <michael.meeks@collabora.com>
2019-10-28 10:48:01 +01:00
Ashod Nakashian
ecdd15d5a9 wsd: allow longer timeout for child forking
We need to give the child more time than we
poll, so we would get a chance to register
prespawned children before we account for
any missing ones. Also, allowing only 5
seconds risks spawning too many children
on a slow, or somewhat loaded system.

This raises the child timeout to twice
the poll timeout, which is 5 seconds,
allowing for 10 seconds for spawning.

Change-Id: I4cc9e2fc656268fa82a935a5cb1626540bd49980
Reviewed-on: https://gerrit.libreoffice.org/80322
Reviewed-by: Andras Timar <andras.timar@collabora.com>
Tested-by: Andras Timar <andras.timar@collabora.com>
(cherry picked from commit afb96fcbad4e36ef4eb07bf6d0b67329b714858e)
Reviewed-on: https://gerrit.libreoffice.org/81560
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Michael Meeks <michael.meeks@collabora.com>
2019-10-28 10:46:44 +01:00
Ashod Nakashian
5dc8e815c4 wsd: logging: color trace differently to improve readability
Change-Id: I5127061f4d62122d07bad32d4757569f1a9cc877
Reviewed-on: https://gerrit.libreoffice.org/80321
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
(cherry picked from commit 48387b00077b9baa72508c05e3fc3228a4569998)
Reviewed-on: https://gerrit.libreoffice.org/81559
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Michael Meeks <michael.meeks@collabora.com>
2019-10-28 10:45:31 +01:00
mert
86489d3dcd Include normalizedViewId to TileCache
Change-Id: Ib23afa023d79189f7fd7aca8b5b0e198c3011fbc
2019-10-15 18:13:03 +03:00
Ashod Nakashian
77e7cae689 wsd: fix password-protect file loading from multiple views
With password-protected files, the first loading attempt
always fails due to missing password. At that point the
client is notified of the missing password and the user
is prompted. The second attempt includes a (hopefully)
correct password and the document loading commences.

Due to the fact that an exception is raised when
the loading fails, this left the loading latch
triggered, which blocked subsequent attempts.

Change-Id: I7cc257a36eb1cc080f460aac8cdb7030783a5914
2019-10-14 21:35:27 +01:00
Michael Meeks
1b7d962607 Avoid lots of re-sizing of token array while parsing.
Change-Id: I009d1203838fa2d585bd415836fc6338f47e2be3
2019-10-08 17:58:57 +01:00
Andras Timar
d1ae1b36c2 typo fixes in comments and code
Change-Id: Idd98516d30d98dea18eda9bbec8ac9777063b553
2019-10-08 11:49:45 +02:00
DarkByt31
dd014e7029 tdf#107038 Poco::Timestamp replacement with std::chrono
Util added getHttpTime
WhiteBoxTests added test for getHttpTime

Change-Id: Ifb6a3fb2dc9b059b925e7b881362b72759a8b56b
Reviewed-on: https://gerrit.libreoffice.org/79754
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Michael Meeks <michael.meeks@collabora.com>
2019-09-28 12:26:32 +02:00
Ashod Nakashian
f7bd2c7e1d wsd: fix variable might be clobbered by ‘longjmp’ or ‘vfork’
As the comment details, this avoids having C++ objects
in the same frame as setjmp, which may reset their
contents without the dtor getting called.

Change-Id: I851ae8bffb4356d465a25dfc815a1fecb489fa30
Reviewed-on: https://gerrit.libreoffice.org/79338
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2019-09-22 20:24:36 +02:00