Commit graph

125 commits

Author SHA1 Message Date
Aron Budea
c4975d01e5 Split config.h into separate headers to avoid extensive rebuild
Eg. if only the version was changed, config.h was updated, and
everything was recompiled.

New structure, maintain these manually:
config.h.in - for configured variables that are stable if
              configuration parameters are unchanged
config_version.h.in - for version/hash related variables

----
config_unused.h.in - still generated by autoheader with all
                     variables, don't use it

Signed-off-by: Aron Budea <aron.budea@collabora.com>
Change-Id: Id9a50a9f1e798a3b3814778d8683b7d7cb57bb29
2022-09-05 12:32:49 +01:00
Ashod Nakashian
5c6516e4e4 wsd: support code-coverage report via --with-coverage
This adds support for code-coverage HTML reporting.
To achieve this, we must use file-linking in jails
so that we can update the coverage data (.gcda files)
from the jails. This means that creating jails is
slower than with bind-mounting and we need to
account for that in our timeouts.

We also can't kill child processes with SIGKILL,
which is un-catchable. Instead, we use SIGTERM
and dump the profile data before exiting.

Change-Id: I16fa534f6ed42f7133014d841bb024423315e0a4
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2022-08-02 14:45:12 -04:00
Ashod Nakashian
4e9b836ea6 wsd: always use Util::forcedExit to properly cleanup
We need to flush and shutdown logging before exiting,
so we can't use _exit/_Exit directly.

In addition, with profiling (e.g. code-coverage) we
must flush the profile data, lest it's all for naught.

Change-Id: I726c5a2f4e699c17dd0d7d5b1c86d856e0118b3c
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2022-08-02 07:51:52 -04:00
Rash419
45891d0ded wsd: alias: handle a case when regex is added in host tag of alias_group
if host has regex and group has no aliases, when we receive first request from host pattern we considered it as original host and all the host following first request which matches the pattern are considered as aliases
for example:
<group>
	<host>https://.*:80</host>
</group>
if we receives first request from asustuf then behaviour will similar to the following config:
<group>
	<host>https://asustuf:80</host>
	<alias>https://.*:80</alias>
</group>

Signed-off-by: Rash419 <rashesh.padia@collabora.com>
Change-Id: I70fb91a4bb7bf38ed79db9efd9fe4bc46db325e1
2022-06-15 11:04:36 +05:30
Michael Meeks
92d3435dd1 Admin: improve smaps parsing performance.
We have around ~8k smaps lines for each process, polled regularly.

Change-Id: I7e78b17e52aeaec55fa17c5192a940aa9e44b045
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
2022-05-26 19:53:11 -04:00
Ashod Nakashian
480fb7b931 wsd: move tokenizer helpers into StringVector
In an attempt to reduce the size of Util.{c,h}pp
which has grown to contain all sorts of unrelated
helpers, we move StringVector helpers into
the StringVector.{c,h}pp files.

This makes the code better organized.

Change-Id: I152f341606807ae66253415b951bc9f89b09df57
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2022-04-12 07:39:07 -04:00
Rash419
c2be0b3a48 wsd: add: support to define multiple domain with '|'
fix: in regex, escaping special character fails to find host of the alias

Signed-off-by: Rash419 <rashesh.padia@collabora.com>
Change-Id: I19bfacc5e45af26832cb8c6a7d249e0c7de56624
2022-04-05 13:47:57 +02:00
Rash419
31d8822758 wsd: cleanup: moved all parsing related static methods and containers to HostUtil.cpp
Signed-off-by: Rash419 <rashesh.padia@collabora.com>
Change-Id: I8555522c6216f893b90ba4c27747314830f7abd2
2022-03-29 12:18:31 +02:00
Michael Meeks
9f5c5dcdf4 Show (E) for experimental mode in help->about if present.
Change-Id: I00d6cb861c3050d5b4f80b0cb41e70012d0fe610
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
2022-03-24 17:25:40 +00:00
Rash419
ea35dbe350 wsd: added regex support for alias tag
Signed-off-by: Rash419 <rashesh.padia@collabora.com>
Change-Id: I2534ea9afa7d3d3bf10f91eeac44f70381acc76a
2022-03-22 14:13:22 +01:00
Ashod Nakashian
832e308e8f wsd: more informative timestamps in dumpState
The timestamps now helpfully include how
long ago they were.

This also improves the conversion between
chrono clocks by adding generic functions.

Change-Id: I18081330564c8d969b9364cf1b9e33546e241280
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2022-03-09 19:24:40 -05:00
Ashod Nakashian
08702df63f wsd: add getSystemClockAsString helper
Change-Id: If6cfa07b48d0d08060987f831fa926f5cf9214a7
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2022-02-17 18:52:48 -05:00
Ashod Nakashian
381d84ea8d wsd: getSteadyClockAsString now includes millis
Change-Id: I1c1e15c5271cd2d9346eaf3a29d11693020e419b
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2022-02-04 10:26:49 -05:00
Yunusemre Şentürk
d15c0a6f0f use gmtime_r instead of std::gmtime for thread safe purposes
Change-Id: I3490c1353d9e2eebc3dc856fe5bd4767df4b1bec
Signed-off-by: Yunusemre Şentürk <yunusemre@collabora.com>
2022-01-21 16:51:46 +01:00
Andras Timar
f07ff8c7e0 rename: remaining lool->cool changes
Signed-off-by: Andras Timar <andras.timar@collabora.com>
Change-Id: Ib7d4e804bebe52dead8d53b0e0bbaed0f08bf3d0
2021-11-18 14:14:11 +01:00
Tor Lillqvist
09942e76c5 Strip shell string syntax quotes from the PRETTY_NAME line in /etc/os-release
Signed-off-by: Tor Lillqvist <tml@collabora.com>
Change-Id: I5cb6f294d6d4bf62c2e16fc88bc6195695595ba5
2021-11-16 13:33:26 +02:00
Ashod Nakashian
31913d56c2 wsd: avoid misbehaving library cleanup
We cleanup all the relevant objects and instances
before exiting the wsd process, however some libraries
may still misbehave and deadlock during this final
cleanup stage. Notably, Poco has been observed to
deadlock during cleaning up the SSL context static
instance (used for the singleton) by waiting on
its internal mutex indefinitely.

Here we exit forcefully after cleaning up.

Change-Id: I76621c1de7f9a4ff918624b3019b9869c08787c2
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
(cherry picked from commit f776d87cf595bab084e53202777251a38e482147)
2021-10-06 12:28:06 -04:00
Ashod Nakashian
28687c0bbf wsd: hexifying helpers and tests
Change-Id: I2f7fa4e6ade3361db612dbb9aa68e4d28b3905d9
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
(cherry picked from commit 348ca073abe2c980f746324589bc58bf5550dc4b)
2021-09-13 09:40:09 -04:00
Michael Meeks
85ae2d668a Get argument count right to spawn.
Why pass a dozen empty arguments each time: someone bitten
again by the default vector constructor.

Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
Change-Id: I31bc6f23f0b5d7352b85d3df4b7b5298199044a1
2021-09-01 07:14:44 +01:00
Tor Lillqvist
4c68a97ce5 Re-factor Trace Event stuff for improved functionality and thread safety
Write to the Trace Event JSON file in just one place, protected by a
mutex.

Handle metadata events separately. We want them to propagate to the
output if Event Trace generation is just enabled, regardless whether
it has been turned on or not.

We can send the metadata for the Kit process only after we have opened
the docuemnt as sending a Trace Event to WSD uses the (singleton)
Document object in Kit.

Introduce some new functions for the above and adapt callers
appropriately.

Generate a Complete event for the handling of one image tile, from
setting the src attribute of the Image object to finishing the onload
function.

Signed-off-by: Tor Lillqvist <tml@collabora.com>
Change-Id: I5978e0fb2495e0d2dc479bf861eb4ac7da03fa50
2021-06-21 12:43:17 +03:00
Tor Lillqvist
d6569159b7 Output a Trace Event metadata event identifying each named thread
Move the generic dummy implementation of
TraceEvent::emitOneRecording() to a source file of its own. (That is
the one which is used in test and tool executables.)

Signed-off-by: Tor Lillqvist <tml@collabora.com>
Change-Id: I81cab07e5a6852b42d278a5446c13c3825cf546e
2021-06-21 12:43:17 +03:00
Tor Lillqvist
73943230d2 Fix build for iOS
Change-Id: I774e8802d959eefecd9863edd693f26f0fd1d863
Signed-off-by: Tor Lillqvist <tml@collabora.com>
2021-05-19 13:33:26 +03:00
Tor Lillqvist
d5ded73b11 Fix FreeBSD compilation error
Change-Id: I0201c8213c9593f225c0aab7c4e95b06bd6a22ec
Signed-off-by: Tor Lillqvist <tml@collabora.com>
2021-05-19 13:33:26 +03:00
Miklos Vajna
a5566baa03 Add version of std::atoi() that needs no extra allocation
std::atoi() assumes a null-terminated string and our strings are not
always null-terminated. So add a version that takes a length parameter,
this way we don't have to copy strings around.

Also switch to this in http::StatusLine::parse().

Signed-off-by: Miklos Vajna <vmiklos@collabora.com>
Change-Id: I449b356c1b9948c562434618596e8e3b38656088
2021-04-26 16:22:27 +02:00
Ashod Nakashian
42f7796e68 wsd: prevent poco from inserting line breaks in hex bytes
An anti-feature, Poco's HexBinaryEncoder inserts
gratuitous line breaks (0x0A) every 32 bytes,
which is neither expected nor necessary.
No idea what use line breaks could have in a
hex encoder (unless it assumes it's only
possible use is to dump data for investigation).

We hadn't observed this because we generate
random hex strings of 8 to 16 bytes long.
But having used it for random URLs that
are up to 1024 bytes long, I started getting
invalid URLs.

Another reason to hasten the removal of Poco,
especially when we have our hex converter
anyway (not that it would matter if we didn't).

Change-Id: Ib674e8ed607db1effef476f1f3478da76c4f6464
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-03-22 10:09:22 -04:00
Ashod Nakashian
5371715a52 wsd: logging improvements
Change-Id: Ic887c42f0ca844e54bfc054e9c0706bbec7ebb6f
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-03-01 21:50:18 -05:00
Ashod Nakashian
f7d61ae0d3 wsd: remove single-use member
And a couple of const cases.
The removed const is to allow move on return.

Change-Id: I7a81b531e75c39379871f5ffeb82d49ba1110ab1
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-02-09 10:45:37 -05:00
Ashod Nakashian
01138d7ae0 wsd: save errno before invoking any other functions
Most C and Posix API clobber errno. By failing to save
it immediately after invoking an API we risk simply
reporting the result of an arbitrary subsequent API call.

This adds LOG_SYS_ERRNO to take errno explicitly.
This is necessary because sometimes logging is not done
immediately after calling the function for which we
want to report errno. Similarly, log macros that log
errno need to save errno before calling any functions.
This is necessary as the argements might contain calls
that clobber errno.

This also converts some LOG_SYS entries to LOG_ERR
because there can be no relevant errno in that context
(f.e. in a catch clause).

A couple of LOG_ macros have been folded into others,
reducing redundancy.

Finally, both of these log macros append errno to the
log message, so there is little point in ending the
messages with a period.

Change-Id: Iecc656f67115fec78b65cad4e7c17a17623ecf43
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2020-12-29 11:40:17 +00:00
Miklos Vajna
10c1885a83 fuzzer-clientsession
The fuzzer ran out of memory, 955443527 bytes (79%) of the used memory
was this map.

Change-Id: I2dd84a094d3dd3d98618667e3c78591e2193bce2
Signed-off-by: Miklos Vajna <vmiklos@collabora.com>
2020-11-30 10:17:43 +01:00
Ashod Nakashian
3724ab4e35 wsd: centralize temporary directory helpers in FileUtil
Change-Id: I9b55d0b8d4c55a68af4baa7e638627de6c73b64d
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2020-11-22 22:26:36 -05:00
Gleb Popov
f7faaad790 Replace all __linux in CPP conditions with __linux__.
Signed-off-by: Gleb Popov <6yearold@gmail.com>
Change-Id: If3e213b1cf1f3c4ab960276fc6edfb71f9416420
2020-11-17 09:21:45 +00:00
Gleb Popov
aa97476f07 Util.cpp: Do not call setpriority for thread handles on non-Linux systems.
Change-Id: I50cdc429cd606e261c70219e51587a54a269b082
Signed-off-by: Gleb Popov <6yearold@gmail.com>
2020-11-16 14:02:23 +00:00
Ashod Nakashian
cb4beaca34 wsd: avoid the using keyword and use C++ size_t
size_t in C and in C++ are not necessarily the same
type. The C++ size_t is in the std namespace. Since
we do include many C headers, and indeed some C++
runtime headers do define size_t for backwards
compatibility, it's easy to mix and match the two
types.

Also, 'using std::size_t;' isn't a great practice,
so removed.

This is not exhaustive, just some low-hanging cases.

Change-Id: I85a36b6fd1acd204274b1869de9bcb94c8b3cf13
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2020-11-15 15:41:41 -05:00
Ashod Nakashian
2ddc1afb69 wsd: resuse Stat where possible
This replaces Util::getFileTimestamp with
FileUtil::Stat::modifiedTimepoint() and fixes a potential bug:
getFileTimestamp had only 1 second precision (it simply dropped
sub-second data). This could mean that any modifications to a file
within a second could not be detected.

Minor simplifications done where possible and overly long lines
have been reformatted.

This is a non-functional change (except that file modified-time
now supports microsecond precision).

Change-Id: I3606638a86fc3e00c0ad5cb602bdbb2b4651867b
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2020-11-15 13:50:16 -05:00
Ayhan Yalçınsoy
c1b8a64fce Remove Poco/TimeStamp.h left-over
Change-Id: I78f76266a50f799306dcfd8ec996db54784acfd8
Signed-off-by: Ayhan Yalçınsoy <ayhanyalcinsoy@pisilinux.org>
2020-10-26 03:30:32 +03:00
Pranam Lashkari
a992cfcd23 wsd: correction in conversion from timestamp to ISO8601
closes #168

Change-Id: Ibdb1aa72a6961d4707c2336382cc593c984d3615
2020-10-16 04:27:53 +05:30
Andrea Gelmini
b2360fec3a Removed duplicated include
Change-Id: Id6ba6e769b3c0ec5493ee32e2abbc00e441388ec
2020-10-07 12:48:04 +02:00
Andras Timar
0002fdfd6c fix license headers
Change-Id: I8623770b32d278a45357dc7f757fabfadd2b4af7
2020-10-01 11:56:43 +02:00
Gülşah Köse
845554a6a3 Revert "wsd: parse headers with Poco::MessageHeader"
This reverts commit dbc562d9ab.

tst-05694-05694 2020-08-26 12:59:14.343136 [ unittest ]
ERR Invalid HTTP header [def]: Malformed message:
Field name too long/no colon found| ../common/Util.cpp:980

Following part of the code tests a request with corrupted http header:
    Authorization auth2(Authorization::Type::Header, "def");
    Poco::Net::HTTPRequest req2;
    auth2.authorizeRequest(req2);
    LOK_ASSERT(!req2.has("Authorization"));

Poco library throws exception.

Change-Id: Ic31a80c0e1e325de27c23059e2bcb3f00d39ad16
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/101887
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Gülşah Köse <gulsah.kose@collabora.com>
2020-09-01 23:17:03 +02:00
Ashod Nakashian
dbc562d9ab wsd: parse headers with Poco::MessageHeader
Our header parses was overly simplistic and
didn't support a number of corner cases that
rfc2616 specifies (folding, for example). The
new approach is to simply normalize the headers by
removing invalid line-breaks and then let the
MessageHeader parser take care of parsing the
headers individually, which we then set on the request.

The new utility setHttpHeaders should be used
whenever we need to set a header in an request
to make sure it are sanitized and valid.

Change-Id: Ifa16fa9364f42183316749276c5d0a4c556cb740
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/96371
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ash@collabora.com>
2020-08-11 20:04:20 +02:00
Tor Lillqvist
d0672cd2b9 Slight refactoring to make planned re-plumbing of iOS app easier
Change-Id: I274cf167c6593de6f073301f7071f2173b40cbab
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/97760
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Tested-by: Jenkins
Reviewed-by: Tor Lillqvist <tml@collabora.com>
2020-07-03 08:41:02 +02:00
Ashod Nakashian
5c9988f2e3 wsd: faster jail setup via bind-mount
loolmount now works and supports mounting and
unmounting, plus numerous improvements,
refactoring, logging, etc..  When enabled,
binding improves the jail setup time by anywhere
from 2x to orders of magnitude (in docker, f.e.).

A new config entry mount_jail_tree controls
whether mounting is used or the old method of
linking/copying of jail contents. It is set to
true by default and falls back to linking/copying.
A test mount is done when the setting is enabled,
and if mounting fails, it's disabled to avoid noise.

Temporarily disabled for unit-tests until we can
cleanup lingering mounts after Jenkins aborts our
build job. In a future patch we will have mount/jail
cleanup as part of make.

The network/system files in /etc that need frequent
refreshing are now updated in systemplate to make
their most recent version available in the jails.
These files can change during the course of loolwsd
lifetime, and are unlikely to be updated in
systemplate after installation at all. We link to
them in the systemplate/etc directory, and if that
fails, we copy them before forking each kit
instance to have the latest.

This reworks the approach used to bind-mount the
jails and the templates such that the total is
now down to only three mounts: systemplate, lo, tmp.

As now systemplate and lotemplate are shared, they
must be mounted as readonly, this means that user/
must now be moved into tmp/user/ which is writable.

The mount-points must be recursive, because we mount
lo/ within the mount-point of systemplate (which is
the root of the jail). But because we (re)bind
recursively, and because both systemplate and
lotemplate are mounted for each jails, we need to
make them unbindable, so they wouldn't multiply the
mount-points for each jails (an explosive growth!)
Contrarywise, we don't want the mount-points to
be shared, because we don't expect to add/remove
mounts after a jail is created.

The random temp directory is now created and set
correctly, plus many logging and other improvements.

Change-Id: Iae3fda5e876cf47d2cae6669a87b5b826a8748df
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92829
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
2020-07-01 05:42:43 +02:00
Jan Holesovsky
2c246eed85 Sanitize the access_header.
The access_header can contain a lot of nonsense, like whitespace around
or additional \n's or \r's.  We used to sanitize that, but then
regressed in e95413d151 where the
"tokenize by any of \n\r" was by mistake replaced with "tokenize by
string '\n\r'".

Unfortunately the unit test didn't uncover that, and the further
refactorings of the related code have hidden that even more.

Change-Id: Ie2bf950d0426292770b599e40ee2401101162ff2
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/96638
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Andras Timar <andras.timar@collabora.com>
2020-06-19 11:48:44 +02:00
Ashod Nakashian
223a1d08f0 wsd: performance-unnecessary-value-param
Change-Id: I1eb092c676da8600e0f8ed70cbc7e1f37fdd5a02
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/95338
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Tested-by: Jenkins
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
2020-06-02 20:01:29 +02:00
Ashod Nakashian
224ef08c7f wsd: single-char string literals -> char
More readable and typically more efficient.

Change-Id: I9bd5bfc91f4ac255bb8ae0987708fb8b56b398f8
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/95285
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
2020-06-02 01:31:26 +02:00
Michael Meeks
92eff552a5 Lower convert-to process priorities by default.
Interactive / editing processes should take precedence over batch
thumbnailing processes to keep responsiveness good.

Change-Id: Ib100409e312cb2ca545586a734711a31a92f110c
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/93323
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-05-02 22:03:36 +02:00
Pranam Lashkari
8b349716eb killpoco: removed Poco::Process completely
Change-Id: Iba67abf9342c11517c69c1d94903bf4752aa87d6
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/83770
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
2020-04-29 14:27:54 +02:00
Michael Meeks
3e4ac31c7c killpoco: remove WebSocket includes from a couple of places.
Change-Id: I06740cd978bec8e6a74beb8ed9ef8f4f970a2535
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92470
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-04-18 18:29:04 +02:00
Miklos Vajna
61790f6bb5 common: it's enough to look up PRETTY_NAME once
Also add the missing newlines before the opening braces, the surrounding
code has that style.

Change-Id: I23bd26ba6d2446858ae3213212e2813a38fd1d46
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92146
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
2020-04-14 13:41:55 +02:00
gokaysatir
9e0594ee2d add pretty name - Android patch
Change-Id: If0de49884954cde26bfbe8ba1dce8844af5b30bf
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/91831
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-04-11 13:43:31 +02:00