This is to defend the sneaking of extra http-headers
in the access_header URI param that was recently fixed.
Change-Id: Ic28cf58854847ac278bed8043f398b107f7992b3
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/96862
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Authorization class now handles the parsing and creation
of its instances, which makes it centralized.
We also avoid repeatedly constructing Authorization objects
in ClientSession and instead do it once at construction
and cache it.
A bunch of new unit-tests added.
Change-Id: I9b5939be51a5957214d07ed8f1096efd179686c6
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/96825
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Leaving behind jails with bind-mount
entries makes build-workspace removal
complicated, and jenkins builds start failing.
The cleanup stage is integrated in Makefiles
and should be transparent.
In the event that manual cleanup is necessary,
'loolwsd --cleanup' can be invoked.
Change-Id: Ia4b99b0c66e56dfa2d50e79b0ba98f714cf32886
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/97470
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
loolmount now works and supports mounting and
unmounting, plus numerous improvements,
refactoring, logging, etc.. When enabled,
binding improves the jail setup time by anywhere
from 2x to orders of magnitude (in docker, f.e.).
A new config entry mount_jail_tree controls
whether mounting is used or the old method of
linking/copying of jail contents. It is set to
true by default and falls back to linking/copying.
A test mount is done when the setting is enabled,
and if mounting fails, it's disabled to avoid noise.
Temporarily disabled for unit-tests until we can
cleanup lingering mounts after Jenkins aborts our
build job. In a future patch we will have mount/jail
cleanup as part of make.
The network/system files in /etc that need frequent
refreshing are now updated in systemplate to make
their most recent version available in the jails.
These files can change during the course of loolwsd
lifetime, and are unlikely to be updated in
systemplate after installation at all. We link to
them in the systemplate/etc directory, and if that
fails, we copy them before forking each kit
instance to have the latest.
This reworks the approach used to bind-mount the
jails and the templates such that the total is
now down to only three mounts: systemplate, lo, tmp.
As now systemplate and lotemplate are shared, they
must be mounted as readonly, this means that user/
must now be moved into tmp/user/ which is writable.
The mount-points must be recursive, because we mount
lo/ within the mount-point of systemplate (which is
the root of the jail). But because we (re)bind
recursively, and because both systemplate and
lotemplate are mounted for each jails, we need to
make them unbindable, so they wouldn't multiply the
mount-points for each jails (an explosive growth!)
Contrarywise, we don't want the mount-points to
be shared, because we don't expect to add/remove
mounts after a jail is created.
The random temp directory is now created and set
correctly, plus many logging and other improvements.
Change-Id: Iae3fda5e876cf47d2cae6669a87b5b826a8748df
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92829
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Seems to not cause any serious regressions in the iOS app or in "make
run", but of course I am not able to run a comprehensive check of all
functionality.
Change-Id: I44a0e8d60bdbc0a885db88475961575c5e95ce88
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/93037
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
The access_header can contain a lot of nonsense, like whitespace around
or additional \n's or \r's. We used to sanitize that, but then
regressed in e95413d151 where the
"tokenize by any of \n\r" was by mistake replaced with "tokenize by
string '\n\r'".
Unfortunately the unit test didn't uncover that, and the further
refactorings of the related code have hidden that even more.
Change-Id: Ie2bf950d0426292770b599e40ee2401101162ff2
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/96638
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Andras Timar <andras.timar@collabora.com>
We used to just timeout when there was no match due to a missing
assignment.
This also shows that some of the tests don't pass currently, so for now
blacklist what's not working.
Change-Id: I777f5acffc83349d7cc0e92af99059495610f612
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/96569
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
re-factor to make it hard not to.
Change-Id: I26ebc48b4660276ede64a22167ac4779cebf5cd4
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/95440
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
...with support for properly extracting the different
fields with unit-test.
URIs are quite complex and varied. For historic reasons
they have all been treated without distinction, which
makes support for all variants difficult. RequestDetails
encapsulates this complexity, and now it is almost
completely documented both descriptively and functionally
(via extensive unit-tests).
Parsing of the URIs is now more structured by having
named fields instead of relying on knowing which
token should contain which field, which is error-prone
and very opaque.
Change-Id: I68d07c2e00baf43f0ade97d20f62691ffb3bf576
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/95292
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
The tokenizer(s) are more generic than the protocol
logic, and are used from contexts that don't involve
the protocol as such.
Change-Id: Ie8c256bf11a91e466bff794021f41603c9596a7f
This avoids depending on LOOLWSD's statics, which
makes adding unit-tests much more difficult due to
the high number of dependencies LOOLWSD pulls.
Adds a number of unit-tests for RequestDetails.
Change-Id: I9f1d56f80a633505c7ff548ec0e33ffe61f59f53
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/95290
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Tested-by: Jenkins
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
StringVector is heavily used for tokenization
and benefits from inlining of small functions.
Also, cat doesn't need to be slower than necessary.
Change-Id: I4ab2ff1b1f1a81092049d2cde64b6df10b34b5f7
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/95287
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Jenkins
More readable and typically more efficient.
Change-Id: I9bd5bfc91f4ac255bb8ae0987708fb8b56b398f8
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/95285
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
This is the new default - do as we're told by the client.
The old setting is left to allow users to force SSL if they are
concerned that they may receive unhelpful URLs.
Change-Id: Idea83aacea6826a8f37264e34d49c7550efe6d27
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/93179
Tested-by: Michael Meeks <michael.meeks@collabora.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
It is not a problem in the multi-process web-based Online, where the
variable exists separately in each KIT process (which handles exactly
one document). But in a mobile app, when we want to be able to handle
multiple document in the single process, we can't have such variables.
Change-Id: I1d3da48316eb3a8c72ff4957cc3fcba8f6870f16
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92582
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
LibreOffice core uses that, too, and we support an even more
restricted set of compilers.
Change-Id: I0d0e2c8608e323eb5ef0f35ee8c46d02ab49a745
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92467
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
The switch away from LOOLWebSocket and the use of a websocket
for talking to forkit removes the need for the pipe code.
Change-Id: Ifb0c6c88681289e7a1709d9bc3281532935c7be4
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92033
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Also add dummy, run-every-build test to validate wopi like proofs.
Change-Id: Ic2dc647a8d61693ae87b6523aaa30632979fd5d6
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/91854
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Not deleting each other's jails mails all the difference.
For now parallelize in smaller groups; some of the tests are
quite heavy resource-wise.
Change-Id: I04e5cdf66ba20cbd4f9f05b9d630141814eb3801
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90982
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
This test sometimes failed on me just because the load was aborted due
to the timeout, doubling the limit makes the problem go away.
Change-Id: Ided724d67a513391b86836065d4e627581857a92
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90771
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
Because the new-style tests are intrustive,
the exception that CppUnit throws on assertion
failures is caught and processed with the
application logic, which is far from ideal,
because it's very difficult to find the
cause of failure.
What we'd like is a way to control what happens
when an test assertion fails, such that we can
properly log/print the failure, and even break
in the debugger.
The new macros allow us to control the behavior
at compile-time and have added flexibility.
For now, they log an assertion failure before
invoking the CPPUNIT macro, and support a
compile-time directive to assert, which is
useful for breaking in the debugger.
Change-Id: If464ba246e3ec747f31496a4215cb73ef735dfaf
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/87625
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Don't check the full URL, but just test some parts of it.
Change-Id: I5367bf4f41dc26f311e03de7ce06349f744d0b85
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90428
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
Essentially we want to be able to separate low-level socket code
for eg. TCP vs. UDS, from Protocol handling: eg. WebSocketHandler
and client sessions themselves which handle and send messages
which now implement the simple MessageHandlerInterface.
Some helpful renaming too:
s/SocketHandlerInterface/ProtocolHandlerInterface/
Change-Id: I58092b5e0b5792fda47498fb2c875851eada461d
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90138
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Allows comparing tokens with C strings without a heap allocation. Do the
same when comparing two tokens from two different StringVectors.
And use it at all places where operator ==() has an argument, which is a
StringVector::operator []() result.
Change-Id: Id36eff96767ab99b235ecbd12fb14446a3efa869
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90201
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
This is meant to reduce lots of small allocations and instead have
pointers into the single string for the various tokens instead.
This has a few requirements, though:
1) It's no longer OK to modify the tokens, changing their length would
invalidate the start/length of other tokens. Rework
DocumentBroker::load() to avoid such mutation.
2) The iterators no longer expose zero-terminated strings, so
Poco::cat() doesn't work anymore: add an own cat() instead and use that
in e.g. ChildSession. The own cat() has the benefit that it won't read
past the end of the array if the begin index is out of bounds to add
more safety.
(This nicely works towards killing Poco usage in general.)
3) If zero-terminated strings for all individual tokens is needed, a
copy has to be made, as done in spawnProcess().
(For all of these requirements, the build fails if there are problems.)
Change-Id: Iea40e4400e630b2d669f5c72aea85cb40edf9a2c
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89711
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
The bulk of this commit just changes std::vector<std::string> to
StringVector when we deal with tokens from a websocket message.
The less boring part of it is the new StringVector class, which is a
wrapper around std::vector<std::string>, and provides the same API,
except that operator[] returns a string, not a string&, and this allows
returning an empty string in case that prevents reading past the end of
the underlying array.
This means in case client code forgets to check size() before invoking
operator[], we don't crash. (See the ~3 previous commits which fixed
such crashes.)
Later the ctor could be changed to take a single underlying string to
avoid lots of tiny allocations, that's not yet done in this commit.
Change-Id: I8a6082143a8ac0b65824f574b32104d7889c184f
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89687
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
Now that we don't re-compress PNG images since core.git commit
8fc1b60f62c213a0476f3acc9f89cd5eccbf335d (sw SVG export: try to reuse
original bitmap data for JPG and PNG bitmaps, 2020-02-27).
Change-Id: I869ae0a21f834e6f05bf9b897b15cf3392320cd8
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89621
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
