Use hash to identify download and pass that to the client.
This allows us to reduce parameters for download requests.
DocBroker maps download ids to URL in the file system.
Change-Id: I254d4f0ccaf3cff9f038a817c8162510ae228bc5
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/101992
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Tested-by: Michael Meeks <michael.meeks@collabora.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Our header parses was overly simplistic and
didn't support a number of corner cases that
rfc2616 specifies (folding, for example). The
new approach is to simply normalize the headers by
removing invalid line-breaks and then let the
MessageHeader parser take care of parsing the
headers individually, which we then set on the request.
The new utility setHttpHeaders should be used
whenever we need to set a header in an request
to make sure it are sanitized and valid.
Change-Id: Ifa16fa9364f42183316749276c5d0a4c556cb740
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/96371
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ash@collabora.com>
This yields a relevant Post request of:
slideshow.svg%3FWOPISrc=https:%252F%252Fshare....
slideshow.svg?WOPISrc=https%3A%2F%2Fshare....
Which ends up trying to serve a file with a ?WOPISRC= suffix.
Since we don't want anything 'odd' being served from the file-system
anyway; get a bit more paranoid and clean that up.
Change-Id: I39993e4629b0d01f34098391a0fa448ef8a9984e
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/100509
Tested-by: Michael Meeks <michael.meeks@collabora.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
... since these are required in proof-key element as per [MS-WOPI].
Change-Id: Ie770271ee911e3f7822375c00a83c6a32cd5f2fc
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/88743
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Tested-by: Jenkins
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
The asan+ubsan build is slow enough that the old timeout started to
result in an assertion failure, while just increasing the timeout seems
to produce stable (passing) results, so do that.
An ever-hanging process is still caught after this.
Change-Id: I53f2c65f9a11da8423f7ff89a4b773fc2b0b1fb5
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/99269
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
* Excised TileCacheDesc to improve performance and simplify code.
* clang-tidy suggestions and auto-rewrite fixes.
* Const-correctness.
* Inlined and improved a couple of trivial functions (that are called
often).
* Reduced some logs from INF to DBG as they are only meaningful to devs.
Change-Id: I1c4eb8c63da49aa061afbf3eb68cae23d4d5e7f3
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/98661
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Tested-by: Jenkins
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
This is to defend the sneaking of extra http-headers
in the access_header URI param that was recently fixed.
Change-Id: Ic28cf58854847ac278bed8043f398b107f7992b3
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/96862
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Authorization class now handles the parsing and creation
of its instances, which makes it centralized.
We also avoid repeatedly constructing Authorization objects
in ClientSession and instead do it once at construction
and cache it.
A bunch of new unit-tests added.
Change-Id: I9b5939be51a5957214d07ed8f1096efd179686c6
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/96825
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Leaving behind jails with bind-mount
entries makes build-workspace removal
complicated, and jenkins builds start failing.
The cleanup stage is integrated in Makefiles
and should be transparent.
In the event that manual cleanup is necessary,
'loolwsd --cleanup' can be invoked.
Change-Id: Ia4b99b0c66e56dfa2d50e79b0ba98f714cf32886
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/97470
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
loolmount now works and supports mounting and
unmounting, plus numerous improvements,
refactoring, logging, etc.. When enabled,
binding improves the jail setup time by anywhere
from 2x to orders of magnitude (in docker, f.e.).
A new config entry mount_jail_tree controls
whether mounting is used or the old method of
linking/copying of jail contents. It is set to
true by default and falls back to linking/copying.
A test mount is done when the setting is enabled,
and if mounting fails, it's disabled to avoid noise.
Temporarily disabled for unit-tests until we can
cleanup lingering mounts after Jenkins aborts our
build job. In a future patch we will have mount/jail
cleanup as part of make.
The network/system files in /etc that need frequent
refreshing are now updated in systemplate to make
their most recent version available in the jails.
These files can change during the course of loolwsd
lifetime, and are unlikely to be updated in
systemplate after installation at all. We link to
them in the systemplate/etc directory, and if that
fails, we copy them before forking each kit
instance to have the latest.
This reworks the approach used to bind-mount the
jails and the templates such that the total is
now down to only three mounts: systemplate, lo, tmp.
As now systemplate and lotemplate are shared, they
must be mounted as readonly, this means that user/
must now be moved into tmp/user/ which is writable.
The mount-points must be recursive, because we mount
lo/ within the mount-point of systemplate (which is
the root of the jail). But because we (re)bind
recursively, and because both systemplate and
lotemplate are mounted for each jails, we need to
make them unbindable, so they wouldn't multiply the
mount-points for each jails (an explosive growth!)
Contrarywise, we don't want the mount-points to
be shared, because we don't expect to add/remove
mounts after a jail is created.
The random temp directory is now created and set
correctly, plus many logging and other improvements.
Change-Id: Iae3fda5e876cf47d2cae6669a87b5b826a8748df
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92829
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Seems to not cause any serious regressions in the iOS app or in "make
run", but of course I am not able to run a comprehensive check of all
functionality.
Change-Id: I44a0e8d60bdbc0a885db88475961575c5e95ce88
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/93037
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
The access_header can contain a lot of nonsense, like whitespace around
or additional \n's or \r's. We used to sanitize that, but then
regressed in e95413d151 where the
"tokenize by any of \n\r" was by mistake replaced with "tokenize by
string '\n\r'".
Unfortunately the unit test didn't uncover that, and the further
refactorings of the related code have hidden that even more.
Change-Id: Ie2bf950d0426292770b599e40ee2401101162ff2
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/96638
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Andras Timar <andras.timar@collabora.com>
We used to just timeout when there was no match due to a missing
assignment.
This also shows that some of the tests don't pass currently, so for now
blacklist what's not working.
Change-Id: I777f5acffc83349d7cc0e92af99059495610f612
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/96569
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
re-factor to make it hard not to.
Change-Id: I26ebc48b4660276ede64a22167ac4779cebf5cd4
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/95440
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
...with support for properly extracting the different
fields with unit-test.
URIs are quite complex and varied. For historic reasons
they have all been treated without distinction, which
makes support for all variants difficult. RequestDetails
encapsulates this complexity, and now it is almost
completely documented both descriptively and functionally
(via extensive unit-tests).
Parsing of the URIs is now more structured by having
named fields instead of relying on knowing which
token should contain which field, which is error-prone
and very opaque.
Change-Id: I68d07c2e00baf43f0ade97d20f62691ffb3bf576
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/95292
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
The tokenizer(s) are more generic than the protocol
logic, and are used from contexts that don't involve
the protocol as such.
Change-Id: Ie8c256bf11a91e466bff794021f41603c9596a7f
This avoids depending on LOOLWSD's statics, which
makes adding unit-tests much more difficult due to
the high number of dependencies LOOLWSD pulls.
Adds a number of unit-tests for RequestDetails.
Change-Id: I9f1d56f80a633505c7ff548ec0e33ffe61f59f53
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/95290
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Tested-by: Jenkins
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
StringVector is heavily used for tokenization
and benefits from inlining of small functions.
Also, cat doesn't need to be slower than necessary.
Change-Id: I4ab2ff1b1f1a81092049d2cde64b6df10b34b5f7
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/95287
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Jenkins
More readable and typically more efficient.
Change-Id: I9bd5bfc91f4ac255bb8ae0987708fb8b56b398f8
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/95285
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
This is the new default - do as we're told by the client.
The old setting is left to allow users to force SSL if they are
concerned that they may receive unhelpful URLs.
Change-Id: Idea83aacea6826a8f37264e34d49c7550efe6d27
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/93179
Tested-by: Michael Meeks <michael.meeks@collabora.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
It is not a problem in the multi-process web-based Online, where the
variable exists separately in each KIT process (which handles exactly
one document). But in a mobile app, when we want to be able to handle
multiple document in the single process, we can't have such variables.
Change-Id: I1d3da48316eb3a8c72ff4957cc3fcba8f6870f16
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92582
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
LibreOffice core uses that, too, and we support an even more
restricted set of compilers.
Change-Id: I0d0e2c8608e323eb5ef0f35ee8c46d02ab49a745
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92467
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
The switch away from LOOLWebSocket and the use of a websocket
for talking to forkit removes the need for the pipe code.
Change-Id: Ifb0c6c88681289e7a1709d9bc3281532935c7be4
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92033
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Also add dummy, run-every-build test to validate wopi like proofs.
Change-Id: Ic2dc647a8d61693ae87b6523aaa30632979fd5d6
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/91854
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Not deleting each other's jails mails all the difference.
For now parallelize in smaller groups; some of the tests are
quite heavy resource-wise.
Change-Id: I04e5cdf66ba20cbd4f9f05b9d630141814eb3801
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90982
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
This test sometimes failed on me just because the load was aborted due
to the timeout, doubling the limit makes the problem go away.
Change-Id: Ided724d67a513391b86836065d4e627581857a92
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90771
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
Because the new-style tests are intrustive,
the exception that CppUnit throws on assertion
failures is caught and processed with the
application logic, which is far from ideal,
because it's very difficult to find the
cause of failure.
What we'd like is a way to control what happens
when an test assertion fails, such that we can
properly log/print the failure, and even break
in the debugger.
The new macros allow us to control the behavior
at compile-time and have added flexibility.
For now, they log an assertion failure before
invoking the CPPUNIT macro, and support a
compile-time directive to assert, which is
useful for breaking in the debugger.
Change-Id: If464ba246e3ec747f31496a4215cb73ef735dfaf
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/87625
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Don't check the full URL, but just test some parts of it.
Change-Id: I5367bf4f41dc26f311e03de7ce06349f744d0b85
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90428
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
