Commit graph

111 commits

Author SHA1 Message Date
Szymon Kłos
27e95bbe7e Wait for async save result on DocumentBroker stop
When async save was started but we want to stop DocumentBroker
- we shouldn't kill the socket which listens to save result.
Because when we kill socket we will destroy DocumentBroker
before saving is completed, this allows for a minimal time
to connect again to the document and receive old content
from the WOPI storage. When we wait for async save result
we will keep DocumentBroker alive and next session can be
connected and receive correct content.

fixes: https://github.com/CollaboraOnline/online/issues/2747

Signed-off-by: Szymon Kłos <szymon.klos@collabora.com>
Change-Id: I5505a5ccc1f3d6928c723bddfea16bf5c5798d1e
2021-07-28 13:00:45 +02:00
Ashod Nakashian
90f0360ada wsd: close the connection after failing to load
Change-Id: I571c9afba3ae2d9be83382dab571902669e8b059
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-07-19 17:31:55 +01:00
Szymon Kłos
27799f23eb HTTP 302 - updates after review
Change-Id: Iaac4384385e05f33472b6678422b4a444319b59c
Signed-off-by: Szymon Kłos <szymon.klos@collabora.com>
2021-07-12 08:44:39 +02:00
Szymon Kłos
fef43e237a HTTP 302 Found - block redirection loops
Change-Id: I686a801a7003d14210a0319d421b326b7f8c107a
Signed-off-by: Szymon Kłos <szymon.klos@collabora.com>
2021-07-12 08:44:39 +02:00
Szymon Kłos
850f3cee71 HTTP 302 Found - remember the correct resource address
remember correct URI so all the following communication
(eg. downloading the file, upload) will be performed on new location.

Signed-off-by: Szymon Kłos <szymon.klos@collabora.com>
Change-Id: Id14c5209eb1d9ffc7c24d7e02f0c7c32a60568b7
2021-07-12 08:44:39 +02:00
Szymon Kłos
f8bea2e9c5 HTTP 302 Found - redirrect support for getWOPIFileInfo
With some loadbalancers it may happen that HTTP 302 Found
response with redirect location will appear.

Change-Id: I7bac807761423adcd827cf775bd1d3d2ba87a6cc
Signed-off-by: Szymon Kłos <szymon.klos@collabora.com>
2021-07-12 08:44:39 +02:00
Ashod Nakashian
f2f713cdaf wsd: rename file after saving to support async uploading
To avoid writing (saving) while reading (uploading) or
any other undesirable racse on the document file,
we atomically rename the file after saving on disk
to pass ownership.

After saving in Core, we rename the document in the
jail to .upload (by appending it). DocumentBroker
looks for the file with that extension and atomically
renames it to .uploading (by appening the 'ing' suffix).
This way, the Kit only renames from the original to
.upload and DocBroker renames only .upload to .uploading.
This guarantees that we never rename the same file
concurrently.

Uploading decision is strictly based on the modified
timestamp of the .uploading file, compared to the
timestamp of the last file we uploaded successfully.

Change-Id: I03520cd8c87605f6dad417e7a978204f76fc0c38
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-06-19 17:44:59 -04:00
Ashod Nakashian
1d0c717d0a wsd: handle async upload callback
Now we can handle the response of the storage
after the async upload is complete (or timed out).

Change-Id: I29d450646bddb07f02bb17d257e7e0fa372ce357
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-06-19 17:44:59 -04:00
Ashod Nakashian
09dd73f6a2 wsd: add async API to Storage
This allows uploading to storage asynchronously.

Change-Id: I83956d8b27cfd24430462b1a6d0a6845bb97b7a5
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-06-19 17:44:59 -04:00
Ashod Nakashian
e8a52959ea wsd: document ModifiedTime getter/setter
Change-Id: I95763c62d5937f0749604ab26d530e85c2ed105e
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-06-13 21:57:43 -04:00
Ashod Nakashian
f5f1a64591 wsd: storage: download with http::Request and http::Session
Change-Id: I11e553fd65faf4d33c51f9b6e7e6e256fd87ba05
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-02-20 16:47:17 -05:00
Ashod Nakashian
c1da9af494 wsd: Loaded -> Downloaded
Disambiguate between document loading
and downloading.

Change-Id: Ia16e580d8239a1aebd9192e01860e2f17cdc3878
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-02-20 16:47:17 -05:00
Ashod Nakashian
d0516b4391 wsd: correct GetFile duration tracking
The download operation itself is done
after reading the response from the Poco
object. Therefore, the time duration that
was captured for the GetFile operation
was inaccurate. Luckily, we only need
to time the download API of the Storage
object, which actually is simpler, and
more accurate.

Change-Id: I05c94a29fa59d5efae215f2daea17672abc6efc7
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-02-18 11:08:38 -05:00
Ashod Nakashian
ce9276f201 wsd: actually compute the CheckFileInfo duration
For some reason, the duration of the CheckFileInfo
request was not set/updated, and therefore it was
always 0. This fixes this oversight and also
moves the duration variable locally, since
the request is synchronous anyway and there is
hardly any reason to store it elsewhere.

Change-Id: I613b24311bc5c8ee9ab98d675057f354990226e8
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-02-18 11:08:38 -05:00
Ashod Nakashian
bafcc98285 wsd: support FileUrl
Change-Id: I433b5f79956be9d8557df1a1f433a99d6ae10f65
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-02-09 10:45:37 -05:00
Ashod Nakashian
ec58e833e8 wsd: refactor downloadStorageFileToLocal
Prepare to support FileUrl.

Change-Id: I0ac4eccb937653ba86e74c2e6cecb7ddd5dd4ffe
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-02-09 10:45:37 -05:00
Ashod Nakashian
f7d61ae0d3 wsd: remove single-use member
And a couple of const cases.
The removed const is to allow move on return.

Change-Id: I7a81b531e75c39379871f5ffeb82d49ba1110ab1
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-02-09 10:45:37 -05:00
Ashod Nakashian
c12313676e wsd: cosmetic
Change-Id: Ie0df45003a0d4221c4e03ad093229cb703e6bf3a
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-01-23 23:55:43 +00:00
Ashod Nakashian
e8a5b4843c wsd: UploadResult cleanup
A minor cleanup of UploadResult to make tidy
it up a little bit and make it less specific.

Single-argument constructors should be explicit
to avoid unexpected conversion and other surprises.

Change-Id: I57599805743dffddac620f501dc6ca79c2217f89
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-01-10 11:13:27 -05:00
Ashod Nakashian
13d7d23cbe wsd: use chrono for readability and conversion safety
Change-Id: If5d478605fef0199b331d91310673a460e70d48e
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2020-12-22 11:24:13 -05:00
Miklos Vajna
8b0ab31864 WopiStorage: fix uninitialized reads
Change-Id: I603a5205f111b7c8f99baecd6ed482fc9502a6d5
Signed-off-by: Miklos Vajna <vmiklos@collabora.com>
2020-12-10 09:35:25 +01:00
Ashod Nakashian
4cd460e239 wsd: avoid chrono::duration<double>
While chrono supports double as a datatype, it
is opaque and doesn't lend itself to any obvious
units of time (presumably seconds). Using
chrono::milliseconds is much more readable and
also safe when converting from seconds or any
other units. Ultimately, we typically convert
to milliseconds anyway, mostly for logging.
There is but one exception where we convert
in seconds, and now that case is documented.

Change-Id: Ide98f45f2ad8da8225d41ae870bbc4bc09a2a0b5
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2020-12-08 09:26:41 +00:00
Ashod Nakashian
0704514730 wsd: label storage operations as upload and download
Using "load" and "save" in the storage was a poor
choice of verbs, in hindsight, because these very
same verbs are also used to describe the loading
and saving of documents in Core.

It is more appropriate to label the storage
operations as download and upload, respectively,
to avoid any confusion. This is especially useful
because when reporting we have for some time now
been reporting the results of each of these
stages separately, there is no longer reason
to label them the same.

We already used "upload" and "download" in
some of the logs, but not all.

Change-Id: I0fac9130032e2c3c6dfb4d671c31130265091f0d
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2020-11-30 18:46:46 -05:00
Ashod Nakashian
06a45e6db9 wsd: extract wopi upload response handling
This is in preparation for asynchronous uploading.

Change-Id: Ibd0ff0fa8edfc08ad2755a45227891ed40e09d1c
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2020-11-30 18:46:46 -05:00
Ashod Nakashian
fad4222a2a wsd: move convert-to docs into the jail
We now download the convert-to files into the
child-root/tmp directory and then move it into
the jail that will convert it. This way ownership
and cleanup become contained within our child-root
and jail subsystems. This reduces the chances of
leaking convert-to files and simplifies the design.

In addition, we avoid an extra file copy and improve
the security of the convert-to API.

Change-Id: I450c24d0d0dc0da447c8072b0701c3b48d07c81b
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2020-11-23 21:29:34 -05:00
Ashod Nakashian
cb4beaca34 wsd: avoid the using keyword and use C++ size_t
size_t in C and in C++ are not necessarily the same
type. The C++ size_t is in the std namespace. Since
we do include many C headers, and indeed some C++
runtime headers do define size_t for backwards
compatibility, it's easy to mix and match the two
types.

Also, 'using std::size_t;' isn't a great practice,
so removed.

This is not exhaustive, just some low-hanging cases.

Change-Id: I85a36b6fd1acd204274b1869de9bcb94c8b3cf13
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2020-11-15 15:41:41 -05:00
Ashod Nakashian
ee0b5203ec wsd: SaveResult final and Result enum class
This makes the code self-documenting and avoids accidental
comparison or assignment of Result variables/values.

Change-Id: I84b8e36aa999191c8704938552b73ddc1c3dc3fc
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2020-11-15 13:50:16 -05:00
Ashod Nakashian
d0dc93d910 wsd: better file size utility
std::ifstream's tellg() returns -1 on error.
This is handled very poorly and shows up as ULONG_MAX.

Luckily, we have Stat class that does the same
both more safely and more efficiently.

Without opening the file, we now get the necessary
information unambiguously.

Change-Id: I2448bc71e01b0f166a9dd66aa38a88ea97a50cdd
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2020-10-25 12:20:58 +01:00
Yunus Bulut
e982a8665b Remove WebDAVStorage in Storage.hpp #111
Change-Id: Ic95b7354ed3448c0ae77cae338d8eb2bbc01e4e5
2020-10-07 14:55:26 +03:00
Andras Timar
0002fdfd6c fix license headers
Change-Id: I8623770b32d278a45357dc7f757fabfadd2b4af7
2020-10-01 11:56:43 +02:00
Ashod Nakashian
c5f9d605e4 wsd: make outgoing connection timeout configurable
The default Poco connection timeout is 60 seconds,
which is probably excessive. The current configurable
default is a more reasonable 30 seconds.

Currently we set this timeout on Storage connections
going out (i.e. WOPI connections).

Change-Id: Ie80a9141ca9bf721addc74baf94e62e0ad72fdd2
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/98913
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ash@collabora.com>
2020-08-11 20:11:14 +02:00
Mike Kaganski
e9c4c0286a Handle failed locking as (temporarily) read-only session
E.g., opening a checked-out document in SharePoint

Change-Id: Ifd5225d8450d7f2f5ba9661f158551c5c16f9b09
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/97596
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Tested-by: Jenkins
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
2020-07-20 15:47:33 +02:00
Samuel Mehrbrodt
9f8fdb7bd7 tdf#131123 Report back real save result
665b1629de was not correct, as it reported back
the save result of the internal save (which usually succeeds).
Instead we want to know the save result of the remote storage (WOPI/Webdav).
So report that back instead.

Change-Id: Iaaa42b8c817a19c2c77935a6f81c1951fdf2216c
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/97637
Tested-by: Jenkins
Reviewed-by: Samuel Mehrbrodt <Samuel.Mehrbrodt@cib.de>
2020-07-01 17:57:53 +02:00
Ashod Nakashian
5c9988f2e3 wsd: faster jail setup via bind-mount
loolmount now works and supports mounting and
unmounting, plus numerous improvements,
refactoring, logging, etc..  When enabled,
binding improves the jail setup time by anywhere
from 2x to orders of magnitude (in docker, f.e.).

A new config entry mount_jail_tree controls
whether mounting is used or the old method of
linking/copying of jail contents. It is set to
true by default and falls back to linking/copying.
A test mount is done when the setting is enabled,
and if mounting fails, it's disabled to avoid noise.

Temporarily disabled for unit-tests until we can
cleanup lingering mounts after Jenkins aborts our
build job. In a future patch we will have mount/jail
cleanup as part of make.

The network/system files in /etc that need frequent
refreshing are now updated in systemplate to make
their most recent version available in the jails.
These files can change during the course of loolwsd
lifetime, and are unlikely to be updated in
systemplate after installation at all. We link to
them in the systemplate/etc directory, and if that
fails, we copy them before forking each kit
instance to have the latest.

This reworks the approach used to bind-mount the
jails and the templates such that the total is
now down to only three mounts: systemplate, lo, tmp.

As now systemplate and lotemplate are shared, they
must be mounted as readonly, this means that user/
must now be moved into tmp/user/ which is writable.

The mount-points must be recursive, because we mount
lo/ within the mount-point of systemplate (which is
the root of the jail). But because we (re)bind
recursively, and because both systemplate and
lotemplate are mounted for each jails, we need to
make them unbindable, so they wouldn't multiply the
mount-points for each jails (an explosive growth!)
Contrarywise, we don't want the mount-points to
be shared, because we don't expect to add/remove
mounts after a jail is created.

The random temp directory is now created and set
correctly, plus many logging and other improvements.

Change-Id: Iae3fda5e876cf47d2cae6669a87b5b826a8748df
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92829
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
2020-07-01 05:42:43 +02:00
Ashod Nakashian
2c73b622d7 wsd: extract WOPI HTTP request construction
This hoists the common parts of the HTTPRequest
for all WOPI requests to avoid errors when changing them.

Change-Id: Ia02ef657a43b7a7d2fc13be3da012836fa0d7650
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/96372
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Tested-by: Jenkins
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
2020-06-20 16:33:20 +02:00
Michael Meeks
d34854f688 Add support for BreadcrumbDocName.
Change-Id: I06c56e92dd3acf9269140ecefb0c8bc731191260
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/95960
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-06-11 19:44:01 +02:00
Michael Meeks
33a5813d84 WOPI: pure re-factor, remove rampant duplication.
Dung out lots of pointless intermediate variables, and overly
verbose code. Vertical space is not a renewable resource.

Most variables had a consistent pattern, except these:

caller var          c'tor parameter         member name

Change-Id: I7910b713b8c4f6950b1e7be9c3a8e4eb4f54e249
----------------------------------------------------------
userId              userid                  _userId
userName            username                _username
canWrite            userCanWrite            _userCanWriter
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/96129
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-06-11 17:38:32 +02:00
Michael Meeks
ff1e14a1d0 WaE: fix 'isAutosave' shadows a member of 'this'
Change-Id: I30fec0e39085f6e4f7049bb690390606d2d1ef15
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/95845
Tested-by: Michael Meeks <michael.meeks@collabora.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-06-08 17:56:27 +02:00
Samuel Mehrbrodt
043e3c5238 Rename: getIsAutosave -> isAutosave
Change-Id: I30b8d52ff33c2ae270b44a1670bacc4a48af4495
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/95807
Tested-by: Jenkins
Reviewed-by: Samuel Mehrbrodt <Samuel.Mehrbrodt@cib.de>
2020-06-08 15:39:00 +02:00
Michael Meeks
f9f392ab5c Storage: add as_scheme to allow auto-determination of whether to use SSL.
This is the new default - do as we're told by the client.
The old setting is left to allow users to force SSL if they are
concerned that they may receive unhelpful URLs.

Change-Id: Idea83aacea6826a8f37264e34d49c7550efe6d27
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/93179
Tested-by: Michael Meeks <michael.meeks@collabora.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-04-30 14:53:25 +02:00
Tor Lillqvist
4eb598711c Use #pragma once
LibreOffice core uses that, too, and we support an even more
restricted set of compilers.

Change-Id: I0d0e2c8608e323eb5ef0f35ee8c46d02ab49a745
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92467
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
2020-04-18 15:00:18 +02:00
Ashod Nakashian
da0f3a950a wsd: per-user cookies
Cookies may be passed from the client to the storage,
in which case each user may have its own unique set
of cookies. These cookies are now preserved in the
ClientSession, which is per connection, and are then
passed to the storage to use when communicating with
the WOPI-like backend.

(cherry picked from commit 6022faf3cc9b622b490c3f8ca91efbff8e542414)

Change-Id: Ic2e13fa541a5ee01b7383939bbbf7d46ea75684b
2020-01-28 18:10:14 +01:00
Miklos Vajna
d0fee8c6ad These can be made const
Change-Id: I952a7566176bda727f8c2e9618d41bfb7bb1240f
Reviewed-on: https://gerrit.libreoffice.org/85197
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
2019-12-16 09:24:08 +01:00
Michael Meeks
0d97efbfcc locking: renew lock after timeout.
Change-Id: I6191ee34239b978292aeb6795be74312a954e240
2019-11-27 19:17:27 +00:00
Gabriel Masei
2164f5207c Add REST endpoint for admin metrics.
Change-Id: I701485631931334d27594c4907cb770f9888e5bf
Reviewed-on: https://gerrit.libreoffice.org/82492
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Michael Meeks <michael.meeks@collabora.com>
2019-11-25 13:06:01 +01:00
Michael Meeks
cabd30a1b8 Initial wopi-like locking implementation.
Change-Id: I3e16cfc40dbd29a24016b09e62afc9ec488300c7
2019-11-21 14:24:58 +00:00
Michael Meeks
68d81dae49 Read and store SupportsLocks property.
Change-Id: Ic405dfa5f6e759937094835f41787fa2cbe29f49
2019-11-20 15:41:06 +00:00
Ashod Nakashian
8c4edb5087 Reuse cookies from the browser
Reviewed-on: https://gerrit.libreoffice.org/78195
Reviewed-by: Andras Timar <andras.timar@collabora.com>
Tested-by: Andras Timar <andras.timar@collabora.com>
(cherry picked from commit 8deecf4ea6966c059458bdc71e365be426238e09)

Change-Id: I96bbdd3e71bc9d0ecfddea7debc0ebcc303a49ae
Reviewed-on: https://gerrit.libreoffice.org/81558
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Michael Meeks <michael.meeks@collabora.com>
2019-10-28 10:45:14 +01:00
Gabriel Masei
d597f22dac Add minimal TLS support for communication with storage
Change-Id: Iafd9946a4240063c07f5c519b8af30b52e23d3e8
Reviewed-on: https://gerrit.libreoffice.org/80373
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Michael Meeks <michael.meeks@collabora.com>
2019-10-08 18:57:36 +02:00
Ashod Nakashian
3dcc68e6f1 Pass Extended Data in Action_Save from the client to the WOPI host
Clients often need to communicate with their wopi hosts when
invoking Action_Save to provide more context when storing the
document in question. Action_Save now support passing arbitrary
string as ExtendedData entry that can be used by client to
pass any context or otherwise flags to the WOPI host, which
will receive it via the X-LOOL-WOPI-ExtendedData custom header.

See reference.html for more details.

Change-Id: I1814d1f3d984a553ffa60cec13d23b014ba59eb3
Reviewed-on: https://gerrit.libreoffice.org/74135
Reviewed-by: Andras Timar <andras.timar@collabora.com>
Tested-by: Andras Timar <andras.timar@collabora.com>
2019-09-03 01:53:12 -04:00