We should have no symlinks in the jail whatsoever, except
those we create to files known to exist in the jail.
Unfortunately, some systems have some of the /etc files
as symlinks. When we create hard-links to these files,
they can't be accessed from the jail, since the path
they point to isn't replicated in systemplate and jails.
First change here is to always link to the source file or,
when copying, to copy the source rather than a symlink.
Next, to detect modifications, we compare not just the
size and timestamp, but also the contents. This way we
can be certain that any modification will be detected.
Finally, when we copy at least one file in the
systemplate/etc directory, we flag it by creating the
'copied' file. This way we have a reliable indicator
and don't need to second guess if the files are
hard-linked or copied.
We also avoid some noisy errors when we fail to update
systemplate when it's read-only by first checking
if systemplate is writable or not and insue a friendly
log instead.
Change-Id: Ie8c3e70ea4ec19ee098309f8666c00639fa7319b
In the old code, if the evaluation first allocates the memory for the
raw pointer, then calls firstLine() and an exception is thrown before
the std::unique_ptr construction, then the memory is leaked. Using
make_unique() has the benefit of avoiding this problem.
Convert only a single usage, so the remaining places can be done as easy
hacks.
Change-Id: Iaf3d8051a8a0627a57fdf1196bde7d5f8612fcff
URIs may or may not have authorization data specified
via access_header or access_token query parameters.
In the event that the host doesn't have such needs
(for example authrorization could be performed by
some other means), we should accept the request
and still go ahead and make the WOPI request
all the same.
This patch effectively reverts the changes from
a019c93d90 which threw
an exception when the authorization method was
undefined. Since there was an assertion to warn
programmers that something is amis, now we simply
log the fact in trace mode and move on.
A new unit-test is added and another one that
expected the now-removed exception has been modified.
Change-Id: I26cc2514d7465b344037a6e32b777c0fe0ba9a2c
This fixes the following issue in Online:
When a save is performed while a cell is still edited the save icon
does not reflect the correct state of the document: it shows that
the document is dirty although it is not.
This works only in conjunction with the following commit from core:
-lok: remove .uno:ModifiedStatus message from deduplication mechanism-
Change-Id: Ie671097ef2b206c8801a5bdfc2b908ee260951fb
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/103084
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
The use of a common threadname suffix in the WSD and Kit
processes is intentional. It is designed to help filter
for a single document's logs across both processes.
The thread name has nothing to do with the classes in
the code, nor is it intended to imply any relationship
except with the process and the document in question.
As the comment in this patch explains, the choice of
the suffix is arbitrary and while it may be changed,
it has to be sensible and common between the two threads
to allow for easy grepping.
Historically, there were in fact dedicated threads
within the respective "broker" classes, but this
fact should be safely ignored, since at the log level
we care less about which part of the code generates a
log entry (that info, if needed, is at the end of each
log entry, in the form of filename and line number),
rather we care more about which document it relates to,
which is crucial in investigating production issues.
Logs and code structure are only incidentally related.
Logs are (or at least should be) designed around
the execution structure, not code architecture.
(This reverts 2a16f34812)
Change-Id: Ic6fe2f9425998824774d2644fe4362e75dea6b88
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/101261
Tested-by: Jenkins
Tested-by: Tor Lillqvist <tml@collabora.com>
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
We now gracefully fallback to copying when/if systemplate
is readonly.
The bulk of the change is to support proper cleanup in
both cases.
First, we had to move as much of the jail bootstrapping
into the loolwsd-systemplate-setup script, so systemplate
will be as complete as possible before it is locked down.
Next, we needed to update the jail with graceful fallback
to linking/copying upon failure. For that, the jail setup
logic in Kit.cpp has been reworked to support not just
update failures, but also more comprehensive mounting
failures as well.
Finally, jail cleanup now is seamless. To support proper
cleanup when we had mounting enabled but had to fallback,
we mark jails that aren't mounted so we can 'rm -rf' the
contents safely and without fear or causing undue damage
(as unlikely as that is, technically we wouldn't want to
rm systemplate files, if mounting read-only had failed).
There are a few minor refactorings of JailUtil to make
it cleaner and more robust.
Change-Id: Iac34869cb84f45acf64fbbc46d46898367b496d2
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/101260
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Andras Timar <andras.timar@collabora.com>
For various reasons, systemplate may be read-only
or under a different owner and therefore impossible
to update the dynamic files in it.
To support such a scenario, we first link the
eight dynamic files in /etc when creating systemplate.
If this fails, we copy the files.
When creating jails, we always check that all the
dynamic files are up-to-date. If they are, nothing
further is necessary and we bind-mount, if enabled
and possible.
However, if the dynamic files are not up-to-date,
we disable bind-mounting and force linking
the files in the jails. Failing that, we copy them,
which is not ideal, but allows us to ensure the
dynamic files are up-to-date as we copy them too.
Ideally, the dynamic files in question would be
hard-link (or at least soft-linked) in systemplate
at creation. From then on we would bind-mount
the jails and everything would work perfectly and
no files would need updating. This patch is fallback
for when this scheme fails, which should be exceedingly
rare anyway, but which still ensures correct operation.
Change-Id: I09c6f057c49396579aaddb1b8bf4af0930dd4247
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/100834
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Tested-by: Jenkins
Reviewed-by: Andras Timar <andras.timar@collabora.com>
Our header parses was overly simplistic and
didn't support a number of corner cases that
rfc2616 specifies (folding, for example). The
new approach is to simply normalize the headers by
removing invalid line-breaks and then let the
MessageHeader parser take care of parsing the
headers individually, which we then set on the request.
The new utility setHttpHeaders should be used
whenever we need to set a header in an request
to make sure it are sanitized and valid.
Change-Id: Ifa16fa9364f42183316749276c5d0a4c556cb740
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/96371
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ash@collabora.com>
When we get a wid match, this helps WSD to cleanup its tile
subscriber list effectively.
Change-Id: I6517039fb3d8c9ad8f53aef549b8adbb79961ce1
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/100348
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
This adds a "view_comment" in addition to "view" and "edit" state
into discovery.xml. In case it is enabled, the filters let the
comment commands through to core.
In addition add "Save Comment" menu action to allow saving the
comments, which is enabled when in "read-only" with "view_comment"
mode.
Change-Id: I3ab3dbee93ee2167ae96adea7025fc0b385f8201
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/99473
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Tested-by: Jenkins
Reviewed-by: Tomaž Vajngerl <quikee@gmail.com>
Avoids the need for PNG encoding (takes significant amount of CPU
time) and Base64 encoding in the app process, transfer to JavaScript
(running in a WebKit process of its own), and corresponding decoding
(in the WebKit process). Instead simply pass the URL of each tile file
to the JavaScript. Remove each BMP file once it has been loaded.
Change-Id: I6e7b9450691679c64813979976c59f1763ec104c
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/98710
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Useful (like LOG_TRC_NOFILE) in cases where the source file location
of the LOG_INF call is uninteresting. (Like if the message to be
logged has originated somewhere else.)
Change-Id: Id6d8c137be073a958943f01b3d7b98143fcd7dfd
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/98735
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
* Excised TileCacheDesc to improve performance and simplify code.
* clang-tidy suggestions and auto-rewrite fixes.
* Const-correctness.
* Inlined and improved a couple of trivial functions (that are called
often).
* Reduced some logs from INF to DBG as they are only meaningful to devs.
Change-Id: I1c4eb8c63da49aa061afbf3eb68cae23d4d5e7f3
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/98661
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Tested-by: Jenkins
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
This is for the benefit of a next-gen iOS app (without FakeSockets and
much of the current Online plumbing).
This is not supposed to cause any functional changes in normal Online
even if code is organised a bit differently.
Change-Id: Ib09a84ff5d3ba858cf3f50553d76757966af7ad2
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/98655
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
It is good practice, and those are needed when it is used in another
context.
Change-Id: I7811cbc6e072ed88ef2f0061994e7a18e797fd0d
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/98588
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
"Disable for now - pushed in error" says the comment added in
e11794da25.
Change-Id: Ia2b72bfe20f8ff16d74d1966d511c74eab3e4417
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/98587
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
We output the informative name we give ourselves to the thread anyway
which is much more useful.
Change-Id: I74721cc0014fa657adfb6ecea05bcd7f846421a4
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/98477
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
When tests are run in parallel, they will all
compete to update and set up the systemplate
directory, which has a handful of files that
need to be up-to-date. This is a source of errors.
Normally, these files are linked (hard- or soft-
link, whichever succeeds). With linking, we
only need to worry about the initial setup,
as the files will never be out-of-date from
then on. However, when linking fails, we need
to copy the files, and update them (by copying
over fresh versions of the files, if necessary)
every time a new kit is forked. Copying over
is tricky, as it's not atomic. To make it
atomic, we copy the files to the destination
directory under a temporary (random) name,
and then rename to the final name (which is
atomic, including replacing the target file,
if it exists).
No such race exists in production, where there
is (or should be) but one instance of loolwsd
(which does the initial setup) and forkit
(which updates systemplate before forking
new kit instances).
This is an issue with parallel tests only.
Change-Id: I6ba1514d00a84da7397d28efeb6378619711d52f
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/97785
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
The new utility is safer and more readable.
Change-Id: I3a86675378d458cb004e5534dbf2b401936d0e57
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/98183
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
String comparisons are costly and much
less readable. Now we have a fast switch
with LOK_CALLBACK enum values.
Change-Id: Icc24b91b174cd9bbb7e0d64039df080c0a4338f2
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/96375
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
This:
while (*pos++);
leaves pos pointing to the char *after* the first null char. Thus we
lost the timestamp, thread name, and message log level.
The log output ended up loooking like:
Mobile-0x42a805To JS: loolserver { "Version": "master..", "Hash": "1e3b28b0", "Protocol": "0.1", "Id": "e42d1a33" }| CODocument.mm:115
with no space between the thread id and the message, while what we
expect is:
Mobile-0x42be9e 2020-07-02 18:59:30.490298 [ main ] INF To JS: loolserver { "Version": "master..", "Hash": "1e3b28b0", "Protocol": "0.1", "Id": "46ebf726" }| CODocument.mm:115
As such it is not necessary to show "Mobile" (the name of the
process's main program) in the log messages. There is just one
process. Will remove that in a later commit.
Change-Id: I55c4a82f2b34e3b9e70e86cc7af8ea42a3108695
This is to defend the sneaking of extra http-headers
in the access_header URI param that was recently fixed.
Change-Id: Ic28cf58854847ac278bed8043f398b107f7992b3
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/96862
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Authorization class now handles the parsing and creation
of its instances, which makes it centralized.
We also avoid repeatedly constructing Authorization objects
in ClientSession and instead do it once at construction
and cache it.
A bunch of new unit-tests added.
Change-Id: I9b5939be51a5957214d07ed8f1096efd179686c6
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/96825
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
loolmount now works and supports mounting and
unmounting, plus numerous improvements,
refactoring, logging, etc.. When enabled,
binding improves the jail setup time by anywhere
from 2x to orders of magnitude (in docker, f.e.).
A new config entry mount_jail_tree controls
whether mounting is used or the old method of
linking/copying of jail contents. It is set to
true by default and falls back to linking/copying.
A test mount is done when the setting is enabled,
and if mounting fails, it's disabled to avoid noise.
Temporarily disabled for unit-tests until we can
cleanup lingering mounts after Jenkins aborts our
build job. In a future patch we will have mount/jail
cleanup as part of make.
The network/system files in /etc that need frequent
refreshing are now updated in systemplate to make
their most recent version available in the jails.
These files can change during the course of loolwsd
lifetime, and are unlikely to be updated in
systemplate after installation at all. We link to
them in the systemplate/etc directory, and if that
fails, we copy them before forking each kit
instance to have the latest.
This reworks the approach used to bind-mount the
jails and the templates such that the total is
now down to only three mounts: systemplate, lo, tmp.
As now systemplate and lotemplate are shared, they
must be mounted as readonly, this means that user/
must now be moved into tmp/user/ which is writable.
The mount-points must be recursive, because we mount
lo/ within the mount-point of systemplate (which is
the root of the jail). But because we (re)bind
recursively, and because both systemplate and
lotemplate are mounted for each jails, we need to
make them unbindable, so they wouldn't multiply the
mount-points for each jails (an explosive growth!)
Contrarywise, we don't want the mount-points to
be shared, because we don't expect to add/remove
mounts after a jail is created.
The random temp directory is now created and set
correctly, plus many logging and other improvements.
Change-Id: Iae3fda5e876cf47d2cae6669a87b5b826a8748df
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92829
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Seems to not cause any serious regressions in the iOS app or in "make
run", but of course I am not able to run a comprehensive check of all
functionality.
Change-Id: I44a0e8d60bdbc0a885db88475961575c5e95ce88
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/93037
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
