==4544==ERROR: AddressSanitizer: odr-violation (0x000002c40140):
[1] size=104 'HostUtil::WopiHosts' ../wsd/HostUtil.cpp:12:34
[2] size=104 'HostUtil::WopiHosts' wsd/HostUtil.cpp:12:34
These globals were registered at these points:
[1]:
#0 0x71f618 in __asan_register_globals.part.13 /home/vmiklos/git/libreoffice/lode/packages/llvm-llvmorg-9.0.1.src/compiler-rt/lib/asan/asan_globals.cc:362
#1 0x7f00cb0f3d7b in asan.module_ctor (/home/vmiklos/git/libreoffice/online-san/test/../test/.libs/unit-base.so+0x10e0d7b)
[2]:
#0 0x71f618 in __asan_register_globals.part.13 /home/vmiklos/git/libreoffice/lode/packages/llvm-llvmorg-9.0.1.src/compiler-rt/lib/asan/asan_globals.cc:362
#1 0x120e2ae in asan.module_ctor (/home/vmiklos/git/libreoffice/online-san/coolwsd+0x120e2ae)
==4544==HINT: if you don't care about these errors you may set ASAN_OPTIONS=detect_odr_violation=0
SUMMARY: AddressSanitizer: odr-violation: global 'HostUtil::WopiHosts' at ../wsd/HostUtil.cpp:12:34
==4544==ABORTING
All of HostUtil was duplicated between the test shared objects and
coolwsd, so remove it from the test objects and rather exclude
RequestDetails::getDocKey() from the test objects instead, which was
linked in but was not used in practice.
Signed-off-by: Miklos Vajna <vmiklos@collabora.com>
Change-Id: Ia2147c89cf4230df97a8f45ac7d509aa11cdca97
When the Kit dies and there is no data saved to
disk to upload, there is nothing to do but unload
DocBroker and log the fact.
With unit-test that simulates the situation by
killing the Kit of a modified document.
Change-Id: I7dcc0583c053d7166935d15d045fcf6624e43b4e
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
so now real locked_host settings will be applied if the document accessed with alias
Signed-off-by: Rash419 <rashesh.padia@collabora.com>
Change-Id: Ib76704a4bf2c3da5dc7b83bbad98fe40b5c03316
Only one of them is needed, and only in COOLWSD.cpp itself.
Signed-off-by: Tor Lillqvist <tml@collabora.com>
Change-Id: I03864afdc13e778a2b8ce8205e135d679dddf13d
We dont upload to storage on mobile we save as locally
and copy the temp file into original on uno save result callback
Doing that once during close is enough since there is no
turning back and it cannot be possibly modified after
Signed-off-by: Mert Tumer <mert.tumer@collabora.com>
Change-Id: I047626be862cdbcb86e083534555e45530f417a2
And improved socket logging in general while
making them more consistent.
Change-Id: I1ed7f2561476ca5370af91079d5d616804396f8e
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
you can switch between 'first' and 'groups' mode
default mode is 'first' it allows only the first host when groups are not defined.
set mode to 'groups' and define group to allow multiple host and its aliases.
also added mode option in JSON format
to make the setup backwards compaitable , auto_host is by default false it won't
resolves aliases and dockey will only use uri's path . To use alias_groups you have
to explicitly set the auto_host allow attribute value to true
Signed-off-by: Rash419 <rashesh.padia@collabora.com>
Change-Id: I3af439edcbc546d9a660d678e52d813951dc237a
==30332==ERROR: AddressSanitizer: odr-violation (0x0000020b9b20):
[1] size=1 'StorageBase::SSLAsScheme' ../wsd/Storage.cpp:67:19
[2] size=1 'StorageBase::SSLAsScheme' wsd/Storage.cpp:67:19
These globals were registered at these points:
[1]:
#0 0x71d0f8 in __asan_register_globals.part.13 lode/packages/llvm-llvmorg-9.0.1.src/compiler-rt/lib/asan/asan_globals.cc:362
#1 0x7f4c362ed33b in asan.module_ctor (online-san/test/../test/.libs/unit-base.so+0x10eb33b)
[2]:
#0 0x71d0f8 in __asan_register_globals.part.13 lode/packages/llvm-llvmorg-9.0.1.src/compiler-rt/lib/asan/asan_globals.cc:362
#1 0x11c709e in asan.module_ctor (online-san/coolwsd+0x11c709e)
==30332==HINT: if you don't care about these errors you may set ASAN_OPTIONS=detect_odr_violation=0
SUMMARY: AddressSanitizer: odr-violation: global 'StorageBase::SSLAsScheme' at ../wsd/Storage.cpp:67:19
==30332==ABORTING
Resolve the conflict by not providing these definitions when building
tests.
Signed-off-by: Miklos Vajna <vmiklos@collabora.com>
Change-Id: I6a8f74bd0b78a76f46b5401acaa816dd0b185aa9
This adds a new config option to enable/disable
experimental features and behavior. The default
value can be controlled at build time by
--enable-experimental.
Change-Id: Iffcb4c71d9e0933a646251b63033b6dadcd3b809
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
When deciding whether the file on disk needs uploading
or not, we compare it with the last modified timestamp.
For this to work they need to be different.
When there is no file to upload (i.e. when there is
nothing new to save), the timestamp we get is
unix-time of 0. This is always different from 'now'.
So we incorrectly think there is data to be uploaded
when there isn't.
Better to initialize the last modified timestamp to
unix-time of 0, so in this case we correctly
conclude that there is nothing to upload.
Change-Id: I545e9021e9c76d009e08758342effaf3e9b964c7
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
When disconnecting from Kit we shouldn't bluntly stop
DocBroker, because even though we cannot save, we
can still upload the latest document version from
disk.
What makes this particularly problematic is that
we request to kill Kit when the last client is
disconnected.
This was caught through unit-tests, albeit it was
not very common or easy to reproduce.
This fixes cases where always_save_on_exit
doesn't save/upload.
Change-Id: Ib66281fc68d4c0264602852b0acdc453b6f79963
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
For some reason the disk-space check wasn't
done for WOPI storage. Here we add the check
and bubble the exception up to stop loading.
UnitStorage has been updated and re-enabled.
Change-Id: I15a093554b662d6c0828da7683ca0190a8225fc9
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
Forkit takes significantly longer to start than
to fork. At the startup of WSD we do wait 4 times
as long for forkit than when forking kits.
Similarly, we should also wait that much when
forkit crashes and we recover.
This stabalizes the forkit crash tests, which
did fail rather frequently (in one example,
2 out of 10 runs had failed locally).
Change-Id: I990c8e595b64c34a8e202933be833c53d7ebfc01
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
This allows us to use multiple hosts using same coolwsd instance.
added aliases configuration to coolwsd.xml to avoid
possibility of opening the same file as two if the
WOPI host is accessed using different aliases
Signed-off-by: Rash419 <rashesh.padia@collabora.com>
Change-Id: I32913015c15fd396cecc702b76e0dcaa8bcafad3
The timestamps now helpfully include how
long ago they were.
This also improves the conversion between
chrono clocks by adding generic functions.
Change-Id: I18081330564c8d969b9364cf1b9e33546e241280
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
And avoid pid of 0 in getKitPids().
Change-Id: I68743a55aed080ed54d7d8576f269974c644e1b2
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
fetch JSON from remote server and apply new config without restarting coolwsd
- Extended the feature_locked configuration & functionality so that it can be set per wopi host ranges
Signed-off-by: Rash419 <rashesh.padia@collabora.com>
Change-Id: Id0e2b4b524a80de88b2b3559dd18c4a95b8163fb
When the document isn't modified and we save,
we get back "unmodified" failure from Core.
This has the unwanted side-effect of skipping
uploading.
As we now decide whether to upload or not
based on the file timestamp on disk,
this logic has no purpose and in fact can
cause a lot of grief.
Removing this outdated logic has the nice
side-effect of simplifying the code, as
the save result doesn't need to be propagated.
Also, save failures are now communicated to
clients at the point of handling the save
response, not when uploading (which is
a distinct stage). And upload failures
are reported separately.
Change-Id: Ia465a0069280fb6bea47e413f0d885565f0bbb3d
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
The closeReason member isn't just for logging,
it's is also broadcast back to clients. This
means that it has to preserve the original
reason for initiating the document closing,
lest the client misbehaves.
Change-Id: Iddee5dba3943d873e8658ce5c1748f4ecd47c486
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
And don't load documents when the ShutdownRequestedFlag
is set, not just the TerminationFlag.
Change-Id: I94c720d6fd7c361549bc905e093886619e8f188b
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
It is possible to lose the last writable session
during unloading and get stuck forever.
This can happen in the following scenario (as observed
in a rare test run):
1. always_save_on_exit is true.
2. The last/only writable session disconnects.
3. We save and upload.
4. We attempt to stop because the clients have disconnected,
but because we are uploading, it doesn't flag to stop.
5. Uploading fails repeatedly until we reach the limit.
6. We endlessly check to save and upload and even though
there is nothing to save, we couldn't even check for
the need to upload, getting stuck.
With this patch the above scenario is not possible.
Change-Id: I19e7b621f2b452a9f18964f5f19d6eb378a48797
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
In 'debug' log-level we expect a detailed, but
still readable output. Having one area with
disproportionately large number of logs reduces
the overall utility of the log output.
This patch reduces a number of redundant log
entries, including errors that are already
logged. It also reduces the level of some
others from 'information' to 'debug' and
from 'debug' to 'trace'.
The goal is to make 'debug' level as useful as
possible to read the progress and be able to
understand what was going on, such that one is
able to decide which area to dig deeper into.
Then, trace level could be used to get more
insight into that area, if necessary. For
example, when investigating a test failure,
one first enables 'debug' logs and reads through.
Once a section between two debug entries is
identified as being of interest, enabling 'trace'
level logs becomes more productive as it's
now possible to easily reach the first DBG
entry and read through until the second one.
It's unfortunate that we don't have per-area
control for enabling/disabling logs, so it
is common to see more and more 'debug' log
entries added all around, making logs
less and less readable.
It is also a limitation of the levels we have
that we really only have 3 usable levels:
one, two, many. That is, 'information' for
the most important events, 'debug' for
technical details needed to investigate issues,
and 'trace' for everything else. ('warning'
and 'error' aren't really 'levels'; they have
semantics that makes them special-cases.)
So we have to avoid degrading one into the
other, or have differences without distinction.
If any of these entries are needed to be
displayed more frequently, changing them
back to 'debug' or even 'information' should
be done. Though for me they seem special
cases that don't benefit most log readings.
Change-Id: Id2c6a9dc027483b81a066b0b4b50a298c5eff449
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
This adds dumpState to DocumentState, SaveManager,
and StorageManager classes, and dumps all the
missing members.
Also, normalize the format and make it symmetric
and consistent.
Change-Id: Ie0cc8e07d13de60c33d64cd621abf4e815a4ef94
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
An initial set of seven of these, easy to add more as/when needed.
Change-Id: I6c65e052d00f9eaa10adee3c9464043e4c594848
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
With --unattended, do not wait for a debugger
upon seg-faulting. This avoids the unnecessary wait
that prolongs failed unit-tests in automated runs.
Now run_unit.sh and Cypress Makefile set this flag.
Note that the wait only happens when in debug
builds, or when envar COOL_DEBUG is set. This
prevents us from waiting when running a debug
build where we can't see the output, or indeed
the run is on a CI build machine.
This flag can also be used by devs when reproducing
failures where there is no interest in attaching
a debugger. The logs are shorter and more
readable, too. At least in trace level.
Change-Id: Ice15482c6724abc47f5955402295198eb7f671ee
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
When saving and/or uploading fails, we have
to give up after a certain number of retries.
A new config entry, per_document.limit_store_failures,
controls the number of consecutive failures
before giving up. This prevents locking-up
documents while uploading.
Includes a new unit-test.
Change-Id: I1296df1691fcaa982df9e861d34cb946a2eda860
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
When the last client connection is closed,
there is UI (or user) to provide input
on the document conflict dialog. In this
case, we detect that the situation is final
and unresolvable and we simply give up.
We log a warning and dump the document state
before terminating.
Change-Id: I111a446b8743a0d16b7ed8b39751a419036c927f
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
When the document has been changed in storage,
we should be extra careful not to inadvertently
clobber it because we needed to force the
upload for some unrelated reason.
When in conflict, only a user can force clobbering.
Change-Id: I498a6d1c86242b059ac722d3e48c31a04a79591b
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
Since we track active sessions in removeSession,
it's possible that the last session is not active.
In that case, we should set the unload flag while
saving. If we've removed it, we should mark to
destroy. And finally, we should only assert
that mark to destroy is set with active session,
not when we are left with disconnected ones.
Change-Id: I275aeafe67875fa75e0fad5438928f2f77b79a9a
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
By waiting at most 2 seconds, we improve the chances
of getting the ModifiedStatus=false before we termiante
the document. This minimizes the noise of warning
of potentially lost data on exit and similarly reduces
the test failures, which test for clear termination.
Change-Id: I284693d1cdd91b1224581fdc29e6e5438ab4c0b2
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
We no store the origina modified time of the document
as we receive from the storage server in string
form and send it back as-is. This avoids any
potential issues with the roundtrip of conversion
to and from a timestamp.
Change-Id: I524bea8f36c3ce62dcd00c4fe6a1e7e083287ed1
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
Probably the last remaining incorrectly labelled
helper that checks for in-progress uploads, but
had retained the old and misleading label of 'save'.
Change-Id: I693275b1559f3dae4e9e3ab2408d997f56ff86e3
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
For example, to skip exporting the first page of a document:
curl -k -F "data=@3page.odg" -F "format=pdf" -F "options={\"PageRange\":{\"type\":\"string\",\"value\":\"2-\"}}" https://localhost:9980/cool/convert-to > out.pdf
https://gerrit.libreoffice.org/c/core/+/128849 has more examples.
Signed-off-by: Miklos Vajna <vmiklos@collabora.com>
Change-Id: I6c4ce25bc580dac041f2865f74c856780d6fe137
use Uri::getAuthority():
"Returns the authority part (userInfo, host and port) of the URI. "
.
Change-Id: I8c63a4c00cc1ad199f6f0cfd15f4e42fac4e7aaa
Signed-off-by: Henry Castro <hcastro@collabora.com>
It will be used to update server check versions
Change-Id: I3acadb575bdcc99daed4c68881d3f35da999a22c
Signed-off-by: Henry Castro <hcastro@collabora.com>
PDFs are view_comment type documents where
editing is not supported, but adding comments
are. This means that we do get ModifiedStatus=true
but we never get ModifiedStatus=false after
.uno:Save. This is because the save command is
handled in a special way in Core by invoking
save-as instead, which doesn't reset the
ModifiedStatus.
The issue with this was that DocBroker was
stuck on trying to save the document
before unloading, thinking it was modified
due to the stuck ModifiedStatus. Here,
we change the definition of isPossiblyModified()
to ignore the ModifiedStatus for such documents.
Instead, we rely on the last save being successful
and that no new user input exists past the last
save request.
In addition, we now have a new Cypress test
that reproduced the failure without the fix
and now passes with the fix.
Change-Id: Ida9d486ac93a588b9007c5e4583d8bf3c090a62d
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
We get a syntax error message in the logs - but have no indication as
to what in the config file caused this issue. Bringing back both more
detail to the browser and logging the extended details.
Signed-off-by: stellarpower <5004545+stellarpower@users.noreply.github.com>
Change-Id: Ib8ddb20a1968f879558e59f50579ee4b18db0f09
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
because the Python macro examples on the top of the page
operate on Writer documwents, and have no effect on Calc documents.
Signed-off-by: Andras Timar <andras.timar@collabora.com>
Change-Id: Ia4f9b28c25689cb2ed9037ffd24086d7fb8c3cf9
When we reject a connection (possibly due to
authorization failure) we will not have a
storage instance to use.
Change-Id: I049c990317e4a78d0e89bc906f1ee48b365cf3c2
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
Recovering from an unloading state has a lot of
corner-cases that make it exceedingly tricky to
handle correctly. Luckily, if we reject new
connections, the client will retry connecting
(or at least it can) and that gives very
reliable results.
Change-Id: I3c06c878cb7a333dd49e153096ce01b356e9815d
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
In case of RTL users we want to switch UI to that mode
early so we don't see flipping in the browser.
Setup correct setting at the serving html step.
Using that approach we don't need to extend LOK API.
We detect that using "lang" parameter passed in the request.
Signed-off-by: Szymon Kłos <szymon.klos@collabora.com>
Change-Id: Iaa2962b01f218cae68a6dfd168bff4dbdae94c8a
And always use the stop() helper so that we properly
log the request to stop.
Change-Id: Iee3be5d8591bbb2febe651815741d9ad9ff5d977
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
(cherry picked from commit ab954b9a34813d143b1d409ae80e7531bec625ed)
This fixes failures in UnitCopyPaste and
TileCacheTests::testWireIDFilteringOnWSDSide where
the documents are closed while isModified() is true.
Change-Id: I6de683530df9b7987ad87897e9ce70b5fc3d3a15
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
(cherry picked from commit 65105263fbed89d787ac59ca77f64896bacf6832)
We make sure that we always check for last-minute
modifications before stopping DocBroker. This covers
the cases where there are modifications that we need
to save and/or saved data that needs uploading.
This save, upload, and stop logic is now in a helper
that cleanly handles this exit-time check.
Change-Id: Ibee0e7769a396b205de955f26700ebcb27d5ac95
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
(cherry picked from commit 69f9d86c8b8a477b5a0cf1910eb1b7498eaceeac)
Relevant for catching test failures, which
went unnoticed if the failure wasn't asserted.
E.g. when a test times out.
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
(cherry picked from commit 426a3ea6eb5d0041432e2f5fa4d99e3cf834f595)
Change-Id: Ic6e5b5aa4344a3048f508a6bec2be7c5a6522f33
When the storage returns modified-in-storage or conflict status
to an upload attempt, we prompt the user to handle the conflict
when the document is modified. Previously, we only prompted
when we had the modified flag set, ignoring the possibility of
having a race with the modified flag, in case the document was
just modified, or potentially modified.
We now prompt the user even when the document is potentially
modified, giving the user the opportunity to decide what to do.
Signed-off-by: Rash419 <rashesh.padia@collabora.com>
Change-Id: I699aaed92d41f76a8eb9de6bdb65608099802663
