Since uploading has become asynchronous, it
might not have finished by the time we
unlock the document. This race can result
in the upload to fail, if the document
is no longer locked.
Now, we ensure that the lock is released
only after uploading, when we remove the
session. With test.
Change-Id: Ic9ba3290dfbfde0eb4af7a70bb5b56ff28711f5c
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
Based on the assumption that the new
number of tiles is correct, the assertion
is updated and the tile tests re-enabled.
Change-Id: I9dabe1ea4ec1b091917c4a94c549f967b9539c13
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
zstd provides for much faster compression server-side, as well
as better decompression.
zstd allows us to de-compress a keyframe and several deltas in a
single call in JS, so it is necessar to add a terminator to the
delta stream so that we can detect when to flush the buffer we
are working on - so the next delta applies to the correct data.
Change-Id: I0e292e3a697b4902d6488b7c04deaba2d1485e94
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
Due to this error:
[ coolwsd ] TST testTileProcessed [testTileProcessed] (+12566ms):
ERROR: Assertion failure: Expected exactly the requested number of
tiles Expected arrivedTile == [25] but got [28]
| TileCacheTests.cpp:1580
Probably since 2cc955f9109c0fc8443c9f93c1bf6bd317043cb5 (core)
Signed-off-by: Aron Budea <aron.budea@collabora.com>
Change-Id: Ie854a57e4241f2c8fbf8e3d769aba8a19981feb1
For some as-yet unknown reason, UnitWOPIStuckSave
is timing out with -j8 for me. It seems that
the issue has to do with the socket not having
the data in a timely fashion. The only thing
that reproduces the issue is the order, and
that isn't always the case. So, it seems
that at some point, on my particular system,
having this test any earlier than where I put
is causing the failure. The fix was to move it.
While I don't yet know the reason, this isn't
happening all the time. It would seem that
after restarting it goes away. So it might
be some system state that is interfering.
Still, I can't explain why this test should
suffer reliably and none of the others.
Worth investigating further, but for now
changing the order.
Change-Id: Ibbb912941f8872c4825468e940e82814b2d14168
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
Now that most tests are rid of poco sockets
and fixed a number of other issues, the
tests run much faster. The relative timings
have changed, so re-ordering can yield
better total time with -j.
The new order gives ~20% shorter execution
time with -j8 (on 8c/16t i9 @ 2.0 Ghz fixed).
Actual time went from ~2:11 down to 1:46.
A saving of more than 25 seconds.
Change-Id: I0f23b86e4218fff187cdbcc756eb77522732a64b
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
And disable it, since it's empty and has
no functional tests in it at all (it just
loads a document and exits the test, without
even waiting for the loading to finish).
We modernize, even though we also disable it,
because otherwise it wouldn't build.
Change-Id: I3735f28fa27af1d00e764f4e0070a25ff36b32d5
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
WhiteBoxTests.cpp is by far the slowest TU
to compile (taking minutes) and gcc started showing
'note: variable tracking size limit exceeded'
and recompiles, which doubles the time.
This moves the StringVector tests into a new
home, which reduces the size of WhiteBoxTests.
Change-Id: Ia4e8daa921fb941815ad7e68f4dbb964fb2cd93e
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
WhiteBoxTests.cpp is by far the slowest TU
to compile (taking minutes) and even gcc gives
'note: variable tracking size limit exceeded'
and recompiles, which doubles the time.
This extracts the RequestDetails tests into
its own file to better parallelize the build.
Change-Id: Ic8d54d06223a122bc1ccf73cfa216e2b3a8f32a5
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
==4544==ERROR: AddressSanitizer: odr-violation (0x000002c40140):
[1] size=104 'HostUtil::WopiHosts' ../wsd/HostUtil.cpp:12:34
[2] size=104 'HostUtil::WopiHosts' wsd/HostUtil.cpp:12:34
These globals were registered at these points:
[1]:
#0 0x71f618 in __asan_register_globals.part.13 /home/vmiklos/git/libreoffice/lode/packages/llvm-llvmorg-9.0.1.src/compiler-rt/lib/asan/asan_globals.cc:362
#1 0x7f00cb0f3d7b in asan.module_ctor (/home/vmiklos/git/libreoffice/online-san/test/../test/.libs/unit-base.so+0x10e0d7b)
[2]:
#0 0x71f618 in __asan_register_globals.part.13 /home/vmiklos/git/libreoffice/lode/packages/llvm-llvmorg-9.0.1.src/compiler-rt/lib/asan/asan_globals.cc:362
#1 0x120e2ae in asan.module_ctor (/home/vmiklos/git/libreoffice/online-san/coolwsd+0x120e2ae)
==4544==HINT: if you don't care about these errors you may set ASAN_OPTIONS=detect_odr_violation=0
SUMMARY: AddressSanitizer: odr-violation: global 'HostUtil::WopiHosts' at ../wsd/HostUtil.cpp:12:34
==4544==ABORTING
All of HostUtil was duplicated between the test shared objects and
coolwsd, so remove it from the test objects and rather exclude
RequestDetails::getDocKey() from the test objects instead, which was
linked in but was not used in practice.
Signed-off-by: Miklos Vajna <vmiklos@collabora.com>
Change-Id: Ia2147c89cf4230df97a8f45ac7d509aa11cdca97
When the Kit dies and there is no data saved to
disk to upload, there is nothing to do but unload
DocBroker and log the fact.
With unit-test that simulates the situation by
killing the Kit of a modified document.
Change-Id: I7dcc0583c053d7166935d15d045fcf6624e43b4e
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
For some reason the disk-space check wasn't
done for WOPI storage. Here we add the check
and bubble the exception up to stop loading.
UnitStorage has been updated and re-enabled.
Change-Id: I15a093554b662d6c0828da7683ca0190a8225fc9
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
This allows us to use multiple hosts using same coolwsd instance.
added aliases configuration to coolwsd.xml to avoid
possibility of opening the same file as two if the
WOPI host is accessed using different aliases
Signed-off-by: Rash419 <rashesh.padia@collabora.com>
Change-Id: I32913015c15fd396cecc702b76e0dcaa8bcafad3
All tests are now ordered to minimize
the time paralell runs take. And the
two longest-running tests are augmented
to take less time. They still take about
20x longer than the fast test, but they
do serve a purpose and we have 50+ other
tests to go through in parallel.
Total time is not much longer than the
longest test with -j8, at around 150
seconds, give or take, on an 8 core,
16 thread CPU.
Duplication in Makefile.am is reduced
with some minor clean ups.
Change-Id: I14530531485bf85d8b59e476aa933e5b2cc26c93
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
An initial set of seven of these, easy to add more as/when needed.
Change-Id: I6c65e052d00f9eaa10adee3c9464043e4c594848
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
Build-time unit-test output is now only
displayed when the test fails. This is to
reduce noise while building when not helpful.
Change-Id: I273d97dae192a24e9a1ae9f662b0fcd7ff555b75
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
When saving and/or uploading fails, we have
to give up after a certain number of retries.
A new config entry, per_document.limit_store_failures,
controls the number of consecutive failures
before giving up. This prevents locking-up
documents while uploading.
Includes a new unit-test.
Change-Id: I1296df1691fcaa982df9e861d34cb946a2eda860
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
Refactor UnitWOPIDocumentConflict.cpp into
WOPIUploadConflictCommon.hpp and reuse for both
DocumentConflict and SaveOnExit tests.
Change-Id: I54ec1e37e5e9c6298b12a2b2e596363683fb2e34
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
Trying to construct a string using sprintf() and std::stringstream
is unnecessarily complicated for something as simple as this,
and it shows up in my profiling. This commit makes the hex values
change to uppercase because that's what hexFromByte() returns,
but I don't think it matters.
Signed-off-by: Luboš Luňák <l.lunak@collabora.com>
Change-Id: I31c7b8e7cf4c2a495eca0bf03ae4cab53b26a04b
this test ensures that double encoded filenames are not a problem.
file names can already be encoded like a%20b and this was causing
problems and fixed with #3786
Signed-off-by: Mert Tumer <mert.tumer@collabora.com>
Change-Id: Ibaf0456f26c3a2e8d3c2863c2b925c7a5b9fc79c
This adds a new service render-search-result, which renders an
image of the position where, the search fund a hit. The search
result contains the information in which object or paragraph,
which is just the node id and node type from the document model.
The service takes the document and the search result (xml file)
as the input and returns a PNG image as the output.
Signed-off-by: Tomaž Vajngerl <tomaz.vajngerl@collabora.co.uk>
Change-Id: Iffc158c5b0a3c9a63f8d05830314c9bc1616b3b1
Signed-off-by: Tomaž Vajngerl <tomaz.vajngerl@collabora.co.uk>
if make is killed or interrupted during the execution
of their recipes, the target is not deleted. So there is
an opportunity to check the logs, why are taking to long
time to finish the test.
Change-Id: I6e8176a48ff721ecb2b2efc01b1f149907dd359b
Signed-off-by: Henry Castro <hcastro@collabora.com>
With some loadbalancers it may happen that HTTP 302 Found
response with redirect location will appear.
Change-Id: I7bac807761423adcd827cf775bd1d3d2ba87a6cc
Signed-off-by: Szymon Kłos <szymon.klos@collabora.com>
Write to the Trace Event JSON file in just one place, protected by a
mutex.
Handle metadata events separately. We want them to propagate to the
output if Event Trace generation is just enabled, regardless whether
it has been turned on or not.
We can send the metadata for the Kit process only after we have opened
the docuemnt as sending a Trace Event to WSD uses the (singleton)
Document object in Kit.
Introduce some new functions for the above and adapt callers
appropriately.
Generate a Complete event for the handling of one image tile, from
setting the src attribute of the Image object to finishing the onload
function.
Signed-off-by: Tor Lillqvist <tml@collabora.com>
Change-Id: I5978e0fb2495e0d2dc479bf861eb4ac7da03fa50
Move the generic dummy implementation of
TraceEvent::emitOneRecording() to a source file of its own. (That is
the one which is used in test and tool executables.)
Signed-off-by: Tor Lillqvist <tml@collabora.com>
Change-Id: I81cab07e5a6852b42d278a5446c13c3825cf546e
This adds a test case for modifying the document after
the previous upload failed and verifying that the
subsequent save also uploads the modified document.
Change-Id: I05e17842399f9af48a029fdde72ce15c023bee3f
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
Tests uploading on document closing when
the previous upload had failed.
Change-Id: Iceeed041bd4e84ba8096be245b4fce1af496eb8f
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
Modelled on how it is done in core. ProfileZone is derived from
NamedEvent which is derived from TraceEvent. Here we don't keep any
separate ProfileZone.hpp, though.
This was needed to introduce generation of "instant" events here, too.
Signed-off-by: Tor Lillqvist <tml@collabora.com>
Change-Id: I6583134e96001641c50339deb4197fca6ab7d5d5
Don't bother collecting them into a vector, like in core. Instead,
just call a static member function ProfileZone::addOneRecording() for
each Event Trace object (string) to be emitted. That function then
needs to be implemented separately in each executable. In WSD it logs
the object to the Event Trace file. In Kit it sends it to WSD for
logging. In Unit tests we use a dummy implementation.
(If Event Trace logging is not enabled at all in loolwsd.xml (the
default), nothing is done, of course.)
When receiving the "traceeventrecording start" or "traceeventrecording
stop" message from the client, turn ProfileZone recoring on or off in
the WSD and Kit process. (Probably in WSD the flag should be
per-client.)
Change-Id: Ie1127d65dd44ed77e7eeab4b0f0a90cce95dc4a2
Signed-off-by: Tor Lillqvist <tml@collabora.com>
This was accidentally removed. However,
we do want to remove this grouping logic
so that the tests execute faster.
Unfortunately, without the grouping some
tests fail randomly. Specifically, UnitHTTP
had an implicit dependency on the jail temp
to be created before it ran, lest it failed.
This is now fixed in the previous commit,
which paves the way to removing the grouping.
Change-Id: If18f7faf8ed0677b718db48781b387c7b37d6b0b
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
This adds support to parse client request
on the server side.
Unit-tests included.
Change-Id: I90e9ad3007a3215682991fd2383362e1c48589a8
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
