Commit graph

769 commits

Author SHA1 Message Date
Michael Meeks
92d3435dd1 Admin: improve smaps parsing performance.
We have around ~8k smaps lines for each process, polled regularly.

Change-Id: I7e78b17e52aeaec55fa17c5192a940aa9e44b045
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
2022-05-26 19:53:11 -04:00
Michael Meeks
c130231379 deltas: track, transmit and cache deltas (disabled for now)
Squashed from feature/deltas-expanded.

TileCache changes:
    + add montonic sequence (wid) numbers to TileData
    + account for sizes of TileData with multiple blobs
    + simplify saving and notifying of tiles

Sends updates (via appendChanges) based on the sequence the
right mix of keyframes and/or deltas required as a single
message, and parse and apply those on the JS side.

We continue to use PNG for slide previews and dialogs,
but remove PngCache - used by document tiles only.

Annotates delta: properly as a binary package for the websocket.

Distinguishes between deltas and keyframes we get from
the Kit based on an initial un-compressed prefix
character which we then discard.

kit can be forced to render a keyframe by oldWid=0

Track invalidity on tiles themselves - to keep the keyframe around.

    We need to be able to track that a tile is invalid, and so subscribe
    to the updated version as/when it is ready - but we also want to
    store the keyframe underneath any deltas.

force rendering of a keyframe for an empty slot in the TileCache.

force tile sequence to be zero for combinedtiles - so the client can
always request standalone tiles with explicit combinedtiles, or tile
requests.

move Blob to Common.hpp

use zero size for un-changed tiles.

remove obsolete render-id, and color deltas in debug mode.

cleanup unit tests for non-png tile results.

Change-Id: I987f84ac4e58004758a243c233b19a6f0d60f8c2
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
2022-05-24 22:16:58 +01:00
Michael Meeks
1eb03824e2 delta: disable PngCache for deltas.
Change-Id: I5019b645ccdc02ab813db5e3e421f920681f05e5
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
2022-05-10 11:48:33 +01:00
Michael Meeks
1d3142feac Session: remove un-used headers.
Change-Id: Icd0d7c1a6abce9157fd23f0c347866a58033e48c
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
2022-05-10 11:48:33 +01:00
Michael Meeks
d8dff82f24 Don't re-use old wire-ids, we want to monotonically increase.
Change-Id: Ie819b73edba175749ed6edf170af326b7b825bac
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
2022-05-10 09:35:47 +01:00
Michael Meeks
f76522c86a Compress raw images too directly with zlib.
Change-Id: Idb54db1c3a3d6970db0782b2a7a2eebbad27f153
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
2022-05-10 09:35:47 +01:00
Michael Meeks
0df3e41879 deltas: make delta creation & caching thread-safe.
Change-Id: If134ac6212850beadb262a4b8f4c084c78d8a44e
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
2022-05-10 09:35:47 +01:00
Michael Meeks
bfc4699ffb deltas: enable monotonic wid incrementing, build deltas based on pos.
We want to always generate a delta vs. the last state we have so we
move linearly forward into the future.

Change-Id: I730d1dfb125a19e2c48b8f84ad5563664d196ab0
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
2022-05-10 09:35:47 +01:00
Michael Meeks
8eebe1e345 Tile-deltas, expose TileData to unit tests.
Change-Id: I5f33a56009927187b460775129f0818f47843368
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
2022-05-10 09:35:47 +01:00
Henry Castro
a56cefc56f common: move "hashSubBuffer" function
Change-Id: If171b7cf4e636e24cb7f757d893136927e8373aa
Signed-off-by: Henry Castro <hcastro@collabora.com>
2022-05-09 16:40:57 +01:00
Michael Meeks
414c36c6f5 Add conditional delta creation back.
Change-Id: I374e4fd0404ee8caa0f81d6de24a55220ae23c91
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
2022-05-09 16:40:57 +01:00
Ashod Nakashian
4bf085f195 wsd: correctly set termination and shutdown flag
We should always set the shutdown flag first.
Otherwise, we run afoul of a race condition.

Change-Id: Ic99793d68b3b943496ff932b4bdafd336fef7f82
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2022-05-06 14:45:34 +01:00
Ashod Nakashian
757529aad2 wsd: cosmetics
Change-Id: I85773b4c06c051ee8c52c353e26dd6149c7843de
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2022-05-06 14:45:34 +01:00
Ashod Nakashian
c6561a99d8 wsd: test: correctly stop SocketPoll in UnitTimeout
Change-Id: I97317e8155e5c65d12e985cc837f07bfcf6a56d6
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2022-05-06 14:45:34 +01:00
Ashod Nakashian
9a69adf766 wsd: test: always set the test name
Change-Id: Ibce5cf049ad1a1df7b11412ed7160d280ece60f3
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2022-05-06 14:45:34 +01:00
Ashod Nakashian
40839e8019 wsd: browser: Reset_Access_Token API to update access-tokens
Change-Id: I92cc424c30824232eaf156ddfe49104932b2ba81
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2022-05-06 11:13:35 +01:00
Rash419
2163e2f043 wsd: cleanup setting firsthost code
we added AllHosts to give admin the err log that host is not in alias_groups but now as we removed the host list entries from configuration we don't need that log

Signed-off-by: Rash419 <rashesh.padia@collabora.com>
Change-Id: I8b5e9e6b7df7df59befb496c12966c7ddc60c707
2022-05-05 11:04:22 +05:30
Ashod Nakashian
fac6c01725 wsd: replace strrchr with constexpr
This avoids the runtime overhead of strrchr used
in logged filenames on iOS, which is unnecessary
on string literals, and adds unit-tests.

We also strip leading './' and '../' in other builds.

Change-Id: I56ec722d1400fad9165eb046d1bd8d4438241ca3
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2022-04-26 11:13:44 +03:00
Ashod Nakashian
14d96957cd wsd: log macro cleanup
* Simplify log macros.
* Reduce expanded lines (faster compilation).
* Avoid flushing (since we always use ostringstream).

Change-Id: Id10c1a75b5af5d8cdba62d768192ab0a6a9e0aca
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2022-04-26 11:13:44 +03:00
Ashod Nakashian
777463d9c5 wsd: automatically include a log prefix if provided
Change-Id: Ic542ec83ffa9c76478a4f1121caf3edf078f40a7
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2022-04-26 11:13:44 +03:00
Ashod Nakashian
b4e8975fbe wsd: minor cleanup of send-frame members
Change-Id: Ic2268780ea815611bf978c0c779ef737facc73fb
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2022-04-20 08:34:59 -04:00
Ashod Nakashian
480fb7b931 wsd: move tokenizer helpers into StringVector
In an attempt to reduce the size of Util.{c,h}pp
which has grown to contain all sorts of unrelated
helpers, we move StringVector helpers into
the StringVector.{c,h}pp files.

This makes the code better organized.

Change-Id: I152f341606807ae66253415b951bc9f89b09df57
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2022-04-12 07:39:07 -04:00
Ashod Nakashian
264bdec250 wsd: test: move Authorization tests to RequestDetailsTests
And cleanup includes to reduce build time.

Change-Id: If2e242e6688b788b91d62351ef555b31aad1ecde
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2022-04-12 07:39:07 -04:00
Ashod Nakashian
4bec395c39 wsd: support serializing STATE_ENUM to ostream
Change-Id: I460e28488184f4f946eec9f5ae4f220af98cca56
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2022-04-06 18:33:45 -04:00
Ashod Nakashian
f5c936ee63 wsd: add LOG_ASSERT to replace assert with extra logging
In NDEBUG builds, where there is no assert macro,
we log at debug level to still find issues without
adding noise to the logs.

However, in ENABLE_DEBUG builds not only do we assert
but we also log at error level.

Change-Id: I773dbf7bb2b459e505e73d91505b13d8ed2ed6d8
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2022-04-06 18:33:45 -04:00
Ashod Nakashian
d8745f288d wsd: logging and minor cosmetics
Change-Id: Ia6ba5b333c8caf742de27be46faf7265a1a777a5
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2022-04-06 18:33:45 -04:00
Ashod Nakashian
1d37371944 wsd: minor improvements to logging and macros
This primarily combines the string-literals
that form the filename and line number to
reduce four calls to operator<< to only one.

Also fix LOK_ASSERT_STATE to use its argument.

Change-Id: I5e8eeb4f862fe8679b1e1f7a57412dc84dc30e92
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2022-04-06 18:33:45 -04:00
Ashod Nakashian
b34de7fad1 wsd: move STATE_ENUM to common/StateEnum.hpp
With some minor modifications.

Change-Id: I5bc06bd96bbf201ccaee87170d052b4ad2a6c778
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2022-04-06 18:33:45 -04:00
Miklos Vajna
212e3c0818 common: isValidUtf8: ensure chunk len is 4 at max
UTF-8 is expected to encode a code point using 1 to 4 bytes.

Also check that the promised chunks are there before accessing them.

Signed-off-by: Miklos Vajna <vmiklos@collabora.com>
Change-Id: I9b4f346738865fc18242e51a33c97e5b38ee90d8
2022-04-06 10:26:24 +02:00
Rash419
c2be0b3a48 wsd: add: support to define multiple domain with '|'
fix: in regex, escaping special character fails to find host of the alias

Signed-off-by: Rash419 <rashesh.padia@collabora.com>
Change-Id: I19bfacc5e45af26832cb8c6a7d249e0c7de56624
2022-04-05 13:47:57 +02:00
Michael Meeks
db63703d0c Validate as utf-8 all Websocket text messages in debug mode.
Ensure that rendersearchresult: is terminated, and identified as binary.

Change-Id: I84e83c63351cd0aac0923bf7c833b14c1be2c051
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
2022-04-04 20:46:11 +01:00
Ashod Nakashian
11f31b0ba4 wsd: test SSL hostname
Change-Id: I0895d90d2c9be8a76afd209446ae39cd33fa5d2e
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2022-04-01 10:33:55 +01:00
Ashod Nakashian
099246d67a wsd: unload when no Kit and nothing to upload
When the Kit dies and there is no data saved to
disk to upload, there is nothing to do but unload
DocBroker and log the fact.

With unit-test that simulates the situation by
killing the Kit of a modified document.

Change-Id: I7dcc0583c053d7166935d15d045fcf6624e43b4e
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2022-03-30 23:18:57 -04:00
Pranam Lashkari
f0026accbf l10n: if English variant is not specified then choose US as default
Signed-off-by: Pranam Lashkari <lpranam@collabora.com>
Change-Id: Iba4f96c75d69e52044f6460c4e26d0ab1d187d3c
2022-03-30 16:51:56 +05:30
Rash419
31d8822758 wsd: cleanup: moved all parsing related static methods and containers to HostUtil.cpp
Signed-off-by: Rash419 <rashesh.padia@collabora.com>
Change-Id: I8555522c6216f893b90ba4c27747314830f7abd2
2022-03-29 12:18:31 +02:00
Michael Meeks
9f5c5dcdf4 Show (E) for experimental mode in help->about if present.
Change-Id: I00d6cb861c3050d5b4f80b0cb41e70012d0fe610
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
2022-03-24 17:25:40 +00:00
Rash419
ea35dbe350 wsd: added regex support for alias tag
Signed-off-by: Rash419 <rashesh.padia@collabora.com>
Change-Id: I2534ea9afa7d3d3bf10f91eeac44f70381acc76a
2022-03-22 14:13:22 +01:00
Pranam Lashkari
defef6a171 util: removed duplicate string trimming function
Signed-off-by: Pranam Lashkari <lpranam@collabora.com>
Change-Id: Ia150975467d1df8a9d05c8543bcb5d7369d29099
2022-03-22 09:07:03 -04:00
Ashod Nakashian
140b4f9c08 wsd: minor improvements to StringVector::equals overloads
Change-Id: Ifa975e579392151bef422ff985a93abf3129dfeb
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2022-03-21 08:57:41 +00:00
Ashod Nakashian
97a6d1cd98 wsd: support experimental features
This adds a new config option to enable/disable
experimental features and behavior. The default
value can be controlled at build time by
--enable-experimental.

Change-Id: Iffcb4c71d9e0933a646251b63033b6dadcd3b809
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2022-03-17 18:46:46 -04:00
Ashod Nakashian
9732689668 wsd: stop loading when low on disk space
For some reason the disk-space check wasn't
done for WOPI storage. Here we add the check
and bubble the exception up to stop loading.

UnitStorage has been updated and re-enabled.

Change-Id: I15a093554b662d6c0828da7683ca0190a8225fc9
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2022-03-16 08:31:54 -04:00
Michael Meeks
4e9d58b0b3 signal handling: print code, and relevant address too.
Change-Id: I8e975886800ce02cb12e840ad36a818145c1112a
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
2022-03-14 16:34:42 +00:00
Ashod Nakashian
88e03a47d3 wsd: support new clang versions for fuzzing
Now we choose static-libasan vs static-libsan
at configuration time, dynamically. This
enables supporting post clang-9 releases
which do not have libasan.

In addition, clang warns about the order
and visibility of operator<< for the chrono
types. Those are fixed to.

Change-Id: I17f5fc4b3e34464194ad686542935c9f596366e4
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2022-03-09 19:24:40 -05:00
Ashod Nakashian
832e308e8f wsd: more informative timestamps in dumpState
The timestamps now helpfully include how
long ago they were.

This also improves the conversion between
chrono clocks by adding generic functions.

Change-Id: I18081330564c8d969b9364cf1b9e33546e241280
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2022-03-09 19:24:40 -05:00
Rash419
50678c5191 wsd: introduce remote configuration
fetch JSON from remote server and apply new config without restarting coolwsd
- Extended the feature_locked configuration & functionality so that it can be set per wopi host ranges

Signed-off-by: Rash419 <rashesh.padia@collabora.com>
Change-Id: Id0e2b4b524a80de88b2b3559dd18c4a95b8163fb
2022-02-25 18:36:06 +03:00
Ashod Nakashian
c9f8edc08a wsd: const correctness and minor cleanups
Change-Id: Ic3dd918e371f74ac59684f09f0b4f5bc0a7b9b0a
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2022-02-24 07:38:13 -05:00
Ashod Nakashian
b53d353779 wsd: reduce debug and generally redundant logs
In 'debug' log-level we expect a detailed, but
still readable output. Having one area with
disproportionately large number of logs reduces
the overall utility of the log output.

This patch reduces a number of redundant log
entries, including errors that are already
logged. It also reduces the level of some
others from 'information' to 'debug' and
from 'debug' to 'trace'.

The goal is to make 'debug' level as useful as
possible to read the progress and be able to
understand what was going on, such that one is
able to decide which area to dig deeper into.
Then, trace level could be used to get more
insight into that area, if necessary. For
example, when investigating a test failure,
one first enables 'debug' logs and reads through.
Once a section between two debug entries is
identified as being of interest, enabling 'trace'
level logs becomes more productive as it's
now possible to easily reach the first DBG
entry and read through until the second one.

It's unfortunate that we don't have per-area
control for enabling/disabling logs, so it
is common to see more and more 'debug' log
entries added all around, making logs
less and less readable.

It is also a limitation of the levels we have
that we really only have 3 usable levels:
one, two, many. That is, 'information' for
the most important events, 'debug' for
technical details needed to investigate issues,
and 'trace' for everything else. ('warning'
and 'error' aren't really 'levels'; they have
semantics that makes them special-cases.)
So we have to avoid degrading one into the
other, or have differences without distinction.

If any of these entries are needed to be
displayed more frequently, changing them
back to 'debug' or even 'information' should
be done. Though for me they seem special
cases that don't benefit most log readings.

Change-Id: Id2c6a9dc027483b81a066b0b4b50a298c5eff449
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2022-02-24 07:38:13 -05:00
Ashod Nakashian
bef9c3ff65 wsd: dump all the state managers in DocBroker
This adds dumpState to DocumentState, SaveManager,
and StorageManager classes, and dumps all the
missing members.

Also, normalize the format and make it symmetric
and consistent.

Change-Id: Ie0cc8e07d13de60c33d64cd621abf4e815a4ef94
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2022-02-24 07:38:13 -05:00
Gleb Popov
cec1a927c2 cleanupJails: Postpone removing tmp directory until the end.
Otherwise, coolmount refuses to unmount devfs hierarchy on FreeBSD, because
the tmp/dev directory doesn't exist anymore.

Signed-off-by: Gleb Popov <6yearold@gmail.com>
Change-Id: I412f6947c4e1f3c04c2504e6cd88ca5645219878
2022-02-21 10:20:01 +01:00
Gleb Popov
34574724b4 Teach coolmount to mount devfs on FreeBSD.
Use it to create /dev/{u}random devices instead of calling mknod().

Signed-off-by: Gleb Popov <6yearold@gmail.com>
Change-Id: I6ae7842c89255f3f43a8ac7f082a275d5496690a
2022-02-21 10:20:01 +01:00