Commit graph

1001 commits

Author SHA1 Message Date
Michael Meeks
29db7db278 logging: calm down dropping of on-the-fly messages.
These tend to be sent at small increments of time, so have some
hysteresis to aggregate drops. Stalled JS such as in a debugger
or a client not responding can cause this for quite some tiles.

Change-Id: Ia807d4ba0ac26a95cad435cba9ed191f22706041
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
2024-01-08 08:32:12 +01:00
Michael Meeks
b10b9276b8 Use /proc/self/smaps_rollup for improved perf #7927
Using the aggregated file saves a hundred+ system-calls per
pid we're getting data for, and presumably also lots of time.

Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
Change-Id: I41c40982ebbec44aba72a1d15dabf24a8986f59e
2024-01-03 20:20:02 +00:00
Jaume Pujantell
35d6e2925f use isKitInProcess() instead of KIT_IN_PROCESS
Removed all uses of KIT_IN_PROCESS, used Utils::isKitInProcess()
when necessary. Also removed the now unused parameter 'limit' from
forkLibreOfficeKit.

Signed-off-by: Jaume Pujantell <jaume.pujantell@collabora.com>
Change-Id: I068d3f55ab49076590f111847c87b3188f4d25d0
2023-12-18 10:34:10 +00:00
Jaume Pujantell
eca990920a add logic to sapwn threads intead of processes
Added code that spawns forkit and lokit in threads instead of new processes.

Signed-off-by: Jaume Pujantell <jaume.pujantell@collabora.com>
Change-Id: I6b6ad6dccfedaf28ebd0d68851205c540b9a0317
2023-12-18 10:34:10 +00:00
Ashod Nakashian
6a42339640 wasm: log to stdout instead of stderr
Change-Id: I747945635c749127d2851cab1f738f4fa58d99f7
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2023-12-09 19:22:24 +00:00
Ashod Nakashian
a96f8a0f92 configure: WASMAPP is now defined for WASM targets
This is in line with MOBILEAPP and GTKAPP and
is independent of __EMSCRIPTEN__.

Replace all application-specific cases, leaving
only toolchain-specific ones (such as include
protection).

Change-Id: Ic44288aa5ace51e0a8b54566170c19120f7bf0dd
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2023-12-09 19:22:24 +00:00
Darshan-upadhyay1110
3f46c1db44 kit-in-process: pure re-factor to a run-time function to flag this.
Avoids a number of compile time conditionals and adds flexibility.

Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
Change-Id: Iff6b294b504526e70715e436ad33d47c8df4752c
2023-11-22 17:06:29 +00:00
Michael Meeks
6a32adcfe8 delta: fix threading gotcha when working.
The ThreadPool::work function can get its condition signalled -very- late.

With bad timing, this can occur after all the work is done, and when the
next batch of work is being fed into the pool.

This can mean that it takes work from the queue, and subverts the:

        bool useThreads = _threads.size() > 1 && _work.size() > 1;

check in ThreadPool::run - which can believe we are in a single
threaded, single tile mode - and not wait for this thread to complete.

That's not good [!] so ensure that threads are only runnable during
ThreadPool::run.

Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
Change-Id: Ifebb0f15cbb4c22ef33ffba06e7c6c87493818be
2023-11-20 19:24:13 +00:00
Michael Meeks
1a4dafa3e5 Fetch available memory and limits from our control group not the system.
To test:

sudo mkdir /sys/fs/cgroup/memory/0
echo "900M" | sudo tee /sys/fs/cgroup/memory/0/memory.limit_in_bytes
echo $$ | sudo tee /sys/fs/cgroup/memory/0/tasks
make run # and check the log.

Change-Id: I81cf5f6212418d1f900a56cdfe476e1594f4fe77
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
2023-11-14 19:36:31 +00:00
Michael Meeks
7183a3d3de spdx: improve machine and human readability of headers.
Change-Id: Ice934380029bf27054e830fffc07a5d037d1430f
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
2023-11-14 19:36:31 +00:00
Andras Timar
78d4fe22b3 fix log date, tm.tm_mon is 0-11
Signed-off-by: Andras Timar <andras.timar@collabora.com>
Change-Id: I93f7ac7a83c9724537b9b214cd8b54e5132c54e4
2023-11-09 13:20:07 +01:00
Caolán McNamara
e0c161c17b These warnings are frequent but harmless
Signed-off-by: Caolán McNamara <caolan.mcnamara@collabora.com>
Change-Id: Id2517f310a1df71dc1357d103ea074bdd12e7e65
2023-10-25 20:08:20 +02:00
Ashod Nakashian
c3fb1d0dde wsd: helper to serialize pairs
Since we don't want to propagate Poco types
as well as our pair containers, it's best
to have these are named functions rather
than as operator<<.

Change-Id: I9772b8f314262147b6c445464bf3ce8cb598b56d
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2023-10-24 21:33:02 -04:00
Ashod Nakashian
44cc7145c2 wsd: helper to concatenate streamable elements together
Change-Id: I0d989b54d5eebbd3efee2502d84a82281ebf62a7
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2023-10-24 09:20:32 +02:00
Caolán McNamara
d6c1916cb2 cool#7374 very little delta threading
We only run one task in each thread when run() is called, then complete
the remainder of tasks in the main thread while the other threads wait.

https: //github.com/CollaboraOnline/online/issues/7374
Signed-off-by: Caolán McNamara <caolan.mcnamara@collabora.com>
Change-Id: Id4fc0a620d98b0bb55310b495eeff7411a4544b5
2023-10-16 09:26:32 +01:00
Caolán McNamara
9ae7c294bc cid#318975 help coverity see assert
Signed-off-by: Caolán McNamara <caolan.mcnamara@collabora.com>
Change-Id: Ibf735e47362715d442c0122b9a29a0225d1cdc9a
2023-10-09 10:24:11 +02:00
Michael Meeks
71213b675e logging: allocate temporary buffer on the stack.
Change-Id: I1805db7c9c6bc6c66afbb912a16e01f6bf0f2699
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
2023-10-06 09:11:15 +02:00
Caolán McNamara
3e67acd361 cid#318942 Unchecked return value from library
use a more standard way to zero the _sb stat struct

Signed-off-by: Caolán McNamara <caolan.mcnamara@collabora.com>
Change-Id: I98973affe6b96a5779cfc3eef3c349019c34d0a0
2023-10-04 09:57:39 +02:00
Skyler Grey
e6d9c1f87c Add support for URP messages in COOLWSD
- Allow COOLWSD client sessions to forward messages with the prefix
  'urp' to the child, and return messages with 'urp:' to the client,
  communicating with binary
- Make COOLWSD child sessions use the FunctionBasedURPConnection from
  https://gerrit.libreoffice.org/c/core/+/155100
  (core change ID I2bda3d0b988bef7883f9b6829eeb5b7ae8075f27) to start a
  new URP session
- Make COOLWSD child sessions submit messages to this URP session,
  stripping and adding the 'urp' and 'urp:' prefixes so the Java client
  from https://gerrit.libreoffice.org/c/core/+/154680
  (core change ID I91ee52922a24688a6b94512cb7e7bc760bf25ec9) can
  use the connection (and to avoid interference with any other websocket
  messages)
- Add a COOLWSD option for enabling/disabling URP given the security
  implications around allowing anyone to write URP (e.g. URP lets you
  run shell commands so a mallicious actor can take over the child
  session)

Signed-off-by: Skyler Grey <skyler.grey@collabora.com>
Change-Id: Idadfe288a78cfd72b01253dfdade150d506e3f05
2023-10-03 10:15:55 +01:00
Ashod Nakashian
7d0276c394 killpoco: replace LocalDateTime
This replaces Poco's LocalDateTime, which
took a lock, called tzset each time, and
did unnecessary Julian-to-Gregorian conversion.

The result is a standards-compliant implementation
that is about an order of magnitude faster (0.3 us
vs 3 us on average).

Change-Id: Iced73056a9f9e6497106224c351110113e21d8d6
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2023-10-02 20:14:53 +01:00
Ashod Nakashian
c5d47b4b20 wsd: log: verify that all threads are stopped
This verifies that in debug-builds the log
thread-local buffers are destroyed, and
therefore the threads are gracefully stopped.

This is necessary to make sure no log entries
are lost by the time we exit the process.

Change-Id: I0db20835109e92d8758a5c4eec6845cd240da025
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2023-10-02 20:14:53 +01:00
Ashod Nakashian
483a602f93 wsd: log: BufferedConsoleChannel with thread-local buffer
This gives each logging thread its own buffer.
We then guarantee that all threads exit clearly
and flush their buffers.

For now, we don't flush buffers when fatally signaled,
primarily because of technical complexities.
Specifically, we don't know if we have a Poco logger
or our Buffered Logger, and there are a host of
threading and signal-safety concerns. To be revisited.

Change-Id: I39674a25feeee3aabd87c8b707ea1adf3c039817
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2023-10-02 20:14:53 +01:00
Ashod Nakashian
150663cc28 wsd: stop the rendering thread pool before exiting
Change-Id: I8a084ee1fd583f6c37ba4ca8c1ebf8abe74288a2
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2023-10-02 20:14:53 +01:00
Ashod Nakashian
370b649b3e killpoco: own implementation of ColorConsoleChannel
This replaces Poco's ColorConsoleChannel with our
own, which is simpler and more efficient. It also
doesn't break log entries arbitrarily (only when
the entry is exceedingly long).

Change-Id: I46ba58d35bc913ca83fe97aaaa958c03b81ae44e
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2023-10-02 20:14:53 +01:00
Ashod Nakashian
066af307ba killpoco: own implementation of BufferedConsoleChannel
This is a buffered version of our ConsoleChannel.

Change-Id: Ie1728e136376fdf82ec28010b67d8344fd812768
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2023-10-02 20:14:53 +01:00
Ashod Nakashian
57a9cf053a wsd: support EnableExperimental in Log
Now that we depend on EnableExperimental in Log.cpp,
we must define it in the standalone binaries that
otherwise don't define it.

Change-Id: Ic56032eaf6df7d0d5d707a60eeddf2d75c9041ac
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2023-10-02 20:14:53 +01:00
Ashod Nakashian
a00fbae7c5 wsd: log: retry on write failure and flush
Change-Id: I7417a74959a52e2a079bfe4502ed238d1517ea60
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2023-10-02 20:14:53 +01:00
Ashod Nakashian
56fd825187 killpoco: our implementation of ConsoleChannel
This replaces Poco's implementation of ConsoleChannel
which has a horrible issue with writing the new line
in a separate syscall. This often results in multiple
log entries on the same line and other horrors.

This implementation does no buffering.

Change-Id: I8d18de1313d8f160270ec04552ba7e7b69d2363e
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2023-10-02 20:14:53 +01:00
Caolán McNamara
3b3b62c95c remove unused used method
which is a little dubious anyway

Signed-off-by: Caolán McNamara <caolan.mcnamara@collabora.com>
Change-Id: If12bd7807ca5f2cff0ca05cad4948962b0cf6688
2023-10-02 17:33:14 +01:00
Patrick Luby
4b71e988b4 Fix compiler warnings when building on macOS Sonoma
Signed-off-by: Patrick Luby <plubius@neooffice.org>
Change-Id: I7e88e0ba272fc00892059c96a2cd0237657e23b9
2023-10-02 08:50:52 +01:00
Caolán McNamara
2f9b0f8d62 cid#323356 Resource leak
Signed-off-by: Caolán McNamara <caolan.mcnamara@collabora.com>
Change-Id: If04703571b2e20c59bd5cd5fa271cb6859fb9887
2023-09-27 10:25:28 +02:00
Caolán McNamara
6208b37a32 cid#318945 Unchecked return value from library
Signed-off-by: Caolán McNamara <caolan.mcnamara@collabora.com>
Change-Id: I37cb5ddd7a1ce2dd1357738130f62950e76bbc6a
2023-09-26 16:08:09 +01:00
Caolán McNamara
0f5c171433 do init_gather_lut at start if simd::init succeeds
and avoid local static in simd_initPixRowSimd

Signed-off-by: Caolán McNamara <caolan.mcnamara@collabora.com>
Change-Id: Idb89d5069da5ff10b346b5e4d767374d4529a96f
2023-09-26 08:39:20 +01:00
Ashod Nakashian
2a58fcaa93 wsd: utility to read small files in memory
Change-Id: I9b98f8706c0c32ed59c4f49844570225532a1a59
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2023-09-25 19:29:12 -04:00
Michael Meeks
cce3767ba8 First cut SIMD wrappers / separation to accelerate RLE code.
Split it out as a C file, to avoid accidental C++ header inclusion,
and C is a cross-platform assembler anyway so a good match.

Change-Id: I6c042781713aecaf143b9663af8377659a7deaf1
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
2023-09-25 16:55:04 +01:00
Ashod Nakashian
3a91ff3e4e wsd: test: exit on failure to load unit-test lib
We now exit immediately with error code when
we are given --unitlib argument but fail to
load the given library.

Change-Id: I4eb5f3a37b9838ecf83bdceef4d09f888afcca83
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2023-09-25 08:58:39 +02:00
Patrick Luby
e7bd50c967 Fix minor iOS build and runtime problems
This patch contains changes to iOS and/or Android code only.

Signed-off-by: Patrick Luby <plubius@neooffice.org>
Change-Id: Ia1461b0b3ca374b682494a7e2e900b832c85ea9d
2023-09-12 00:49:18 +02:00
Miklos Vajna
790a19ab25 kit: improve date format of UNO commands in crashreports
Old format:

	kit-27634-02839 2023-08-30 11:08:43.590564 +0000 [ kitbroker_17a ] SIG   Fatal signal received: SIGSEGV code: 1 for address: 0x4f00000007
	      unoCommand(2552f) : ToolbarMode?Mode:string=notebookbar_online.ui - Wed, 30 Aug 2023 11:08:31

New format:

	kit-30199-30197 2023-09-06 14:30:21.417817 +0200 [ kitbroker_001 ] SIG   Fatal signal received: SIGABRT code: 18446744073709551610 for address: 0x3e8000075f7
	Recent activity:
		unoCommand(064) : ToolbarMode?Mode:string=notebookbar_online.ui - 2023-09-06 12:29:56

Signed-off-by: Miklos Vajna <vmiklos@collabora.com>
Change-Id: Idf62ea18c75c453d188d8c25723a43824d5fc147
2023-09-11 10:07:19 +02:00
Paris Oplopoios
e67fa117f6 Revert "Fix getting canonical view id of 1000"
This reverts commit 6448c8be0d6a6a58f46c6e1ec2755138b9e7e3d1.

Signed-off-by: Paris Oplopoios <paris.oplopoios@collabora.com>
2023-09-06 10:28:26 +02:00
Caolán McNamara
24317d555f ensure we don't split a utf-8 sequence when logging message
Given a well-formed utf-8 string 'message' of messageLen bytes and a
desire to truncate to approximately abbrevLen bytes return the shortest
string greater or equal to abbrevLen that does not split a utf-8
sequence.

Signed-off-by: Caolán McNamara <caolan.mcnamara@collabora.com>
Change-Id: Ie623d8c1027bb4724485cab5b0bb6d3d1cd3d9ab
2023-08-25 08:41:46 +02:00
Caolán McNamara
3e7ae50ec1 messages to be logged might not end in a new line
they sometimes do, and sometimes not, we can't assume that the last
byte is a newline and can be discarded without truncating message
text.

Signed-off-by: Caolán McNamara <caolan.mcnamara@collabora.com>
Change-Id: Ie5ea00f927a558b43b2ae38e1cac96c805e93fcc
2023-08-25 08:41:46 +02:00
Caolán McNamara
a3b32c5203 use the same level of building blocks in each getAbbreviatedMessage impl
Signed-off-by: Caolán McNamara <caolan.mcnamara@collabora.com>
Change-Id: Ia38dc3a8318c0d2adad9bf7f879b7b8a55e5ad77
2023-08-25 08:41:46 +02:00
Caolán McNamara
a4e74dedab std::string variant of getAbbreviatedMessage would include trailing newline
for a short non-abbreviated message if it ended in a newline which the
more common variant would not.

Signed-off-by: Caolán McNamara <caolan.mcnamara@collabora.com>
Change-Id: I4105fb8eb67b3a38aebc745dfb1e79aa99f2d878
2023-08-25 08:41:46 +02:00
Caolán McNamara
a3f5ca0057 one of the getAbbreviatedMessage variants creates results 1 char longer
the std::string variant uses:

Util::getDelimiterPosition(message.data(), std::min<size_t>(message.size(), maxNonAbbreviatedMsgLen + 1),...

while the more commonly used const char* version has

getFirstLine(message, std::min(length, maxNonAbbreviatedMsgLen)

where getFirstLine is...

return Util::getDelimitedInitialSubstring(message, length,...

and getDelimitedInitialSubstring has

const size_t size = getDelimiterPosition(message, length, delim);
return std::string(message, size);

so the std::string variant's condition is one char longer for the same input

Signed-off-by: Caolán McNamara <caolan.mcnamara@collabora.com>
Change-Id: I5b8c4523844501a43731ff3b4a939b76ebe472c3
2023-08-25 08:41:46 +02:00
Caolán McNamara
ca6044cd40 move max 500/501 len into a single place
Signed-off-by: Caolán McNamara <caolan.mcnamara@collabora.com>
Change-Id: I71f44b88b62bd1d2daa7d5019e14e58cc15c7e02
2023-08-25 08:41:46 +02:00
Caolán McNamara
afe6c1bed8 reduce cost of TileDesc::parse
perf reported 6.58% of time in collaborative multi user test
on 2023-08-24 was spent in TileDesc::parse and much of that
in std::unordered_map.

There are only 12 arguments in the map we care about here so
we can just used a sorted array, look by name on write, and
read by index.

Signed-off-by: Caolán McNamara <caolan.mcnamara@collabora.com>
Change-Id: Iadfc2001e298d8f4d46200c8488f0eb4cd8734c2
2023-08-24 22:22:43 +01:00
Caolán McNamara
da54b23538 cid#322632 COPY_INSTEAD_OF_MOVE
Signed-off-by: Caolán McNamara <caolan.mcnamara@collabora.com>
Change-Id: If53b75521fa07d89828aaff06834f741e2b64a5a
2023-08-24 08:30:40 +02:00
Caolán McNamara
492f678996 add and use 'N_ELEMENTS' in lieu of std::size
which isn't available in the current android toolchain

Signed-off-by: Caolán McNamara <caolan.mcnamara@collabora.com>
Change-Id: I5834adb7c6211c7aad38f5977a7e425d9ca257fd
2023-08-23 12:06:31 +02:00
Caolán McNamara
6d5c78dec0 move "combined" bit inside TileCombined class
Signed-off-by: Caolán McNamara <caolan.mcnamara@collabora.com>
Change-Id: I274c5844660acbf69e50587cce3f4ddcae414723
2023-08-21 20:48:31 +01:00
Caolán McNamara
851c232531 cid#318843 COPY_INSTEAD_OF_MOVE
Signed-off-by: Caolán McNamara <caolan.mcnamara@collabora.com>
Change-Id: Icb757fdc094f428a351df5db6afa53da04be51d1
2023-08-16 15:15:59 +02:00