Cue up some basic state for being signal safe dumped on crash/abort.
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
Change-Id: Ibc6713aef2a0e7b878b178b05f0e13c8d40b47fc
This enables our own logging (much faster)
out of the experimental flag and enables
permanently.
Not for backporting.
Change-Id: I15161ff4f82a2e298942f81f35aea305bb06f683
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
so remove one in favor of the other
Signed-off-by: Caolán McNamara <caolan.mcnamara@collabora.com>
Change-Id: I47778f7bce24f0687565aa179b7a3bbea9d95120
This moves the clipboard resque logic earlier to
make sure it is cached before the Kit process
exits.
By putting the clipboard-fetching command before
the save-before-exit, we should get the clipboard
in most-all cases. There are still edge-cases
that can slip through, but they should be much
less frequent, and they will require much more
involved logic (tbd).
Change-Id: I0eff95c28524002a6131fa0a57ef8e09ae21ca59
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
Avoid using LOG_TRC etc. during static initializers before logging
is setup:
#0 0x00007ffff7362183 in std::string::size() const () from /lib64/libstdc++.so.6
#1 0x00000000007d1acc in std::operator==<char> (__lhs="", __rhs=<error reading variable: Cannot access memory at address 0xffffffffffffffe8>)
at /opt/rh/devtoolset-12/root/usr/include/c++/12/bits/basic_string.h:3586
#2 0x00000000008fdc04 in Poco::Logger::unsafeGet (name="") at src/Logger.cpp:289
#3 0x00000000008fdb81 in Poco::Logger::get (name="") at src/Logger.cpp:280
#4 0x000000000066a30f in Log::logger () at /opt/rh/devtoolset-12/root/usr/include/c++/12/bits/new_allocator.h:90
#5 0x0000000000692f8e in Util::rng::getBytes (length=16) at ../common/Util.cpp:122
#6 0x0000000000693312 in Util::rng::getSeed () at ../common/Util.cpp:91
#7 0x000000000044f987 in __static_initialization_and_destruction_0 (__initialize_p=1, __priority=65535) at ../common/Log.hpp:87
Change-Id: I6a32a0d3f50a8e769994737d4c09befe11599b4e
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
And improve quality of low-grade random numbers significantly.
_rd.entropy() ? is not your friend.
Change-Id: I477557245949334bec517cdeae653c7452ed2049
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
we want the normal ones, so we see the output in our own console, but we
don't want the others.
It would be better to use posix_spawn_file_actions_addinherit_np here,
but that seems to be currently macos-only.
Signed-off-by: Noel Grandin <noel.grandin@collabora.co.uk>
Change-Id: Ifd42c51f68ca2f340b78066b381cdbc6a23f0abe
using fork(), the kernel needs to copy the VM data structures,
which can be quite large for the main COOL processes.
I looked into things like vfork(), clone() and using a helper-process,
but it looks like posix_spawn is sufficient for our needs.
Internally it uses clone(CLONE_VM) which means we avoid the VMA copies.
Also, simplify the arguments of spawnProcess,
we don't use the optional params anymore.
Signed-off-by: Noel Grandin <noel.grandin@collabora.co.uk>
Change-Id: I8f943541dc4d7f56c2d36ccbf7b78c40ec14b8e0
Eight is a little few with multiple users around.
Change-Id: I0fe3e8a463463b9d7f2bb3e9efdc4aa12e7777e9
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
Add a way to search with an offset in findInVector() and then we can
avoid strstr() which assumes a null-terminated input, which may not be
the case.
READ of size 55 at 0x606000068a56 thread T8 (docbroker_001)
#0 0x55d0d6718a5a in StrstrCheck(void*, char*, char const*, char const*) /home/abuild/rpmbuild/BUILD/llvm-15.0.7.src/build/../projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:652:5
#1 0x55d0d6718882 in __interceptor_strstr /home/abuild/rpmbuild/BUILD/llvm-15.0.7.src/build/../projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:669:5
#2 0x55d0d71872c9 in ClientSession::postProcessCopyPayload(std::shared_ptr<Message> const&)::$_4::operator()(std::vector<char, std::allocator<char>>&) const /home/vmiklos/git/collaboraonline/online-23.05-san/wsd/ClientSession.cpp:1631:31
During unit-uno-command.
Signed-off-by: Miklos Vajna <vmiklos@collabora.com>
Change-Id: Id42669d912bab4746d9f7d99a192c4c3f1d175f7
/usr/bin/../lib64/gcc/x86_64-suse-linux/12/../../../../include/c++/12/bits/stl_vector.h:1124:9: runtime error: reference binding to null pointer of type 'char'
#0 0x557ce8c06055 in std::vector<char, std::allocator<char>>::operator[](unsigned long) /usr/bin/../lib64/gcc/x86_64-suse-linux/12/../../../../include/c++/12/bits/stl_vector.h:1124:2
#1 0x557ce8f16e3f in FileUtil::readFile(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, int) /home/vmiklos/git/collaboraonline/online-23.05-san/test/../common/FileUtil.cpp:449:37
While running toplevel 'make'. Both 'st.st_size' and 'off' were 0, so
this is an edge case.
Signed-off-by: Miklos Vajna <vmiklos@collabora.com>
Change-Id: I4899bffa48a761c84b6ee7d42776453b641ff5a7
But re-write them anyway to be more sensible.
Change-Id: Ie146f4f84b539ab7e826a1c1b947497acde7e384
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
These tend to be sent at small increments of time, so have some
hysteresis to aggregate drops. Stalled JS such as in a debugger
or a client not responding can cause this for quite some tiles.
Change-Id: Ia807d4ba0ac26a95cad435cba9ed191f22706041
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
Using the aggregated file saves a hundred+ system-calls per
pid we're getting data for, and presumably also lots of time.
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
Change-Id: I41c40982ebbec44aba72a1d15dabf24a8986f59e
Removed all uses of KIT_IN_PROCESS, used Utils::isKitInProcess()
when necessary. Also removed the now unused parameter 'limit' from
forkLibreOfficeKit.
Signed-off-by: Jaume Pujantell <jaume.pujantell@collabora.com>
Change-Id: I068d3f55ab49076590f111847c87b3188f4d25d0
Added code that spawns forkit and lokit in threads instead of new processes.
Signed-off-by: Jaume Pujantell <jaume.pujantell@collabora.com>
Change-Id: I6b6ad6dccfedaf28ebd0d68851205c540b9a0317
This is in line with MOBILEAPP and GTKAPP and
is independent of __EMSCRIPTEN__.
Replace all application-specific cases, leaving
only toolchain-specific ones (such as include
protection).
Change-Id: Ic44288aa5ace51e0a8b54566170c19120f7bf0dd
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
Avoids a number of compile time conditionals and adds flexibility.
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
Change-Id: Iff6b294b504526e70715e436ad33d47c8df4752c
The ThreadPool::work function can get its condition signalled -very- late.
With bad timing, this can occur after all the work is done, and when the
next batch of work is being fed into the pool.
This can mean that it takes work from the queue, and subverts the:
bool useThreads = _threads.size() > 1 && _work.size() > 1;
check in ThreadPool::run - which can believe we are in a single
threaded, single tile mode - and not wait for this thread to complete.
That's not good [!] so ensure that threads are only runnable during
ThreadPool::run.
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
Change-Id: Ifebb0f15cbb4c22ef33ffba06e7c6c87493818be
To test:
sudo mkdir /sys/fs/cgroup/memory/0
echo "900M" | sudo tee /sys/fs/cgroup/memory/0/memory.limit_in_bytes
echo $$ | sudo tee /sys/fs/cgroup/memory/0/tasks
make run # and check the log.
Change-Id: I81cf5f6212418d1f900a56cdfe476e1594f4fe77
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
Since we don't want to propagate Poco types
as well as our pair containers, it's best
to have these are named functions rather
than as operator<<.
Change-Id: I9772b8f314262147b6c445464bf3ce8cb598b56d
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
We only run one task in each thread when run() is called, then complete
the remainder of tasks in the main thread while the other threads wait.
https: //github.com/CollaboraOnline/online/issues/7374
Signed-off-by: Caolán McNamara <caolan.mcnamara@collabora.com>
Change-Id: Id4fc0a620d98b0bb55310b495eeff7411a4544b5
use a more standard way to zero the _sb stat struct
Signed-off-by: Caolán McNamara <caolan.mcnamara@collabora.com>
Change-Id: I98973affe6b96a5779cfc3eef3c349019c34d0a0
- Allow COOLWSD client sessions to forward messages with the prefix
'urp' to the child, and return messages with 'urp:' to the client,
communicating with binary
- Make COOLWSD child sessions use the FunctionBasedURPConnection from
https://gerrit.libreoffice.org/c/core/+/155100
(core change ID I2bda3d0b988bef7883f9b6829eeb5b7ae8075f27) to start a
new URP session
- Make COOLWSD child sessions submit messages to this URP session,
stripping and adding the 'urp' and 'urp:' prefixes so the Java client
from https://gerrit.libreoffice.org/c/core/+/154680
(core change ID I91ee52922a24688a6b94512cb7e7bc760bf25ec9) can
use the connection (and to avoid interference with any other websocket
messages)
- Add a COOLWSD option for enabling/disabling URP given the security
implications around allowing anyone to write URP (e.g. URP lets you
run shell commands so a mallicious actor can take over the child
session)
Signed-off-by: Skyler Grey <skyler.grey@collabora.com>
Change-Id: Idadfe288a78cfd72b01253dfdade150d506e3f05
This replaces Poco's LocalDateTime, which
took a lock, called tzset each time, and
did unnecessary Julian-to-Gregorian conversion.
The result is a standards-compliant implementation
that is about an order of magnitude faster (0.3 us
vs 3 us on average).
Change-Id: Iced73056a9f9e6497106224c351110113e21d8d6
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
This verifies that in debug-builds the log
thread-local buffers are destroyed, and
therefore the threads are gracefully stopped.
This is necessary to make sure no log entries
are lost by the time we exit the process.
Change-Id: I0db20835109e92d8758a5c4eec6845cd240da025
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
This gives each logging thread its own buffer.
We then guarantee that all threads exit clearly
and flush their buffers.
For now, we don't flush buffers when fatally signaled,
primarily because of technical complexities.
Specifically, we don't know if we have a Poco logger
or our Buffered Logger, and there are a host of
threading and signal-safety concerns. To be revisited.
Change-Id: I39674a25feeee3aabd87c8b707ea1adf3c039817
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
This replaces Poco's ColorConsoleChannel with our
own, which is simpler and more efficient. It also
doesn't break log entries arbitrarily (only when
the entry is exceedingly long).
Change-Id: I46ba58d35bc913ca83fe97aaaa958c03b81ae44e
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
This is a buffered version of our ConsoleChannel.
Change-Id: Ie1728e136376fdf82ec28010b67d8344fd812768
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
Now that we depend on EnableExperimental in Log.cpp,
we must define it in the standalone binaries that
otherwise don't define it.
Change-Id: Ic56032eaf6df7d0d5d707a60eeddf2d75c9041ac
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
This replaces Poco's implementation of ConsoleChannel
which has a horrible issue with writing the new line
in a separate syscall. This often results in multiple
log entries on the same line and other horrors.
This implementation does no buffering.
Change-Id: I8d18de1313d8f160270ec04552ba7e7b69d2363e
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
