A number of call-sites, eg. clipboard, or admin-ws were
writing to sockets assuming they could return all the data
in a single series of writes, without needing to poll. As
such they failed to addSocketToPoll on the new poll - eg.
the docBroker. Unfortunately this meant that on EAGAIN
writes, the socket would be closed and the last parts
of a message lost.
Browsers would give net::ERR_CONTENT_LENGTH_MISMATCH 200 (OK)
The situation is/was intermittent, so painful to debug.
On under-loaded developer machines, socket buffers are larger,
so this was seldom seen.
The re-factor forces a transfer to another SocketPoll via
the disposition, except for a couple of corner cases.
Change-Id: I2f1b2f99f179c4fda84464c9241fe434fa527725
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
Avoid continually shuffling large output buffers down in memory,
it kills CPU caches, does nothing useful, and we can just advance
an offset instead. Of course, we do need to free memory - so do
that for lots of common cases.
Change-Id: I63240661ca87bdc0574acbce856771d9dde3fa94
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
For large transfers eg. image previews, particularly with SSL's
protocol limit of 16k byte blocks, we see lots of inefficiency
repeatedly copying a 20Mb image and shuffling it down a
std::vector as we write data out.
Change-Id: I620568cad2e6f41684c35289b0ee77cf7f59c077
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
Otherwise, there is no way to know which connection
a given log belongs to, when websrv_poll handles
multiple simultaneous connections.
Also removed a duplicate log entry.
Change-Id: Id74e6e7b66062daf15a716b9fea8e7995fd75af0
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
This is non-functional change to have all HttpHelper
members in a single place (HttpHelper.cpp).
Clang-format is applied and FileUtil::Stat is used
instead of a naked stat(2).
Change-Id: I663e2eb3070e873bd1f4477b60e45b0086d29f4e
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
Per the rfc (https://tools.ietf.org/html/rfc6455#section-5.5.2):
"An endpoint MAY send a Ping frame any time after the connection
is established and before the connection is closed."
And "Upon receipt of a Ping frame, an endpoint MUST send a Pong
frame in response, unless it already received a Close frame."
Here we allow for pings to come from clients and we respond
to them by pongs, as required by rfc 6455.
Change-Id: I8e285f095526e4b67373ecb3ae1efc9c8717d756
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/102948
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Andras Timar <andras.timar@collabora.com>
Because Admin derives from it, and UnitPHPProxy holds a
shared_ptr<SocketPoll>, so the non-virtual dtor was invoked for a class
whcih is not (and can't be) final.
Found by clang's -Werror,-Wdelete-non-abstract-non-virtual-dtor.
Change-Id: I9b31bb58e79b2da3a3896789c52258f315fdd3d6
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/102355
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
Sometimes multiple messages are processed in a single iteration
at socket level. This happens in WebSocketHandler and when draining
Document queue.Just covered these cases.
Change-Id: Ifa46f5d484b67015cca64008b2c89426cc839e64
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/99387
Tested-by: Jenkins
Tested-by: Gabriel Masei <gabriel.masei@1and1.ro>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Reviewed-by: Gabriel Masei <gabriel.masei@1and1.ro>
The map._activate, among other actions, is sending indirectly some messages
to the server like clientzoom and clientvisiblearea. If these messages are send
before the document finishes processing the load message then there is
a chance that a nodocloaded error will be thrown because there is a
chance that the messages will be processed in parallel with load. This happens
constantly for xlsx files. This is generated by the Unipoll mechanism which,
in case of xlsx files, triggers a parallel processing.
To avoid the above scenario a mechanism of disabling parallel processing of
messages in kit was implemented and is used for load and save messages, for now.
Change-Id: I4c83e72e600f92d0bb4f1f18cebe694e326256d0
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/98519
Tested-by: Jenkins
Tested-by: Michael Meeks <michael.meeks@collabora.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
We probably used to have circular references that made KitSocketPoll
and KitWebSocketHandler objects hang around forever, or something.
(Not a problem in web-based Online where kit processes have a
restricted lifetime.)
Change-Id: Ia6eebc51f4a4a8fb4f69a2c83a0131de921ea1d6
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/98744
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
loolmount now works and supports mounting and
unmounting, plus numerous improvements,
refactoring, logging, etc.. When enabled,
binding improves the jail setup time by anywhere
from 2x to orders of magnitude (in docker, f.e.).
A new config entry mount_jail_tree controls
whether mounting is used or the old method of
linking/copying of jail contents. It is set to
true by default and falls back to linking/copying.
A test mount is done when the setting is enabled,
and if mounting fails, it's disabled to avoid noise.
Temporarily disabled for unit-tests until we can
cleanup lingering mounts after Jenkins aborts our
build job. In a future patch we will have mount/jail
cleanup as part of make.
The network/system files in /etc that need frequent
refreshing are now updated in systemplate to make
their most recent version available in the jails.
These files can change during the course of loolwsd
lifetime, and are unlikely to be updated in
systemplate after installation at all. We link to
them in the systemplate/etc directory, and if that
fails, we copy them before forking each kit
instance to have the latest.
This reworks the approach used to bind-mount the
jails and the templates such that the total is
now down to only three mounts: systemplate, lo, tmp.
As now systemplate and lotemplate are shared, they
must be mounted as readonly, this means that user/
must now be moved into tmp/user/ which is writable.
The mount-points must be recursive, because we mount
lo/ within the mount-point of systemplate (which is
the root of the jail). But because we (re)bind
recursively, and because both systemplate and
lotemplate are mounted for each jails, we need to
make them unbindable, so they wouldn't multiply the
mount-points for each jails (an explosive growth!)
Contrarywise, we don't want the mount-points to
be shared, because we don't expect to add/remove
mounts after a jail is created.
The random temp directory is now created and set
correctly, plus many logging and other improvements.
Change-Id: Iae3fda5e876cf47d2cae6669a87b5b826a8748df
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92829
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Seems to not cause any serious regressions in the iOS app or in "make
run", but of course I am not able to run a comprehensive check of all
functionality.
Change-Id: I44a0e8d60bdbc0a885db88475961575c5e95ce88
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/93037
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
There aren't multiple processes that would need to "ping" each others.
Ifdef out the related member variables and code completely. Having
them partially in caused lots of FakeSocket polling with zero timeout
which is less than ideal.
Change-Id: Ibdfa4980d6d4fc9c00ea5146ca8d75ca0df81f1d
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/97021
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
Set a FAKESOCKET_LOG_LEVEL environment variable to "2" for more
verbose logging. This is how it used to be, and is indeed very
verbose, as each poll, read, and write operation is logged.
(Normally the FakeSocket logging does not get displayed, though, as it
is passed to LOG_INF() and the default LOOL_LOGLEVEL is "warning". To
see it, either set FAKESOCKET_LOG_ALWAYS_STDERR or set LOOL_LOGLEVEL
appropriately.)
With the default log level 1 only creation, connection, and closing of
FakeSockets is logged, and the state of all active ones is displayed
after each established connetion and when a FakeSocket has been
closed. This is usually enough to get a basic trace of how the
plumbing works.
Change-Id: Id87bd32ce06105af561aa6a75cf365b41c079713
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/96943
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
re-factor to make it hard not to.
Change-Id: I26ebc48b4660276ede64a22167ac4779cebf5cd4
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/95440
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
More readable and typically more efficient.
Change-Id: I9bd5bfc91f4ac255bb8ae0987708fb8b56b398f8
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/95285
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Sometimes kit process goes into a heavy processing state (or even hangs)
and is not able to report its memory usage. Thus we can't implement cleanup
of problematic kit processes based on memory information reported by kit.
By moving memory reporting to admin module we avoid this problem.
Change-Id: Icf274e3a3a97b33623a93f9d2dc1e640ad9b7d99
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92752
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
The current thread's id is returned by std::this_thread::get_id().
std:🧵:id() (the result of the std::thead::id constructor), on
the other hand, does not represent any thread.
Change-Id: I13f2831810282109d7bce984f1d040595c466712
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92881
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
Weak pointers can be null and must be
checked before using. This fixes at least
one segfault and prevents a number of others.
Also, minimizes locking of weak pointers
in the message handlers.
Change-Id: I306501c26c3441d7bd6812d51fa17e7356126f32
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92828
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
