A number of call-sites, eg. clipboard, or admin-ws were
writing to sockets assuming they could return all the data
in a single series of writes, without needing to poll. As
such they failed to addSocketToPoll on the new poll - eg.
the docBroker. Unfortunately this meant that on EAGAIN
writes, the socket would be closed and the last parts
of a message lost.
Browsers would give net::ERR_CONTENT_LENGTH_MISMATCH 200 (OK)
The situation is/was intermittent, so painful to debug.
On under-loaded developer machines, socket buffers are larger,
so this was seldom seen.
The re-factor forces a transfer to another SocketPoll via
the disposition, except for a couple of corner cases.
Change-Id: I2f1b2f99f179c4fda84464c9241fe434fa527725
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
For large transfers eg. image previews, particularly with SSL's
protocol limit of 16k byte blocks, we see lots of inefficiency
repeatedly copying a 20Mb image and shuffling it down a
std::vector as we write data out.
Change-Id: I620568cad2e6f41684c35289b0ee77cf7f59c077
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
This is non-functional change to have all HttpHelper
members in a single place (HttpHelper.cpp).
Clang-format is applied and FileUtil::Stat is used
instead of a naked stat(2).
Change-Id: I663e2eb3070e873bd1f4477b60e45b0086d29f4e
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
Because Admin derives from it, and UnitPHPProxy holds a
shared_ptr<SocketPoll>, so the non-virtual dtor was invoked for a class
whcih is not (and can't be) final.
Found by clang's -Werror,-Wdelete-non-abstract-non-virtual-dtor.
Change-Id: I9b31bb58e79b2da3a3896789c52258f315fdd3d6
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/102355
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
The map._activate, among other actions, is sending indirectly some messages
to the server like clientzoom and clientvisiblearea. If these messages are send
before the document finishes processing the load message then there is
a chance that a nodocloaded error will be thrown because there is a
chance that the messages will be processed in parallel with load. This happens
constantly for xlsx files. This is generated by the Unipoll mechanism which,
in case of xlsx files, triggers a parallel processing.
To avoid the above scenario a mechanism of disabling parallel processing of
messages in kit was implemented and is used for load and save messages, for now.
Change-Id: I4c83e72e600f92d0bb4f1f18cebe694e326256d0
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/98519
Tested-by: Jenkins
Tested-by: Michael Meeks <michael.meeks@collabora.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
We probably used to have circular references that made KitSocketPoll
and KitWebSocketHandler objects hang around forever, or something.
(Not a problem in web-based Online where kit processes have a
restricted lifetime.)
Change-Id: Ia6eebc51f4a4a8fb4f69a2c83a0131de921ea1d6
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/98744
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
loolmount now works and supports mounting and
unmounting, plus numerous improvements,
refactoring, logging, etc.. When enabled,
binding improves the jail setup time by anywhere
from 2x to orders of magnitude (in docker, f.e.).
A new config entry mount_jail_tree controls
whether mounting is used or the old method of
linking/copying of jail contents. It is set to
true by default and falls back to linking/copying.
A test mount is done when the setting is enabled,
and if mounting fails, it's disabled to avoid noise.
Temporarily disabled for unit-tests until we can
cleanup lingering mounts after Jenkins aborts our
build job. In a future patch we will have mount/jail
cleanup as part of make.
The network/system files in /etc that need frequent
refreshing are now updated in systemplate to make
their most recent version available in the jails.
These files can change during the course of loolwsd
lifetime, and are unlikely to be updated in
systemplate after installation at all. We link to
them in the systemplate/etc directory, and if that
fails, we copy them before forking each kit
instance to have the latest.
This reworks the approach used to bind-mount the
jails and the templates such that the total is
now down to only three mounts: systemplate, lo, tmp.
As now systemplate and lotemplate are shared, they
must be mounted as readonly, this means that user/
must now be moved into tmp/user/ which is writable.
The mount-points must be recursive, because we mount
lo/ within the mount-point of systemplate (which is
the root of the jail). But because we (re)bind
recursively, and because both systemplate and
lotemplate are mounted for each jails, we need to
make them unbindable, so they wouldn't multiply the
mount-points for each jails (an explosive growth!)
Contrarywise, we don't want the mount-points to
be shared, because we don't expect to add/remove
mounts after a jail is created.
The random temp directory is now created and set
correctly, plus many logging and other improvements.
Change-Id: Iae3fda5e876cf47d2cae6669a87b5b826a8748df
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92829
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
re-factor to make it hard not to.
Change-Id: I26ebc48b4660276ede64a22167ac4779cebf5cd4
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/95440
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
More readable and typically more efficient.
Change-Id: I9bd5bfc91f4ac255bb8ae0987708fb8b56b398f8
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/95285
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Sometimes kit process goes into a heavy processing state (or even hangs)
and is not able to report its memory usage. Thus we can't implement cleanup
of problematic kit processes based on memory information reported by kit.
By moving memory reporting to admin module we avoid this problem.
Change-Id: Icf274e3a3a97b33623a93f9d2dc1e640ad9b7d99
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92752
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
The current thread's id is returned by std::this_thread::get_id().
std:🧵:id() (the result of the std::thead::id constructor), on
the other hand, does not represent any thread.
Change-Id: I13f2831810282109d7bce984f1d040595c466712
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92881
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
LibreOffice core uses that, too, and we support an even more
restricted set of compilers.
Change-Id: I0d0e2c8608e323eb5ef0f35ee8c46d02ab49a745
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92467
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
This mends several problems from commit
5710c86323.
Change-Id: I1b29f29ca81679608a2692488fa1ef22b2e62dfd
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92032
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
With the change that is reverted here, the editing session on Android
returns without proper cleanup which leads to occassional hangs when
the user tries to open a new editing session quickly.
Also, in the iOS app, with the change that is reverted, when closing
the document we never get the LOK_CALLBACK_UNO_COMMAND_RESULT for the
.uno:Save and thus we never get to save it properly from the system's
point of view.
This reverts commit a73590d81f.
Change-Id: Ia77fe2fd9b59d30c343ca1e10f69d5a434bc3628
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/91915
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/91965
Reviewed-by: Tor Lillqvist <tml@collabora.com>
Add checking up the stack to detect when we have clean queues
and buffers so we can exit fast.
Change-Id: I82c3843f816bbe869094c21f070774e6d034ac65
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90358
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
It took both an std::string and a length. Take a char* and a length
instead.
Change-Id: Id37dfa67fe1baae09b69819680848a0a8a1d80ed
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90552
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
But opening a second document now hangs.
Sigh, the plumbing in the mobile apps is so extremely fragile. But
that is to be expected when turning a multi-process structure (where
one class of processes exit as soon as they have done their job) into
a single process running forever.
Change-Id: I0fdb751f44e16efb42843189969e049bf14816f0
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90443
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
Also, the comment in it was misleading. We don't have just a single
buffer for a FakeSocket any longer.
Change-Id: I8f45fba2342ef42040e467b631739a56664ce6e8
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90440
Tested-by: Tor Lillqvist <tml@collabora.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
Essentially we want to be able to separate low-level socket code
for eg. TCP vs. UDS, from Protocol handling: eg. WebSocketHandler
and client sessions themselves which handle and send messages
which now implement the simple MessageHandlerInterface.
Some helpful renaming too:
s/SocketHandlerInterface/ProtocolHandlerInterface/
Change-Id: I58092b5e0b5792fda47498fb2c875851eada461d
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90138
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
The initial bits to serving some page with provision
to ingest the different formats into a table.
Doesn't yet link with the actual document, but it's
a start.
The link will have some unique id(s) in it to
reference the document in question, which will
be some hash (possibly changing with some logic for
security reasons). This hash will have to be
something valid that WSD will use to locate
the DocBroker in question, connect to it and
fetch the formats supported and generate unique
links for each. When the user clicks on a link,
the contents will be downloaded in the given
format and copied to the user's clipboard.
The clipboard.html template is based on loleaflet.html
as we're very likely to use the same customization,
branding, localization, and javascript bits.
We would probably want to add a brandable title
with logo etc. and possibly some more reasable
background (ideally, an image enlarged and blured
to give the page some semblance of having content).
Change-Id: If0550184d4423bef1e98fecbb072bdf8df07701b
After a document is closed, the prisoner_poll, accept_poll,
websrv_poll, docbroker_nnn, and lokit_main threads that served that
document are supposed to go away as quickly and reliably as possible.
This change improves that significantly.
Change-Id: If658bea74f70a77d2d537f0ec6455a6b36f2667e
Unfortunately processing multiple events from the Kit socket
is causing massive document invalidations, for unknown
reasons. As such, for now we have to process one event
at a time, until the source of the invalidations is found
and fixed.
Without the invalidation, the average tile rendering
roundtrip is about 3x faster than with the invalidations
and the maximum roundrip is at least 2x faster.
Change-Id: Iafbf9ccc2b80656cb71c208b598080f72d201ca2
Reviewed-on: https://gerrit.libreoffice.org/70906
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Unix Domain Sockets are inaddressable remotely, and more efficient,
as well as allowing future SCM_CREDENTIALS / SCM_RIGHTS.
Change-Id: Ia2472260f75feb43e9022cdfa0fe005ccd489454
