Impress deletes the current view and creates a new one during that call,
so make sure at the end our registered callback is not lost. With this,
the order matches how it's done in gtktiledviewer.
This fixes the HTTPWSTest::testInsertDelete() test when per-view
callbacks are enabled.
Also remove the assert from TileCacheTests::testTileInvalidateWriter()
that would assume something is invalidated right after load: the hope is
that it's possible to load a document without invalidating anything...
:-)
Since rendering moved to centralized WSD<->Kit
processing, it runs on a different thread than
those processing the communication between
ClientSession and ChildSession. This introduces
a new race between events and tile rendering.
The shared ChildSession lock prevents this race
such that no events are processed while a tile
is rendered and, more importantly, response
is prepared and sent back. That preparation
could be lengthy due to png compression.
The typical race happens when two keystrokes
are entered in quick succession such that the
same tile is invalidated while it's rendered.
If the invalidation is processed in parallel
to rendering, it will find no cached image to
remove. It will reach the client, who will
request a new tile. Meanwhile, the first tile
is rendered and cached. By the time the second
request arrives we have a cache hit, albeit on
an outdated tile, missing the second character.
By locking until the tile response is sent we
ensure that the invalidate event will follow
it, and by then the image will have been cached.
The invalidation then removes the cached image
and the second tile request is forced to place
a new tile render request.
There is some inefficiency, it would seem, but
that is not really true, as Core is really
sequential anyway and we shouldn't process
events in parallel in the first place.
Change-Id: Id8170a41a7e69bca6ac8b520029b7cdb2d96c880
Reviewed-on: https://gerrit.libreoffice.org/25662
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Tiles can be rendered in large batches and sent back
as combined payloads, all reduce latency and overhead
significantly.
Initial tests show a reduction in total latency by 2x.
This without sending tiles back to the client in combined
form, which will reduce latency further.
Change-Id: Iee06503f2a6c741fadb2c890266ea514c809c0dc
Reviewed-on: https://gerrit.libreoffice.org/25339
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
A source file (.cpp) must include its own header first.
This insures that the header is self-contained and
doesn't depend on arbitrary (and accidental) includes
before it to compile.
Furthermore, system headers should go next, followed by
C then C++ headers, then libraries (Poco, etc) and, finally,
project headers come last.
This makes sure that headers and included in the same dependency
order to avoid side-effects. For example, Poco should never rely on
anything from our project in the same way that a C header should
never rely on anything in C++, Poco, or project headers.
Also, includes ought to be sorted where possible, to improve
readability and avoid accidental duplicates (of which there
were a few).
Change-Id: I62cc1343e4a091d69195e37ed659dba20cfcb1ef
Reviewed-on: https://gerrit.libreoffice.org/25262
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Also, don't let these tile previews change part of the document
permanently. This is a temporary solution till we have some
better API from LOKit to deal with such a situation.
Change-Id: I8dfefd2b7ad8cf3e7a57afb95b57994ef0bb3b6c
editlock= is always the last parameter. This fixes the slide
previews in impress documents for editing session.
Change-Id: I3531c7f52e09e655524fa0afd6fe66da504b5d70
Obviously this is dangerous, since the id is not part of the
subscription key (the filename) so different clients could
have different ids on the same part, but in practice I
expect this not to happen. Though that clearly depends on
how clients use the id.
Change-Id: I52a0b043c9b5e5ad1111b692e4216cc9ffec5b2b
Reviewed-on: https://gerrit.libreoffice.org/24680
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
bccu#1433, bccu#1757 related.
Piggyback editlock information to tile messages so that kit can
use that information to allow changing parts only for messages
with editlock.
... and decline tile render request for tile messages without editlock
information
Change-Id: I9cedb870cd977741375665cb258d04c818481a14
For the moment, it will allow running 'make check' that does not conflict with
an already running loolwsd (eg. from 'make run'). Later we can consider
running more tests in parallel.
WSD's DocumentBroker and Kit's Document now handle
the communication of tiles as well as all aspects
of rendering, caching, unifying requests and
distributing results etc.
Change-Id: Ie22fbaaae26b91185ee348ec7077f30430eec0f6
Reviewed-on: https://gerrit.libreoffice.org/24640
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
While there are two separate callbacks registered
(one with lokit and the other with lokitDocument,)
there is no reason why they should be handled
separately (and indeed differently).
The lokit callback only sends notifications on
status indicator (during loading and saving)
and document password type (if protected).
Due to the different callback handlers
the status indicator was only sent to the
first client, not all (as one expects).
Furthermore, because the lokit callback
was processed on the Core thread, it
was bound to cause performance and
thread-safety issues. Specifically it
deadlocked when another callback was
in flight when a save issued status
indicator callback.
By unifying the callbacks and putting
all callback messages on the message
queue we avoid all of the above and
simplify the code.
Change-Id: I5bd790b6ce88b7939186c1ec1dead7fb6cabf7e0
Reviewed-on: https://gerrit.libreoffice.org/24522
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
When loading a document first we set the rendering
options. Beyond that, the document is shared and
we shouldn't change the rendering options.
Change-Id: I0d2ac6fc43553b8395111ba2b8a3cc2796d2f0a4
Reviewed-on: https://gerrit.libreoffice.org/24470
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
The closing handshake.
Either peer can send a control frame with data containing
a specified control sequence to begin the closing handshake.
Upon receiving such a frame, the other peer sends a
Close frame in response, if it hasn't already sent one.
When inside the chroot, what we would need to do is remove everything
below / . But doing that is a bit too risky, in case some developer
screws up some detail and that code happens to run outside the chroot
after all, and the developer's machine gets trashed. So just remove
paths we can reasonably assume won't exist as global pathnames on a
developer machine: loSubPath and JAILED_DOCUMENT_ROOT.
Currently the actual complete cleanup of loolkit jails happens in
loolwsd when it is exiting. That is a bug and will have to be
fixed. It should be done in loolforkit as soon as possible after the
loolkit process has exited.
This ensures that bundled fonts in instdir/share end up resolved to
the same path that they were in when the forkit font config was setup.
It may also help locate other pre-inited resources.
Also copy in ~/.fonts in debug mode - can't hurt.
SocketProcessor doesn't need to take response
instance, since by the time it is called we
are already upgraded to WebSocket and it's
too late to set a request-level status.
Change-Id: Id95087e60354a50148c88427130613356679cf82
Reviewed-on: https://gerrit.libreoffice.org/24110
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Connection thread should not attempt to disconnect the session,
which in turn will try to unload the document, which will
wait on the connection to destroy. The latter will never
happen since the connection destructor must, correctly,
wait for its thread to finish, which is waiting on itself now.
Since the session disconnect is already called from the session
destructor, there is no need to explicitly invoke it here.
Change-Id: Iaf9e8a10d4caa9001208084e909a14b4d4c5105e
Reviewed-on: https://gerrit.libreoffice.org/23966
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
This seems to get rid of the "terminate called after throwing an
instance of 'Poco::SystemException'" problem for me at least.
Sigh. Why can't the compiler warn about such things? I build with
clang++ -Wall -Wextra. The Connection class is fully defined inside
the LOOLKit.cpp so it should be able to, right?
Presumably it is only developers that are interested in signals, and
terms like SEGV or ABRT are more precise than their textual
descriptions like "Segmentation violation" or "Aborted".
Sleep a second before exiting in case we get a fatal signal just when
about to finish, which sadly seems to happen often. (In fact, if
handleFatalSignal() is running at the same time, it will kill the
process so we never get to the _Exit() call.)
The logging functions already display the thread name on all output
lines. No need to mention it another time in the thread start and
finish logging messages.
A call to Log::error() should be enough to indicate that it is an
error. We don't need to prefix the message with the string "Error: "
in some cases but not others. (If we do want such a prefix for all
errors, surely then we should add it in the actual Log::error()
function.)
Also, change some more Log::error() calls to Log::syserror() where
appropriate.
Admin no longer needs a pipe as it's notified
from WSD. It is now a singleton with improved
locking.
The tracking of documents and views still needs
improvement and corrections.
Change-Id: If614331de6dd595c6dd4443f480d4ab588ca4551
Reviewed-on: https://gerrit.libreoffice.org/23860
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Preiniting LibreOfficeKit and forking kit processes (instead of
spawning) has worked fine for a while, and has been the default way
this works.
No 'loolkit' program gets built any more.
Broker is now just a simple spawn-machine.
It only gets requests from WSD to spawn
new kit instances, which it doesn't even
track.
Once a kit instance is initialized, it
connects to WSD with a control WS.
From there on, it's up to WSD to manage
the kit process.
Also was removed the benchmark logic
since it can no longer function.
Change-Id: I1bf56bc6416c9eadafba637276bbb8b3107e5727
Reviewed-on: https://gerrit.libreoffice.org/23790
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
WSD now communicates on a WebSocket directly
with kit processes. ChildProcess encapsulates
kit processes and the control WS, which itself
is owned by DocumentBroker.
Change-Id: Ica209aaa07974739b8e51a14e11325d084e193f6
Reviewed-on: https://gerrit.libreoffice.org/23789
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
In face of exceptions, the lock was not released
and the condition variable was not signalled,
thereby causing all subsequent views on the
same document to fail loading.
Change-Id: I18d3cefcc74a158facefe1e74a9c802ee048b014
Reviewed-on: https://gerrit.libreoffice.org/23785
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
of the 10k files still linked into the jail; 5700 are from usr/
so bind mount just that directory, also set noatime, ro, and
some other helpful looking options.
Change-Id: I28d2d5cbbdf33fb57ea0f0c0915cb267603ee16d
Reviewed-on: https://gerrit.libreoffice.org/23777
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Create directories top-down and not bottom up for more efficiency too.
Skip the sdk (if we have it) - ~20k files, and misc. other
pieces that we don't need; still more to go there.
Change-Id: Iccd9ebac495fba414d268b339ef82a161d98a9ca
Reviewed-on: https://gerrit.libreoffice.org/23770
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
This reverts commit 97c8f35ddf.
Since the Broker design has been extremely simplified,
all communication between Broker <-> Kit are gone.
Only a pipe between WSD and Broker remain.
Temporarily reverting this to apply the Broker redesign,
after which this patch can be reviewed and merged.
This will be easier than trying to merge the redesigned
Broker on top of this.
Change-Id: Ia901fad604008654c01841df62e88918adad45e1
Reviewed-on: https://gerrit.libreoffice.org/23769
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Had to add a shared pointer to the BasicTileQueue for the session to
the MasterProcessSession object, and restructure the coe a a bit to
allocate BasicTileQueue objects dynamically. Possibly just passing a
reference to a BasicTileQueue in the stack would have worked, but why
risk it?
The actual logic when to do auto / idle save is not quite right still,
did not change that in this commit.
It is not a good idea to have the same string somewhat arbitrarily
both as a static const members of the LOOLWSD class and then as a
file-local static const in another file. Or defined as a separate
local const static in each compilation unit that includes
Common.hpp. Use constexpr instead, in Common.hpp.
When the Document lock cannot be taken
purging doesn't block (which would block
the kit-broker pipe). Instead, purging
is done only when the lock is taken,
otherwise we try again later.
Change-Id: Id201f1c67803d9b1e765e8c55f85206795fe53c0
Reviewed-on: https://gerrit.libreoffice.org/23448
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Ideally, we will have a randomized path for the jails.
Unfortunately, this will make it harder to cleanup
after an ungraceful exit of a child, including recovery
of docs etc.
Having a PID for the jailId makes this issue easier by
implicitly implying the jail path for a given child.
To prevent security leaks, we should at least randomize
the doc directory within the jail, as such:
/chroot/<pid>/user/docs/<rand>/
For now we use jailId=pid=rand.
Change-Id: I948fba0aaef725c9c059780df0a184a86569d898
Reviewed-on: https://gerrit.libreoffice.org/23223
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
The current format is more flexible and standard.
/loolwsd/child?sessionId=xxx&jailId=yyy
The sessionId is the client-specific connection ID (which
is originally passed to the child via the Broker.
The jailId is the PID of the child.
Change-Id: I69c88e84114f9678addf795896ca2da15ca1221b
Reviewed-on: https://gerrit.libreoffice.org/23211
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Much simpler.
Also, don't duplicate the code informing that LD_BIND_NOW or
LOK_VIEW_CALLBACK are not set. Not that I understand why we need to
inform about that? If the "normal" thing should be that they are set,
why don't make it so by default then?
I occasionally saw a deadlock when running 'make check' where one
thread holds the ChildProcessSession::Mutex and wants the _mutex for a
Document, while another thread holds that _mutex and wants the
Mutex. In particular, it is the Document::onUnload() that wants the
_mutex. So avoid the deadlock by having Document::onUnload() first
take the ChildProcessSession::Mutex.
The callbacks from documentLoad() are made in the same thread.
Sure, as such it is not a good thing to use recursive mutexes. If we
switch back to non-recursive mutexes, we will have to stop taking the
lock in callbacks from documentLoad(), i.e. make sure we know those
functions aren't used elsewhere, in places where a lock would be
needed. Or something.
If a client session closes just after sending a load message to load a
document, and another session then fairly immediately connects and
sends a load message for the same document, the latter session gets
handled by the same kit process. Also, the same Document object is
apparently used. In that kit process, the first documentLoad() might
easily still be in progress. The handler for the new session still
calls onLoad(), too, and as the first onLoad() had dropped the lock
for the duration of the documentLoad() call, the new onLoad can take
the lock and call documentLoad(), too, while the first documentLoad()
call in the other thread still is in progress. This leads to
interesting problems.
Admin web sessions are added as subscribers to AdminModel. Live
notification fill up the AdminModel, and notifies to
subscribers, if present any. AdminModel can also be queried to
fetch any previous data since the start of the server including
expired documents/views with timestamps for analysis.
There is lot of stuff that can be added in future. This commit
just lays the foundation of appropriate classes.
Change-Id: Ifcf6c2896ef46b33935802e79cd28240fd4f980e
Reviewed-on: https://gerrit.libreoffice.org/22869
Reviewed-by: Tor Lillqvist <tml@collabora.com>
Tested-by: Tor Lillqvist <tml@collabora.com>
Turns out the callbacks from documentLoad() are in general done on
another thread, not necessarily the same one that did the
documentLoad() call. Not sure why it happened to be the same thread
for me, but oh well. Need to fix the problem described in
3671abf89b in some other way then.
This reverts commit 2fab757462.
The callbacks from documentLoad() are made in the same thread.
Sure, as such it is not a good thing to use recursive mutexes. If we
switch back to non-recursive mutexes, we will have to stop taking the
lock in callbacks from documentLoad(), i.e. make sure we know those
functions aren't used elsewhere, in places where a lock would be
needed. Or something.
Not needed after all. It was a red herring. The device files work fine
even if not owned by root:root and with mode 664. The actual problem
was that I used a file system mounted with nodev when testing loolwsd.
This reverts commit 509314d559
We are not interested in the variable being assigned an incremented
value. Its value is not used any more. We are just interested in the
value of the variable plus one. Using pre-increment gives the wrong
impression.
Sure, this is nit-picking.
Sorry, could not resist. Obviously not very important.
In retrospect, maybe it would have been better to have as policy to
*not* use any 'using Poco::Foo'. Now there is an inconsistent mix of
writing out the complete type and using a 'using'. Plus copy-pasted
long lists of 'usings'. And of course, one should never have 'using'
in an include file. Oh well.
This is possible by moving all the password handling logic to
Document container class. When a user opens a password protected
document the first time, it saves all possible data such as
password, password type etc. Upon opening the same document
again, password is matched with the cached password saved in the
document container class before allowing the new view access to
this document.
Change-Id: Id1f2b6e06de806564bf865e83fed51b01c9a0fbc
Reviewed-on: https://gerrit.libreoffice.org/22208
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Sometimes there are situations when `connection` map is filled
with dead sessions. We don't want to deal with those dead ones.
Change-Id: I00dda77c39b5adbba69421eace0be0159e02505c
Reviewed-on: https://gerrit.libreoffice.org/22207
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
There are 5 LOK callbacks currently that are triggered on LOKit,
and not on LOKitDocument. These include status indicators, and
document password callbacks during document load. Lets move all
the callbacks called during document load in the Document container
itself, and keep the callbacks called after document load in the
Child session.
Change-Id: I8e43c2baaa12023b34822954dd494780ee6dd7ca
Reviewed-on: https://gerrit.libreoffice.org/22206
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Our DocumentCallback is smart enough which checks all the
running connections, and send the callback notification to all of
them. Registering the callback only during the first loadDocument
call should be enough.
Change-Id: I82bcb9525814dae14def3bfb6c088337d0d0ea3c
Reviewed-on: https://gerrit.libreoffice.org/22202
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Create a corresponding symlink inside the chroot.
Now a LOK_PREINIT=1 loolwsd starts up for me successfully. (Note: I
did not say whether it actually works. I don't know yet.)
A load document operation may fail, for example, when user enters
a wrong password or no password. In such cases ToPrisoner sends a
disconnect to child. Child do not need to take any steps in this
case, such as, decrementing the view counter, because nothing was
loaded.
Change-Id: Ib71f871aa1728c6355563362987800dece1679d3
Reviewed-on: https://gerrit.libreoffice.org/22203
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
