Relying on the filesystem to tell us when the document
was last modified (to decide whether to upload to storage or not,)
proved unreliable.
Now we always upload to storage if there is only one client.
This both minimizes the risk and also avoids the file timestamp
check as a workaround to the problem of re-uploading documents
as many time as there were clients. Since with one client we
can only upload no more than once per save, which is reasonable.
Furthermore, when a client disconnects we auto-save automatically
as a matter of precaution. However, when there are other clients
still connected, we don't wait for the save to complete, rather
we let that job to the very last one.
Change-Id: I94a2e4bddaed30a6c9c0e69f8006667d33c5b8ee
Reviewed-on: https://gerrit.libreoffice.org/24767
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
WSD server must be up and listening to incoming
connections before firing up ForKit and spawning
children.
Having the order reversed caused tests to fail
randomly when WSD was slow to listen to incoming
connections from children already initialized.
Change-Id: I4eaf4a658c65da024101efc096c39222ebfa3c00
Reviewed-on: https://gerrit.libreoffice.org/24745
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
For the moment, it will allow running 'make check' that does not conflict with
an already running loolwsd (eg. from 'make run'). Later we can consider
running more tests in parallel.
...child from DocumentBroker""
Restore the communication with child from DocumentBroker.
This reverts commit 20ab6e8ae7.
Change-Id: I248bededff7074d8fb482b2cdd172048f80c02b2
Reviewed-on: https://gerrit.libreoffice.org/24639
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
The WebSocket that each child created with WSD is not used
except to request the child to load the document a client
requests. Beyond this point, it was not utilized for anything.
In fact, there are no handlers in WSD for messages coming
from the child; it is a one-way communication.
That is until now. With the move to unify communication
between WSD and each child, DocumentBroker can now
receive and handle messages from its ChildProcess.
Change-Id: Ie7f030a92db8303cd7087fff2325f136a49bc7fc
Reviewed-on: https://gerrit.libreoffice.org/24581
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
This makes for amore compact API and avoids
a race between issuing the save and waiting for it.
Also added force flag and autoSave now checks the
modified state of the document. If a document is
not modified, nor save forced, autoSave checks
the last activity on the document and only
if there is any since last save does it issue
a save command.
Change-Id: I962e36df18d7edf5f658992e97b5def5f6247dc3
Reviewed-on: https://gerrit.libreoffice.org/24382
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Sessions are referrenced in DocumentBroker instances,
which themselves are referrenced in a container.
When exceptions are thrown either while creating a new
session, or during the lifetime of one, these references
must be correctly cleaned up, otherwise we introduce
internal instability in addition to stalling the client.
Change-Id: I3177e45564860897528da6d7fbcbe346d3bd1c75
Reviewed-on: https://gerrit.libreoffice.org/24338
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
All changes are supposed to be persistent. This simplifies the tile
caching code quite a lot.
The TileCache object no longer needs to keep any state whether the
document is being edited or whether it has been modified without
saving etc.
Update the modtime.txt file after saving the document. Otherwise the
tile cache would wrongly be considered invalid next time.
As a sanity check, we put a flag file 'unsaved.txt' into the cache
directory whenever we get a callback that indicates the document has
been modified, and remove it when the document is saved. If the flag
file is present when we take an existing tile cache into use, we can't
trust it.
Even after these changes, we still don't use an existing tile cache as
much (or at all?) as we could, though. The INVALIDATE_TILES EMPTY
callback that LO does early on in a conection causes us to remove all
cached tiles...
This also avoids the feedback loop that results from the kit
thinking the previously inactive client is now active and
sending commands (.uno:Save).
Change-Id: I47074b35a922da15592d550032d494ba1efab83e
Reviewed-on: https://gerrit.libreoffice.org/24287
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
It is still possible to access them directly via loleaflet/dist/<something>,
but such use can lead to unexpected behaviour due to various caching in the
browsers etc.
The closing handshake.
Either peer can send a control frame with data containing
a specified control sequence to begin the closing handshake.
Upon receiving such a frame, the other peer sends a
Close frame in response, if it hasn't already sent one.
At least, that is the value of the num_prespawn_children element in
the loolwsd.xml as shipped. But maybe that is not what is meant with
"default"? It is unclear to me what the "default" attribute means.
When a new view is created on a document that is
in the process of unloading, all sorts of things
can go wrong. This is especially problematic when
the document needs to be saved before unloading,
which takes significantly longer than otherwise.
Change-Id: Ib33a18cafa9d5a3a17f6bd8c6145f9331ae54044
Reviewed-on: https://gerrit.libreoffice.org/24184
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Normally, when each client view closes, the
session count is decremented until the last
view is closed. However this doesn't work
when the kit child process terminates.
Due to a race condition between the last
client disconnecting, and the internal
structure destructing, and the next
client connecting (on the same doc),
the Admin loses track of the doc and pid.
This is an issue of assuming a document
and its pid are unique and will always
remain unchanged.
This patch adds a new API to remove a
doc and all its views unconditionally
to try to avoid the above issues.
Change-Id: I0c181260679875b0464dd9b6548b29b8d6a361f7
Reviewed-on: https://gerrit.libreoffice.org/24183
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Standardized error handling in request-handlers.
There is a new family of internal exeptions designed
to signify the type of error and how to handle it.
All handlers must throw one of those errors
and they will be translated to the correct HTTP
response when caught.
Since some requests send a response as part of their
handling (convert-to, for example) those handlers
must return a flag signlaning whether or not they
sent a response. If not, HTTP OK response is sent
at the end of the handler.
To complicate things, some requests upgrade the
connection to WebSocket. In those cases errors
must be sent via the WebSocket and not as an
HTTP response. The error message sent can (and
in most cases should) be displayed to the end-user.
A new file, UserMessages.hpp, has been added to
hold user-visible messages that can be
reviewed and translated.
Change-Id: Icc725f3313446d4514cf6d092635158ee7171f5d
Reviewed-on: https://gerrit.libreoffice.org/24133
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
SocketProcessor doesn't need to take response
instance, since by the time it is called we
are already upgraded to WebSocket and it's
too late to set a request-level status.
Change-Id: Id95087e60354a50148c88427130613356679cf82
Reviewed-on: https://gerrit.libreoffice.org/24110
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Some messages are not forwarded to the client session, this is caused
by the time the client session is assigned, the prison session,
it is already forwarding to not assigned peer session.
Do not distinguish between normal shutdown or abnormal shutdown.
Also remove 'disconnect' frame to indicate normal shutdown.
Change-Id: I98fd9f5a219feb1097c57302dba14e08ad9bf143
That global flag is checked all over the place, so setting it will
actually make the threads eventually finish. (All polling is done with
timeout, I think, and then checking TerminationFlag whenever the poll
times out.)
Sure, it would be much better to use an eventfd and poll that, too,
instead of timing out from the polls all the time to check a plain old
boolean flag.
Enforce user being 'lool' for setcap binaries loolmount and loolforkit.
Add warnings if configured without --enable-debug.
Developers should pass --enable-debug to configure.
The loolwsd process created it and opened it for reading, but nothing
opened it for writing.
There is still documentation for it in README, that needs to be either
rewritten to match reality or removed.
Comes in handly in some testing situations where you don't want to
send a signal to get loolwsd to finish. Option is present only in an
--enable-debug build.
Last client disconnection now correctly issues a save
and waits for the confirmation before tearing down
the sockets, queues and threads.
Change-Id: I28c28d79a17d359e9aa1fe67b983ca9fb592b847
Reviewed-on: https://gerrit.libreoffice.org/23978
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
The sessions container already has the number of sessions.
No need for separate counters to track them.
Change-Id: I838865e2b8a843e87e81a6cc1226bcacd774b032
Reviewed-on: https://gerrit.libreoffice.org/23964
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Autosaving is done by DocumentBroker, which
tracks the last save time.
There are two triggers: idle and auto save.
The first triggers when sufficient time passes
after the last interaction the user had with
the UI (currently 30 seconds).
The second triggers when it's been more than
5 minutes since the last save.
Both triggers are conditional on the user
being active after the last save.
The new code auto-saves doesn't issue
a save command per session, but only
one per doc.
Change-Id: Iada15c16002e70710d2c13a3dcfdab036d8935c6
Reviewed-on: https://gerrit.libreoffice.org/23951
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Presumably it is only developers that are interested in signals, and
terms like SEGV or ABRT are more precise than their textual
descriptions like "Segmentation violation" or "Aborted".
The logging functions already display the thread name on all output
lines. No need to mention it another time in the thread start and
finish logging messages.
A call to Log::error() should be enough to indicate that it is an
error. We don't need to prefix the message with the string "Error: "
in some cases but not others. (If we do want such a prefix for all
errors, surely then we should add it in the actual Log::error()
function.)
Also, change some more Log::error() calls to Log::syserror() where
appropriate.
Much better than assuming that errno would be relevant at all
Log::error() calls (or alternatively, having to remember to append a
false parameter to the Log::error() call, which had not been done a
single time anyway.)
Call log::syserror() right after a system call has returned an
error. Don't call it otherwise.
Loading documents from the local filesystem
opens the door to security issues.
By default filesystem storage is disabled,
even if enabled in the config file. The
only way to enable it is to set the
allowlocalstorage command-line argument.
Change-Id: Ib8f57377260817436d101a16757aab38276cbdcd
Reviewed-on: https://gerrit.libreoffice.org/23881
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Most all configuration values can now be defined
in the configuration XML. The command-line arguments
can be used to override some of these values (for
the convenience of developement and testing) and,
in a few cases, as a security measure to avoid
storing sensitive data in the configuration file.
Change-Id: I040b807b1a59a3537bb94646150d3c7d711f8b62
Reviewed-on: https://gerrit.libreoffice.org/23880
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
A new command-line argument, admincreds, must be provided
to set the Admin Console credentials.
The new command-line argument specifies the username
and password in the following format: username/password
If not provided, Admin Console is disabled for security
reasons. A warning is emitted at startup and an error
on every invocation of Admin Console is logged when
no credentials are defined.
Change-Id: I348623949fd0b292f5066e4955759c708204540f
Reviewed-on: https://gerrit.libreoffice.org/23878
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
loolstat now uses `pgrep loolwsd$` instead
of relying on pid dumping in a temp file.
With the Admin Console this tool (loolstat)
is less useful, so this cleanup is probably
a stepping stone to removing it altogether.
Change-Id: Ib7732a00c3d3ea54dffcb71e9fe1a56c4a88016e
Reviewed-on: https://gerrit.libreoffice.org/23877
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Admin no longer needs a pipe as it's notified
from WSD. It is now a singleton with improved
locking.
The tracking of documents and views still needs
improvement and corrections.
Change-Id: If614331de6dd595c6dd4443f480d4ab588ca4551
Reviewed-on: https://gerrit.libreoffice.org/23860
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Add a function to determine whether a client message indicates user
interaction. We need that distinction when deciding when to do an
automatic ("idle" or "auto") save of document being edited.
"Interaction" is a loose term, possibly what we actually want is to
see whether the user is actively doing an edit that changes the
contents of meta-data of the document.
I easily got the thing into a state where it kept forking new
processes without limit even if no client was doing anything. The log
output contained lines like:
wsd-09526-04 00:00:48.302029 [client_ws_001b ] MasterToBroker: spawn -1
brk-09528-00 00:00:48.279945 [loolbroker ] Broker command: [spawn -1].
brk-09528-00 00:00:48.279964 [loolbroker ] Spawning -1 childs per request.
brk-09528-00 00:00:48.279974 [loolbroker ] Creating -1 new child.
This fixes that, but there are still several issues remaining.
WSD now communicates on a WebSocket directly
with kit processes. ChildProcess encapsulates
kit processes and the control WS, which itself
is owned by DocumentBroker.
Change-Id: Ica209aaa07974739b8e51a14e11325d084e193f6
Reviewed-on: https://gerrit.libreoffice.org/23789
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
This reverts commit 97c8f35ddf.
Since the Broker design has been extremely simplified,
all communication between Broker <-> Kit are gone.
Only a pipe between WSD and Broker remain.
Temporarily reverting this to apply the Broker redesign,
after which this patch can be reviewed and merged.
This will be easier than trying to merge the redesigned
Broker on top of this.
Change-Id: Ia901fad604008654c01841df62e88918adad45e1
Reviewed-on: https://gerrit.libreoffice.org/23769
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Had to add a shared pointer to the BasicTileQueue for the session to
the MasterProcessSession object, and restructure the coe a a bit to
allocate BasicTileQueue objects dynamically. Possibly just passing a
reference to a BasicTileQueue in the stack would have worked, but why
risk it?
The actual logic when to do auto / idle save is not quite right still,
did not change that in this commit.
... which can be taken only one at a time. Others can only view,
not edit. When a session with edit lock exits, the edit lock is
handed over to the next alive session.
Change-Id: I712a4e70369f1d07c1d83af416a0f5c288b05c7d
We act as a client when we communicate with owncloud server.
For time being, just accept whatever certificates owncloud server
gives us. We might want to get more strict here in future.
Change-Id: I4813d19412b66ecf57d6cdef9c3ac94fbbaa521f
All messages now pass through the queue.
This resolves a race between single-line
messages and multi-line ones.
Previously, single-line messages were
processed on the queue (on a background
thread) while multi-line ones were handled
immediatly. This resulted in order-inversion
due to a race between the queue thread and the
next multi-line message, which caused stability
issues every so often.
Change-Id: Ia220791d1d75c4f3e3e0965dd0c6f81bae63a296
Reviewed-on: https://gerrit.libreoffice.org/23583
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
It is not a good idea to have the same string somewhat arbitrarily
both as a static const members of the LOOLWSD class and then as a
file-local static const in another file. Or defined as a separate
local const static in each compilation unit that includes
Common.hpp. Use constexpr instead, in Common.hpp.
This is C++, not Java. Or is there a school of thought for C++ style
that says one should avoid plain "C-style" file-local static
variables, and instead put everything always in a class, even as
static members? Do we want to follow that?
Configuration XML is added with SSL as sample use-case.
A 'desc' attribute can be used to describe the fields,
and another 'type' to help define the corresponding data
type in the code.
Since Poco allows accessing group nodes (that have the
same name) by index, order can be preserved.
SSL initialization refactored and cert/key file
paths moved to the config file.
Change-Id: I259826a19697bd851587bebcc4f0cd233ab6848b
Reviewed-on: https://gerrit.libreoffice.org/23464
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Just a skeleton, actual saving not yet implemented. Also, not sure
the logic when to trigger save is as intended.
Note that no separate timer classes or objects are used. The existing
watpid/sleep loop that wakes up once every two seconds currently is
used. If that loop is re-factored to be less silly, the auto/idle save code
must be implemented differently.
When no child process has died, I don't see the point in calling
waitpid() eleven times in quick succession (with WNOHANG), doing
nothing else, and then sleeping for a bit. Let's call waitpid() just
once, and sleep only if the return value indicates that no child
process has died.
Use same port (9989) for all client connections. This includes
admin panel, static file serving and normal client websocket
connections.
Change-Id: Idcfd7dd8925523c36e884717c41a3b6a827f6ff3
... and use SSL for client connections. Also fix our test suite
to use HTTPS now.
Change-Id: Id396a7c2d1830da8d3b0ce446522403363ac17c1
Reviewed-on: https://gerrit.libreoffice.org/23395
Reviewed-by: Tor Lillqvist <tml@collabora.com>
Tested-by: Tor Lillqvist <tml@collabora.com>
... and use SSL for client connections.
Change-Id: Id396a7c2d1830da8d3b0ce446522403363ac17c1
Reviewed-on: https://gerrit.libreoffice.org/23395
Reviewed-by: Tor Lillqvist <tml@collabora.com>
Tested-by: Tor Lillqvist <tml@collabora.com>
This is to distinguish the deliberate close of connection, and timeout,
connection drop, or forced close.
When the last session is closed non-deliberately, force a save so that the
edits are not lost.
