Commit graph

15318 commits

Author SHA1 Message Date
Ashod Nakashian
f2f713cdaf wsd: rename file after saving to support async uploading
To avoid writing (saving) while reading (uploading) or
any other undesirable racse on the document file,
we atomically rename the file after saving on disk
to pass ownership.

After saving in Core, we rename the document in the
jail to .upload (by appending it). DocumentBroker
looks for the file with that extension and atomically
renames it to .uploading (by appening the 'ing' suffix).
This way, the Kit only renames from the original to
.upload and DocBroker renames only .upload to .uploading.
This guarantees that we never rename the same file
concurrently.

Uploading decision is strictly based on the modified
timestamp of the .uploading file, compared to the
timestamp of the last file we uploaded successfully.

Change-Id: I03520cd8c87605f6dad417e7a978204f76fc0c38
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-06-19 17:44:59 -04:00
Ashod Nakashian
266fd6e981 wsd: set the timestamp of the file in storage after downloading
This makes sure that if the document isn't modified, we
will detect no change from the storage as well and
avoid making an unnecessary upload.

Change-Id: I8ba2850921d75c96b2aa4f6deb86891b4fad0ea0
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-06-19 17:44:59 -04:00
Ashod Nakashian
cbf05e0eff wsd: handle renameFile command with async uploading
The client always issued a save before renameFile.
This worked fine until we make uploading asynchronous.

Now that uploading is done asychronously, the renameFile
command arrives while uploading is still in progress
and fails because renaming is done via the same
storage API used for uploading.

It is best to do both the saving (including uploading)
and renaming directly in WSD, instead of relying on
the client to save explicitly before renaming.

Change-Id: I99f6294f4c83130bc13ec8e3b013ba0192f0f84c
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-06-19 17:44:59 -04:00
Ashod Nakashian
cbee71a6da wsd: track user issued activities on the document
Some user-initiated activities, such as renaming
the document filename, is either a multi-stage
activity or involves async operations.

Using a new Activity enum in DocumentState, we
now track each of the different activities that
needs special attention or handling.

Change-Id: I9cd2d04b10a97387ce02f5c3cc98b41a1732af6a
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-06-19 17:44:59 -04:00
Ashod Nakashian
cb077179b0 wsd: logs and assertion in upload completion handler
Change-Id: I7abec95525e73d7d9a9621f8c389a63b7dd66afc
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-06-19 17:44:59 -04:00
Ashod Nakashian
0810234480 wsd: simply passing the authorization object to the storage
Change-Id: I57cf952918edcb5bf2ddeab66c534a6805eec7e6
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-06-19 17:44:59 -04:00
Ashod Nakashian
2ac35de6df wsd: name session variable instead of opaque iterator
Change-Id: Ic2c16ef94c67c90307f2aae2c35dba0d92fb4271
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-06-19 17:44:59 -04:00
Ashod Nakashian
16a3c43871 wsd: stop after successfully uploading
With async uploading we have to process the
stop-condition only after we get the response from
the async operation. For now, we don't stop
if we fail to upload, giving a chance to retrying.

Timeout should be used in the poll loop to
limit how long we retry.

Change-Id: I8e8f8618f92f34ef2f1853e499f4f8d6ce932b40
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-06-19 17:44:59 -04:00
Ashod Nakashian
35bf4a32ed wsd: DocumentBroker::needToUploadToStorage now returns an enum
This is to also include the 'force' case that is
needed when unloading a document.

Now that the force flag is reliably set, we don't
need to override it in needToUploadToStorage.

Change-Id: Ic0672e6c7bcb53a17ec6a0cbd2d2f8b434fbc300
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-06-19 17:44:59 -04:00
Ashod Nakashian
13543f45d6 wsd: test: add async upload test to modify after fail
This adds a test case for modifying the document after
the previous upload failed and verifying that the
subsequent save also uploads the modified document.

Change-Id: I05e17842399f9af48a029fdde72ce15c023bee3f
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-06-19 17:44:59 -04:00
Ashod Nakashian
bd3b862148 wsd: test: add async upload test on closing
Tests uploading on document closing when
the previous upload had failed.

Change-Id: Iceeed041bd4e84ba8096be245b4fce1af496eb8f
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-06-19 17:44:59 -04:00
Ashod Nakashian
48e76c3a94 wsd: better logging of autosave, upload, and modified states
Change-Id: I7249c3e57568acbf24c754db7097c263573fb6bf
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-06-19 17:44:59 -04:00
Ashod Nakashian
f32fd0756b wsd: better logging of upload state
Change-Id: Id95c69a09c14e8d07e0a64b113bd7ab7e760850e
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-06-19 17:44:59 -04:00
Ashod Nakashian
161cf090e2 wsd: upload after saving only if necessary
Change-Id: I4037030329815a87edf8e15d151dbec2491e9973
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-06-19 17:44:59 -04:00
Ashod Nakashian
1d0c717d0a wsd: handle async upload callback
Now we can handle the response of the storage
after the async upload is complete (or timed out).

Change-Id: I29d450646bddb07f02bb17d257e7e0fa372ce357
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-06-19 17:44:59 -04:00
Ashod Nakashian
39bb098601 wsd: support asynchronous upload requests to storage
Add the necessary support for handling async
document upload to storage in DocumentBroker.

Change-Id: Id7a58c245d729cef58669bafb60db450f7ebbba2
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-06-19 17:44:59 -04:00
Ashod Nakashian
09dd73f6a2 wsd: add async API to Storage
This allows uploading to storage asynchronously.

Change-Id: I83956d8b27cfd24430462b1a6d0a6845bb97b7a5
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-06-19 17:44:59 -04:00
Muhammet Kara
41bfcd510c Update Gitpod config to core co-2021
Signed-off-by: Muhammet Kara <muhammet.kara@collabora.com>
Change-Id: Idf9c43120e5e3b2907688d6390fdcb3675101661
2021-06-18 14:42:57 +03:00
Muhammet Kara
449d5ffcd9 Remove repeated assignment
Signed-off-by: Muhammet Kara <muhammet.kara@collabora.com>
Change-Id: Ic8b69da652adde02417d2e4b605ad319fa670299
2021-06-17 13:45:12 +03:00
Michael Meeks
f4302e50c8 Remove 1ms timeout thrash while loading resources asynchronously.
Saves spinning the mainloop while waiting for images to load.

Change-Id: I7978d30443493047325ad20e3278ba75d0533133
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
2021-06-17 07:46:30 +01:00
Michael Meeks
08392b782e Avoid re-parsing object events un-necessarily.
Change-Id: Idd53244c723d8c92d1fd66854678a8e4ba39b91a
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
2021-06-17 07:46:30 +01:00
Michael Meeks
c865d47a34 Split up _extractTextImg to make it easier to profile.
On Android was taking ~100ms to convert a few tiles to base64 URLs
which is unexpected. So:

* split it into bits so we can get a better view of where the problem is
* use Array.indexOf if we can to accelerate scanning for newline.

Change-Id: Ie7c6336f2a867afca16579b119322283c4b4f350
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
2021-06-17 07:46:30 +01:00
Mike Kaganski
cb917d7eae Bail out of getPixmap gracefully, when rendering failed.
And only insert the new pixmap after the check.

Signed-off-by: Mike Kaganski <mike.kaganski@collabora.com>
Change-Id: I310d321443152967b3aed8992ece3b55287d01cc
2021-06-16 17:42:26 +02:00
Szymon Kłos
d869243352 notebookbar: keep old toolbox implementation
structure for JSDialogBuilder has been changed and doesn't
fit notebookbar needs

Signed-off-by: Szymon Kłos <szymon.klos@collabora.com>
Change-Id: Ib040edd32f2e039cb8d0395abdc2bfb39a3b43d8
2021-06-16 11:56:56 +02:00
Tor Lillqvist
34f345f1ee Emit a Trace Event (when enabled and turned on) in ChildSession::_handleInput
Signed-off-by: Tor Lillqvist <tml@collabora.com>
Change-Id: I8fab2f6f8e86d01fafba0f7bb799c503b5331018
2021-06-16 07:17:44 +03:00
Tor Lillqvist
a38732d30d Also emit a Trace Event (when enabled and turned on) in L.TileSectionManager.postKeyboardEvent
Signed-off-by: Tor Lillqvist <tml@collabora.com>
Change-Id: I851b7096685ca0f522cdb8052a3f27abb8d8efaf
2021-06-16 07:17:44 +03:00
Szymon Kłos
712d5c9fa1 cypress: adjust apply_font tests to recent core changes
Signed-off-by: Szymon Kłos <szymon.klos@collabora.com>
Change-Id: I0664a4a2d1d52703cbd249ad72af2f0646f89848
2021-06-15 19:38:24 +02:00
Pedro Pinto Silva
14a7508f82 Busypopup: Create overlay when in presence of modal dialog
Adds additional element (overlay) to emphasize that the UI
is currently blocked (in a waiting state) with the following CSS class
: 'jsdialog-overlay'. Uses the same CSS properties' values
as 'vex-overlay'.

Fixes overlay background color so it's the same as other modal dialogs
(example about)

Signed-off-by: Pedro Pinto Silva <pedro.silva@collabora.com>
Change-Id: Id68302e7e366b68d709b1507ab3948e37c55457b
2021-06-15 19:38:24 +02:00
Pedro Pinto Silva
94c869af0f Busypopup: Less padding, fix text capitalization and style
container's border and shadow

Signed-off-by: Pedro Pinto Silva <pedro.silva@collabora.com>
Change-Id: I2238bfba48b14e6b0543c9fb8ea07c11360b8d6c
2021-06-15 19:38:24 +02:00
Michael Meeks
7227c7c48a Remove dupliate forward definition.
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
2021-06-15 15:44:21 +01:00
Tor Lillqvist
3b32aa667e Emit Trace Event (if enabled and turned on) for handling of key events
Signed-off-by: Tor Lillqvist <tml@collabora.com>
Change-Id: Ib38252197d9a6597d88622b0b965ad026bb53fcb
2021-06-15 14:18:31 +03:00
Pedro Pinto Silva
93ca2c8c84 Block UI: Add spinnerImgControl and use it for busypopup
Utilize svg and css animation of a paper (document) being
saved (stored) instead of a generic spinner (save that for
other cases though).

SVG paths' color change depending on document type.

Signed-off-by: Pedro Pinto Silva <pedro.silva@collabora.com>
Change-Id: Ie352f5947b3bbc1951d2cd6c5b438737521f8b1b
2021-06-15 10:29:17 +02:00
Szymon Kłos
2d5838ab46 Block also Save and Insert Image functionality
Change-Id: Iaed0af0b4a652370e0cee696882f1171c6ab1f0f
Signed-off-by: Szymon Kłos <szymon.klos@collabora.com>
2021-06-15 10:29:17 +02:00
Szymon Kłos
29c6cdd3d6 Block document editing when UI is blocked
Signed-off-by: Szymon Kłos <szymon.klos@collabora.com>
Change-Id: I0079637648ea0ad1c576730299f8e050b13bb954
2021-06-15 10:29:17 +02:00
Szymon Kłos
1f4ad5c30c Use jsdialog popup for busy state
Signed-off-by: Szymon Kłos <szymon.klos@collabora.com>
Change-Id: I42ceaf59dc4e8a06cd233a6d046f8b7ef3867b21
2021-06-15 10:29:17 +02:00
Szymon Kłos
13ca421398 Block UI when command arrives from the server
Signed-off-by: Szymon Kłos <szymon.klos@collabora.com>
Change-Id: Ia6fa54ed9990d7424377735bda98c06db5304c0f
2021-06-15 10:29:17 +02:00
Szymon Kłos
14aab9ee72 Add UI blocking functions to UIManager
- when UI is blocked any interactions with sidebar/dialogs,
  jsdialogs or menu/toolbar items are blocked.

to use: fire 'blockUI' / 'unblockUI' events
or call directly: map.uiManager.blockUI() / .unblockUI()

Change-Id: If96b066fecc610f4a6f4d3b4457f6dfce9a681cd
Signed-off-by: Szymon Kłos <szymon.klos@collabora.com>
2021-06-15 10:29:17 +02:00
Ashod Nakashian
823f349383 wsd: http: break cyclic dependency
The Socket instance owns the protocol handler
by design. This implies that the Session,
which is also the protocol handler, cannot
itself own the Socket.

Here we change that ownership into a weak_ptr
and pass the exclusive ownership of the Socket
to SocketPoll. We track the connection state,
so we can create a new socket when we are
disconnected.

Change-Id: Id6bf6506b0c8e2e0564f91977e1e7189d934a68f
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2021-06-14 23:02:39 -04:00
Henry Castro
ef2d14c494 loleaflet: introduce console.logException
log javascript exceptions to server side

Change-Id: I3f31a052ede8dbb467ddba9dc18a1020b38a16a3
Signed-off-by: Henry Castro <hcastro@collabora.com>
2021-06-14 14:17:46 -04:00
Henry Castro
d7fa80854d loleaflet: fix app.definitions instance of Socket
Change-Id: I9aad1f792c7a2c07e9e9fa95c0d6b80aa0eef44a
Signed-off-by: Henry Castro <hcastro@collabora.com>
2021-06-14 14:17:46 -04:00
Henry Castro
d751e37e1d loleaflet: introduce logServer function
required to log exceptions

Change-Id: Ic63b31080b252042284b2826e82d4c5300e17f6d
Signed-off-by: Henry Castro <hcastro@collabora.com>
2021-06-14 14:17:46 -04:00
Henry Castro
2f45fa7340 loleaflet: fix mocha tests
Error: No test files found: "mocha_tests/**/*.js"

Change-Id: I57e86689011dceebd1100005b8e800b6155bb7c1
Signed-off-by: Henry Castro <hcastro@collabora.com>
2021-06-14 14:15:33 -04:00
Tor Lillqvist
9813c393dd Remove leftover incorrect usage hint
loleaflet does not look for a &enabletraceeventlogging=yes query
parameter. That was removed many weeks ago.

Signed-off-by: Tor Lillqvist <tml@collabora.com>
Change-Id: I33b36191f1d21c2c0177061ff704aa9bcd5f9ef3
2021-06-14 18:45:40 +03:00
Henry Castro
a11cc85a99 loleaflet: fix overriding recipe for target 'check'
warning: overriding recipe for target 'check'

Change-Id: I9584ffaa9258d779a4e6da3de8838729fec54e5e
Signed-off-by: Henry Castro <hcastro@collabora.com>
2021-06-14 10:29:11 -04:00
Michael Meeks
05c9ab54dc Update perf test, and add make perf-test rule.
Change-Id: I473550b82e0f0e64b89320329b648005509d7215
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
2021-06-14 09:31:46 +01:00
Michael Meeks
b31b44167f Largeish performance test document.
Change-Id: I7839f85e584ed04a2984c3076f70c3a266ee0d8c
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
2021-06-14 09:31:46 +01:00
Michael Meeks
f8a5edb995 Transfer LOK callback processing into our main thread if necessary.
While we are processing data in the poll handler, we can have
another helpful thread from the core sending things to us:

loolforkit(Socket::assertCorrectThread(char const*, int))
loolforkit(WebSocketHandler::sendFrame(std::shared_ptr<StreamSocket> const&, char const*, unsigned long, unsigned char, bool) const)
loolforkit(WebSocketHandler::sendMessage(char const*, unsigned long, WSOpCode, bool) const)
loolforkit(Document::postMessage(char const*, int, WSOpCode) const)
loolforkit(Document::sendFrame(char const*, int, WSOpCode))
loolforkit(ChildSession::sendTextFrame(char const*, int))
loolforkit(ChildSession::loKitCallback(int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&))
loolforkit(Document::GlobalCallback(int, char const*, void*))
/core/instdir/program/libsofficeapp.so()
/core/instdir/program/../program/libfwklo.so()
/core/instdir/program/libscfiltlo.so()
/core/instdir/program/libexpwraplo.so()
/core/instdir/program/libooxlo.so(oox::core::FastParser::parseStream(com::sun::xml::sax::InputSource const&, bool))
/core/instdir/program/libooxlo.so(oox::core::FastParser::parseStream(com::sun::uno::Reference<com::sun::io::XInputStream> const&, rtl::OUString const&))
/core/instdir/program/libooxlo.so(oox::core::XmlFilterBase::importFragment(rtl::Reference<oox::core::FragmentHandler> const&, oox::core::FastParser&))
/core/instdir/program/libscfiltlo.so()
/core/instdir/program/libcomphelper.so(comphelper::ThreadTask::exec())
/core/instdir/program/libcomphelper.so()
/core/instdir/program/libuno_salhelpergcc3.so.3(salhelper::Thread::run())
/core/instdir/program/libuno_salhelpergcc3.so.3()
/core/instdir/program/libuno_sal.so.3()
/lib/x86_64-linux-gnu/libpthread.so.0()
/lib/x86_64-linux-gnu/libc.so.6(clone)

This is most likely from eg. threaded parsing of various file formats,
or progress messages while loading / saving, sent while the SolarMutex
is locked.

We assume that:
  + there will only ever be one thread running inside kitPoll
    this is enforced by vcl/headless.
  + and so, we can safely mutate SocketPoll and Socket buffer
    state from this thread.
  + there are only two entry points for callbacks from threads
    in core - which are both instrumented, and if necessary
    their work is shifted to this thread.

Memory corruptions matching the above trace, suggesting that
multiple writes have collided between kitPoll and an event
callback have been seen in the wild.

Change-Id: I5b084cbfec1ea326b6e17c9e5590a8c8e35b3854
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
2021-06-14 09:31:26 +01:00
Dennis Francis
c812edc7d3 fix avoid putting empty tiles to cache
Signed-off-by: Dennis Francis <dennis.francis@collabora.com>
Change-Id: Ifb564a9779071d1e559a004150efc8eb937e10dc
(cherry picked from commit f7a8502d8a3ed2691acf5c8c2545ea4baa624b5d)
2021-06-14 13:33:13 +05:30
Dennis Francis
98c78caa4d fix: watermark not present in fully transparent tiles
Calc has transparent tiles due to client side grid/bg rendering, so do
alpha blending in calc for transparent pixels. As before do alpha
blending for the font rendering phase, no change there.

Signed-off-by: Dennis Francis <dennis.francis@collabora.com>
Change-Id: I30ab21475b6654a9375574be825e01edbdabf82a
(cherry picked from commit 7b7a73c17d4f6213e78f0b8737ce70cbaccbb56d)
2021-06-14 13:33:13 +05:30
Miklos Vajna
f1c796bd29 doc: freenode -> libera
Signed-off-by: Miklos Vajna <vmiklos@collabora.com>
Change-Id: I1ae690ae6c879f100236020eb369ff457b8bbb66
2021-06-14 09:14:02 +02:00