Commit graph

670 commits

Author SHA1 Message Date
Pranav Kant
160c1f555a Save unsaved documents when memory consumption is high
These saved documents are then removed to free some memory when next
memory cleanup is triggered in the admin loop.

Change-Id: Ia789a65dc204b546f1d3627a8006c4590bdca371
2017-07-11 17:51:47 +05:30
Pranav Kant
ccce64eeef This is percentage of total sys memory, not total consumed memory
Change-Id: I16b8b3d217cd0fabed83a9c63a5966cc6fe4d8fb
2017-07-11 17:51:47 +05:30
Pranav Kant
aa6b96aaed Skip memory cleanup when memproportion is not configured
Change-Id: I3799d914b5155c37f37bb497ac48e76c1a68761b
2017-07-10 20:41:49 +05:30
Pranav Kant
5ef7606316 Update the iterator too
Change-Id: Ia715fa11eb21a25935a2075d09c674bd45472e96
2017-07-08 01:52:46 +05:30
Pranav Kant
fde57adbbf Introduce hard mode when we are OOM
Start killing documents when memory usage goes above threshold.

Also make it possible to close documents from admin instance.
In DocumentBroker::closeDocument, just set the _stop flag and wake
up the polling thread which will terminate the children, instead of
manually terminating the children.

Change-Id: Ie70e05b3fb6ea816a87b6dcfaed92cdddb94aa90
2017-07-07 21:14:53 +05:30
Pranav Kant
13a541e4bb Lets do away with this
Consistency.

Change-Id: Iea670b77ab5f3f668183eac52f3ae8db92d79a40
2017-07-07 12:18:37 +05:30
Pranav Kant
3eaf31f81b Document missing protocol message
Change-Id: I79b2ceb1dafb884356b2b0a0b6fb59b493784fe4
2017-07-06 19:23:14 +05:30
Miklos Vajna
01bf6ab344 wsd: use the more effective overload accepting a character in Storage
I guess checking for "{" in parseJSON() happens for performance reasons,
but then it's a good idea to also avoid constructing a temporary string
for no reason.

Change-Id: I285ba0ed615054fc014796f7d8971a2b9783b075
2017-07-06 11:17:33 +02:00
Aditya Dewan
bc7fa9a812 admin: convert document list to drop down
Change-Id: I23635599b008fb0bfa694b4e73cb8f18f49c8ab7
Reviewed-on: https://gerrit.libreoffice.org/39170
Reviewed-by: pranavk <pranavk@collabora.co.uk>
Tested-by: pranavk <pranavk@collabora.co.uk>
2017-07-06 09:18:09 +02:00
Aditya Dewan
c29815f42c Assign unique editorId to localhost users
Change-Id: I1c8949fa7acca1b11b35b48629cc20a1170d8caa
Reviewed-on: https://gerrit.libreoffice.org/39442
Reviewed-by: pranavk <pranavk@collabora.co.uk>
Tested-by: pranavk <pranavk@collabora.co.uk>
2017-07-06 09:17:23 +02:00
Miklos Vajna
c07efecdcf wsd: avoid string concatenation resulting in allocation of unnecessary temporary strings in FileServer
By using operator+= instead, which does not have this problem.

Change-Id: I6dd54cde7e430f2f1d549642dddd3ed1a0e9eefd
2017-07-05 11:22:06 +02:00
Miklos Vajna
3e6680209e wsd: remove unused using declarations in FileServer
Change-Id: Ia56e5af384791df036411fc4acd4b1c69afe30f4
2017-07-04 10:19:45 +02:00
Ashod Nakashian
3582012333 wsd: don't resurrect forkit when shutting down
Change-Id: I88ebe5e07a5a9f6c0f52cb12e6acb688de763192
Reviewed-on: https://gerrit.libreoffice.org/39466
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2017-07-03 05:43:38 +02:00
Ashod Nakashian
c9b0dc0424 wsd: cleanup deflate data and free memory
With help from Valgrind to find and verify
these leaks. Also some minor cleanup to the
deflate logic.

Change-Id: I3bb3f1e01cef7025c45874ce52cfc922dfd19e21
Reviewed-on: https://gerrit.libreoffice.org/39465
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2017-07-03 05:43:22 +02:00
Ashod Nakashian
4f8ebca0cb wsd: free leaking memory
With help from Valgrind to find and verify
these leaks.

Change-Id: I3afeed89dc4bcd714a222f81822144477a346fb0
Reviewed-on: https://gerrit.libreoffice.org/39464
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2017-07-03 05:42:38 +02:00
Pranav Kant
a03f69a16a Update PostMessageOrigin if SSL termination is on too
Change-Id: I887ab368fec62a5efefde4da3762c47dd0bf66d7
Reviewed-on: https://gerrit.libreoffice.org/39100
Reviewed-by: pranavk <pranavk@collabora.co.uk>
Tested-by: pranavk <pranavk@collabora.co.uk>
2017-06-26 12:32:49 +02:00
Ashod Nakashian
6591f219ed wsd: use logging macros and const ref where possible
Change-Id: I2f56e1887ea9193083043763d7c876bfb785a7f8
Reviewed-on: https://gerrit.libreoffice.org/39249
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2017-06-26 05:19:42 +02:00
Ashod Nakashian
8689d11f5c wsd: don't busy wait when getNewChild fails fast
If Forkit is dead, getNewChild will fail fast.

Try not to busy loop in that case and yield
the CPU for a nominal duration, which currently
is the tenth of the normal getNewChild timeout.

Change-Id: I1a94dfedbf2a4f4fc12e4d33d1307f70c307987a
Reviewed-on: https://gerrit.libreoffice.org/39248
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2017-06-26 05:18:14 +02:00
Pranav Kant
c21de44524 Bin unused parameter, rude
Change-Id: I39f55f55f8a0c5432350625d9720ea2367955a8c
Reviewed-on: https://gerrit.libreoffice.org/39085
Reviewed-by: pranavk <pranavk@collabora.co.uk>
Tested-by: pranavk <pranavk@collabora.co.uk>
2017-06-23 10:42:55 +02:00
Pranav Kant
6c6ddbe425 Remove redundant function call
This is called later after the while loop. Breaking out of while loop
should be enough.

Change-Id: I04979d3af1f475c05b5a43d7afe47770ff69ee25
Reviewed-on: https://gerrit.libreoffice.org/39086
Reviewed-by: pranavk <pranavk@collabora.co.uk>
Tested-by: pranavk <pranavk@collabora.co.uk>
2017-06-23 10:42:44 +02:00
Michael Meeks
d43589c343 Replace now un-used locking with thread affinity assertions.
Change-Id: I8c08d1618404740e9dc1d5ff2cb7d9d460ca2be5
2017-06-22 18:41:16 +01:00
Pranav Kant
2c9966ed2f Bin unused header
Change-Id: I2067e793fe54e7c713b6d319c1f13b95a09923a3
2017-06-22 18:28:24 +05:30
Pranav Kant
2e42ae8c2f Enable HTTP key pinning when ssl termination is on too
Change-Id: Id2d7a34374236f50e28551ff9c57433d9153e2fd
2017-06-22 18:27:32 +05:30
Pranav Kant
d192fa4944 Obsolete comment
Change-Id: I30b1bcbb5c4bd040ab1205030d16867fa488bdb4
2017-06-22 12:56:23 +05:30
Michael Meeks
73a87493f0 Use WireIds instead of long hashes to identify tiles efficiently.
Changes protocol to use 'wid' instead of 'hash' everywhere. Wire-ids
are monotonically increasing integers that can be mapped to hash
values for all of the hash values and tiles we cache internally.

Change-Id: Ibcb25817bab0f453e93d52a6f99d3ff65059e47d
2017-06-20 21:49:44 +01:00
Ashod Nakashian
bc067c7b29 wsd: always prespawn children when creating forkit
Also, no need for the DocBrokerMutex to be locked
in createForkit.

Change-Id: I651c50c8b5518f7b0ee0d8b15c413fc070e30443
Reviewed-on: https://gerrit.libreoffice.org/38993
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2017-06-20 05:16:54 +02:00
Ashod Nakashian
c28992758d wsd: avoid deadlock when restoring forkit
LOOLWSD::checkAndRestoreForKit() eventually takes
NewChildrenLock, which is also taken in getNewChild_Blocks().

Luckily, we can defer that to the Prisoner poll, which we do.

Change-Id: Ib65c0be08870d1c4c68efd72192dac2e73c79f7c
Reviewed-on: https://gerrit.libreoffice.org/38992
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2017-06-20 05:16:23 +02:00
Ashod Nakashian
7ae37aff0d wsd: fix access-after-free error
Valgrind spotted one case, and the other is possible but
not common it seems.

Change-Id: Id5e41145f597c3564263adb25b7b765db1c90bf7
Reviewed-on: https://gerrit.libreoffice.org/38991
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2017-06-20 05:15:29 +02:00
Pranav Kant
c88780b2d4 wsd: Don't show dialog if document is unmodified
Change-Id: Ibb9e2122461c577863d1f713070748bdda6129d1
2017-06-19 20:01:31 +05:30
Michael Meeks
7b90179fe8 Check for spare space in the right place.
ie. destination not source of the local file.

Change-Id: I4d4c4e56deb039a44d8c2ab2886b8ca52bdb2ef8
2017-06-16 20:25:23 +01:00
Michael Meeks
7ea5f71a7b Units: allow filtering of user alerts.
Catch things that kill tests like disk-full earlier.

Change-Id: Ib50b516063305f2fc93b662ec3ad9ab6c52c6c92
2017-06-16 20:25:23 +01:00
Pranav Kant
db13014e00 Update modifiedTime for local files unconditionally
Also teach logger to print timestamp object.

Change-Id: Ia13836814c195cef92f5dafd8245c9958600876d
2017-06-16 19:49:37 +05:30
Aditya Dewan
681138ab54 tdf#106451 admin: graph to monitor network activity
Change-Id: Id71ef4e2a9d16e72f4df442fbf646a39213b61d5
Reviewed-on: https://gerrit.libreoffice.org/38621
Reviewed-by: pranavk <pranavk@collabora.co.uk>
Tested-by: pranavk <pranavk@collabora.co.uk>
2017-06-16 09:55:01 +02:00
Jan Holesovsky
4f89ce23b2 wsd: Honor the hosting/discovery read-only setting in LocalStorage too.
To make the debugging easier...

Change-Id: I7c6e748e72a595a6c3a5942a20874339b8456f19
Reviewed-on: https://gerrit.libreoffice.org/38781
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
Tested-by: Jan Holesovsky <kendy@collabora.com>
2017-06-14 14:02:23 +02:00
Ashod Nakashian
44d8876f27 Remove data rlimit
Seems to have no effect, so gone in
favor of RLIMIT_AS (virtual memeory).

Change-Id: I210879ec9285f420c9f9839cdabf45c42d865fb3
Reviewed-on: https://gerrit.libreoffice.org/38720
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2017-06-13 06:27:12 +02:00
Pranav Kant
5c7b9ce6e7 Consistency
Change-Id: I16889728d0062c058ea5831ff8892025e1719c3e
2017-06-12 16:09:10 +05:30
Pranav Kant
54ea436026 CSP - Add object-src: , this is required for printing documents
Change-Id: I4a759086f2b503dfa9df4000267d920984cfe422
2017-06-12 15:47:03 +05:30
Ashod Nakashian
459249b177 wsd: apply rlimits set from Admin Console to ForKit
Change-Id: I425c28ce08c5ecd659c4fe8eaa1a7ed3634e7f18
Reviewed-on: https://gerrit.libreoffice.org/38678
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2017-06-12 06:20:13 +02:00
Ashod Nakashian
9e11cd7db7 Support setting rlimits from Admin Console
Change-Id: Ia0d45948998d7a5612a1828a90ad20908d07639e
Reviewed-on: https://gerrit.libreoffice.org/38677
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2017-06-12 06:19:53 +02:00
Ashod Nakashian
6de3adcf9c wsd: pass default rlimits to forkit from config
Change-Id: I84d271f460f0fb1d03a973107c32265d84bf2841
Reviewed-on: https://gerrit.libreoffice.org/38675
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2017-06-12 06:18:31 +02:00
Ashod Nakashian
9cb82cebe3 wsd: add rlimit config entries and defaults
Change-Id: I8cb498d01bc1a7a55d168e49c754bb1bba80aea1
Reviewed-on: https://gerrit.libreoffice.org/38673
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2017-06-12 06:17:44 +02:00
Aditya Dewan
9502741590 tdf#107278 admin console: adding graph to track CPU load
Change-Id: Idb07fe4139dd639a49ce1545cc15895f74876b06
Reviewed-on: https://gerrit.libreoffice.org/38425
Reviewed-by: pranavk <pranavk@collabora.co.uk>
Tested-by: pranavk <pranavk@collabora.co.uk>
2017-06-08 12:58:12 +02:00
Jan Holesovsky
ca39f61330 Don't crash on Shift+reload.
Change-Id: I1706fc0ff748fa74f2d2b29f103c167acf36c706
Reviewed-on: https://gerrit.libreoffice.org/38516
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
Tested-by: Jan Holesovsky <kendy@collabora.com>
2017-06-07 16:35:57 +02:00
Ashod Nakashian
194faa6d69 wsd: force readonly for view file types per discovery.xml
File extensions marked as view (as opposed to edit)
in discovery.xml are now forced to be read-only,
regardless of what the client tries to request.

Change-Id: I3eb00c33ff716800dc317f7377281c6d5f0909d7
Reviewed-on: https://gerrit.libreoffice.org/38480
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2017-06-07 06:02:30 +02:00
Ashod Nakashian
44f4ba2730 wsd: discovery.xml now preprocessed and cached
Change-Id: Ie2f274452e9bd3a65061e24582a7280646162cdf
Reviewed-on: https://gerrit.libreoffice.org/38479
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2017-06-07 06:01:13 +02:00
Pranav Kant
2d0ec13249 CSP - allow WOPI host as image sources
This is needed to avoid CSP error when loading the avatar image URL from
the WOPI hosts.

Change-Id: I6bd8bd846f81bc799192e7fbc16b2d7ecc9f555e
2017-06-06 17:36:47 +05:30
Ashod Nakashian
80b6e97824 Set EverModified on saving the document
When saving while editing a cell, the
document is saved with the new cell contents
(as the cell editing is interrupted and applied),
but the modified status is not published to the
client (loleaflet).

This has the unfortunate side-effect that the
EverModified flag is not set in this scenario.

To avoid this, this patch sets the EverModified
upon successful saving. This is easier than
forcing LO Core to emit ModifiedStatus=true
in the scenario above. Though it suffers from
the false-positive result when the user forces
a save. Although this is probably defensible,
since technically the document changes upon
saving (though not the content if there was
nothing new to save). On the other hand,
it's less harmful to signal modification
due to save than to ignore genuine
modification to the document.

Change-Id: I7ccba62731f534b69819f7924eeb9231d874c111
Reviewed-on: https://gerrit.libreoffice.org/38398
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2017-06-05 06:52:24 +02:00
Michael Meeks
eeaf436d53 Admin: show cumulative bandwidth sent / recv'd over all time.
Change-Id: I3f9f398d1de19d54e0aa4c51bc44c597019dc839
2017-06-03 22:53:57 +01:00
Michael Meeks
3101fa510d Accumulate I/O stats per document.
Change-Id: Ie2f5647e65070ddd828f048820efd38b600f9133
2017-06-02 20:41:25 +01:00
Pranav Kant
a0710c9613 Inform the current session about document change too
Change-Id: I9947eb8b23e5a698cc2cbf39bfde4e1941aae0f0
2017-06-01 22:07:25 +05:30