Commit graph

59 commits

Author SHA1 Message Date
Michael Meeks
e3864a060e Dump DocumentBroker state more completely.
Change-Id: I3477fe70ba25d6e9a95c12f30138c3353994e384
2019-11-27 19:17:27 +00:00
Gabriel Masei
2164f5207c Add REST endpoint for admin metrics.
Change-Id: I701485631931334d27594c4907cb770f9888e5bf
Reviewed-on: https://gerrit.libreoffice.org/82492
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Michael Meeks <michael.meeks@collabora.com>
2019-11-25 13:06:01 +01:00
Ashod Nakashian
d038ceb118 test: Improve iso8601ToTimestamp and tests
Using double caused all sorts of rounding issues,
especially with random unit-test failures.
Luckily, we don't need doubles and can do everything
with integers.

Also added a new function to print time_point as
iso8601 string, for logging and convenience.

Change-Id: I1c2040c02d1143282dbde0dadef32613b77c330d
Reviewed-on: https://gerrit.libreoffice.org/81578
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2019-10-29 02:39:34 +01:00
Ashod Nakashian
77e7cae689 wsd: fix password-protect file loading from multiple views
With password-protected files, the first loading attempt
always fails due to missing password. At that point the
client is notified of the missing password and the user
is prompted. The second attempt includes a (hopefully)
correct password and the document loading commences.

Due to the fact that an exception is raised when
the loading fails, this left the loading latch
triggered, which blocked subsequent attempts.

Change-Id: I7cc257a36eb1cc080f460aac8cdb7030783a5914
2019-10-14 21:35:27 +01:00
Andras Timar
d1ae1b36c2 typo fixes in comments and code
Change-Id: Idd98516d30d98dea18eda9bbec8ac9777063b553
2019-10-08 11:49:45 +02:00
DarkByt31
dd014e7029 tdf#107038 Poco::Timestamp replacement with std::chrono
Util added getHttpTime
WhiteBoxTests added test for getHttpTime

Change-Id: Ifb6a3fb2dc9b059b925e7b881362b72759a8b56b
Reviewed-on: https://gerrit.libreoffice.org/79754
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Michael Meeks <michael.meeks@collabora.com>
2019-09-28 12:26:32 +02:00
Tor Lillqvist
a7362d52ac Use portable format conversion specifier for uint64_t
Avoids a warning when compiling for iOS: format specifies type
'unsigned long' but the argument has type 'uint64_t' (aka 'unsigned
long long').

Change-Id: I8b5205dd0c3a8ae2f531f1647b3e3bac27ea6065
Reviewed-on: https://gerrit.libreoffice.org/78985
Reviewed-by: Tor Lillqvist <tml@collabora.com>
Tested-by: Tor Lillqvist <tml@collabora.com>
2019-09-16 13:52:14 +02:00
Michael Meeks
79637655f6 Avoid PRIx64 format - we are 64bit only, so lx will do.
Fixes RHEL6 compilation problems around inttypes inclusion / definition.

Change-Id: I4f4647c7954c32d90dad94a43c630af24d551d78
2019-09-05 22:53:09 +01:00
DarkByt31
22f1656e08 tdf#107038 Poco::Timestamp replacement with std::chrono
Added functions to get file timestamp and to convert
chrono timestamp in ISO8601 fraction format and some
test cases.

Change-Id: I58961a31f7262b367cff9f33cffdec7571a2f8f7
2019-09-02 15:50:37 -04:00
Ashod Nakashian
919a93cd4b wsd: improved anonymization algorithm
Better hashing algorithm based on FNV-1a.
Adds support for salting the hash, and
for providing salt via configuration.

More unit-tests added, and better formatting.

Change-Id: I2be42675d0cdbaa73c3d7faed99e07631a9c20fc
Reviewed-on: https://gerrit.libreoffice.org/70034
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Reviewed-on: https://gerrit.libreoffice.org/71091
2019-08-17 04:26:35 +02:00
Miklos Vajna
13884468c3 common: fix UB in vectorAppend()
unit-copy-paste with ubsan failed with:

common/Util.hpp:751:43: runtime error: null pointer passed as argument 2, which is declared to never be null
/usr/include/string.h:43:28: note: nonnull attribute specified here
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior common/Util.hpp:751:43 in

Change-Id: I71ae9b43a63f979c300d704419afc9a14cd303cd
2019-08-15 08:38:07 +02:00
Michael Meeks
149b861975 clipboard: a more helpful calc test.
Change-Id: Ibe0e583a2019e7bc5968dd8f73cf4e4a07d363b2
2019-08-05 21:18:27 -04:00
Michael Meeks
9726ce6f77 Various copy/paste fixes, unit test passing, parsing of result.
Change-Id: I9168853dd011e86896c3bd474a4d05ee82c0e336
2019-08-05 21:18:12 -04:00
Michael Meeks
7f4a7404cc Add hard random/hex key generation utility.
Change-Id: I8bb95153a5bd63a6b41edf65a0fdf0ebd52ad070
2019-08-05 21:08:07 -04:00
Michael Meeks
c36da1cd97 Implement getClipboard and start of a unit test.
includes Change-Id: I47d38f9285c855dcc6fbeeb73d35f22a46a30c05

Change-Id: I60b84bef1cf832ec5c191d664afd90a080b7ffbc
2019-08-05 20:25:49 -04:00
Michael Meeks
52e477e57e Switch to text/html for paste where we can.
Build special URLs to detect the same host being in-use, and much more.

Change-Id: I0ca639ea416cb78bf5e5274eac4400542b6b2cda
2019-08-05 15:47:47 -04:00
Michael Meeks
9d723cb230 Initial chunked transfer encoding.
Important for convert-to on larger documents and/or with newer curls.

Change-Id: Id18be6d22741a3af7cee39a069c509e4f662977b
2019-05-22 11:07:42 +01:00
DarkByt31
8e34705fe2 tdf#107038 Poco::DateTimeFormatter with Util::getHttpTimeNow()
Added a function to Util to get current time in HTTP
format using std::chrono.

Change-Id: I9e7a732f585c1758c9348c450a01713a66f1e7b7
Reviewed-on: https://gerrit.libreoffice.org/72585
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Michael Meeks <michael.meeks@collabora.com>
2019-05-20 11:53:13 +02:00
Michael Meeks
289894c66c debug: show WebSocketURI & a unique host id in help -> about.
Rather useful for debugging clustering issues.

Change-Id: I6d5f224bf8a3e4034c419137c8ad2b17fdf265ed
2019-05-17 13:24:32 +01:00
Jan Holesovsky
f76b36193d android: #if(n)def MOBILEAPP -> #if (!)MOBILEAPP for better reliability.
Change-Id: I5f9c9420b6c83601db1c8fdba4ae5a10b17b2107
2019-02-12 12:20:11 +01:00
Tor Lillqvist
5f01fbdf81 The iOS memrchr() replacement should of course be marked inline 2018-10-17 12:34:49 +03:00
Tor Lillqvist
0d8912ffe9 Test also looking for the last char 2018-10-17 11:55:19 +03:00
Tor Lillqvist
ccc7d2133d "The memrchr() function is a GNU extension, available since glibc 2.1.91."
Thus it doesn't exist on iOS, so add a trivial implementation. Include
an (ifdeffed out) unit test for it.
2018-10-17 11:46:56 +03:00
Jan Holesovsky
9a8e4fc0e8 wsd: safer string splitting
Reviewed-on: https://gerrit.libreoffice.org/57644
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
(cherry picked from commit 17d1fdda7a0c29df12c43b956418c83b59bfe0da)

Change-Id: I88b82a3754c4f5e280f00be8e27614c3fe49eff8
2018-10-16 20:12:23 -04:00
Ashod Nakashian
b6333ada8d wsd: anonymization improvements and unittests
Also support anonymization of downloadas documents
and renaming of documents.

Reviewed-on: https://gerrit.libreoffice.org/57541
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
Tested-by: Jan Holesovsky <kendy@collabora.com>
(cherry picked from commit 78248a542c9ca31bf9ad4cad9b55d78690384395)

Change-Id: I81a80e6290217659987d73f625e5f0fb81cb7ef2
2018-10-16 20:12:23 -04:00
Ashod Nakashian
053161b3f5 wsd: move string utilities into Util
Reviewed-on: https://gerrit.libreoffice.org/57539
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
Tested-by: Jan Holesovsky <kendy@collabora.com>
(cherry picked from commit fb2671c4145edd4d4e359f0dcf5cc84835487cd4)

Change-Id: Idc578dff4e8ee5e48c1b7780d3feb2d21c6a9b13
2018-10-16 20:12:23 -04:00
Ashod Nakashian
4587dde56f wsd: anonymize filename by using the WOPI file ID
Reviewed-on: https://gerrit.libreoffice.org/57254
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
Tested-by: Jan Holesovsky <kendy@collabora.com>
(cherry picked from commit 5e3568ff1029da948f05d1c0e0c56c6d0706690e)

Change-Id: I869cae3846c8630b192246bc68cc90e70c50d1fd
2018-10-16 20:12:23 -04:00
Ashod Nakashian
6c18aa7e88 kit: anonymize usernames and filenames in Kit
Change-Id: Id7928136db71ded7bf6b1a5e8e387db7251f8a35
2018-10-16 20:12:23 -04:00
Ashod Nakashian
f12883bc07 wsd: add anonymization helpers
Change-Id: Ic479218ab1b6e580c288a984f35795e1d0d6e8ad
2018-10-16 20:12:23 -04:00
Tor Lillqvist
b59d160a08 Intermediate commit of work in progress on an iOS app
The app is unimaginatively called "Mobile" for now.

Runs but crashes pretty quickly after loading the document by the LO
core. Will need some heavy changes to get a ClientSession object
created in there, too, to handle the (emulated) WebSocket messages
from the JavaScript. It would then handle some of these messages
itself, and forwards some to the ChildSession, which in this case is
in the same process. Now the messsages from the JavaScript go to a
ChildSession, which is wrong. As the assertion says, "Tile traffic
should go through the DocumentBroker-LoKit WS"
2018-09-12 18:32:05 +03:00
Tor Lillqvist
75438baa70 More mobile app stuff, very much early state of work in progress
Re-think Linux vs mobile ifdefs a bit. Use #ifdef __linux only to
surround code that actually is Linux-specific. Use #ifdef MOBILEAPP
for code that is for a mobile version (with no separste wsd, forkit,
and kit processes, and with no WebSocket protocol used).

Bypass UnitFoo for mobile. Possibly we do want the UnitFoo stuff after
all on mobile, to run in some special testing mode? Hard to say, let's
skipt it for now.
2018-09-10 15:13:43 +03:00
Tor Lillqvist
39072dbae5 Make this file compile for iOS 2018-08-29 20:41:16 +03:00
Tor Lillqvist
40f4cf17c9 Add a symbolicErrno() function to Util
For a developer, it is much nicer to see "EXDEV" in a log than "18",
for instance. (Sure, we often also log strerror(), but might that
perhaps be localised? And the symbolic names are what one uses when
coding anyway.)

Change-Id: I456a8c2589147dcad87f1b4c3a20b3bd5a35d097
2018-07-11 12:54:27 +03:00
Michael Meeks
03bbe52665 Ensure the forkit does not have lingering threads after preinit.
Change-Id: I216a03a0ada628189355ebfd4016bf6033430eef
2018-04-16 20:04:23 +01:00
Jan Holesovsky
c8ef63253a Sanity-check the scheme and host for frame ancestor, POCO does not do that.
Change-Id: Ieea9532ccd2a11e74f370a340e68f46122469848
2018-04-04 12:50:53 +02:00
Michael Meeks
6803ee4087 Use a hard random /tmp directory for document intermediates.
Requires a lok tweak to reset the tmpdir globals on init.

Change-Id: I602705f20fa016ef415088be8d1ada6d465d2a8d
2018-03-15 11:01:51 +00:00
Ashod Nakashian
5befd0803a wsd: improved wopi info parsing
Better logging during wopi info parsing,
especially upon failures.
Refactored the code from Storage.cpp into
JsonUtil.hpp.
Minor optimizations.

Add unit-tests for the parsing logic.

Change-Id: Ifebc3f6b7030a6c7b3b399786633f6b5e8737478
Reviewed-on: https://gerrit.libreoffice.org/49927
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2018-02-23 16:54:39 +01:00
Miklos Vajna
9eacfe4fcb common: spell out non-trivial autos to improve readability
Change-Id: Id13bc0e48cec845f5b05171128be5b4efc05c6bc
2018-02-07 10:18:12 +01:00
Michael Meeks
d3c17510ed Implement an improved fork/exec wrapper.
* logs helpful messages for various error corner-cases.
* optimized file descriptor closing for large fd counts.

Change-Id: I8cba9ecb3d71ddc6e22e20d89368d8c6b9b5097f
2018-01-29 15:20:10 +00:00
Pranav Kant
c16d198560 Print humanized bytes in the logs
Change-Id: I2ebdea0c3a360be1573ae13fa9cbe6f432436f29
2018-01-26 20:03:22 +05:30
Tor Lillqvist
7bf59d8f64 Add Util::startsWith() to check whether a std::string starts with another
Odd that we have managed without such a helper for so long, or did I
just not find it?
2018-01-15 13:39:48 +02:00
Ashod Nakashian
59398af621 wsd: simpler and more efficient OOM killing
Change-Id: I118abdffba4e7ab57fe6a29a3a9fc420d871bdc0
Reviewed-on: https://gerrit.libreoffice.org/47738
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2018-01-11 08:12:09 +01:00
Michael Meeks
864b07f5b3 Annotate string pointers in hex dumps to make sense of memory.
Change-Id: I7934ccf379dbca9862ddb911865aaeff692261bf
2017-12-15 22:45:04 +00:00
Michael Meeks
26876e6165 loolmap - read the parent's page data for unshared pages too.
Change-Id: I23bb319fa4694638efc82c7af86224d2e041160c
2017-11-30 13:33:25 +00:00
Michael Meeks
0778b72697 Dump hex contents of un-shared memory.
Change-Id: I47dfa8a0b48d0ba34b7462301b83e2721617814d
2017-11-29 03:33:16 +00:00
Pranav Kant
fde57adbbf Introduce hard mode when we are OOM
Start killing documents when memory usage goes above threshold.

Also make it possible to close documents from admin instance.
In DocumentBroker::closeDocument, just set the _stop flag and wake
up the polling thread which will terminate the children, instead of
manually terminating the children.

Change-Id: Ie70e05b3fb6ea816a87b6dcfaed92cdddb94aa90
2017-07-07 21:14:53 +05:30
Aditya Dewan
9502741590 tdf#107278 admin console: adding graph to track CPU load
Change-Id: Idb07fe4139dd639a49ce1545cc15895f74876b06
Reviewed-on: https://gerrit.libreoffice.org/38425
Reviewed-by: pranavk <pranavk@collabora.co.uk>
Tested-by: pranavk <pranavk@collabora.co.uk>
2017-06-08 12:58:12 +02:00
Ashod Nakashian
da2d3cbc92 wsd: support per-user links and commands
userextrainfo is a json array that contains
extra user-specific links.

Currently 'avatar' is assumed to hold the
image url for the user's avatar.

'mail' and other links can also be added.

Change-Id: I37c4c68bfa0b7ee659e017b4867dcb8cf5c2ca2f
Reviewed-on: https://gerrit.libreoffice.org/38120
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2017-05-29 06:45:18 +02:00
Pranav Kant
7a4bc5b95a admin-console: Check the password against hashed value in config
The new password hash property is called secure_password in the config
file. `loolconfig` tool should be used to set the password hash in
appropriate format with desired salt length, password length, number of
iterations in PBKDF2.

To be backward compatible, plain-text password for admin-console in
config file is still accepted in case secure_password property is
missing from the config file.

Change-Id: If229999dac62856e368555c0242c4aa6f8061fba
2017-05-25 01:18:31 +05:30
Ashod Nakashian
484fcd63a5 wsd: corrections to replace and unittests
Change-Id: I0e03deaa710ba722a97c9f857b514f8ecb0e1682
Reviewed-on: https://gerrit.libreoffice.org/37609
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2017-05-15 05:13:21 +02:00