Commit graph

57 commits

Author SHA1 Message Date
Ashod Nakashian
e7cec81846 Fix build - potential use before initialization
Change-Id: I5d9db13b0241af64fa8e9eb9528a6e460166f280
2019-08-05 21:18:57 -04:00
Michael Meeks
7f4a7404cc Add hard random/hex key generation utility.
Change-Id: I8bb95153a5bd63a6b41edf65a0fdf0ebd52ad070
2019-08-05 21:08:07 -04:00
Michael Meeks
52e477e57e Switch to text/html for paste where we can.
Build special URLs to detect the same host being in-use, and much more.

Change-Id: I0ca639ea416cb78bf5e5274eac4400542b6b2cda
2019-08-05 15:47:47 -04:00
DarkByt31
8e34705fe2 tdf#107038 Poco::DateTimeFormatter with Util::getHttpTimeNow()
Added a function to Util to get current time in HTTP
format using std::chrono.

Change-Id: I9e7a732f585c1758c9348c450a01713a66f1e7b7
Reviewed-on: https://gerrit.libreoffice.org/72585
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Michael Meeks <michael.meeks@collabora.com>
2019-05-20 11:53:13 +02:00
Michael Meeks
289894c66c debug: show WebSocketURI & a unique host id in help -> about.
Rather useful for debugging clustering issues.

Change-Id: I6d5f224bf8a3e4034c419137c8ad2b17fdf265ed
2019-05-17 13:24:32 +01:00
Ashod Nakashian
8136bbc1b5 wsd: parse empty json string as empty map
Change-Id: I11335e82e42e9f896ae33c2e57cb9d79b642171e
Reviewed-on: https://gerrit.libreoffice.org/69633
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Reviewed-on: https://gerrit.libreoffice.org/71097
Reviewed-by: Szymon Kłos <szymon.klos@collabora.com>
Tested-by: Szymon Kłos <szymon.klos@collabora.com>
2019-05-08 20:38:27 +02:00
Ashod Nakashian
ed0efde542 wsd: use thread_local instead of __thread
The former is the standard C++ approach
and is reportedly faster than __thread
(at least with gcc).

Change-Id: Ibdefd32172774a280637f73dd062282b7bf62025
Reviewed-on: https://gerrit.libreoffice.org/71019
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2019-04-23 03:00:44 +02:00
Michael Meeks
5627c767ab Anonymization: don't log when it is disabled.
Also fix unexpected concatenation error in Poco::URI::encode generating
eg. authorid=localhost0 xauthorid=localhost0localhost0 in the output.

Change-Id: I560e47e31884eeb1c662f468436ed7541cfb082d
2019-03-15 11:27:38 +01:00
Jan Holesovsky
f76b36193d android: #if(n)def MOBILEAPP -> #if (!)MOBILEAPP for better reliability.
Change-Id: I5f9c9420b6c83601db1c8fdba4ae5a10b17b2107
2019-02-12 12:20:11 +01:00
Jan Holesovsky
e60421e5d5 android: No execinfo.h on Android (yet). 2019-02-12 10:50:40 +01:00
Ashod Nakashian
b6333ada8d wsd: anonymization improvements and unittests
Also support anonymization of downloadas documents
and renaming of documents.

Reviewed-on: https://gerrit.libreoffice.org/57541
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
Tested-by: Jan Holesovsky <kendy@collabora.com>
(cherry picked from commit 78248a542c9ca31bf9ad4cad9b55d78690384395)

Change-Id: I81a80e6290217659987d73f625e5f0fb81cb7ef2
2018-10-16 20:12:23 -04:00
Ashod Nakashian
00a44d6e81 wsd: flush logs before existing
This is important for when we abort with some explanation.
Often said explanation doesn't show up anywhere to be useful.

Also, issue fatal logs for abnormal exist and use SFL to log errno.

Reviewed-on: https://gerrit.libreoffice.org/57540
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
Tested-by: Jan Holesovsky <kendy@collabora.com>
(cherry picked from commit ad7964393eadb68873b820e0a620fb40f1e1b06a)

Change-Id: Ic67064ef40ef6e93d26e5847ecd32bdd49c3cc8b
2018-10-16 20:12:23 -04:00
Ashod Nakashian
b516891815 wsd: prevent anonymization to empty strings
Change-Id: Ib4f90db5d39e7bf2e2f0b6566b1927363e6afcec
Reviewed-on: https://gerrit.libreoffice.org/57377
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
Tested-by: Jan Holesovsky <kendy@collabora.com>
(cherry picked from commit 84245aa61e89cda6a9075a7059b5a7d839389719)
2018-10-16 20:12:23 -04:00
Ashod Nakashian
4587dde56f wsd: anonymize filename by using the WOPI file ID
Reviewed-on: https://gerrit.libreoffice.org/57254
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
Tested-by: Jan Holesovsky <kendy@collabora.com>
(cherry picked from commit 5e3568ff1029da948f05d1c0e0c56c6d0706690e)

Change-Id: I869cae3846c8630b192246bc68cc90e70c50d1fd
2018-10-16 20:12:23 -04:00
Ashod Nakashian
f12883bc07 wsd: add anonymization helpers
Change-Id: Ic479218ab1b6e580c288a984f35795e1d0d6e8ad
2018-10-16 20:12:23 -04:00
Ashod Nakashian
3158d4c31c wsd: always use signal-safe calls
The async-signal-safe functions to get thread-id
and thread-name, which cache the results, are
faster, cleaner, and signal-safe. No reason why
we shouldn't always use them.

Especially since it appears the logic was
inverted in Log::prefix, such that the signal
un-safe calls were made during signal-handling,
and the safe ones were called otherwise!

Instead of passing the signal-safe flag to
Log::prefix, we pass the buffer size, for
improved security.

Furthermore, reduce header dependencies
and reduce clutter.

Change-Id: I697689b2f0a290b6d8cce4babc3ac1e576141da6
2018-10-16 20:12:23 -04:00
Tor Lillqvist
58db979291 Start on a gtk+-based workalike to the iOS app
The idea is that it would work sufficiently identically, so that even
people without a Mac and without an iOS device could participate in
development of the non-iOS-specific bits, like the JavaScript, or the
online MOBILEAPP-specific plumbing. Which would be great.

No, this doesn't do anything sane yet. It does compile the same online
C++ files as the iOS app, though. (Some minor tweaks were needed in a
couple of them to silence gcc warnings.)

There is a plain Makefile, but I should change to using autofoo, too.
Eventually, this will need to be built in a separate tree from a
normal online, just like when using the --enable-iosapp configure
switch. (But for now, doesn't matter.)

Change-Id: I13e4d921acb99d802d2f9da4b0df4a237ca60ad6
2018-10-17 00:45:35 +03:00
Tor Lillqvist
42948f6d75 On iOS, Util::getThreadId() just wraps std::this_thread::get_id() 2018-10-16 23:04:33 +03:00
Tor Lillqvist
b59d160a08 Intermediate commit of work in progress on an iOS app
The app is unimaginatively called "Mobile" for now.

Runs but crashes pretty quickly after loading the document by the LO
core. Will need some heavy changes to get a ClientSession object
created in there, too, to handle the (emulated) WebSocket messages
from the JavaScript. It would then handle some of these messages
itself, and forwards some to the ChildSession, which in this case is
in the same process. Now the messsages from the JavaScript go to a
ChildSession, which is wrong. As the assertion says, "Tile traffic
should go through the DocumentBroker-LoKit WS"
2018-09-12 18:32:05 +03:00
Tor Lillqvist
75438baa70 More mobile app stuff, very much early state of work in progress
Re-think Linux vs mobile ifdefs a bit. Use #ifdef __linux only to
surround code that actually is Linux-specific. Use #ifdef MOBILEAPP
for code that is for a mobile version (with no separste wsd, forkit,
and kit processes, and with no WebSocket protocol used).

Bypass UnitFoo for mobile. Possibly we do want the UnitFoo stuff after
all on mobile, to run in some special testing mode? Hard to say, let's
skipt it for now.
2018-09-10 15:13:43 +03:00
Tor Lillqvist
7d850bfa6e Make this file compile for iOS
Change-Id: I1d82fed408818a6945cbf2b7743ffcdbe67f6079
2018-08-29 20:41:07 +03:00
Tor Lillqvist
57326ae27c Third parameter to prefix() is a bool
Passing syscall(SYS_gettid) as a bool is equivalent to passing true,
as far as I understand.

Change-Id: I31bb15000a9e6c95b657d58bc78df4f3da0fe687
2018-08-29 18:36:16 +03:00
Michael Meeks
03bbe52665 Ensure the forkit does not have lingering threads after preinit.
Change-Id: I216a03a0ada628189355ebfd4016bf6033430eef
2018-04-16 20:04:23 +01:00
Jan Holesovsky
c8ef63253a Sanity-check the scheme and host for frame ancestor, POCO does not do that.
Change-Id: Ieea9532ccd2a11e74f370a340e68f46122469848
2018-04-04 12:50:53 +02:00
Michael Meeks
6803ee4087 Use a hard random /tmp directory for document intermediates.
Requires a lok tweak to reset the tmpdir globals on init.

Change-Id: I602705f20fa016ef415088be8d1ada6d465d2a8d
2018-03-15 11:01:51 +00:00
Ashod Nakashian
7be98efd93 ut: log timestamp to track timeouts better
And improve the logging support in unit-tests to
help troubleshoot issues faster and more accurately.
Also makes the code more readable (hopefully).

Change-Id: I4f8aafb5245e2f774b03231591a74544f9ec84aa
Reviewed-on: https://gerrit.libreoffice.org/48645
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2018-02-11 19:21:15 +01:00
Miklos Vajna
9eacfe4fcb common: spell out non-trivial autos to improve readability
Change-Id: Id13bc0e48cec845f5b05171128be5b4efc05c6bc
2018-02-07 10:18:12 +01:00
Henry Castro
f269a9321c follow up, avoid logging when closing file descriptors
Change-Id: I38432ae1d042094551d22c2512a416c3b345556b
2018-01-31 17:53:58 -04:00
Henry Castro
b1f1500b56 util: avoid logging when closing file descriptors
eventually the log file descriptor will be closed and unhandled exception it will throw it

"terminate called after throwing an instance of 'Poco::WriteFileException'
  what():  Cannot write file"

Change-Id: I1d6ae3a4d4d4910f2ed2cdc80b162c27f93d55d9
Reviewed-on: https://gerrit.libreoffice.org/49055
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Michael Meeks <michael.meeks@collabora.com>
2018-01-31 22:06:57 +01:00
Miklos Vajna
222399e99a common: avoid pointless copy in Util
Change-Id: I40097203eba6fede78a17caf057452f4704b2bd4
2018-01-30 16:09:33 +01:00
Michael Meeks
d3c17510ed Implement an improved fork/exec wrapper.
* logs helpful messages for various error corner-cases.
* optimized file descriptor closing for large fd counts.

Change-Id: I8cba9ecb3d71ddc6e22e20d89368d8c6b9b5097f
2018-01-29 15:20:10 +00:00
Pranav Kant
c16d198560 Print humanized bytes in the logs
Change-Id: I2ebdea0c3a360be1573ae13fa9cbe6f432436f29
2018-01-26 20:03:22 +05:30
Ashod Nakashian
59398af621 wsd: simpler and more efficient OOM killing
Change-Id: I118abdffba4e7ab57fe6a29a3a9fc420d871bdc0
Reviewed-on: https://gerrit.libreoffice.org/47738
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2018-01-11 08:12:09 +01:00
Pranav Kant
f63858433b loplugin:includeform
Change-Id: Ib62a7aa61062f00698aa3e8a144438de5c57e53d
2017-12-20 21:21:05 +05:30
Miklos Vajna
bb7ac21d44 Make sure that loop index type is never narrower than the length type
Change-Id: I19aac80850c0371085ca6364412d5ac9207220d0
2017-11-06 09:14:38 +01:00
Miklos Vajna
a3d6dee503 Remove unused strings
Change-Id: I621c462bca38222dcf26dfa1e414cf27e1a3e088
2017-11-03 15:25:09 +01:00
Pranav Kant
fde57adbbf Introduce hard mode when we are OOM
Start killing documents when memory usage goes above threshold.

Also make it possible to close documents from admin instance.
In DocumentBroker::closeDocument, just set the _stop flag and wake
up the polling thread which will terminate the children, instead of
manually terminating the children.

Change-Id: Ie70e05b3fb6ea816a87b6dcfaed92cdddb94aa90
2017-07-07 21:14:53 +05:30
Aditya Dewan
9502741590 tdf#107278 admin console: adding graph to track CPU load
Change-Id: Idb07fe4139dd639a49ce1545cc15895f74876b06
Reviewed-on: https://gerrit.libreoffice.org/38425
Reviewed-by: pranavk <pranavk@collabora.co.uk>
Tested-by: pranavk <pranavk@collabora.co.uk>
2017-06-08 12:58:12 +02:00
Ashod Nakashian
da2d3cbc92 wsd: support per-user links and commands
userextrainfo is a json array that contains
extra user-specific links.

Currently 'avatar' is assumed to hold the
image url for the user's avatar.

'mail' and other links can also be added.

Change-Id: I37c4c68bfa0b7ee659e017b4867dcb8cf5c2ca2f
Reviewed-on: https://gerrit.libreoffice.org/38120
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2017-05-29 06:45:18 +02:00
Pranav Kant
7a4bc5b95a admin-console: Check the password against hashed value in config
The new password hash property is called secure_password in the config
file. `loolconfig` tool should be used to set the password hash in
appropriate format with desired salt length, password length, number of
iterations in PBKDF2.

To be backward compatible, plain-text password for admin-console in
config file is still accepted in case secure_password property is
missing from the config file.

Change-Id: If229999dac62856e368555c0242c4aa6f8061fba
2017-05-25 01:18:31 +05:30
Ashod Nakashian
484fcd63a5 wsd: corrections to replace and unittests
Change-Id: I0e03deaa710ba722a97c9f857b514f8ecb0e1682
Reviewed-on: https://gerrit.libreoffice.org/37609
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2017-05-15 05:13:21 +02:00
Ashod Nakashian
9798eafb8c wsd: random jail paths instead of pid
Jail paths are now generate from a PRNG
instead of using the PID of the kit process.

The PRN is converted to base-64 and used
as the directory name where a given
kit is jailed.

Change-Id: I8e4bc35d9ccdfdae0e542ab707c417cd29ad52f3
Reviewed-on: https://gerrit.libreoffice.org/37372
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2017-05-08 04:59:12 +02:00
Ashod Nakashian
e0822f8516 wsd: better thread affinity logging
Change-Id: I9e4bc3fe864aa409dc4874a9d6fc4ab22bfea592
Reviewed-on: https://gerrit.libreoffice.org/36029
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2017-04-03 05:11:57 +00:00
Michael Meeks
e79c5f9083 Use process name for a thread if no name is set (yet). 2017-03-31 11:21:35 +01:00
Michael Meeks
ae0dba1088 Cleanup prctl / gettid system-call thrash on logging.
Makes the strace look much prettier.
2017-03-30 18:14:40 +01:00
Ashod Nakashian
7096133f07 wsd: log thread affinity violations
Change-Id: Ib1317bc71f9162f005e0ce9b8c715bbce656db73
2017-03-20 00:41:53 -04:00
Michael Meeks
0eaef6c896 config.h - get includes right: must always be the first include. 2017-03-10 10:47:43 +01:00
Ashod Nakashian
1e42b530e6 wsd: read memory RSS directly from /proc/pid/stat
There is no need to use `ps` here as reading
directly is trivial and has far less overhead.

Change-Id: I27d0432c1f3a9d35763d67fc445d8bd828f1b27e
Reviewed-on: https://gerrit.libreoffice.org/34052
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2017-02-09 04:35:24 +00:00
Michael Meeks
e3e2a5eeee WaE: add missing bracket. 2017-02-07 20:07:37 +00:00
Michael Meeks
9ec0466c57 Admin - measure dirty memory of kits, and RSS of forkit.
Gives a more accurate real memory consumption for each process.
2017-02-07 18:55:32 +00:00