Commit graph

22 commits

Author SHA1 Message Date
Michael Meeks
0599db3576 copyFile: de-poco-ize and handle EINTR and short writes.
Change-Id: I2046881c786a9f31f45c53f282de9ddd9a9cebcf
2020-01-21 15:07:54 +00:00
Michael Meeks
e6a3364e93 Move file url anonymization down from LOOLWSD into FileUtil.
Change-Id: I415c73b10621d5c7c942367bbf38a3bbd9bf8f27
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/87024
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
2020-01-18 20:05:35 +01:00
Ashod Nakashian
1ae9ce8ad2 test: thread-safe common shared test data
This protects against memory corruptions,
and a cascade of issues, such as the following:

 Attaching to process 56245
 [New LWP 56246]
 [New LWP 56252]
 [New LWP 56253]
 [New LWP 56254]
 [New LWP 56362]
 [New LWP 56364]
 [Thread debugging using libthread_db enabled]
 Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
 0x00007fda228f7d2d in __GI___pthread_timedjoin_ex (threadid=140574279595776, thread_return=0x0, abstime=0x0, block=<optimized out>) at pthread_join_common.c:89
 89	pthread_join_common.c: No such file or directory.

 Thread 7 (Thread 0x7fda197fa700 (LWP 56364)):
 #0  __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
 #1  0x00007fda228f9023 in __GI___pthread_mutex_lock (mutex=0x55e9937567e0 <SigHandlerTrap>) at ../nptl/pthread_mutex_lock.c:78
 #2  0x000055e9934cd4fe in __gthread_mutex_lock (__mutex=0x55e9937567e0 <SigHandlerTrap>) at /usr/include/x86_64-linux-gnu/c++/7/bits/gthr-default.h:748
 #3  std::mutex::lock (this=0x55e9937567e0 <SigHandlerTrap>) at /usr/include/c++/7/bits/std_mutex.h:103
 #4  std::unique_lock<std::mutex>::lock (this=<synthetic pointer>) at /usr/include/c++/7/bits/std_mutex.h:267
 #5  std::unique_lock<std::mutex>::unique_lock (__m=..., this=<synthetic pointer>) at /usr/include/c++/7/bits/std_mutex.h:197
 #6  SigUtil::handleFatalSignal (signal=11) at common/SigUtil.cpp:214
 #7  <signal handler called>
 #8  std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_capacity (this=<optimized out>, __capacity=0) at /usr/include/c++/7/bits/basic_string.h:200
 #9  std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string (__str=..., this=<optimized out>) at /usr/include/c++/7/bits/basic_string.h:542
 #10 std::_Construct<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > (
     __p=<optimized out>) at /usr/include/c++/7/bits/stl_construct.h:75
 #11 std::__uninitialized_copy<false>::__uninit_copy<std::move_iterator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*> (__result=0x7fd9f40017b0, __last=..., __first=...) at /usr/include/c++/7/bits/stl_uninitialized.h:83
 #12 std::uninitialized_copy<std::move_iterator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*> (__result=<optimized out>, __last=..., __first=...) at /usr/include/c++/7/bits/stl_uninitialized.h:134
 #13 std::__uninitialized_copy_a<std::move_iterator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > (__result=<optimized out>, __last=..., __first=...)
     at /usr/include/c++/7/bits/stl_uninitialized.h:289
 #14 std::__uninitialized_move_if_noexcept_a<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > (__alloc=..., __result=<optimized out>, __last=0x55e995662850,
     __first=<optimized out>) at /usr/include/c++/7/bits/stl_uninitialized.h:312
 #15 std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >::_M_realloc_insert<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&> (
     this=this@entry=0x55e9937566f0 <FileUtil::getTempFilePath(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::fileDeleter>,
     __position=<error: Cannot access memory at address 0x3735003238323a>, __args#0="/tmp/loadTorture_viewcursor.odp_725a6013_viewcursor.odp") at /usr/include/c++/7/bits/vector.tcc:424
 #16 0x000055e9934aa829 in std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >::push_back (__x="/tmp/loadTorture_viewcursor.odp_725a6013_viewcursor.odp",
     this=0x55e9937566f0 <FileUtil::getTempFilePath(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::fileDeleter>) at /usr/include/c++/7/bits/stl_vector.h:948
 #17 (anonymous namespace)::FileDeleter::registerForDeletion (file="/tmp/loadTorture_viewcursor.odp_725a6013_viewcursor.odp",
     this=0x55e9937566f0 <FileUtil::getTempFilePath(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::fileDeleter>) at common/FileUtil.cpp:69
 #18 FileUtil::getTempFilePath (srcDir=..., srcFilename=..., dstFilenamePrefix=...) at common/FileUtil.cpp:97
 #19 0x00007fda216b51c9 in helpers::getDocumentPathAndURL (docFilename="viewcursor.odp", documentPath="", documentURL="", prefix="loadTorture_viewcursor.odp_") at ./helpers.hpp:151
 #20 0x00007fda216ad007 in UnitLoadTorture::loadTorture (this=<optimized out>, testname="loadTorture_viewcursor.odp ", docName="viewcursor.odp", thread_count=thread_count@entry=3,
     max_jitter_ms=<optimized out>, max_jitter_ms@entry=75) at UnitLoadTorture.cpp:41
 #21 0x00007fda216ad7dd in UnitLoadTorture::<lambda()>::operator() (__closure=0x55e994dbc758) at UnitLoadTorture.cpp:186
 #22 std::__invoke_impl<void, UnitLoadTorture::testLoadTorture()::<lambda()> > (__f=...) at /usr/include/c++/7/bits/invoke.h:60
 #23 std::__invoke<UnitLoadTorture::testLoadTorture()::<lambda()> > (__fn=...) at /usr/include/c++/7/bits/invoke.h:95
 #24 std:🧵:_Invoker<std::tuple<UnitLoadTorture::testLoadTorture()::<lambda()> > >::_M_invoke<0> (this=0x55e994dbc758) at /usr/include/c++/7/thread:234
 #25 std:🧵:_Invoker<std::tuple<UnitLoadTorture::testLoadTorture()::<lambda()> > >::operator() (this=0x55e994dbc758) at /usr/include/c++/7/thread:243
 #26 std:🧵:_State_impl<std:🧵:_Invoker<std::tuple<UnitLoadTorture::testLoadTorture()::<lambda()> > > >::_M_run(void) (this=0x55e994dbc750) at /usr/include/c++/7/thread:186
 #27 0x00007fda22de366f in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
 #28 0x00007fda228f66db in start_thread (arg=0x7fda197fa700) at pthread_create.c:463
 #29 0x00007fda2261f88f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

 Thread 6 (Thread 0x7fd9fffff700 (LWP 56362)):
  #0  __lll_lock_wait_private () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:95
 #1  0x00007fda225952db in __GI___libc_malloc (bytes=bytes@entry=3635) at malloc.c:3063
 #2  0x00007fda2262f364 in __backtrace_symbols (array=array@entry=0x7fd9ffffc540, size=size@entry=26) at backtracesyms.c:69
 #3  0x000055e9934cc439 in SigUtil::dumpBacktrace () at common/SigUtil.cpp:253
 #4  0x000055e9934cd5ae in SigUtil::handleFatalSignal (signal=6) at common/SigUtil.cpp:236
 #5  <signal handler called>
 #6  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
 #7  0x00007fda2253e801 in __GI_abort () at abort.c:79
 #8  0x00007fda22587897 in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7fda226b4b9a "%s
") at ../sysdeps/posix/libc_fatal.c:181
 #9  0x00007fda2258e90a in malloc_printerr (str=str@entry=0x7fda226b2e0e "malloc(): memory corruption") at malloc.c:5350
 #10 0x00007fda22592994 in _int_malloc (av=av@entry=0x7fda00000020, bytes=bytes@entry=32) at malloc.c:3738
 #11 0x00007fda225952ed in __GI___libc_malloc (bytes=32) at malloc.c:3065
 #12 0x00007fda22db9258 in operator new(unsigned long) () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
 #13 0x00007fda244adaae in __gnu_cxx::new_allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::allocate (this=0x7fd9ffffe740, __n=1)
     at /usr/include/c++/5/ext/new_allocator.h:104
 #14 0x00007fda244ad300 in __gnu_cxx::__alloc_traits<std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >::allocate (__a=..., __n=1)
     at /usr/include/c++/5/ext/alloc_traits.h:182
 #15 0x00007fda244ac858 in std::_Vector_base<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >::_M_allocate (this=0x7fd9ffffe740, __n=1) at /usr/include/c++/5/bits/stl_vector.h:170
 #16 0x00007fda244ab7c7 in std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >::_M_insert_aux (this=0x7fd9ffffe740, __position=non-dereferenceable iterator for std::vector, __x="home") at /usr/include/c++/5/bits/vector.tcc:353
 #17 0x00007fda244aada2 in std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >::push_back (this=0x7fd9ffffe740, __x="home") at /usr/include/c++/5/bits/stl_vector.h:925
 #18 0x00007fda2350ddcb in Poco::Path::pushDirectory (this=0x7fd9ffffe6c0, dir="home") at src/Path.cpp:471
 #19 0x00007fda2350ea27 in Poco::Path::parseUnix (this=0x7fd9ffffe6c0, path="/home/ash/prj/lo/online/test/") at src/Path.cpp:671
 #20 0x00007fda2350cd6b in Poco::Path::assign (this=0x7fd9ffffe6c0, path="/home/ash/prj/lo/online/test/") at src/Path.cpp:182
 #21 0x00007fda2350c1be in Poco::Path::Path (this=0x7fd9ffffe6c0, path="/home/ash/prj/lo/online/test/") at src/Path.cpp:54
 #22 0x00007fda2350d33a in Poco::Path::makeAbsolute (this=0x7fd9ffffe8f0) at src/Path.cpp:318
 #23 0x00007fda216b52ca in helpers::getDocumentPathAndURL (docFilename="setclientpart.ods", documentPath="/tmp/loadTorture_setclientpart.ods_ae70d2e_setclientpart.ods", documentURL="",
     prefix="loadTorture_setclientpart.ods_") at ./helpers.hpp:153
 #24 0x00007fda216ad007 in UnitLoadTorture::loadTorture (this=<optimized out>, testname="loadTorture_setclientpart.ods ", docName="setclientpart.ods", thread_count=thread_count@entry=3,
     max_jitter_ms=<optimized out>, max_jitter_ms@entry=75) at UnitLoadTorture.cpp:41
 #25 0x00007fda216ad7dd in UnitLoadTorture::<lambda()>::operator() (__closure=0x55e994cffe18) at UnitLoadTorture.cpp:186
 #26 std::__invoke_impl<void, UnitLoadTorture::testLoadTorture()::<lambda()> > (__f=...) at /usr/include/c++/7/bits/invoke.h:60
 #27 std::__invoke<UnitLoadTorture::testLoadTorture()::<lambda()> > (__fn=...) at /usr/include/c++/7/bits/invoke.h:95
 #28 std:🧵:_Invoker<std::tuple<UnitLoadTorture::testLoadTorture()::<lambda()> > >::_M_invoke<0> (this=0x55e994cffe18) at /usr/include/c++/7/thread:234
 #29 std:🧵:_Invoker<std::tuple<UnitLoadTorture::testLoadTorture()::<lambda()> > >::operator() (this=0x55e994cffe18) at /usr/include/c++/7/thread:243
 #30 std:🧵:_State_impl<std:🧵:_Invoker<std::tuple<UnitLoadTorture::testLoadTorture()::<lambda()> > > >::_M_run(void) (this=0x55e994cffe10) at /usr/include/c++/7/thread:186
 #31 0x00007fda22de366f in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
 #32 0x00007fda228f66db in start_thread (arg=0x7fd9fffff700) at pthread_create.c:463
 #33 0x00007fda2261f88f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Change-Id: Ib60954e49e692082e686cb3fdc5fc9d5cfbc83c9
Reviewed-on: https://gerrit.libreoffice.org/83047
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Michael Meeks <michael.meeks@collabora.com>
2019-11-18 06:47:20 +01:00
Jan Holesovsky
07d5f8cf2a killpoco: Remove POCO usage from FileUtil.cpp when C++17 is available.
This introduces basic C++17 support, because the functionality needed
here is easy to implement using std::filesystem.

Adds also the necessary checks to ./configure.  The code still uses POCO
when C++17 is not available in the compiler.

Change-Id: I03353834d10201bf0a13ea72715560b9b9b16265
Reviewed-on: https://gerrit.libreoffice.org/82294
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
Tested-by: Jan Holesovsky <kendy@collabora.com>
2019-11-08 13:56:09 +01:00
Samuel Mehrbrodt
6e87ebc91e 1GB should be enough for development
Change-Id: I7567f99c4cf54a25084e25b44887b7906bbc7b06
Reviewed-on: https://gerrit.libreoffice.org/72827
Reviewed-by: Samuel Mehrbrodt <Samuel.Mehrbrodt@cib.de>
Tested-by: Samuel Mehrbrodt <Samuel.Mehrbrodt@cib.de>
2019-05-23 12:38:31 +02:00
Jan Holesovsky
f76b36193d android: #if(n)def MOBILEAPP -> #if (!)MOBILEAPP for better reliability.
Change-Id: I5f9c9420b6c83601db1c8fdba4ae5a10b17b2107
2019-02-12 12:20:11 +01:00
Miklos Vajna
b1f06d1c0a FileUtil: make fs members private
Change-Id: I921118c42a20117725074169f6425e9ff87bc54b
2018-12-10 08:55:39 +01:00
Tor Lillqvist
95eb849217 Still more iOS app and related Online C++ code hacking
Re-think the plumbing between the different parts of the C++ Online
code. Do try to have it work more like in real Online on all but the
lowest socket level. Except that we don't have multiple processes, but
threads inside the same process. And instead of using actual system
sockets for WebSocket traffic between the threads, we use our own
FakeSocket things, with no WebSocket framing of messages.

Reduce the amount of #ifdef MOBILEAPP a bit also by compiling in the
UnitFoo things. Hardcode that so that no unit testing is ever
attempted, though. We don't try to dlopen any library.

Corresponding changes in the app Objective-C code. Plus fixes and
functionality improvements.

Now it gets so far that the JavaScript code thinks it has the document
tiles presented, and doesn't crash. But it hangs occasionally. And all
tiles show up blank.

Anyway, progress.

Change-Id: I769497c9a46ddb74984bc7af36d132b7b43895d4
2018-09-19 11:31:18 +03:00
Tor Lillqvist
75438baa70 More mobile app stuff, very much early state of work in progress
Re-think Linux vs mobile ifdefs a bit. Use #ifdef __linux only to
surround code that actually is Linux-specific. Use #ifdef MOBILEAPP
for code that is for a mobile version (with no separste wsd, forkit,
and kit processes, and with no WebSocket protocol used).

Bypass UnitFoo for mobile. Possibly we do want the UnitFoo stuff after
all on mobile, to run in some special testing mode? Hard to say, let's
skipt it for now.
2018-09-10 15:13:43 +03:00
Tor Lillqvist
8209d6cd0e Make this file compile for iOS 2018-08-29 20:38:40 +03:00
Ashod Nakashian
3d6e373992 wsd: reduce lock scope and log diskspace checks
Change-Id: If093670ae83de5596a86a116ba6224aa0badbcbe
Reviewed-on: https://gerrit.libreoffice.org/52415
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Michael Meeks <michael.meeks@collabora.com>
2018-06-11 22:15:57 +02:00
Miklos Vajna
9eacfe4fcb common: spell out non-trivial autos to improve readability
Change-Id: Id13bc0e48cec845f5b05171128be5b4efc05c6bc
2018-02-07 10:18:12 +01:00
Pranav Kant
f63858433b loplugin:includeform
Change-Id: Ib62a7aa61062f00698aa3e8a144438de5c57e53d
2017-12-20 21:21:05 +05:30
Michael Meeks
2bca560feb Add hook for disk space check. 2017-03-31 17:18:41 +01:00
Michael Meeks
0eaef6c896 config.h - get includes right: must always be the first include. 2017-03-10 10:47:43 +01:00
Jan Holesovsky
7eff2294d9 We should be able to run with 5G of free space just fine.
Change-Id: Ie5df96388eeaa0ddf1a41564ec7d52c283e6b1bb
2017-02-21 19:55:04 +01:00
Ashod Nakashian
0d4d506ea3 wsd: faster jail directory cleanup
Around 1.5x faster than Poco,
which first enumerates files into
a container, then iterates over
them and stats before unlinking.

Here we enumerate and unlink in
a single pass.

Change-Id: I79d1c0f1b5ec557ccc4f0e2ec7a0609051d8d212
Reviewed-on: https://gerrit.libreoffice.org/33680
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2017-01-30 05:10:09 +00:00
Ashod Nakashian
4580ed4e27 wsd: prefix the test docs with the test name
This makes debugging much easier as one can
readily match WSD logs with a given test.

Change-Id: I8f2c83d67189038699af3f24dee205bc7efb5c28
Reviewed-on: https://gerrit.libreoffice.org/32860
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2017-01-09 06:17:43 +00:00
Pranav Kant
62a666c06a wsd: better handling of low storage situation
There is no way to let the user of document currently being
opened, in case of failure, know that disk is low on space.
We check the disk space when forking children after which we try
to alert all users but this would end up doing nothing for
current document because document broker is not registered at
this time (we iterate through doc brokers when alerting). Another
conditional disk check is performed just before opening the
document but this is performed only if last disk check was
performed greater than 60 seconds which would never be the case
because document open is always preceded by a child fork (when
rebalancing children).

Lets not cache the disk check when forking the children to
prevent above mentioned situation while still minimizing the
number of disk checks performed.

Change-Id: Id3add998f94e23f9f8c144f09e5efe9f0b63821c
2017-01-04 14:21:02 +05:30
Noel Grandin
f53669cf9d loplugin:unreffun
Change-Id: I06b13a5c035f6ae4dd4b414f49a0987c019feea5
Reviewed-on: https://gerrit.libreoffice.org/32365
Reviewed-by: Tor Lillqvist <tml@collabora.com>
Tested-by: Tor Lillqvist <tml@collabora.com>
2016-12-27 10:26:23 +00:00
Ashod Nakashian
1376856107 wsd: logging updates
Change-Id: Icbee0349a3cfda5a56f8d681c779484e18b98ab6
Reviewed-on: https://gerrit.libreoffice.org/32287
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
2016-12-21 15:01:20 +00:00
Michael Meeks
cca657c8f2 Apply the pre-branch rename script to re-organize the source. 2016-11-25 09:58:48 +00:00
Renamed from loolwsd/common/FileUtil.cpp (Browse further)