office-gobmx/pyuno
Stephan Bergmann b3b18ed165 Don't make UNO calls from PyThreadAttached regions (i.e., with the GIL locked)
...which caused e.g. UITest_writer_tests3 to occasionally hang with the
python.bin process thread 1 at

>   File "sw/qa/uitest/writer_tests3/insertPageFooter.py", line 30, in delete_footer
[...]

i.e.,

> #4  ___pthread_cond_wait at /usr/src/debug/glibc-2.35-4.fc36.x86_64/nptl/pthread_cond_wait.c:618
> #5  0x00007ff9d6ad9c43 in cppu_threadpool::JobQueue::enter(void const*, bool) at cppu/source/threadpool/jobqueue.cxx:72
> #6  0x00007ff9d6ae8e79 in cppu_threadpool::ThreadPool::enter(rtl::ByteSequence const&, void const*) at cppu/source/threadpool/threadpool.cxx:303
> #7  0x00007ff9d6ae9278 in uno_threadpool_enter(uno_ThreadPool, void**) at cppu/source/threadpool/threadpool.cxx:407
> #8  0x00007ff9c7d016eb in binaryurp::Bridge::makeCall(rtl::OUString const&, com::sun::uno::TypeDescription const&, bool, std::__debug::vector<binaryurp::BinaryAny, std::allocator<binaryurp::BinaryAny> >&&, binaryurp::BinaryAny*, std::__debug::vector<binaryurp::BinaryAny, std::allocator<binaryurp::BinaryAny> >*) at binaryurp/source/bridge.cxx:604
> #9  0x00007ff9c7d322fc in binaryurp::Proxy::do_dispatch_throw(_typelib_TypeDescription const*, void*, void**, _uno_Any**) const at binaryurp/source/proxy.cxx:168
> #10 0x00007ff9c7d338cb in binaryurp::Proxy::do_dispatch(_typelib_TypeDescription const*, void*, void**, _uno_Any**) const at binaryurp/source/proxy.cxx:101
> #11 0x00007ff9d63222e0 in cpp2uno_call(bridges::cpp_uno::shared::CppInterfaceProxy*, typelib_TypeDescription const*, typelib_TypeDescriptionReference*, sal_Int32, typelib_MethodParameter*, void**, void**, void**, sal_uInt64*) at bridges/source/cpp_uno/gcc3_linux_x86-64/cpp2uno.cxx:191
> #12 0x00007ff9d6322b59 in cpp_vtable_call(sal_Int32, sal_Int32, void**, void**, void**, sal_uInt64*) at bridges/source/cpp_uno/gcc3_linux_x86-64/cpp2uno.cxx:389
> #13 0x00007ff9d63336ca in privateSnippetExecutor at instdir/program/libgcc3_uno.so
> #14 0x00007ff9d6241f38 in (anonymous namespace)::ImplIntrospectionAccess::hasByName(rtl::OUString const&) at stoc/source/inspect/introspection.cxx:1114
> #15 0x00007ff9d6b856b2 in pyuno::PyUNO_getattr(PyObject*, char*) at pyuno/source/module/pyuno.cxx:1392
[...]

doing a remote call to soffice.bin with GIL locked, and thread 4 at

> #6  0x00007ff9e51efa28 in take_gil at workdir/UnpackedTarball/python3/Python/ceval_gil.h:207
> #7  0x00007ff9e51f00d3 in PyEval_AcquireThread at workdir/UnpackedTarball/python3/Python/ceval.c:316
> #8  0x00007ff9d6b9b1e8 in pyuno::Adapter::invoke(rtl::OUString const&, com::sun::uno::Sequence<com::sun::uno::Any> const&, com::sun::uno::Sequence<short>&, com::sun::uno::Sequence<com::sun::uno::Any>&) at pyuno/source/module/pyuno_adapter.cxx:181
> #9  0x00007ff9d632b3e4 in gcc3::callVirtualMethod(void*, unsigned int, void*, _typelib_TypeDescriptionReference*, bool, unsigned long*, unsigned int, unsigned long*, double*) at bridges/source/cpp_uno/gcc3_linux_x86-64/callvirtualmethod.cxx:77
> #10 0x00007ff9d632a4f1 in cpp_call(bridges::cpp_uno::shared::UnoInterfaceProxy*, bridges::cpp_uno::shared::VtableSlot, typelib_TypeDescriptionReference*, sal_Int32, typelib_MethodParameter*, void*, void**, uno_Any**) at bridges/source/cpp_uno/gcc3_linux_x86-64/uno2cpp.cxx:233
> #11 0x00007ff9d632a9e6 in bridges::cpp_uno::shared::unoInterfaceProxyDispatch(uno_Interface*, typelib_TypeDescription const*, void*, void**, uno_Any**) at bridges/source/cpp_uno/gcc3_linux_x86-64/uno2cpp.cxx:413
> #12 0x00007ff9d621384a in stoc_invadp::(anonymous namespace)::AdapterImpl::invoke at stoc/source/invocation_adapterfactory/iafactory.cxx:457
> #13 stoc_invadp::adapter_dispatch(uno_Interface*, typelib_TypeDescription const*, void*, void**, uno_Any**) at stoc/source/invocation_adapterfactory/iafactory.cxx:605
> #14 0x00007ff9c7d1a7db in binaryurp::IncomingRequest::execute_throw(binaryurp::BinaryAny*, std::__debug::vector<binaryurp::BinaryAny, std::allocator<binaryurp::BinaryAny> >*) const at include/typelib/typedescription.hxx:155
> #15 0x00007ff9c7d1d65f in binaryurp::IncomingRequest::execute() const at binaryurp/source/incomingrequest.cxx:78
> #16 0x00007ff9c7d33ad3 in binaryurp::(anonymous namespace)::request(void*) at ~/gcc/trunk/inst/include/c++/12.0.1/bits/unique_ptr.h:172
> #17 0x00007ff9d6ad9e1b in cppu_threadpool::JobQueue::enter(void const*, bool) at cppu/source/threadpool/jobqueue.cxx:100
> #18 0x00007ff9d6adbb80 in cppu_threadpool::ORequestThread::run() at cppu/source/threadpool/thread.cxx:164
[...]

processing an unrelated incoming call from soffice.bin and blocked trying to
lock the GIL.  (The reason that thread 1 doesn't make progress is that the
soffice.bin process is also blocking with thread 6 at

> #8  osl::Guard<comphelper::SolarMutex>::Guard(comphelper::SolarMutex&) at include/osl/mutex.hxx:142
> #9  SolarMutexGuard::SolarMutexGuard() at include/vcl/svapp.hxx:1368
> #10 sw::(anonymous namespace)::XStyleFamily::hasByName(rtl::OUString const&) at sw/source/core/unocore/unostyle.cxx:923
> #11 0x00007f089d3d53e4 in gcc3::callVirtualMethod(void*, unsigned int, void*, _typelib_TypeDescriptionReference*, bool, unsigned long*, unsigned int, unsigned long*, double*) at bridges/source/cpp_uno/gcc3_linux_x86-64/callvirtualmethod.cxx:77
> #12 0x00007f089d3d44f1 in cpp_call(bridges::cpp_uno::shared::UnoInterfaceProxy*, bridges::cpp_uno::shared::VtableSlot, typelib_TypeDescriptionReference*, sal_Int32, typelib_MethodParameter*, void*, void**, uno_Any**) at bridges/source/cpp_uno/gcc3_linux_x86-64/uno2cpp.cxx:233
> #13 0x00007f089d3d49e6 in bridges::cpp_uno::shared::unoInterfaceProxyDispatch(uno_Interface*, typelib_TypeDescription const*, void*, void**, uno_Any**) at bridges/source/cpp_uno/gcc3_linux_x86-64/uno2cpp.cxx:413
> #14 0x00007f089c1be7db in binaryurp::IncomingRequest::execute_throw(binaryurp::BinaryAny*, std::__debug::vector<binaryurp::BinaryAny, std::allocator<binaryurp::BinaryAny> >*) const at include/typelib/typedescription.hxx:155
> #15 0x00007f089c1c165f in binaryurp::IncomingRequest::execute() const at binaryurp/source/incomingrequest.cxx:78
> #16 0x00007f089c1d7ad3 in binaryurp::(anonymous namespace)::request(void*) at ~/gcc/trunk/inst/include/c++/12.0.1/bits/unique_ptr.h:172
> #17 0x00007f08a18ade1b in cppu_threadpool::JobQueue::enter(void const*, bool) at cppu/source/threadpool/jobqueue.cxx:100
> #18 0x00007f08a18afb80 in cppu_threadpool::ORequestThread::run() at cppu/source/threadpool/thread.cxx:164
[...]

processing the incoming call from python.bin thread 1 and blocked trying to lock
the SolarMutex, while thread 1 at

> #4  ___pthread_cond_wait at /usr/src/debug/glibc-2.35-4.fc36.x86_64/nptl/pthread_cond_wait.c:618
> #5  0x00007f08a18adc43 in cppu_threadpool::JobQueue::enter(void const*, bool) at cppu/source/threadpool/jobqueue.cxx:72
> #6  0x00007f08a18bce79 in cppu_threadpool::ThreadPool::enter(rtl::ByteSequence const&, void const*) at cppu/source/threadpool/threadpool.cxx:303
> #7  0x00007f08a18bd278 in uno_threadpool_enter(uno_ThreadPool, void**) at cppu/source/threadpool/threadpool.cxx:407
> #8  0x00007f089c1a56eb in binaryurp::Bridge::makeCall(rtl::OUString const&, com::sun::uno::TypeDescription const&, bool, std::__debug::vector<binaryurp::BinaryAny, std::allocator<binaryurp::BinaryAny> >&&, binaryurp::BinaryAny*, std::__debug::vector<binaryurp::BinaryAny, std::allocator<binaryurp::BinaryAny> >*) at binaryurp/source/bridge.cxx:604
> #9  0x00007f089c1d62fc in binaryurp::Proxy::do_dispatch_throw(_typelib_TypeDescription const*, void*, void**, _uno_Any**) const at binaryurp/source/proxy.cxx:168
> #10 0x00007f089c1d78cb in binaryurp::Proxy::do_dispatch(_typelib_TypeDescription const*, void*, void**, _uno_Any**) const at binaryurp/source/proxy.cxx:101
> #11 0x00007f089d3cc2e0 in cpp2uno_call(bridges::cpp_uno::shared::CppInterfaceProxy*, typelib_TypeDescription const*, typelib_TypeDescriptionReference*, sal_Int32, typelib_MethodParameter*, void**, void**, void**, sal_uInt64*) at bridges/source/cpp_uno/gcc3_linux_x86-64/cpp2uno.cxx:191
> #12 0x00007f089d3ccb59 in cpp_vtable_call(sal_Int32, sal_Int32, void**, void**, void**, sal_uInt64*) at bridges/source/cpp_uno/gcc3_linux_x86-64/cpp2uno.cxx:389
> #13 0x00007f089d3dd6ca in privateSnippetExecutor at instdir/program/libgcc3_uno.so
> #14 0x00007f08a6b5d170 in operator() at sfx2/source/notify/globalevents.cxx:491
> #15 comphelper::OInterfaceContainerHelper4<com::sun::document::XDocumentEventListener>::forEach<(anonymous namespace)::SfxGlobalEvents_Impl::implts_notifyListener(const com::sun::document::DocumentEvent&)::<lambda(const com::sun::uno::Reference<com::sun::document::XDocumentEventListener>&)> > at include/comphelper/interfacecontainer4.hxx:285
> #16 at sfx2/source/notify/globalevents.cxx:488
> #17 0x00007f08a6b04c8b in (anonymous namespace)::NotifySingleListenerIgnoreRE<com::sun::document::XDocumentEventListener, com::sun::document::DocumentEvent>::operator() (listener=Python Exception <class 'gdb.error'>: Dwarf Error: Cannot find DIE at 0x0 referenced in module instdir/program/libmergedlo.so
> #18 comphelper::OInterfaceContainerHelper2::forEach<com::sun::document::XDocumentEventListener, at include/comphelper/interfacecontainer2.hxx:271
> #19 SfxBaseModel::postEvent_Impl(rtl::OUString const&, com::sun::uno::Reference<com::sun::frame::XController2> const&, com::sun::uno::Any const&) at sfx2/source/doc/sfxbasemodel.cxx:3253
> #20 0x00007f08a6b066fe in SfxBaseModel::Notify(SfxBroadcaster&, SfxHint const&) at sfx2/source/doc/sfxbasemodel.cxx:2902
> #21 0x00007f08a6f15def in SfxBroadcaster::Broadcast(SfxHint const&) at svl/source/notify/SfxBroadcaster.cxx:39
> #22 0x00007f087d640650 in SwLayIdle::SwLayIdle(SwRootFrame*, SwViewShellImp*) at sw/source/core/layout/layact.cxx:2389
> #23 0x00007f087dc0832b in SwViewShell::LayoutIdle() at sw/source/core/view/viewsh.cxx:821
> #24 0x00007f087d2ba194 in sw::DocumentTimerManager::DoIdleJobs(Timer*) at sw/source/core/inc/rootfrm.hxx:206
> #25 0x00007f08a89fbd26 in Scheduler::CallbackTaskScheduling() at vcl/source/app/scheduler.cxx:472
> #26 0x00007f08a8ca8780 in SalTimer::CallCallback() at vcl/inc/saltimer.hxx:54
> #27 SvpSalInstance::CheckTimeout(bool) at vcl/headless/svpinst.cxx:212
> #28 0x00007f08a8ca99bd in SvpSalInstance::ImplYield(bool, bool) at vcl/headless/svpinst.cxx:452
> #29 0x00007f08a8ca9db1 in SvpSalInstance::DoYield(bool, bool) at vcl/headless/svpinst.cxx:524
> #30 0x00007f08a8a27d4c in ImplYield(bool, bool) at vcl/source/app/svapp.cxx:474
> #31 0x00007f08a8a28445 in Application::Execute() at vcl/source/app/svapp.cxx:452
[...]

is doing the remote call to python.bin thread 4 with the SolarMutex erroneously
locked.  But lets concentrate on just fixing the python.bin deadlock in this
commit.)

The PyThreadDetach antiguard just covering the xInvocation->getValue call in the
"or a property" case had been added with
e0a1cd4dc7 "fixed a deadlock", presumably without
taking into account that the xInvocation->hasMethod/hasProperty calls are
equally problematic.

Change-Id: I2483949df8213c3397a674be190224ee58c95c02
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/132428
Tested-by: Jenkins
Reviewed-by: Stephan Bergmann <sbergman@redhat.com>
2022-04-01 16:56:58 +02:00
..
demo
doc
inc
qa/pytests
source
zipcore
CustomTarget_python_shell.mk
CustomTarget_pyuno_pythonloader_ini.mk
Executable_python.mk
IwyuFilter_pyuno.yaml
Library_pythonloader.mk
Library_pyuno.mk
Library_pyuno_wrapper.mk
Makefile
Module_pyuno.mk
Package_python_scripts.mk
Package_python_shell.mk
Package_pyuno_pythonloader_ini.mk
PythonTest_pyuno_pytests_insertremovecells.mk
PythonTest_pyuno_pytests_testcollections.mk
PythonTest_pyuno_pytests_testssl.mk
Rdb_pyuno.mk
README.md

Python UNO Bindings

UNO bindings for the Python programming language.

To have much joy debugging Python extensions you need to:

  • a) edit pythonloader.py in your install setting DEBUG=1 at the top
  • b) touch pyuno/source/module/pyuno_runtime.cxx and make debug=true in pyuno

Then you'll start to see your exceptions on the console instead of them getting lost at the UNO interface.

Python also comes with a gdb script libpython$(PYTHON_VERSION_MAJOR).$(PYTHON_VERSION_MINOR)m.so.1.0-gdb.py that is copied to instdir and will be auto-loaded by gdb; it provides commands like py-bt to get a Python-level backtrace, and py-print to print Python variables.

Another way to debug Python code is to use pdb: edit some initialization function to insert import pdb; pdb.set_trace() (somewhere so that it is executed early), then run soffice from a terminal and a command-line Python debugger will appear where you can set Python-level breakpoints.