office-gobmx/vcl/CppunitTest_vcl_gtk3_a11y.mk
Michael Weghorn ca01deadcb gtk3 a11y test: Let xvfb-run auto-determine free server num
As the xvfb-run manpage says, it uses a default server number of 99:

>      -n servernumber, --server-num=servernumber
>             Use servernumber as the server number (but see the -a, --auto-servernum option above).  The default is 99.

The gtk3 a11y tests use xvfb-run. Running the test multiple
times in parallel (e.g. when doing two separate builds with
tests, as happens on CI) would fall like this on Debian testing:

    $ make CppunitTest_vcl_gtk3_a11y
    make -j 12 -rs -f /home/michi/development/git/libreoffice/Makefile.gbuild CppunitTest_vcl_gtk3_a11y
    [CUT] vcl_gtk3_a11y
    xvfb-run: error: Xvfb failed to start
    Hint: You are currently not seeing messages from other users and the system.
          Users in groups 'adm', 'systemd-journal' can see all messages.
          Pass -q to turn off this notice.
    No coredumps found.

    Error: a unit test failed, please do one of:

    make CppunitTest_vcl_gtk3_a11y CPPUNITTRACE="gdb --args"
        # for interactive debugging on Linux
    make CppunitTest_vcl_gtk3_a11y VALGRIND=memcheck
        # for memory checking
    make CppunitTest_vcl_gtk3_a11y DEBUGCPPUNIT=TRUE
        # for exception catching

    You can limit the execution to just one particular test by:

    make CPPUNIT_TEST_NAME="testXYZ" ...above mentioned params...

However, it behaves differently on Alma Linux 8, where it
looks like xvfb-run seems then to reuse the existing
Xfvb session, causing the tests to potentially find
the LibreOffice instance from another test run, and
then inspecting the wrong one, leading to failures
like the one in [1]:

    equality assertion failed
    - Expected: ecclectic.fodt — LibreOfficeDev Writer
    - Actual  : ecclectic.fodt — LibreOfficeDev Writer 24.2 [9bc2aede99]

The fact that the second instance of xvfb-run
reuses the session of the firs one and finds the
LibreOffice there can also be reproduced on Alma Linux 8
by just running these 2 commands in parallel:

1) start LO in xfvb-run session:

    $ xvfb-run dbus-launch libreoffice

2) run below Python script that lists the
   running applications:

    $ xvfb-run dbus-launch python3 /home/vagrant/atspi-list-apps.py
    --------start--------
    [application | soffice]
    --------end--------
    /usr/bin/xvfb-run: line 186: kill: (27078) - No such process

This *should not* list the applications from
the other xfvb-run.

It works as expected when passing `--auto-servernum`:

    $ xvfb-run --auto-servernum dbus-launch python3 /home/vagrant/atspi-list-apps.py
    --------start--------
    --------end--------

Therefore, pass the `--auto-servernum` arg to xvfb-run, so that it
automatically determines a free server number instead.

>      -a, --auto-servernum
>             Try to get a free server number, starting at 99, or the argument to --server-num.

Python script mentioned above used for testing:

    $ cat atspi-list-apps.py
    #!/usr/bin/python3

    import pyatspi

    registry = pyatspi.registry.Registry()
    apps = list(registry.getDesktop(0))

    print('--------start--------')

    for app in apps:
        print(f'{app}')

    print('--------end--------')

[1] https://ci.libreoffice.org/job/gerrit_linux_gcc_release/151750/consoleFull#-1985341263d893063f-7f3d-4b7e-b56f-4e0f225817cd

Change-Id: I0673212fb8fed5c9698c9f797b7bf49ba51033b0
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/158021
Tested-by: Jenkins
Reviewed-by: Colomban Wendling <cwendling@hypra.fr>
Reviewed-by: Michael Weghorn <m.weghorn@posteo.de>
2023-10-17 07:50:37 +02:00

63 lines
1.9 KiB
Makefile

# -*- Mode: makefile-gmake; tab-width: 4; indent-tabs-mode: t -*-
#
# This file is part of the LibreOffice project.
#
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
#
# hack plugging into the CppunitTest machinery yet using a xvfb-wrapper GTK3 run
$(call gb_CppunitTest_get_target,vcl_gtk3_a11y) : gb_TEST_ENV_VARS += SAL_USE_VCLPLUGIN=gtk3
# force running with the X11 Gdk backend also when running on Wayland
$(call gb_CppunitTest_get_target,vcl_gtk3_a11y) : gb_TEST_ENV_VARS += GDK_BACKEND=x11
ifeq (,$(VCL_GTK3_TESTS_NO_XVFB))
$(call gb_CppunitTest_get_target,vcl_gtk3_a11y) : \
ICECREAM_RUN += $(XVFB_RUN) --auto-servernum $(DBUS_LAUNCH) --exit-with-session
endif
$(eval $(call gb_CppunitTest_CppunitTest,vcl_gtk3_a11y))
$(eval $(call gb_CppunitTest_add_exception_objects,vcl_gtk3_a11y, \
vcl/qa/cppunit/a11y/atspi2/atspiwrapper \
vcl/qa/cppunit/a11y/atspi2/atspi2 \
vcl/qa/cppunit/a11y/atspi2/atspi2text \
))
$(eval $(call gb_CppunitTest_set_include,vcl_gtk3_a11y,\
$$(INCLUDE) \
$$(ATSPI2_CFLAGS) \
))
$(eval $(call gb_CppunitTest_add_libs,vcl_gtk3_a11y,\
$$(ATSPI2_LIBS) \
))
$(eval $(call gb_CppunitTest_use_libraries,vcl_gtk3_a11y, \
sal \
cppu \
subsequenttest \
test \
i18nlangtag \
unotest \
vcl \
))
$(eval $(call gb_CppunitTest_use_externals,vcl_gtk3_a11y,\
boost_headers \
))
$(eval $(call gb_CppunitTest_use_api,vcl_gtk3_a11y,\
offapi \
udkapi \
))
$(eval $(call gb_CppunitTest_use_sdk_api,vcl_gtk3_a11y))
$(eval $(call gb_CppunitTest_use_rdb,vcl_gtk3_a11y,services))
$(eval $(call gb_CppunitTest_use_ure,vcl_gtk3_a11y))
$(eval $(call gb_CppunitTest_use_vcl,vcl_gtk3_a11y))
$(eval $(call gb_CppunitTest_use_instdir_configuration,vcl_gtk3_a11y))
$(eval $(call gb_CppunitTest_use_common_configuration,vcl_gtk3_a11y))
# vim: set noet sw=4 ts=4: