Commit graph

59 commits

Author SHA1 Message Date
Tamás Zolnai
96819f81cb cypress: generate test names dynamically for the parallel build.
Also improve afterAll(). Having test files with the same suffix
led to test failure. We can have better result with a regex.

Change-Id: I8d678aa17eb0e28e643cc86f584aab713ca153ca
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92157
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
2020-04-14 16:22:35 +02:00
Tamás Zolnai
b394c5a8f7 cypress: introduce a second chance for the tests to pass.
In parallel build try to run the specific test files
for a second time if they fail. Let's be realist,
testing an online application always has some
unreliabilities, let's make things more reliable with
running test more times and abort the build only if
the test fails consistently.

This is enabled only for parallel builds now, so we
have it on Jenkins. A simple make check will be more
deterministic. I also added an option to disable second
chance for parallel build: see CYPRESS_NO_SECOND_CHANCE
make file variable.

Change-Id: Ib7c554a238ded7ee15908b9f0c2d2c92ef0dcf32
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/91987
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
2020-04-14 11:38:44 +02:00
Tamás Zolnai
56c6f5cc48 cypress: mobile: add some tests for alignment properties in Calc.
Change-Id: I6a0209a484041a3df44e5472a4b0679193dd38c1
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/91912
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
2020-04-08 14:30:13 +02:00
Tamás Zolnai
bdef8193b8 cypress: simplify output error matcher.
Change-Id: I18392d8aabd9f3f4ba43139f9e951faead8b9667
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/91876
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
2020-04-08 11:24:38 +02:00
Tamás Zolnai
efdc8b25b4 cypress: run JS error check also for make run commands.
Change-Id: I0a00d64e2222cbad5fc7e89711a89bb3620f793f
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/91875
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
2020-04-08 11:22:20 +02:00
Tamás Zolnai
aeb48c3ad5 cypress: Catch any "Error:" in log.
Change-Id: Ia82f896e7ff4299e25ac38283f0e518bef25f875
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/91808
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
2020-04-07 12:40:16 +02:00
Tamás Zolnai
d41348e841 cypress: add the issue link about hash values.
Change-Id: Ied7d05a2b5d112c222076d5799e44dd89a2f8641
2020-04-04 17:09:41 +02:00
Tamás Zolnai
4246d1e73a cypress: Catch also timout error of the new cypress-wait-until package.
Change-Id: I0b9856bbf933040a107c6cb8fe738d7a9f30a4a1
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/91653
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
2020-04-03 18:19:33 +02:00
Tamás Zolnai
ab10cc1c83 cypress: mobile: number formatting tests in Calc.
Change-Id: I07b919f4bc06f5f368c23b5fed014cb5011c8e68
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/91411
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
2020-03-31 13:30:04 +02:00
Tamás Zolnai
732a69e69b cypress: use hardcoded value as WSD hash.
cypress test framework has a problem with some
specific hashes (for example: 769e98018) and it
do not pass these values to the test code, which
make all test fail with 404 error.

Change-Id: I385b1621de25fdf90e6169bdff25eaf3c18cf81e
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/91101
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
2020-03-26 15:58:22 +01:00
Tamás Zolnai
3b1ca68a21 cypress: mobile: add spellchecking tests for impress.
Change-Id: I7b3909c79ca1fad27c5e3754a36f38f659e467e1
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90712
Tested-by: Tamás Zolnai <tamas.zolnai@collabora.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
2020-03-19 14:48:42 +01:00
Tamás Zolnai
6e96d4f6df cypress: catch "Uncaught TypeError" error in the log.
During parallel build.

Change-Id: I18d78250650e5d16ff9a4ff8588a2e955a934f44
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90721
Tested-by: Tamás Zolnai <tamas.zolnai@collabora.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
2020-03-19 12:30:54 +01:00
Tamás Zolnai
29aa5e15d2 cypress: calc-mobile: add tests for spellchecking menu.
Change-Id: If0c1af21c69b49452ccdc24e3c908396a045e03d
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90615
Tested-by: Tamás Zolnai <tamas.zolnai@collabora.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
2020-03-17 18:23:10 +01:00
Tamás Zolnai
8c6bc949da cypress: better parallel build detection.
Now it works both with '-j' and with '-j <number>'.

Change-Id: I41ebace6fd799f103ce557a8cf8419fea786faca
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90496
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
2020-03-14 22:04:32 +01:00
Tamás Zolnai
f49783b7be cypress: add package.json as a dependency of local npm installation.
So packages will be reinstalled, when packages.json changes.

Change-Id: I4731b4995bc969a474ea3fcd36e1135ed03ec62c
2020-03-12 19:02:32 +01:00
Michael Meeks
9e0ec98aaf cypress: rename the user-agent to not include a space.
Makes some combinations of chrome etc. happy it seems.

Change-Id: Ica8063e42a8719f2ba45de9084ad1fddd6fb58bd
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90419
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
2020-03-12 18:51:30 +01:00
Tamás Zolnai
d0fe43bffa cypress: catch also AssertionError.
Change-Id: I78a5d0f5a160a68197019735565689f61a5cdb39
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90372
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
2020-03-12 12:06:55 +01:00
Tamás Zolnai
bbf31dc62a cypress: mobile: test changing font properties in calc.
Change-Id: I23b26ed9d9866bace0b8f83a7e54674d8c49d451
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90292
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
2020-03-10 19:48:43 +01:00
Tamás Zolnai
ae759479f7 cypress: better way to find out test failure.
Change-Id: I2a398714976f65977cc64490206e7e57c1da5d86
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90256
Tested-by: Tamás Zolnai <tamas.zolnai@collabora.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
2020-03-10 14:56:51 +01:00
Tamás Zolnai
79073f6ce1 cypress: mobile: add first insertion related tests for calc.
Change-Id: I57561ef5687119cd53d99ea46e9ca3a269548422
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90237
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
2020-03-10 11:18:04 +01:00
Tamás Zolnai
90c5717466 cypress: mobile: avoid hard coded coordinates for calc focus tests.
Change-Id: I9da74fcd090371cbea5b3a8c8836bf9236f709b3
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90224
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
2020-03-09 15:39:40 +01:00
Tamás Zolnai
68924966c0 cypress: mobile: merge styles tests to font tests.
There is not much more we can test about styles,
since now we have only an apply style button.

Change-Id: I92a1a6dc60701743af44928966475cf4471b317b
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/90108
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
2020-03-06 15:19:34 +01:00
Tamás Zolnai
d9a23d9668 parallel-cypress: improve console output of parallel test run.
Change-Id: I29440bafdba101c3f4e86de7d587330b972289de
2020-03-03 16:00:30 +01:00
Tamás Zolnai
033e353e9d parallel-cypress: restore original sequential build.
Running all tests in one command will display a better output
and also might be faster.

Change-Id: Ib061fad28eeb1246c774bfb7fd958c3f5b9c317b
2020-03-03 12:41:54 +01:00
Yunusemre Şentürk
e18b7ca425 Wait until Xvfb says it is ready
Change-Id: I3af4428802d6e0ee380536e8dbad1de8aa6d9337
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89727
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
2020-03-03 12:41:54 +01:00
Tamás Zolnai
2cfd990e75 parallel-cypress: Start own Xvfb in CI environment.
It's not necessary when we build with one job, but when
there are more jobs they try to start their own Xvfb
and they might use the same DISPLAY number for that
which leads to a failure or a freeze.
See also:
https://docs.cypress.io/guides/guides/continuous-integration.html#Xvfb

Change-Id: I2fb301b1d8e889b2898d6792fb38cece303129a6
2020-03-03 12:41:54 +01:00
Tamás Zolnai
eb3b3e54d4 parallel-cypress: during parallel build don't write out execution line-by-line
Because in this case, the output of the different threads will be
mixed in the command line.
For sequential build keep the original behavior.

Change-Id: Ibf9bf7c6111f8b5b0cf188fab66b97dedd613c65
2020-03-03 12:41:54 +01:00
Tamás Zolnai
be9277d700 parallel-cypress: initial parallelization
- Run all spec files separately so we can run them
in parallel.

Change-Id: I521feeac6e1ac4c3d2c87e989c11d4231945e568
2020-03-03 12:41:46 +01:00
Tamás Zolnai
5ff34b20c5 cypress: let's make the pid file under workdir.
Change-Id: Ib927b7d4c29c623de8cd4bf083843a7a6a7c9c2f
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89374
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
2020-02-24 19:43:23 +01:00
Tamás Zolnai
a7da409624 cypress: make sure that npm packages are not installed more times.
When we build with more jobs.

Change-Id: I1051825ec8db4ccc7e10bd4a599480a97334538d
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89373
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
2020-02-24 19:40:24 +01:00
Tamás Zolnai
75451d9fb6 cypress: use parentheses consistently in the make file.
Change-Id: I3d679fe99749222a6ab56c952d5d3fa09a6dd3d7
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89238
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
2020-02-21 20:25:43 +01:00
Tamás Zolnai
3f0afa9cfa cypress: better handling of npm package dependencies
For incremental builds, it's not enough to check the node_modules
folder, we also need to check whether all used binaries are in
place.

Change-Id: I9c5f380f3845195bfa2dbfb03ab269ce4659c4ae
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89233
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
2020-02-21 19:47:22 +01:00
Tamás Zolnai
fdcdcedbfa cypress: don't wait on loolwsd server forever.
Use one minutes timeout.

Change-Id: Ia4e351088d7bd152b4c7f47c66dce2a88f14970c
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89232
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
2020-02-21 19:43:27 +01:00
Tamás Zolnai
be9333bf5b cypress: try harder to kill loolwsd server.
pkill uses SIGTERM signal by default, which is good because
using that makes loolwsd to clean up things before shut down.
For example it removes the pid file.
However, when loolwsd stuck for any reason, then SIGTERM might
not be effective. In this case use SIGKILL.

Change-Id: I3ed4bd24d6e73169b4d04dd7c9b4ac03bee8fdd1
2020-02-20 13:13:52 +01:00
Tamás Zolnai
eb4f5abf52 cypress: kill only our own loolwsd process.
Change-Id: I9c6f71571183c7583bd3c4c122c3c3e2c04696ea
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89070
Tested-by: Tamás Zolnai <tamas.zolnai@collabora.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
2020-02-20 05:45:25 +01:00
Tamás Zolnai
39150fca06 cypress: better way to generate allowed port numbers.
Change-Id: I1900df07f1fe76d39a35ab8cab0d62c0c53a35f9
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89069
Tested-by: Tamás Zolnai <tamas.zolnai@collabora.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
2020-02-20 05:44:57 +01:00
Tamás Zolnai
eb53dd5a8c cypress: Always search for an available port number.
Change-Id: Ibfb80d43cd7f7d4c8cab8e3621bebfc2201d4b2d
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89066
Tested-by: Tamás Zolnai <tamas.zolnai@collabora.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
2020-02-20 04:49:38 +01:00
Tamás Zolnai
f7d232d758 cypress: Create workdir in the builddir.
Change-Id: Ief5a74412b7e12350d83b3af4345bf813288065c
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/88628
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
2020-02-13 20:14:18 +01:00
Henry Castro
a420aef10d cypress: makefile: fix running specific test
buildir != srcdir

Change-Id: If3f33b546d9c6a95ba214d2d26213ca95fe52e2d
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/88605
Tested-by: Henry Castro <hcastro@collabora.com>
Reviewed-by: Henry Castro <hcastro@collabora.com>
2020-02-13 15:31:04 +01:00
Henry Castro
c2de19b368 cypress: makefile: fix build dir expansion
builddir != srcdir

Change-Id: I8243fb28782bbe123b65ec86a36770323092b9f8
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/88561
Tested-by: Henry Castro <hcastro@collabora.com>
Reviewed-by: Henry Castro <hcastro@collabora.com>
2020-02-13 15:29:51 +01:00
Tamás Zolnai
1a65e4040f cypress: Check JS errors in all test code.
Change-Id: Ifb33b98b5da61ac69cce91b9985e313863764d5c
2020-01-30 15:24:14 +01:00
Tamás Zolnai
2a82aa6db9 cypress: mobile: Add one use case to check font changes.
Use the clipboard code to generate the HTML of
the selected text, so we can check the font properties
in the document.

Change-Id: I43e5325b7fe0a9fda073c43335231e014374d6c2
2020-01-28 18:01:36 +01:00
Tamás Zolnai
0145c03ed6 cypress: Introduce an --enable-cypress config option.
Removing the cypress_test subdir from top level
folder made the packaging process to fail.
So better to use a flag to enable cypress tests.

Change-Id: Iead4b7cbbea5c6aaba18c0b85f23d67a4fbe920b
2020-01-23 14:55:43 +01:00
Miklos Vajna
ed586d9601 cypress: accept chromium next to chrome
Change-Id: Id2a577b1413227840e2af29abe4cc96e5a81dd82
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/87191
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
Tested-by: Miklos Vajna <vmiklos@collabora.com>
2020-01-22 18:59:46 +01:00
Tamás Zolnai
33245f42ab cypress: Disable SSL for the test loolwsd server.
Change-Id: I613768b3a91e5f34c64495cb5ca13312210f3549
2020-01-22 12:55:13 +01:00
Tamás Zolnai
d87fab2acf cypress: Don't use hardcoded wsd hash.
Change-Id: Iff7296c6a9def8df9ddb6d78a29e0adf64436b7e
2020-01-22 12:55:13 +01:00
Tamás Zolnai
b06ec9c71b cypress: let's have these two commands in separate lines.
Change-Id: Idf7856518121ca2ce84a9eb8f77b6f596a21ca18
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/87156
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
Tested-by: Tamás Zolnai <tamas.zolnai@collabora.com>
2020-01-21 18:47:02 +01:00
Tamás Zolnai
c782c033b0 cypress: Use a better wait method for loolwsd server
Use the wait-on npm module for this.

Change-Id: I602f95aa79d46cf7b55bcc598e612bbc5342c52b
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/87150
Reviewed-by: Tamás Zolnai <tamas.zolnai@collabora.com>
Tested-by: Tamás Zolnai <tamas.zolnai@collabora.com>
2020-01-21 17:37:28 +01:00
Tamás Zolnai
d5f698acd5 cypress: Don't run npm install unconditionally.
Change-Id: I51ff6e7706075d15c213617fede89088499226ed
2020-01-21 16:13:01 +01:00
Tamás Zolnai
a410b54c47 cypress: Introduce more make rules to make easier to run tests.
Change-Id: Ic8ae881864ae931fe6abed73a6a6086d6e0e591d
2020-01-21 16:13:01 +01:00