|
1 | | -The `cypress/` directory holds Cypress tests and the `tests/` directory holds Playwright tests. |
| 1 | +The `tests/browser` directory holds Playwright tests. |
2 | 2 |
|
3 | | -The following upstream projects have Playwright tests |
| 3 | +The following upstream projects have Playwright tests: |
4 | 4 |
|
5 | 5 | * JupyterLab (https://github.com/jupyterlab/jupyterlab/tree/main/galata) |
6 | 6 | * code-server (https://github.com/coder/code-server/tree/main/test) |
7 | 7 |
|
8 | | -Honorable mentions include |
| 8 | +Honorable mentions include: |
9 | 9 |
|
10 | 10 | * VSCode uses custom framework where Playwright is one of the possible runners (https://github.com/microsoft/vscode/wiki/Writing-Tests) |
11 | 11 | * RStudio components have Playwright tests (https://github.com/rstudio/shinyuieditor, https://github.com/rstudio/xterm.js) |
12 | 12 | * Some RStudio tests are implemented in private repository https://github.com/rstudio/rstudio/issues/10400, possibly in R https://github.com/rstudio/rstudio/tree/main/src/cpp/tests/automation with https://github.com/rstudio/chromote) |
13 | 13 |
|
14 | | -The following upstream projects have Cypress tests |
| 14 | +The following upstream projects have Cypress tests: |
15 | 15 |
|
16 | 16 | * Elyra (https://github.com/elyra-ai/elyra/tree/main/cypress) |
17 | 17 | * ODH Dashboard (https://github.com/opendatahub-io/odh-dashboard/tree/main/frontend/src/__tests__/cypress) |
18 | 18 |
|
19 | | -# Cypress |
20 | | - |
21 | | -The Cypress part was added after the Playwright part below. |
22 | | -Therefore, we are starting with an existing pnpm project folder. |
23 | | - |
24 | | -```shell |
25 | | -pnpm add --save-dev cypress |
26 | | -``` |
27 | | - |
28 | | -Since pnpm skips running build scripts by default, just run `cypress install` manually. |
29 | | - |
30 | | -``` |
31 | | -╭ Warning ───────────────────────────────────────────────────────────────────────────────────╮ |
32 | | -│ │ |
33 | | -│ Ignored build scripts: cypress. │ |
34 | | -│ Run "pnpm approve-builds" to pick which dependencies should be allowed to run scripts. │ |
35 | | -│ │ |
36 | | -╰────────────────────────────────────────────────────────────────────────────────────────────╯ |
37 | | -``` |
38 | | - |
39 | | -```shell |
40 | | -pnpm cypress install |
41 | | -``` |
42 | | - |
43 | | -## Getting started |
44 | | - |
45 | | -> https://learn.cypress.io/testing-your-first-application/installing-cypress-and-writing-your-first-test |
46 | | -
|
47 | | -Cypress operates in two modes, |
48 | | -the noninteractive `run` mode and the interactive `open` mode that is useful for development. |
49 | | - |
50 | | -```shell |
51 | | -pnpm cypress run |
52 | | -pnpm cypress open |
53 | | -``` |
54 | | - |
55 | | -To specify base URL, set the environment variable. |
56 | | - |
57 | | -```shell |
58 | | -BASE_URL=https://nb_name.apps.oc_domain/notebook/ns_name/nb_name pnpm cypress open --e2e --browser chrome |
59 | | -``` |
60 | | - |
61 | | -Upon first run, `cypress open` will ask to begin with either E2E or Component testing. |
62 | | -Choose E2e, and the following files are created if they did not exist before: |
63 | | - |
64 | | -* `cypress.config.ts`: The Cypress config file for E2E testing. |
65 | | -* `cypress/support/e2e.ts`: The support file that is bundled and loaded before each E2E spec. |
66 | | -* `cypress/support/commands.ts`: A support file that is useful for creating custom Cypress commands and overwriting existing ones. |
67 | | -* `cypress/fixtures/example.json`: Added an example fixtures file/folder. |
68 | | - |
69 | | -For any subsequent run, Cypress offers a choice of three test environments: |
70 | | - |
71 | | -1. Chrome |
72 | | -2. Electron |
73 | | -3. Firefox |
74 | | - |
75 | | -Pick Chrome and click `Start E2E Testing in Chrome` to confirm. |
76 | | - |
77 | | -If there are no tests (specs) detected, Cypress offers to `Scaffold example specs` or to `Create new spec`. |
78 | | -To experience this and maybe experiment with example specs, |
79 | | -temporarily delete everything under `cypress/e2e/` and let Cypress refresh. |
80 | | - |
81 | | -## Developing tests |
82 | | - |
83 | | -Start `cypress open` in E2E mode with Chrome |
84 | | - |
85 | | -```shell |
86 | | -BASE_URL=... pnpm cypress open --e2e --browser chrome |
87 | | -``` |
88 | | - |
89 | | -The `open` mode can be further enhanced by enabling the (currently experimental) Cypress Studio. |
90 | | - |
91 | | -Use this to quickly scaffold the test steps and then refactor them to use page objects. |
92 | | - |
93 | | -* https://docs.cypress.io/app/guides/cypress-studio |
94 | | -* https://www.selenium.dev/documentation/test_practices/encouraged/page_object_models/ |
95 | | -* https://docs.cypress.io/app/core-concepts/best-practices#Organizing-Tests-Logging-In-Controlling-State |
96 | | - |
97 | | -```typescript |
98 | | -// cypress.config.ts |
99 | | -import { defineConfig } from 'cypress' |
100 | | - |
101 | | -export default defineConfig({ |
102 | | - e2e: { |
103 | | - experimentalStudio: true, |
104 | | - }, |
105 | | -}) |
106 | | -``` |
107 | | - |
108 | | -## Execution model |
109 | | - |
110 | | -Cypress execution model can be tricky. |
111 | | - |
112 | | -Do read the introductory docs page, then the retry-ability, |
113 | | -and then the conditional testing page to appreciate the ramifications. |
114 | | - |
115 | | -* https://docs.cypress.io/app/core-concepts/introduction-to-cypress |
116 | | -* https://docs.cypress.io/app/core-concepts/retry-ability |
117 | | -* https://docs.cypress.io/app/guides/conditional-testing |
118 | | - |
119 | | -Cypress is not a general purpose web browser automation framework, |
120 | | -that was sufficiently clarified in the introduction docs, and also read the following. |
121 | | - |
122 | | -* https://docs.cypress.io/app/references/trade-offs |
123 | | -* https://docs.cypress.io/app/guides/cross-origin-testing |
124 | | - |
125 | | -Also do check out: |
126 | | - |
127 | | -* https://docs.cypress.io/app/core-concepts/best-practices |
128 | | - |
129 | | -## Problems and how to solve them |
130 | | - |
131 | | -See above for the execution model notes, and the Cypress trade-offs documentation. |
132 | | - |
133 | | -### Browser runs out of memory |
134 | | - |
135 | | -Often, the `cypress open` browser crashes with the following error message. |
136 | | - |
137 | | -``` |
138 | | -We detected that the Chrome Renderer process just crashed. |
139 | | -
|
140 | | -We have failed the current spec but will continue running the next spec. |
141 | | -
|
142 | | -This can happen for a number of different reasons. |
143 | | -
|
144 | | -If you're running lots of tests on a memory intense application. |
145 | | - - Try increasing the CPU/memory on the machine you're running on. |
146 | | - - Try enabling experimentalMemoryManagement in your config file. |
147 | | - - Try lowering numTestsKeptInMemory in your config file during 'cypress open'. |
148 | | -
|
149 | | -You can learn more here: |
150 | | -
|
151 | | -https://on.cypress.io/renderer-process-crashed |
152 | | -``` |
153 | | - |
154 | | -The advice helps somewhat, but Elyra still keeps crashing from time to time in `cypress open`. |
155 | | - |
156 | | -### Cross-origin testing |
157 | | - |
158 | | -Prior to Cypress 14, the [`document.domanin`](https://developer.mozilla.org/en-US/docs/Web/API/Document/domain) would be automatically set by Cypress. |
159 | | -Now that it is no loger true, it is as the documentation says: |
160 | | - |
161 | | -> You can visit two or more origins in different tests without needing cy.origin(). |
162 | | -> (https://docs.cypress.io/app/guides/cross-origin-testing#What-Cypress-does-under-the-hood) |
163 | | -
|
164 | | -This is especially annoying when Dashboard, Workbench, |
165 | | -and OAuth server each live in a separate origin and one test needs to visit all three. |
166 | | - |
167 | | -#### Solutions for cross-origin testing |
168 | | - |
169 | | -* The origin for each test is pinned by wherever the first `cy.visit()` ends up going, taking redirects into account. |
170 | | - * Always `cy.visit()` first the origin where the test needs to spend the most time. |
171 | | -* Use `cy.origin()` when needed. Beware that custom commands don't work on secondary origins unless `Cypress.require()` (experimental) is called! |
172 | | -* Reconfigure oauth-proxy to allow bearer token authentication, or skip auth altogether and expose workbench container directly. |
173 | | - * https://github.com/openshift/oauth-proxy/issues/179#issuecomment-1202279241 |
174 | | - * https://github.com/openshift/oauth-proxy/blob/8d8daec87683f43a15c1d74f05cb0f2635dba04e/main.go#L76 |
175 | | -* Write the tests so that only one origin needs to be touched in the test. |
176 | | - * `cy.session()` can hold login cookies established in a `before` step. |
177 | | - * `cy.request()` is not bound by origin restrictions, attempt to log in through API. |
178 | | - |
179 | 19 | # Playwright |
180 | 20 |
|
181 | 21 | This is a basic Playwright in Typescript that was setup like this |
|
0 commit comments