Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DEV: Minimal first pass of rails system test setup #16311

Merged
merged 49 commits into from Sep 28, 2022

Conversation

martin-brennan
Copy link
Contributor

@martin-brennan martin-brennan commented Mar 29, 2022

This commit introduces rails system tests run with chromedriver, selenium,
and headless chrome to our testing toolbox.

We use the webdrivers gem and selenium-webdriver which is what
the latest Rails uses so the tests run locally and in CI out of the box.

You can use SELENIUM_VERBOSE_DRIVER_LOGS=1 to show extra
verbose logs of what selenium is doing to communicate with the system
tests.

By default JS logs are verbose so errors from JS are shown when
running system tests, you can disable this with
SELENIUM_DISABLE_VERBOSE_JS_LOGS=1

You can use SELENIUM_HEADLESS=0 to run the system
tests inside a chrome browser instead of headless, which can be useful to debug things
and see what the spec sees. See note above about bin/ember-cli to avoid
surprises.

I have modified bin/turbo_rspec to exclude spec/system by default,
support for parallel system specs is a little shaky right now and we don't
want them slowing down the turbo by default either.

PageObjects and System Tests

To make querying and inspecting parts of the page easier
and more reusable inbetween system tests, we are using the
concept of PageObjects in
our system tests. A "Page" here is generally corresponds to
an overarching ember route, e.g. "Topic" for /t/324345/some-topic,
and this contains logic for querying components within the topic
such as "Posts".

I have also split "Modals" into their own entity. Further down the
line we may want to explore creating independent "Component"
contexts.

Capybara DSL should be included in each PageObject class,
reference for this can be found at https://rubydoc.info/github/teamcapybara/capybara/master#the-dsl

For system tests, since they are so slow, we want to focus on
the "happy path" and not do every different possible context
and branch check using them. They are meant to be overarching
tests that check a number of things are correct using the full stack
from JS and ember to rails to ruby and then the database.

CI Setup

Whenever a system spec fails, a screenshot
is taken and a build artifact is produced after the entire CI run is complete,
which can be downloaded from the Actions UI in the repo.

Most importantly, a step to build the Ember app using Ember CLI
is needed, otherwise the JS assets cannot be found by capybara:

- name: Build Ember CLI
  run: bin/ember-cli --build

A new --build argument has been added to bin/ember-cli for this
case, which is not needed locally if you already have the discourse
rails server running via bin/ember-cli -u since the whole server is built and
set up by default.

@martin-brennan martin-brennan marked this pull request as ready for review Aug 31, 2022
@martin-brennan martin-brennan marked this pull request as draft Aug 31, 2022
@martin-brennan martin-brennan marked this pull request as ready for review Sep 1, 2022
@martin-brennan
Copy link
Contributor Author

martin-brennan commented Sep 1, 2022

I will uncomment all the other test runs before merging this.

davidtaylorhq and others added 3 commits Sep 25, 2022
* Simplify selenium launching using `webdrivers` gem

This is the default way of handling system tests when spinning up a new Rails app via `rails new`. It means there is no need to manually launch Selenium.

This commit also removes the docker container approach. Personally I think using the locally installed version of Chrome is fine - it's what we do for qunit tests, and it will be significantly faster than the dockerised version on non-linux machines. Also, the `selenium/standalone-chrome-debug` image appears to be abandoned - it hasn't been updated for more than a year. If we want to use this approach we should use a more up-to-date image like `selenium/standalone-chrome`.

* Update GitHub actions config

* Revert ChromeInstalledChecker changes

This refactoring doesn't seem to be required any more - reverting to clean up the diff

* Simplify host, port and WebMock configuration

Now that we're using the `webdrivers` gem, the defaults work without too much tweaking

* Remove SYSTEM_SPEC_RUN flag

All the setup is now cheap, so we can do it unconditionally. This means that our system tests will work alongside regular rspec tests/tooling.
@martin-brennan martin-brennan merged commit 57caf08 into main Sep 28, 2022
11 checks passed
@martin-brennan martin-brennan deleted the dev/experimental-rails-system-tests branch Sep 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
3 participants