Running Nightwatch tests in Acquia Code Studio

Last updated March 5, 2024
1 minute read

Goal

Learn how to run Nightwatch tests to test your Drupal application on Acquia Code Studio.

Overview

Nightwatch is an automated testing framework for web applications and websites. It uses the W3C WebDriver API to simulate real user interactions in a web browser environment, such as Chrome, to test the complete flow of an application, end-to-end.

In this tutorial, we'll see how we can extend the Acquia AutoDevOps template that comes out-of-the-box with Code Studio to also run Nightwatch tests for our application so we can ensure critical features remain functional as we further develop and maintain it.

A simple Nightwatch test
To start, let's first add a simple Nightwatch test to verify Drupal is up and running by navigating to the Drupal login page and checking to see if the expected fields are present. If you already have Nightwatch tests for your application, you can go ahead and skip to the next step.

At the root of our project, we will create the directories "tests/src/Nightwatch/Tests" and "tests/src/Nightwatch/Commands". For the purposes of this tutorial, we will only be using the first one to hold our simple test that checks whether we can access Drupal's login screen. In the other, you can put any custom Nightwatch commands you may have that help with running your tests.

So, once we have our tests directory, let's go ahead and create a file called "checkDrupalLoginScreen.js" and add the following inside:
```
module.exports = {
  '@tags': ['myproject'],

  before(browser) {
    browser
      .deleteCookies()
      .resizeWindow(1400, 1024);

    browser.globals.drupalSitePath = 'sites/default';
  },

  'Test login screen': (browser) => {
    browser
      // Navigate to the Drupal login page.
      .drupalRelativeURL('/user/login')
      // Wait for the body of the page to be visible.
      .waitForElementVisible('body', 1000)
      // Check for the username field.
      .assert.visible('input[name="name"]', 'Login input field is visible')
      // Check for the password field.
      .assert.visible('input[name="pass"]', 'Password input field is visible')
      // Check for the login button.
      .assert.visible('input#edit-submit', 'Submit button is visible')
  },
};
```
Make note of the tag "myproject" we added to our test as we will use it later to run it.
Extend the Acquia AutoDevOps template
Navigate to your project page on https://code.acquia.com and do the following:
1. On the left hand sidebar, find and click Settings > CI/CD.
2. At the top of the page there should be a "General pipelines" section. Click Expand.
3. In the fields that show up you will see one called "CI/CD configuration file", which should be pre-filled with the path to the Acquia AutoDevOps template. Make note of the template path in case you need to restore it later (in case you haven't, it's "gitlab-ci/Auto-DevOps.acquia.gitlab-ci.yml@acquia/standard-template").
4. Delete the Acquia template entry and either:
  - Leave it empty. This will pick up the .gitlab-ci.yml file from the root directory of our project (this is what we will do for our purposes here).
  - Add a path to the directory where you want to store your .gitlab-ci file relative to the root of the code repository (e.g. my/path/.gitlab-ci.yml).
5. In our project's root directory, we will now go ahead and create our .gitlab-ci.yml file.
At the top of our .gitlab-ci.yml file we will add:
```
include:
  - project: 'acquia/standard-template'
    file:
      - '/gitlab-ci/Auto-DevOps.acquia.gitlab-ci.yml'
```
Using include, Gitlab allows us to include external YAML files into our own CI file. In our case, we are including the same file we removed from the Code Studio UI earlier. This allows us to use the predefined pipeline configuration in our project while also extending it to add our own pieces, such as a step for running Nightwatch tests.

Variables

Next we will define variables for our project's pipeline.

It is important to note here that Acquia recommends adding variables through the Code Studio UI. Variables added through the Code Studio UI take higher precedence than variables defined in the .gitlab-ci.yml file. As such, if we have a variable defined in both places, the one added through the Code Studio UI will take effect, which may lead to unexpected results.

For the purposes of this tutorial, we will add our variables in our .gitlab-ci.yml while making sure we are not overriding those same variables in the Code Studio UI.

variables:
  # ACQUIA
  #We want Code Studio to build the Drupal code base (e.g run composer to bring in our application's dependencies).
  ACQUIA_JOBS_BUILD_DRUPAL: 'true'
  #We will execute the Code Studio tests stage which will test Drupal.
  ACQUIA_JOBS_TEST_DRUPAL: 'true'
  # We will not create a CDE to deploy our codebase (feel free to change this to true if you have a CDE entitlement and want Code Studio to deploy the build artifact to it.
  ACQUIA_JOBS_CREATE_CDE: 'false'
  # We will install Drupal during the pipeline execution.
  ACQUIA_TASKS_SETUP_DRUPAL: 'true'
  # We will install a fresh copy of Drupal.
  ACQUIA_TASKS_SETUP_DRUPAL_STRATEGY: 'install'
  # We will use the "standard" install profile 
  ACQUIA_TASKS_SETUP_DRUPAL_PROFILE: 'standard'
  # We will not install the site from configuration or import configuration during the setup process.
  ACQUIA_TASKS_SETUP_DRUPAL_CONFIG_IMPORT: 'false'

  # DATABASE
  # Username for the MySQL database.
  MYSQL_USER: drupal
  # Password for the MySQL database.
  MYSQL_PASSWORD: drupal
  # Name of the database to use for Drupal.
  MYSQL_DATABASE: drupal
  # Hostname of the database container.
  DB_HOST: mysql

  # OTHER
  # Where we want to store our test reports.
  REPORT: '$CI_PROJECT_DIR/tests/reports'

  # Nightwatch
  # The URL where Drupal will be accessible during our tests.
  DRUPAL_TEST_BASE_URL: http://127.0.0.1:8080
  # The Drupal database URL Drupal will use during our tests.
  DRUPAL_TEST_DB_URL: mysql://$MYSQL_USER:$MYSQL_PASSWORD@$DB_HOST:3306/$MYSQL_DATABASE
  # Hostname of the WebDriver server (Chromedriver in our case) that will be used for testing.
  DRUPAL_TEST_WEBDRIVER_HOSTNAME: 127.0.0.1
  # Port number of the WebDriver.
  DRUPAL_TEST_WEBDRIVER_PORT: "9515"
  # Whether ChromeDriver should start automatically during tests.
  DRUPAL_TEST_CHROMEDRIVER_AUTOSTART: "false"
  # Arguments for the Chrome browser when used by WebDriver. 
  DRUPAL_TEST_WEBDRIVER_CHROME_ARGS: --disable-gpu --headless --no-sandbox --disable-dev-shm-usage --disable-extensions
  # Where to store the Nightwatch reports.
  DRUPAL_NIGHTWATCH_OUTPUT: $REPORT/nightwatch
  # Directories to ignore when running tests.
  DRUPAL_NIGHTWATCH_IGNORE_DIRECTORIES: node_modules,vendor,.*,sites/*/files,sites/*/private,sites/simpletest
  # Path to the directory where Nightwatch should search for tests. In our case the route of our repository since that's where we defined our "tests" directory.
  DRUPAL_NIGHTWATCH_SEARCH_DIRECTORY: ../

The Acquia variables are included here for completeness of the example and are not mandatory. They can also be configured via the Code Studio UI.

Nightwatch
Once we've extended the Acquia AutoDevOps template and defined our variables, it's time to add our Nightwatch step.

We want Nightwatch to execute as part of the "Test Drupal" stage that Code Studio already executes as part of the standard pipeline.
```
"Nightwatch":
  stage: "Test Drupal"
```
We will use the mysql service because we want to be able to install Drupal using MySQL (alternatively you can use sqlite). To make things easier, we will also use the Chromedriver image that Drupal CI uses.
```
services:
  - mysql:5.7
  - drupalci/webdriver-chromedriver:production
```
We want our Nightwatch tests to run if our CUSTOM_RUN_TESTS flag is set to true and we will run them only if the "Build Code" and "Manage Secrets" steps have completed successfully. Lastly, we want the pipeline to fail if the Nightwatch tests run into an issue and fail so we can catch code that breaks our application and prevent its deployment.
```
  extends: .cache_strategy_pull
  needs: ["Build Code" , "Manage Secrets"]
  rules:
  - if: ($CUSTOM_RUN_TESTS == "true")
    when: on_success
    allow_failure: false
```
While our tests are executing, they may run into issues and fail. If that happens, we want to store the Nightwatch generated reports as artifacts so we can view from the Code Studio UI:
```
  artifacts:
    when: always
    expose_as: 'Nightwatch'
    paths:
        - "$REPORT/nightwatch/"
    expire_in: 1 week
```
To prepare our tests to run, we will use the scripts from the Acquia AutoDevOps template to build the Drupal codebase and install Drupal. We will install yarn, which we can then use to install the Drupal javascript dependencies and run our tests, and we will make sure Chromedriver is updated (as the one that ships with Drupal may be old).
```
  before_script:
    - !reference [.clone_standard_template, script]
    - . "$STANDARD_TEMPLATE_PATH"/ci-files/scripts/stages/build/job_build_code.sh
    - . "$STANDARD_TEMPLATE_PATH"/ci-files/scripts/utility/install_drupal.sh
    - start_section install_yarn "Installing yarn and updating chromedriver"
    # Install yarn
    - curl -o- -L https://yarnpkg.com/install.sh | bash
    # Make available in the current terminal
    - export PATH="$HOME/.yarn/bin:$HOME/.config/yarn/global/node_modules/.bin:$PATH"
    # Update chromedriver as Drupal package.json can be late on chrome version.
    - |
      yarn --cwd $CI_PROJECT_DIR/docroot/core upgrade \
      chromedriver@$(curl -s http://$DRUPAL_TEST_WEBDRIVER_HOSTNAME:$DRUPAL_TEST_WEBDRIVER_PORT/status | jq '.value.build.version' | tr -d '"' | cut -d. -f1)
    - exit_section install_yarn
```
If you noticed above, we've also included some echo statements such as:
```
- start_section section_name "Section name"
- exit_section section_name
```
These add any output from the commands between them into collapsable sections in the Code Studio logs so the logs can be easier to read and kept relatively clean, since what we care about the most is the output of our tests. We wrap everything else in sections like these as well. If there's an issue with any of the commands at any point, we can expand the section to take a look.

Finally, we can configure Drupal's Nightwatch environment variables by creating the expected .env file in the core directory and adding them inside. We will check the status of Chromedriver and Drupal to ensure we are ready and install Drupal core's Javascript dependencies and run our tests. We will start the PHP server that is included with Drupal core and configure it to serve Drupal on the same address and port we told Nightwatch to use as the Drupal base URL (http://127.0.0.1:8080). We will install the Javascript dependencies and run our tests.

Make note that we will run only the tests we previously tagged with "myproject" as we don't want to run other tests that may be included in core itself or contributed modules.
```
script:
  - start_section configure_nightwatch "Configure Nightwatch"
  # Configure Nightwatch for Drupal.
  # see: https://git.drupalcode.org/project/gitlab_templates/-/blob/main/includes/include.drupalci.main.yml#L639
  - touch $CI_PROJECT_DIR/docroot/core/.env
  - |
    cat <<EOF > $CI_PROJECT_DIR/docroot/core/.env
    DRUPAL_TEST_BASE_URL='${DRUPAL_TEST_BASE_URL}'
    DRUPAL_TEST_CHROMEDRIVER_AUTOSTART=${DRUPAL_TEST_CHROMEDRIVER_AUTOSTART}
    DRUPAL_TEST_DB_URL='${DRUPAL_TEST_DB_URL}'
    DRUPAL_TEST_WEBDRIVER_HOSTNAME='${DRUPAL_TEST_WEBDRIVER_HOSTNAME}'
    DRUPAL_TEST_WEBDRIVER_CHROME_ARGS='${DRUPAL_TEST_WEBDRIVER_CHROME_ARGS}'
    DRUPAL_TEST_WEBDRIVER_PORT='${DRUPAL_TEST_WEBDRIVER_PORT}'
    EOF
  - exit_section configure_nightwatch
  # Start Drupal's PHP server and surpress its logging output to keep the test output clean.
  - cd $CI_PROJECT_DIR/docroot && php -S 127.0.0.1:8080 .ht.router.php >/dev/null 2>&1 &
  # Run nightwatch tests
  - yarn --cwd $CI_PROJECT_DIR/docroot/core install
  - yarn --cwd $CI_PROJECT_DIR/docroot/core test:nightwatch --tag myproject
```

Putting it all together

Putting all of the above together our final .gitlab-ci.yml file should look like this:

include:
  - project: 'acquia/standard-template'
    file:
      - '/gitlab-ci/Auto-DevOps.acquia.gitlab-ci.yml'

variables:
  # ACQUIA
  #We want Code Studio to build the Drupal code base (e.g run composer to bring in our application's dependencies).
  ACQUIA_JOBS_BUILD_DRUPAL: 'true'
  #We will execute the Code Studio tests stage which will test Drupal.
  ACQUIA_JOBS_TEST_DRUPAL: 'true'
  # We will not create a CDE to deploy our codebase (feel free to change this to true if you have a CDE entitlement and want Code Studio to deploy the build artifact to it.
  ACQUIA_JOBS_CREATE_CDE: 'false'
  # We will install Drupal during the pipeline execution.
  ACQUIA_TASKS_SETUP_DRUPAL: 'true'
  # We will install a fresh copy of Drupal.
  ACQUIA_TASKS_SETUP_DRUPAL_STRATEGY: 'install'
  # We will use the "standard" install profile 
  ACQUIA_TASKS_SETUP_DRUPAL_PROFILE: 'standard'
  # We will not install the site from configuration or import configuration during the setup process.
  ACQUIA_TASKS_SETUP_DRUPAL_CONFIG_IMPORT: 'false'

  # DATABASE
  # Username for the MySQL database.
  MYSQL_USER: drupal
  # Password for the MySQL database.
  MYSQL_PASSWORD: drupal
  # Name of the database to use for Drupal.
  MYSQL_DATABASE: drupal
  # Hostname of the database container.
  DB_HOST: mysql

  # OTHER
  # Where we want to store our test reports.
  REPORT: '$CI_PROJECT_DIR/tests/reports'

  # Nightwatch
  # The URL where Drupal will be accessible during our tests.
  DRUPAL_TEST_BASE_URL: http://127.0.0.1:8080
  # The Drupal database URL Drupal will use during our tests.
  DRUPAL_TEST_DB_URL: mysql://$MYSQL_USER:$MYSQL_PASSWORD@$DB_HOST:3306/$MYSQL_DATABASE
  # Hostname of the WebDriver server (Chromedriver in our case) that will be used for testing.
  DRUPAL_TEST_WEBDRIVER_HOSTNAME: 127.0.0.1
  # Port number of the WebDriver.
  DRUPAL_TEST_WEBDRIVER_PORT: "9515"
  # Whether ChromeDriver should start automatically during tests.
  DRUPAL_TEST_CHROMEDRIVER_AUTOSTART: "false"
  # Arguments for the Chrome browser when used by WebDriver. 
  DRUPAL_TEST_WEBDRIVER_CHROME_ARGS: --disable-gpu --headless --no-sandbox --disable-dev-shm-usage --disable-extensions
  # Where to store the Nightwatch reports.
  DRUPAL_NIGHTWATCH_OUTPUT: $REPORT/nightwatch
  # Directories to ignore when running tests.
  DRUPAL_NIGHTWATCH_IGNORE_DIRECTORIES: node_modules,vendor,.*,sites/*/files,sites/*/private,sites/simpletest
  # Path to the directory where Nightwatch should search for tests. In our case the route of our repository since that's where we defined our "tests" directory.
  DRUPAL_NIGHTWATCH_SEARCH_DIRECTORY: ../

"Nightwatch":
  stage: "Test Drupal"
  services:
    - mysql:5.7
    - drupalci/webdriver-chromedriver:production
  extends: .cache_strategy_pull
  needs: ["Build Code" , "Manage Secrets"]
  rules:
  - when: on_success
    allow_failure: false
  artifacts:
    when: always
    expose_as: 'Nightwatch'
    paths:
        - "$REPORT/nightwatch/"
    expire_in: 1 week
  before_script:
    - !reference [.clone_standard_template, script]
    - . "$STANDARD_TEMPLATE_PATH"/ci-files/scripts/stages/build/job_build_code.sh
    - . "$STANDARD_TEMPLATE_PATH"/ci-files/scripts/utility/install_drupal.sh
    - start_section install_yarn "Installing yarn and updating chromedriver"
    # Install yarn
    - curl -o- -L https://yarnpkg.com/install.sh | bash
    # Make available in the current terminal
    - export PATH="$HOME/.yarn/bin:$HOME/.config/yarn/global/node_modules/.bin:$PATH"
    # Update chromedriver as Drupal package.json can be late on chrome version.
    - |
      yarn --cwd $CI_PROJECT_DIR/docroot/core upgrade \
      chromedriver@$(curl -s http://$DRUPAL_TEST_WEBDRIVER_HOSTNAME:$DRUPAL_TEST_WEBDRIVER_PORT/status | jq '.value.build.version' | tr -d '"' | cut -d. -f1)
    - exit_section install_yarn
  script:
    - start_section configure_nightwatch "Configure Nightwatch"
    # Configure Nightwatch for Drupal.
    # see: https://git.drupalcode.org/project/gitlab_templates/-/blob/main/includes/include.drupalci.main.yml#L639
    - touch $CI_PROJECT_DIR/docroot/core/.env
    - |
      cat <<EOF > $CI_PROJECT_DIR/docroot/core/.env
      DRUPAL_TEST_BASE_URL='${DRUPAL_TEST_BASE_URL}'
      DRUPAL_TEST_CHROMEDRIVER_AUTOSTART=${DRUPAL_TEST_CHROMEDRIVER_AUTOSTART}
      DRUPAL_TEST_DB_URL='${DRUPAL_TEST_DB_URL}'
      DRUPAL_TEST_WEBDRIVER_HOSTNAME='${DRUPAL_TEST_WEBDRIVER_HOSTNAME}'
      DRUPAL_TEST_WEBDRIVER_CHROME_ARGS='${DRUPAL_TEST_WEBDRIVER_CHROME_ARGS}'
      DRUPAL_TEST_WEBDRIVER_PORT='${DRUPAL_TEST_WEBDRIVER_PORT}'
      EOF
    - exit_section configure_nightwatch
    # Start Drupal's PHP server and surpress its logging output to keep the test output clean.
    - cd $CI_PROJECT_DIR/docroot && php -S 127.0.0.1:8080 .ht.router.php >/dev/null 2>&1 &
    # Run nightwatch tests
    - yarn --cwd $CI_PROJECT_DIR/docroot/core install
    - yarn --cwd $CI_PROJECT_DIR/docroot/core test:nightwatch --tag myproject

Troubleshooting
If for some reason the tests are not running or failing without any clear reason, we can use the following to verify the status of Drupal, Chromedriver, as well as the environment variables that are being used to run the tests.
```
# Verify chromedriver is running before doing anything
- curl http://$DRUPAL_TEST_WEBDRIVER_HOSTNAME:$DRUPAL_TEST_WEBDRIVER_PORT/status | jq '.'
# Confirm Drupal is setup
- $CI_PROJECT_DIR/vendor/bin/drush status
# Confirm Nightwatch environment variables.
- cat $CI_PROJECT_DIR/docroot/core/.env
```
In addition, we can adapt the Nightwatch command to run in verbose mode to see if there's any other issues that we can't immediately identify.
```
- yarn --cwd $CI_PROJECT_DIR/docroot/core test:nightwatch --verbose --tag myproject
```

Running Nightwatch tests in Acquia Code Studio

Goal

Overview

A simple Nightwatch test

Extend the Acquia AutoDevOps template

Variables

Nightwatch

Putting it all together

Troubleshooting

Additional Resources

Similar Tutorials

Removing BLT on Acquia Cloud: How to Get Started

Running Nightwatch tests in Acquia Cloud IDEs

De-index/Block bulk pages from search engines for Drupal sites

Automating Error Detection with Git Bisect Run

How To Poll for Changes in Acquia DAM

Local environment for Acquia Site Factory

Running Nightwatch tests in Acquia Code Studio