Tests are why most teams build a pipeline in the first place. But a pipeline that runs npm test and prints 3,000 lines of output is only half the value. GitLab can read your test results, show failures right in the merge request, track coverage line by line and spread a slow suite across several machines. This lesson wires all of that into Orbit’s pipeline.
The test pyramid in a pipeline
| Layer | What it checks | Speed | Runs |
|---|---|---|---|
| Static checks | Formatting, lint, types | Seconds | Every push, first |
| Unit tests | Functions and modules in isolation | Seconds to a minute | Every push |
| Integration tests | The API with a real database and cache | A few minutes | Every merge request |
| End-to-end tests | The deployed app through a browser or HTTP | Minutes | After deploying to a review app or staging |
Order matters: cheap checks run first and fail fast, expensive ones run only after cheap ones pass. With needs, the static checks do not even wait for the build.
Show test results in the merge request
Most test runners can write JUnit XML. Upload it as a report and GitLab shows a Test summary on the merge request: how many tests ran, which failed, the failure message, and which tests are newly failing compared with the target branch.
unit-tests:
stage: test
image: node:24-alpine
needs: [build-api]
script:
- npm ci
- node --test --test-reporter=junit --test-reporter-destination=junit.xml --test-reporter=spec --test-reporter-destination=stdout
artifacts:
when: always
reports:
junit: junit.xml
Two details matter:
when: alwaysuploads the report even when tests fail, which is exactly when you need it.- Using two reporters keeps the readable output in the job log and writes XML for GitLab.
Every major test framework has a JUnit option: pytest --junitxml, Jest with jest-junit, Go with gotestsum --junitfile, Maven Surefire by default.
Coverage
GitLab tracks coverage in two ways.
The percentage
The coverage keyword takes a regular expression; GitLab reads the job log, extracts the number and shows it on the job, the merge request and the pipeline graphs:
unit-tests:
script:
- npx c8 --reporter=text-summary --reporter=cobertura npm test
coverage: '/Lines\s*:\s*(\d+\.?\d*)%/'
Line-by-line coverage in the diff
Upload a Cobertura XML report and GitLab paints the merge request diff: a green bar next to lines covered by tests, orange next to lines that are not. Reviewers see at a glance whether the new code is tested.
artifacts:
when: always
reports:
junit: junit.xml
coverage_report:
coverage_format: cobertura
path: coverage/cobertura-coverage.xml
Coverage is a signal, not a goal. 90% coverage of trivial getters is worth less than 60% that covers every error path. Watch the trend and the diff, not a hard number that people learn to game.
Integration tests with services
Lesson 8 introduced services. Here is the full integration job, with a wait for the database before tests start:
integration-tests:
stage: test
image: node:24-alpine
needs: [build-api]
services:
- name: postgres:17-alpine
alias: db
variables:
POSTGRES_DB: orbit_test
POSTGRES_USER: orbit
POSTGRES_PASSWORD: orbit-test-only
DATABASE_URL: postgres://orbit:orbit-test-only@db:5432/orbit_test
before_script:
- apk add --no-cache postgresql17-client
- until pg_isready -h db -U orbit; do sleep 1; done
- npm ci
script:
- npm run db:migrate
- npm run test:integration -- --test-reporter=junit --test-reporter-destination=integration.xml
artifacts:
when: always
reports:
junit: integration.xml
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
The password is visible in the YAML on purpose: the database lives only inside this job and is destroyed with it. Never reuse real credentials for test services.
End-to-end tests against a review app
End-to-end tests exercise the whole deployed system. Run them after the review app from lesson 7 is up, against its URL:
e2e-tests:
stage: verify
image: mcr.microsoft.com/playwright:v1.56.0-noble
needs: [deploy-review]
variables:
BASE_URL: https://$CI_COMMIT_REF_SLUG.review.orbit.example.com
script:
- npm ci
- npx playwright test --reporter=junit,line
artifacts:
when: always
paths: [playwright-report/, test-results/]
reports:
junit: results.xml
expire_in: 14 days
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
Keep the Playwright image version equal to the @playwright/test version in package-lock.json, otherwise the browsers will not match. Saving traces and screenshots as artifacts turns “it failed in CI” into “here is exactly what the browser saw”.
Run jobs in parallel
parallel: N — split one suite
When the unit tests take eight minutes, run four copies of the job and give each a quarter of the files. GitLab sets CI_NODE_INDEX (1 to N) and CI_NODE_TOTAL (N) in each copy:
unit-tests:
parallel: 4
script:
- npm ci
- FILES=$(find test -name '*.test.js' | sort | awk "NR % $CI_NODE_TOTAL == $CI_NODE_INDEX - 1")
- node --test $FILES
Many runners split by timing instead of file count for better balance: Jest has --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL, Playwright has --shard, and pytest has plugins such as pytest-split.
parallel:matrix — the same job, different settings
Test Orbit against the two Node.js versions it supports and two databases, in one definition:
compat-tests:
stage: test
image: node:$NODE_VERSION-alpine
services:
- name: $DB_IMAGE
alias: db
parallel:
matrix:
- NODE_VERSION: ["22", "24"]
DB_IMAGE: ["postgres:16-alpine", "postgres:17-alpine"]
script:
- npm ci
- npm run test:integration
That creates four jobs, named after their variables, all running at the same time.
Flaky tests
A flaky test passes and fails on the same code. It is worse than no test, because people learn to click retry and stop trusting red pipelines. A pragmatic policy:
- Measure. The merge request test summary shows how many times a failing test also failed recently on other pipelines, which is how flaky tests give themselves away.
- Quarantine. Move a known flaky test to a separate job with
allow_failure: trueso it stops blocking merges while still being visible. - Fix or delete within a deadline, usually one sprint. A quarantined test nobody fixes is a deleted test with extra steps.
- Use
retryonly for infrastructure failures (runner_system_failure,stuck_or_timeout_failure), not forscript_failure.
Keep feedback fast
| Technique | Lesson |
|---|---|
| Cache dependency downloads | 3 |
needs so jobs start as soon as their inputs are ready | 3 |
Run lint with needs: [] so it starts instantly | 3 |
rules:changes to skip unaffected parts on merge requests | 6 |
Cancel outdated pipelines with interruptible | 6 |
Split slow suites with parallel | 9 |
| Push slow end-to-end tests to after the merge or to a nightly schedule | 9 |
Aim for a merge request pipeline under ten minutes. Measure it under Analyze → CI/CD analytics, which shows pipeline duration and success rate over time.
Orbit’s test stage
stages: [build, test, deploy, verify]
lint:
extends: .node-job
stage: test
needs: []
script: npm run lint
unit-tests:
extends: .node-job
stage: test
needs: [build-api]
parallel: 2
script:
- npx c8 --reporter=text-summary --reporter=cobertura node --test --test-shard=$CI_NODE_INDEX/$CI_NODE_TOTAL --test-reporter=junit --test-reporter-destination=junit.xml --test-reporter=spec --test-reporter-destination=stdout
coverage: '/Lines\s*:\s*(\d+\.?\d*)%/'
artifacts:
when: always
reports:
junit: junit.xml
coverage_report:
coverage_format: cobertura
path: coverage/cobertura-coverage.xml
Node’s built-in runner supports --test-shard, so the split needs no extra tools.
Exercise
Your unit test job fails, but the merge request shows no test summary. The job has artifacts:reports:junit: junit.xml. What is the most likely cause?
Show answer
Answer: B. Artifacts default to when: on_success, so a failing job never uploads its report. Setting when: always makes the report available exactly when you need to see which test failed.
Exercise
A job defines parallel:matrix with NODE_VERSION [“22”, “24”] and DB_IMAGE with two values. How many jobs does GitLab create?
Show answer
Answer: B. A matrix creates one job for every combination of the listed values: 2 Node.js versions times 2 database images is 4 jobs, all able to run in parallel.
Key takeaways
- Order tests from cheapest to most expensive and fail fast.
- Upload JUnit reports with
when: alwaysto see failures in the merge request. coverageshows a percentage; a Cobertura report shows covered lines in the diff.- Services give integration tests a fresh real database per job.
parallelsplits a slow suite;parallel:matrixtests several configurations at once.- Quarantine and fix flaky tests; never hide them with retries.
FAQ
Do test reports need a paid GitLab tier?
No. JUnit test summaries, coverage percentage and Cobertura coverage visualisation in merge requests work on the Free tier.
How do I see which test is slow?
The test report on the pipeline page lists every test with its duration. Sort by time and start with the slowest file.
Should end-to-end tests block a merge?
A small, reliable smoke suite should. A large, slower suite usually runs after merge on staging or on a nightly schedule, with failures alerting the team.
Can I run only the tests affected by a change?
Some tools support it (Jest’s --changedSince, Nx and Turborepo affected graphs, Bazel). Combine them with a full run on main so nothing slips through.
How do I test the Dockerfile itself?
Build the image in the merge request pipeline without pushing a release tag, then run a container-structure test or a quick docker run smoke test that hits the health endpoint.