Flaky Test Quarantine: Owners, Expiry, and Nonblocking CI Results
A confirmed flaky test can block unrelated changes without providing a dependable release signal. Removing that obstruction is reasonable only if someone still sees…
A checkout test says “delivery works,” but two reviewers disagree about whether a discount should remove free shipping. Adding more clicks to the test will not settle that question. The team first needs an agreed rule, a concrete example and an outcome that exposes an incorrect interpretation.
Gherkin gives those examples a recognizable structure: Given describes the starting context, When identifies an event and Then states the expected result. In Cucumber, a Scenario is a concrete example of a business rule, not merely a high-level item on a coverage checklist. Clear wording lets product, development and QA review the same behavior before anyone implements the automation.
Use this hypothetical shop throughout the examples. Standard delivery to an eligible domestic address costs USD 4.00 when the merchandise subtotal is below USD 50.00. At USD 50.00 or more, standard delivery costs USD 0.00. Eligibility uses the merchandise subtotal after discounts and before tax. These amounts and policies are invented teaching assumptions, not a recommendation for a real retailer.
That short contract answers several questions hidden inside “free delivery above the threshold.” Does exactly USD 50.00 qualify? Are discounts subtracted first? Does the rule apply to express delivery or every address? The examples answer the first two questions and deliberately limit the delivery method and address. Express delivery and other regions need their own agreed rules.
Discuss the rule with a product reviewer and a developer before choosing step wording. List concrete examples beneath it, then keep unanswered questions separately. If nobody can explain how a discount affects eligibility, writing a precise-looking Then would manufacture a requirement rather than clarify one.
Choose USD 49.99, USD 50.00 and USD 50.01 as initial examples at cent precision. They expose a potential error at the threshold without pretending to cover every cart. Tax treatment, rounding, excluded products and changing an address may need more examples after the team agrees what those behaviors should be.
Start with the exact threshold. The example below stops at delivery pricing; placing an order and confirming payment are different behaviors. Each displayed line is a separate Gherkin line. Later snippets belong under this same Feature, rather than being complete files on their own. The surrounding explanation is not part of the feature file.
Feature: Standard delivery pricing
Scenario: Free standard delivery starts at the qualifying subtotal
Given a cart has a merchandise subtotal of USD 50.00 after discounts and before tax
And the delivery address is eligible for standard delivery
When the customer selects standard delivery
Then the displayed delivery charge is USD 0.00
The Given establishes a known starting state, including the calculation basis that decides eligibility. The And adds another precondition without repeating Given. When names the event that applies the rule. Then identifies something the customer can observe and a value against which it can be checked.
Feature groups related examples, and Scenario names this particular example. Notice the colons after those labels, but not after Given, When, Then or And. Follow the demonstrated English forms rather than adding punctuation to every keyword. A scenario name should explain the behavior, so a report containing its name is more informative than “Checkout test 1.”
One behavior does not require exactly one Then assertion. If a delivery selection must also appear on the order review, an additional And can check that related outcome. Adding payment authorization, stock allocation and confirmation email checks would make this pricing example harder to diagnose. Split those into examples with their own triggers and conditions.
“Then the delivery is correct” leaves the tester to invent correctness. “Then the displayed delivery charge is USD 0.00” fixes the expectation. If the application exposes the pricing decision through an agreed API response instead of a screen, name that response and its expected charge. Choose an output relevant to the user or consuming system, rather than a private implementation detail.
A procedural draft might say: open the shop, click a product, add it to the cart, open checkout, select an address, click standard delivery and check the result. It describes navigation but still omits the subtotal basis and expected charge. A layout change could invalidate the wording even when the delivery rule stays the same.
Move incidental navigation, selectors and fixture construction into the implementation or a separate manual procedure. Keep the business facts that explain the decision in the scenario. Declarative wording is useful when it removes irrelevant mechanics; it becomes vague when it removes the amount, customer state or eligibility condition that makes the outcome meaningful.
Consider a second illustrative example that checks the discount basis explicitly:
Scenario: A discount takes the cart below the free-delivery threshold
Given a cart has merchandise worth USD 55.00 before discounts and before tax
And a merchandise discount of USD 5.01 has been applied
And the delivery address is eligible for standard delivery
When the customer selects standard delivery
Then the displayed delivery charge is USD 4.00
The after-discount subtotal is USD 49.99, so the fictional below-threshold charge applies. This example would distinguish an implementation that wrongly uses the original USD 55.00 subtotal. Its arithmetic and expected output are visible enough for another reviewer to challenge them.
Keep interaction details when the interaction itself is the requirement, such as keyboard access to the delivery selector. That deserves an appropriately focused accessibility example. Removing all UI language by rule would lose the very behavior that test needs to specify.
The Then implementation also needs an assertion. Reading a charge and printing it in a log does not compare it with USD 4.00. A check of an internal eligibility flag alone could miss a wrong customer-facing charge. Select the observable output first, then implement the assertion that can reveal the failure.
The three subtotal boundary examples share their context, trigger and type of result. A Scenario Outline expresses that common structure while Examples supplies the changing input and expected value:
Scenario Outline: Price standard delivery at the subtotal boundary
Given a cart has a merchandise subtotal of USD <subtotal> after discounts and before tax
And the delivery address is eligible for standard delivery
When the customer selects standard delivery
Then the displayed delivery charge is USD <charge>
Examples:
subtotal | charge |
|---|---|
49.99 | 4.00 |
50.00 | 0.00 |
50.01 | 0.00 |
The table is formatted for reading in this document. In a feature file, write the header and each data row on its own line, with a vertical bar before, between and after the values. For example, the first data row is | 49.99 | 4.00 |. Keep the Examples label immediately above those rows and do not copy a Markdown separator row into the feature file.
The <subtotal> and <charge> names refer to the table's matching column headers. Cucumber substitutes a row's values before matching the steps to their definitions. The template represents three intended executions here, one for each data row, not three recorded passes. Each row needs its own independent setup and actual result when run.
Select rows because they distinguish plausible mistakes. USD 50.00 checks whether the comparison includes the threshold; the neighboring amounts check the outcomes on either side. Adding many similar qualifying amounts may contribute less than an example for a confirmed discount rule. None of these rows establishes coverage of all currencies, addresses or delivery methods.
Do not compress unrelated flows into one outline by adding columns such as “if authorized, pay; otherwise, log in again.” Different actions and failure mechanisms are a reason to separate scenarios. The same selection principle helps with parameterized test cases in Jira, although manual-test datasets and called steps are app-specific mechanisms, not another spelling of Gherkin Examples.
A table beneath a step serves a different purpose from an Examples table. Suppose one scenario needs a cart containing several items. The Given can receive the following Data Table as its argument:
Scenario: Price delivery for one prepared cart
Given the cart contains these items
item | merchandise amount in USD |
|---|---|
Notebook | 30.00 |
Pen set | 20.00 |
And no merchandise discount is applied
And the delivery address is eligible for standard delivery
When the customer selects standard delivery
Then the displayed delivery charge is USD 0.00
Use the same pipe-delimited row convention when transferring this table into a feature file, without the Markdown separator. Both item rows describe one cart for this scenario. The table is passed to the Given's step definition; its rows do not independently generate a scenario each. The implementation must interpret the columns consistently and construct that cart. A Data Table alone does not supply that implementation.
Shared context needs similar care. If all examples require the standard-delivery pricing policy to be enabled, a short Background can express that common prerequisite before the scenarios. Cucumber runs Background steps before each scenario. It is not a one-time setup that lets a later scenario inherit the cart left by an earlier one.
Keep decision-relevant conditions close to the example. Moving the subtotal, address eligibility and discount basis into a long Background makes readers scroll away to understand why a charge applies. Hidden hooks can make the same mistake: technical fixture cleanup belongs there, but a business condition should remain visible when it explains the result. Reset data between examples so the order of execution cannot decide whether they pass.
Writing these examples does not execute the shop. Cucumber needs step definitions that bind step text to implementation code, plus suitable test data, an execution environment and assertions. The implementation may prepare state through controlled fixtures and observe behavior through the application's supported interfaces. Review those choices separately from the wording of the business rule.
Before automation, another person can still read the scenario and challenge its assumptions. For a manual check, supply the verified navigation and environment information the tester needs, then record what happened separately from the expected result. None of the hypothetical examples here is a report of a performed browser, Jira or Cucumber test.
When the threshold changes, revisit the rule, boundary values, fixtures and expected charges together. A previous pass against the old rule does not establish the new behavior. Preserve the earlier execution evidence rather than editing its expected result to match today's policy.
For Jira-based teams, Sherlock Test Management describes an editable testing brief and source-referenced test-case proposals that a person reviews before they enter an approved baseline. A team could use that review boundary to challenge a proposed delivery-threshold change before approving related cases. These are app capabilities, not native Jira functionality or a claim that Sherlock imports or executes the Gherkin shown here. Keep the specification, approved case and eventual execution result distinguishable.
Ask someone who did not write the scenario to explain why its Then follows from its Given and When. If they must guess a fee, discount policy or starting state, resolve that gap before creating more variants. If they can explain the rule but cannot tell what output would fail, improve the assertion.
Use a short review checklist on the actual example:
Begin with one rule that reviewers can challenge and a tester can check without inventing an expectation. Expand the examples when a new condition or risk requires a different decision, not because the syntax makes another row easy to add.
A confirmed flaky test can block unrelated changes without providing a dependable release signal. Removing that obstruction is reasonable only if someone still sees…
A tester reviewing a Jira issue may need a fresh smoke run without opening GitLab and choosing pipeline parameters. A manually triggered Jira Cloud…
A regression-results worker receives HTTP 429 while reading Jira issues. Sleeping that worker may be necessary, but it will not help if other workers…
Create traceable test cases, prepare release-specific test runs, and keep approved coverage current as the product changes.