A Jira story is Done, a test case is linked, and the related bug is Closed. Can the release owner conclude that the story's acceptance criteria passed on the build being shipped? Those records alone do not identify what was asserted, which specification was used, or whether anyone retested the fix.
A Jira requirements traceability matrix connects a defined set of requirements to test cases, execution results, and defects. Its value comes from preserving enough context to distinguish planned coverage from current evidence. For a QA lead, the useful output is a list of requirements that still need design, execution, investigation, or an explicit release decision.
Start with the unit you intend to count. A story can contain several acceptance criteria, and one linked test does not establish that every criterion is asserted. For detailed coverage, identify each testable clause within its source story and retain the reviewed requirement revision. A story-level summary can sit above those clauses, but it should not conceal their gaps.
Keep five identities separate:
A case specifies what should happen; its result records what happened during a particular execution. Creating the case or scheduling it in a run supplies no passing evidence. The distinction between test cases, suites, and runs helps when reusable definitions and release records have become mixed together.
Allow many-to-many mappings when the assertions justify them. One clause may need positive, negative, and boundary checks. One end-to-end case may assert several clauses. Record which assertions support each relationship rather than linking a broad checkout test to every story it touches. A failed case covering several clauses also needs enough result detail to identify which expectation failed.
Build the requirement inventory independently of the tests. Otherwise, a report assembled only from existing case references cannot show a requirement nobody linked. Preserve the selected source keys and revisions alongside the selection rule so later edits do not silently change the denominator.
Consider a fictional, vendor-neutral checkout release. The reviewed requirement baseline is B3, the release is 4.2, the candidate build is 701, and the environment is QA-EU. RUN-701 is the selected testing round. The eight clauses below are the complete requirement inventory for this illustration, not a claim about complete checkout coverage.
All linked cases are selected for this round. Their reviewed definitions are revision 1; TC-05 is the exception that still reflects baseline B2 and needs revision. There is one required configuration per case. Evidence identifiers refer to invented explanatory records, not tests performed in an application.
Requirement Clause | Behavior to Verify | Case | Run and Outcome | Evidence or Defect | Next Action |
|---|---|---|---|---|---|
R1: ORD-101/AC1 | Authorized checkout creates one order | TC-01 v1 | RUN-701: Pass | E-01: expected order observed | Retain current evidence |
R1: ORD-101/AC1 | Timeout recovery still creates one order | TC-02 v1 | RUN-701: Not Run | No result | Execute recovery check |
R2: ORD-101/AC2 | Unauthorized checkout is rejected | TC-03 v1 | RUN-701: Pass | E-03: rejection observed | Retain current evidence |
R3: ORD-102/AC1 | Payment retry never captures twice | TC-04 v1 | RUN-701: Fail | E-04: duplicate capture; BUG-900 Closed | Retest deployed fix |
R4: ORD-103/AC1 | Expired session cannot place an order | No case | No selected execution | No evidence | Design and review a case |
R5: ORD-104/AC1 | Partial refund follows the revised amount rule | TC-05 v1 | RUN-700: historical Pass; no current result | E-05: build 700, baseline B2 | Update case and execute |
R6: ORD-105/AC1 | Refund reaches the original payment method | TC-06 v1 | RUN-701: Blocked | E-06: sandbox unavailable | Restore prerequisite and execute |
R7: ORD-106/AC1 | Timeout recovery writes the required audit entry | TC-02 v1 | RUN-701: Not Run | Same unexecuted recovery case | Execute and inspect audit assertion |
R8: ORD-107/AC1 | Currency conversion uses the agreed rounding rule | TC-08 v1 | RUN-701: Skipped | Recorded scheduling decision | Execute or seek a scoped exception |
R1 occupies two rows because the team requires both ordinary checkout and timeout recovery. TC-02 appears under R1 and R7 because its specification contains separate order-count and audit-entry assertions. It remains one selected case, not two executions. Passing TC-01 therefore does not satisfy every required check for R1.
The gaps require different actions. R4 is uncovered because no case exists. R7 has a linked case but no execution result. R6 has a recorded blocker, and R8 has an explicit skip. Neither blocked nor skipped is a passing behavioral observation. Keeping those categories separate lets the QA lead assign design work, environment repair, execution, and exception decisions to the right owners.
R5 demonstrates stale evidence. Its old pass remains useful history, but it tested a different build and requirement baseline. The association to TC-05 counts as a link; it does not establish that the current refund rule has a suitable case or passing evidence.
For R3, closing BUG-900 does not replace E-04 with a pass. A new qualifying result must show whether the deployed correction resolves the duplicate capture. Conversely, a historical defect link can remain after successful confirmation, so the existence of a linked bug does not by itself mean the current test is failing. Read the result, build, defect history, and retest together.
Agree on reporting predicates before reviewing percentages. For each selected case and configuration, this example uses its latest recorded state in RUN-701 at the review cutoff, retaining earlier attempts as history. That state counts as a current completed execution only when it records Pass or Fail against build 701, QA-EU, the applicable B3 requirement, and an accepted case specification. Historical, Blocked, Skipped, and Not Run entries do not satisfy that predicate. A later blocked attempt cannot silently inherit an earlier pass.
Executed requirement coverage means a distinct clause has at least one qualifying completed check. It does not mean every required check for that clause completed. For fully passed requirement coverage, the example team uses a stricter policy: every required case and configuration for the clause must have a qualifying current Pass. Missing or stale evidence prevents that classification. These are declared example policies, not universal Jira, AIO Tests, or TestRail aggregation rules.
Metric | Numerator | Denominator | Example Result |
|---|---|---|---|
Link coverage | Distinct clauses with at least one associated case | All eight in-scope clauses | 7/8 = 87.5% |
Executed requirement coverage | Distinct clauses with at least one current completed check | All eight in-scope clauses | 3/8 = 37.5% |
Fully passed requirement coverage | Distinct clauses satisfying the all-required-checks policy | All eight in-scope clauses | 1/8 = 12.5% |
Selected-case completion | Distinct selected cases with a current Pass or Fail | All seven selected cases | 3/7, approximately 42.9% |
Pass rate among completed cases | Distinct current passing cases | The three current completed cases | 2/3, approximately 66.7% |
R1, R2, and R3 have current completed checks. Only R2 satisfies the full-pass policy: R1 still needs TC-02, and R3 failed. The current completed cases are TC-01, TC-03, and TC-04. TC-02's repeated appearance must not inflate either the case count or execution count.
Count distinct requirement identities rather than displayed join rows. This matrix has nine display rows, including the no-case row, but eight clauses and seven selected cases. Joining requirements to multiple cases, attempts, configurations, or defects can multiply rows without adding any requirements.
These figures describe the selected inventory and its evidence, not code coverage or release safety. A high link percentage can coexist with a missing security case, outdated expectations, and blocked critical behavior. Even full passing coverage under a declared policy cannot prove that the requirements or chosen assertions cover every relevant risk.
Requirements can remain in Jira while specifications and results live in different systems. Choose the matrix implementation by where those records are maintained, how the links are resolved, and which reporting settings determine the answer.
Jira Cloud supports configurable work-item relationships. Linking must be enabled, and users need the relevant linking permission. A team can agree on a tested-by relationship and use it consistently, but that name is a configured convention rather than a universal default.
A custom Test work type can represent a manual test specification. Creating that type does not automatically provide structured steps, case versions, run instances, execution outcomes, or a formal RTM report. If the team maintains those records manually, define their fields and retention procedure explicitly. A Jira story reaching Done is not execution evidence.
For a small controlled scope, maintain the matrix from the complete requirements inventory, then attach the case and result references. Keep requirements with no matches visible. Treat the matrix as a reviewed snapshot with a cutoff and owner, not a collection of links assumed to remain accurate indefinitely.
Native JQL can retrieve direct linked work items through linkedIssues for an identified key. Do not treat that function as an automatic multihop join across requirements, cases, results, and defects. Enhanced functions supplied by apps such as ScriptRunner belong to that app's search model, not native JQL.
AIO Tests supplies an app-managed testing and reporting layer. Its Traceability Summary and Traceability Detail reports accept requirements through an issue list, JQL, or a saved filter. Their settings also determine which child issues, case versions, archived cases, and execution cycles appear.
The Summary documentation defines coverage as requirements mapped to cases and presents execution distribution separately. The Detail report can show requirements, cases, execution results, and defects, but run and defect retrieval must be included in the selected report configuration. A two-column requirement-to-case view cannot establish execution coverage.
Review history settings deliberately. Detail can show only the last run or multiple runs; Summary offers Last Run and All Runs strategies across selected cycles. A current-state view and a historical view answer different questions. Neither should be interpreted without the intended build, environment, and scope, and a later valid pass should not erase access to the earlier failure.
TestRail uses a separate model: cases contain specifications, runs select tests for a testing round, and results record outcomes. Jira requirement keys in a case's References field establish requirement associations. Jira bug keys in a result's Defects field establish defect relationships. Those two links serve different purposes.
Its reports also separate questions. Coverage for References (Cases) examines case associations; Comparison for References (Results) examines results for selected runs; Summary for References (Defects) examines related defects. When looking for untested Jira requirements, supply the independently reviewed requirement list rather than assuming all references already found on cases represent the complete release. Review report filters and size limits as well.
Configuration affects what is visible. TestRail's Cloud integration requires suitable administrative privileges and separately configures References and Defects. Current Cloud documentation describes selected Jira field display and optional webhook synchronization; that does not establish automatic synchronization of every case, result, or requirement version. Data Center setup has different deployment and connectivity prerequisites.
Check access boundaries before sharing the matrix. TestRail warns that stored Jira field values may be visible to users who can access the relevant TestRail entity regardless of their Jira permissions. Seeing results inside Jira also depends on the configured integration and appropriate TestRail access. Verify the documented behavior for your deployment rather than assuming the same settings apply everywhere.
Sherlock Test Management offers another app-specific approach to preserving source and execution context. Its current product description separates source-linked, approved baseline versions from release-specific runs and says finished runs remain attached to the baseline version executed. For the stale refund case, that separation can help reviewers distinguish an old specification from a proposed update. The Sherlock product workflow describes this model; it does not establish a dedicated RTM report or an AIO Tests or TestRail integration. AI-proposed edits still require human acceptance and are not execution evidence.
When the refund requirement changes from B3 to B4, identify the affected assertions, review the proposed case revision, and decide which runs need new evidence. Keep the old source and executed specification recoverable. Replacing a case's expected amount must not make a build-700 pass appear to validate the new rule. A test-case approval and change-control workflow helps define which revision was reviewed and when approval needs reconsideration.
Review traceability in both directions. From each requirement, inspect the associated assertions, selected execution context, current results, and unresolved gaps. From each selected case, inspect its source clauses and ask whether its steps still test the intended behavior. A case with no source may represent a missing requirement, legitimate cross-cutting risk coverage, or obsolete work; investigate rather than deleting it automatically.
For this release example, assign owners to R4's missing design, R5's stale specification, R6's sandbox dependency, and the unexecuted or skipped checks. Retest R3 on the corrected candidate while preserving its failure record. If scope changes or an exception is accepted, retain the decision, owner, affected clause, rationale, and remaining uncertainty instead of silently removing the row.
Before signing off, choose one critical clause and ask another reviewer to recover its approved source, required checks, executed revisions, candidate, result evidence, and any confirmation after a defect fix. If that chain cannot be recovered, repair the evidence or record the unresolved risk before relying on a green coverage percentage.
A green automated test history can help a developer identify a safe-looking change while leaving a QA lead unable to answer a different question:…
A checkout test appears under Payments, again in a Smoke folder, and once more under the latest release. When the payment rules change, someone…
A refund feature can pass technical tests while leaving a finance team unable to complete its daily work. The calculation may be correct, but…
Create traceable test cases, prepare release-specific test runs, and keep approved coverage current as the product changes.
