Jira Test Execution Burndown: Track Remaining Tests, Not Pass Rates

A test-execution burndown can reach zero while failed results still demand investigation. In the AIO Tests documentation's worked example, both passed and failed cases reduce the pending count. A falling line therefore answers a narrower question than a release reviewer usually needs: how much of the selected execution work remains?

For a QA lead, the useful response is to read remaining work alongside outcomes, scope and dates. Keep the execution-progress conversation separate from whether defects are acceptable, required coverage is complete and a person with release authority has approved the remaining risk.

Identify the Work Behind Each Signal

AIO Tests is an installed testing app in Jira. Its current task-specific documentation calls the report the Multi Cycle Execution Burndown Report and describes case counts over the selected cycle timeline. Native Jira sprint burndown has a different population and completion rule. A dashboard that places them side by side should name those populations rather than label both charts as testing completion.

Atlassian's company-managed Scrum documentation describes a board-specific chart: it includes work items matching the board's saved filter, follows a selected sprint and uses the configured estimation statistic. A work item counts as Done through its status mapping to the board's rightmost column. That workflow mapping does not establish a passed app test. Story Points on subtasks are excluded; subtask time-estimate behavior also depends on the board's remaining-estimate and time-spent setting. Do not transfer those rules to an AIO case count.

Use the following boundaries when explaining a report to someone who did not configure it.

Signal	Counted Work and Scope	Decision Boundary
AIO Tests execution burndown	Cases remaining in selected app cycles; verify the report and status rules.	A failed execution can reduce pending work without becoming a Pass.
Native Jira sprint burndown	Board-filtered sprint work items and the board estimation statistic.	Rightmost-column Done is a Jira workflow mapping, not an app test result.
Local illustrative pass percentage	Latest Pass divided by Pass plus Fail in the declared fictional membership set.	This denominator is authored here, not asserted as an AIO formula.
Release readiness	Required coverage, unresolved defects, retesting and authorized risk acceptance.	A zero pending count is insufficient evidence for release approval.

The local percentage in this comparison is defined for the fictional example below. It is not a claim about an undocumented AIO pass-rate calculation. Before comparing charts, confirm whether each number counts cases, case memberships in cycles, recorded attempts or Jira work-item estimates. Matching dates do not make different units interchangeable.

Configure the AIO Report Before Comparing Lines

Check the report name and available inputs in your tenant before following a recipe. AIO's vendor blog contains both an Execution Burndown procedure and a Multi Cycle procedure. The current task-specific knowledge-base page supplies the Multi Cycle steps used here; those two descriptions do not prove that every edition or tenant exposes identical screens.

For a compatible Multi Cycle report, use the documented inputs deliberately:

1. Open the report using the chevron next to Multi Cycle Execution Burndown.
2. Select the desired projects and cycles. Narrow the cycle list with the relevant filters, such as Component, Release, Cycle Tags or Folders, when needed. Check that the resulting selection matches the execution scope you intend to discuss.
3. Review Show Cycle Plan data, which is selected by default. Record whether you are including plan data so another reviewer can interpret the same comparison.
4. If appropriate, specify a Guideline end date for completing execution across the selected cycles. Otherwise, the documentation says the guideline is evaluated using cycle end dates.
5. Select Generate, then check the cycle overview and the reported scope before interpreting the curve.

Treat this configuration as part of the report's meaning. A later selection of different cycles is a different comparison, even if the chart title looks familiar. Keep the selected projects, cycles, filters and date assumptions with the snapshot used in a review.

Check Dates and Plan Data

The knowledge base says the guideline is not shown if one or both dates are missing. It also says the guideline appears only when the specified cycle has no plan, and describes a straight line for a cycle without a plan. Check both dates and plan state before treating an absent straight guideline as a broken report.

The page describes the end date's 6 pm as the endpoint, but the selected text does not identify a timezone. Confirm the relevant time boundary before comparing an end-of-day export, a tester's local date and the chart. Do not assume midnight or a universal timezone from that wording.

For several cycles with different dates or plans, the documentation reviewed here does not establish a complete interpolation or aggregation algorithm. An optional common end date is documented; a universal formula for every mixed-cycle configuration is not. Validate an ambiguous comparison in the applicable report before using the apparent distance from a line as evidence of delay.

Reconcile Pending Cases With Recorded Results

AIO's published example starts with 125 pending cases. On September 18, 10 pass and 1 fails; on September 19, another 17 pass and none fail. The pending calculation is 125 minus the 28 cases with those recorded results, leaving 97 pending cases. The example's cumulative outcomes are 27 Pass and 1 Fail. The failed case contributes to reducing pending work even though its result remains a failure.

The page describes Day Total as the number of cases passed or failed on a specific day. Its Cumulative Total wording is inconsistent: the prose refers to summing remaining cases, while the adjacent worked example adds passed counts and failed counts separately across days. Use the explicit pending arithmetic above, and reconcile the actual column labels and values in your report before building a cumulative formula. That wording conflict does not justify treating every cumulative column as either remaining cases or execution attempts.

A practical reconciliation starts with an unchanged scope and checks which recorded results account for the decrease. If the count differs from your expectation, inspect membership, filters and result history before concluding that the arithmetic is wrong. The selected documentation does not settle how every custom status, repeated attempt, historical correction or case shared across cycles affects the calculation. A locally convenient rule is not evidence of the app's behavior.

Follow a Falling Burndown With Worsening Results

Consider a fictional, unexecuted design illustration for a checkout regression cycle C1. It contains 12 unique case memberships, T01 through T12, with one membership per case and no scope additions or removals. These are authored inputs and calculated illustrations, not results observed in Jira, AIO Tests or an application.

The local rule removes a membership from pending when it receives Pass or Fail. Not started, Blocked and In progress remain pending in this illustration only; their treatment is not asserted as an AIO status mapping. The latest-membership pass percentage is Pass divided by Pass plus Fail. It excludes pending memberships from that denominator and is undefined when there are no Pass or Fail results.

Read the Daily Inputs and Denominators

At day 0, all 12 memberships are Not started, so 12 remain pending. There are no executed memberships and no defined pass percentage. Converting that empty denominator into a perfect score would hide the absence of evidence.

On day 1, T01, T02 and T03 receive Pass, while T04 receives Fail. Fictional day 1 has 8 pending memberships, 3 Pass and 1 Fail. The latest-membership percentage is 3 divided by 4 executed memberships, or 75%. The remaining-work count has fallen by 4, although one of those first executions failed.

On day 2, T05, T06, T07 and T08 all receive Fail. Day 2 has 4 pending memberships, 3 Pass and 5 Fail. The latest-membership percentage is now 3 divided by 8 executed memberships, or 37.5%. Pending work has halved since day 1, while the proportion of executed memberships with a passing latest result has fallen.

On day 3, T09 receives Pass and T10, T11 and T12 receive Fail. Day 3 reaches zero pending with 4 Pass and 8 Fail. The latest-membership percentage is 4 divided by 12 executed memberships, or 33.33%, rounded to two decimal places. Every membership has a first result; most latest results are failures. Zero pending has not resolved those failures or authorized a release.

For this illustration alone, a constant-rate guideline is 12 multiplied by (3 minus day), divided by 3, for days 0 through 3. Its day-end pending targets are 12, 8, 4 and 0 memberships, respectively. The fictional actual count matches that line at every snapshot while latest-result quality worsens. This authored line is neither an AIO algorithm nor a forecast, SLA or commitment, and it is not extended beyond day 3.

Keep a Recheck Separate From First Execution

On day 4, the illustration rechecks T04 and records Pass in place of its latest Fail. Scope remains fixed at 12 memberships. Pending stays at zero, while the latest outcomes become 5 Pass and 7 Fail. The latest-membership percentage is 5 divided by 12, or 41.67%, rounded to two decimal places.

Retaining every recorded attempt gives a different denominator: the original 12 first attempts plus the additional recheck make 13 attempts, of which 5 are Pass. The latest-result fraction is 5/12 while its attempt fraction is 5/13. That attempt-based percentage is 38.46%, rounded to two decimal places. Both fractions are correct within their declared local populations, but they answer different questions.

This recheck changes outcome evidence without reducing an already empty pending set. It illustrates why retesting work needs explicit visibility beyond first-execution progress. It does not establish how AIO rewrites historical points, counts attempts or handles reruns in a particular configuration. Nor does one improved result establish that the remaining failures, required coverage or release risks are acceptable.

Investigate the Cause Before Changing the Plan

A curve shows a count relative to a timeline; it does not identify the cause of a deviation. AIO's vendor guidance lists blockers, environment issues, defects, limited capacity and scope changes as possible reasons to investigate. Treat them as hypotheses until the relevant execution evidence supports one.

When the report looks unexpected, follow a short diagnostic sequence rather than immediately moving the finish date:

• For a flat line, check whether selected cases remain unexecuted and whether results have been recorded promptly. Ask about unavailable builds, data or access, but do not infer a specific blocker from the line alone.
• For an actual line above the applicable guideline, confirm scope, dates and plan inputs first. Then identify the outstanding memberships and the constraint preventing their execution before proposing a capacity or priority change.
• For a sudden count change, compare cycle membership and filters with the prior snapshot, then inspect result updates. Do not describe a lower count as completed testing if work was removed from scope.
• For a missing guideline, inspect dates and cycle-plan state against the documented conditions. Do not substitute native Jira's guideline troubleshooting for an AIO report with different inputs.

Where a case appears in several cycles, establish the report's identity and counting rule before adding cycle totals together. Likewise, check custom statuses and retest handling before comparing an app export with a locally built calculation. Document what remains unknown instead of forcing the observed figures into the fictional rule above.

If a scope or date change is needed, preserve the reason and the before-and-after basis. Otherwise, a revised baseline can make the chart look healthier while obscuring the original execution problem. A plan change should support an explicit decision about work, capacity or risk, not merely reduce the visible gap.

Use Remaining Work in a Release Review

Bring a bounded snapshot to the review: selected cycles and filters, counted unit, observation time and timezone, pending count, recorded outcomes, and the applicable plan or guideline dates. Keep the status and denominator definitions beside any percentage. This lets another reviewer explain a change without guessing whether the population or the result history changed.

Pair that progress snapshot with evidence about required coverage, unresolved defects, their severity and impact, retest obligations, and any proposed acceptance of residual risk. AIO's own vendor guidance recommends combining burndown with defect trends, execution status and coverage. None of those inputs is replaced by a line reaching the horizontal axis.

The person maintaining execution results can explain what remains; the authorized release decision-maker must assess the evidence against the team's actual criteria. Before the next status meeting, reconcile one unexpected point to its selected cases and recorded results, then state separately what the curve establishes and what still requires a quality or release decision.