A regression window is closing, and your Test Plan contains more checks than the team can finish. An AI-generated order can help decide where to begin. It cannot establish that the selected tests cover the release, that a low-ranked check is safe to skip, or that anything has executed.
Xray Cloud's AI Test Prioritization, powered by Sembi IQ, re-ranks existing Tests within a Test Plan or Test Execution. Use that order as an input to a review of release risk. The useful decision is which evidence to obtain first, while keeping mandatory checks and untested scope visible.
This is functionality supplied by Xray in Jira Cloud, rather than a native Jira AI feature. Xray's June 2026 introduction associates AI Test Prioritization with Xray Cloud 15.0.0. Check the capability exposed in your current tenant, subscription and app configuration before relying on a documented route. Do not assume the same feature or limits apply to Xray Data Center.
The dedicated feature guide requires Xray installed on Jira Cloud and configuration by a Jira administrator. Administrators enable or disable prioritization through the AI Hub at the global settings level and can make it available to all users or specific groups. If the control is missing, confirm app availability, enablement and group access with the administrator before investigating the quality of a prompt.
Keep neighboring AI features separate. Xray's Rovo Test Plan Summarizer and AI-generated scripts address different tasks. The Rovo installation requirement described for the summarizer is not listed as a prerequisite for AI Test Prioritization in its dedicated guide.
The supported scope is a Test Plan or Test Execution. After processing, Xray displays AI Priority results and reorders Tests from top to bottom. The documentation describes an applied re-ranking, not a required preview-and-approve stage. Arrange human review around that behavior rather than assuming the application will hold the new order pending approval.
Sembi describes the approach as combining machine-learning analysis of historical risk with AI-driven semantic analysis. The Xray guide lists the user prompt, Test summary and description, categorical metadata, selected labels and historical execution information among the inputs. Metadata includes Test type, kind, labels, component, priority and status. Historical factors include recent and overall failure rates, flakiness, recency of execution and failure, and estimated effort or duration.
These inputs explain why AI prioritization is broader than sorting one field. They do not disclose a complete scoring formula. The reviewed documentation does not establish fixed factor weights, a calibrated defect probability or a guarantee that repeated requests produce identical orders. Keep the generated AI Priority distinct from a team-defined deterministic rule, such as always scheduling a mandatory payment-rejection check before optional compatibility checks.
The Settings control lets you choose an analysis timeframe. Xray states that data older than that timeframe is excluded, and that processing a longer history may take longer. Inspect the choices available in your tenant rather than assuming a particular set of day ranges from a broader product announcement.
Choose the window according to what the history represents. A short window may reflect the current architecture well but omit an older failure that still matters. A longer window may retain useful rare failures while also including incidents caused by a retired environment or obsolete test data. Review those conditions before interpreting a historically failure-prone test as evidence of current product risk.
Flakiness needs similar care. A test that fails intermittently may expose a real timing defect, unreliable infrastructure or a flawed assertion. A ranking can bring it to your attention; it does not establish which explanation is correct. Inspect the relevant results and defect context before changing the execution plan.
Sembi also describes semantic analysis as useful when little or no execution history exists. Treat that as a vendor-described capability, not proof that a new Test receives the right position. When a release introduces new behavior, examine the new checks explicitly instead of letting the absence of past failures stand in for evidence of low risk.
Xray allows up to three selected Jira labels. Tests containing any of those labels receive a slight boost to their priority score. The guide does not define that boost numerically or promise a fixed position for a matching Test.
A label therefore does not establish that all matching Tests execute first, that unmatched Tests are excluded, or that a compliance obligation is enforced. Broad labels applied to nearly every Test also provide little useful differentiation. Check that the selected labels describe the release concern and that their assignment is current.
An illustrative prompt could be: Prioritize verification of changed payment-rejection and retry behavior for the checkout release. Consider recent failures and relevant defect history, while keeping new acceptance criteria visible for review. This supplies context, but it does not create missing tests or enforce a mandatory execution block. Record those requirements separately as team decisions.
The current feature guide and Xray's Cloud explainer document these maximum Test counts for AI Test Prioritization:
Xray Cloud Tier | Documented Maximum Tests |
|---|---|
Standard | 75 |
Advanced | 150 |
Enterprise | 300 |
These figures describe the number of Tests supported by the prioritization feature. They are not stated as monthly credits, execution allowances or total repository-size limits. Do not infer how an oversized request is rejected, truncated or charged without checking current product behavior and documentation.
Compare the intended Test Plan or Test Execution with the applicable cap before preparing the request. If the scope is larger, decide which coherent release scope needs review and confirm the supported handling with your administrator. Avoid silently dropping low-visibility checks merely to fit the limit.
Separately prioritizing smaller groups also introduces a planning problem: a position within one group does not establish its position against a Test in another group. Do not treat separately generated rankings as a globally comparable score table. The person assembling the execution schedule still needs to reconcile dependencies, mandatory coverage and remaining risk across the groups.
The documented entry point is Prioritize Tests on a supported work item. The guide also describes opening a Test Plan or Test Execution through the Testing Board. Labels and screen placement may vary with the tenant's available version, so establish the supported route before making it part of a team procedure.
Use a review sequence that preserves the difference between what Xray changes and what your team approves:
Explanations support investigation; they do not validate the decision. Xray says the hover details help explain some factors behind the assigned priorities. That is narrower than a complete account of model reasoning or proof that the proposed order maximizes defect detection.
After each use, the documented History entry provides visibility into changes to the Test Plan or Test Execution and a high-level overview of the prioritization strategy. Add the contextual information your decision needs, such as the chosen timeframe, labels, release concern, review rationale and required follow-up. Do not assume the History entry alone supplies a reproducible model replay or every detail of your local approval process.
The guide also states that navigating away does not stop an ongoing prioritization. If you intend to cancel it, use the documented cancellation interaction rather than treating a closed page as cancellation. Coordinate ownership so another reviewer does not mistake an in-progress or newly reordered list for an approved execution schedule.
Consider a fictional checkout release that changes how rejected payments and repeated submissions are handled. The Test Plan includes payment-rejection checks, a retry/idempotency check, an older refund regression and a browser smoke test with intermittent failures. This is an illustrative review situation, not an observed Xray output or customer result.
The recent history may draw attention to the browser smoke test because it has failed repeatedly. Before promoting it ahead of everything else, the reviewer investigates whether those failures represent a checkout defect or unstable setup. The new retry requirement may have little history, so its significance must also be evaluated against the changed acceptance criteria.
An older refund failure creates a different question. If the selected timeframe excludes it, is that reasonable because the affected behavior is unchanged and covered elsewhere, or does it hide a still-relevant risk? The timeframe decision should follow that assessment. Extending the window without understanding the failure simply adds more history to interpret.
Suppose the team also uses a payment label across both critical rejection checks and low-impact display checks. Selecting it can provide a boost, but cannot express the full business distinction between those cases. Improve the descriptions and review the individual checks instead of expecting the label to encode the release policy.
The team may decide to schedule mandatory rejection and retry checks first, investigate the intermittent smoke failures, and retain the refund check in the remaining release scope. Those are human decisions based on the fictional release conditions. They are not a predicted Sembi IQ ranking, an invented score or evidence that the tool enforces that schedule.
Re-ranking Tests does not execute them. A Test Execution is a planning and results context; the existence of an AI-generated order does not establish that a manual tester followed it or that an automation runner consumed it. Confirm any downstream execution connection independently before assuming the displayed order controls a CI job.
For manual work, assign the approved checks and record what was attempted and what happened. For automated work, verify the actual selection, scheduling, result collection and failure handling of the execution system. A successful prioritization request is not a passed test result, and a completed subset is evidence only about the checks that actually ran.
Keep execution ownership and triggers explicit when turning the reviewed order into work. A broader test automation strategy helps define the runner boundary, required evidence and response to missing or unreliable results. Those operating decisions still need to be established separately from the ranking feature.
At release review, show the executed scope, actual results, unresolved failures and checks that remain unexecuted. A low position in the list is not a reason by itself to accept their risk. The responsible decision-maker needs the requirement or business consequence behind any omission, along with the person accepting it.
Begin with an approved Test Plan or Test Execution whose scope and history the team understands. Retain the prior scope, prioritization inputs, resulting order, explanations examined and review decisions. Observe whether the suggestions help reviewers identify relevant checks and whether they repeatedly overlook new or mandatory coverage.
Judge later outcomes against actual execution evidence, not the plausibility of an explanation or the absence of defects in a small sample. Keep review effort and unresolved risks visible alongside any observed benefit. Expand use only when the team can explain how it handles misleading history, missing context and required checks that rank unexpectedly low.
The release owner should be able to answer two questions before work proceeds: which checks must run regardless of the suggested order, and who accepts the risk of those that will not? AI prioritization is useful when it helps that decision without obscuring either answer.
Two reports can describe the same symptom while disagreeing about where the failure occurs. If a reviewer accepts a duplicate suggestion because the titles…
A checkout test times out while waiting for an order confirmation. The runner reports an assertion failure, the application records a failed request, and…
A recipient-preview rule changes, but the regression suite still checks the earlier behavior. An AI agent may choose a different route through the editor…
Create traceable test cases, prepare release-specific test runs, and keep approved coverage current as the product changes.
