A checkout requirement changes after regression testing has started. Your QA lead needs to update the reusable test, preserve the failed execution, and show which release still needs retesting. Both Xray and Zephyr can participate in that workflow. The buying decision depends on how their test objects, execution records, permissions, and licensed capabilities fit your Jira setup.
Xray is a strong candidate when testing artifacts need to follow Jira work-item workflows. Current Zephyr Cloud offerings deserve close evaluation when teams prefer an app-managed test library and cycle-based planning, particularly if built-in no-code execution is a requirement. Neither choice follows from the product name alone: edition, deployment, and the exact task determine the comparison.
The capabilities and Cloud subscription examples below were checked on October 16, 2026. They describe vendor-documented behavior, not a hands-on performance benchmark.
“Zephyr” can refer to different generations and deployments. SmartBear now presents a family comprising Zephyr Essential, Zephyr Standard, and Zephyr Advanced. Older references to Squad and Scale still appear in documentation, integrations, and procurement records. Some current documentation calls the middle offering “Zephyr” rather than “Standard.”
For Cloud, SmartBear distinguishes the current Essential experience from Legacy Zephyr Squad Cloud. A renamed subscription does not prove that an existing site has moved to the newer experience. Check the installed app and actual interface with the administrator before using a feature matrix or migration guide.
Xray has a different commercial structure. Xray Test Management for Jira offers Standard and Advanced editions, while Xray Enterprise has an independent Marketplace listing. Current Cloud guidance says Enterprise does not require a second paid base-app license. During evaluation, confirm which subscription will remain active rather than adding the two prices together by default.
Keep Jira Cloud and Data Center evaluations separate. Xray documents different Cloud and Data Center APIs and authentication. SmartBear also publishes deployment-specific differences, including Cloud no-code automation that its Data Center comparison does not list as available. A Cloud edition's entitlement is not evidence of the same capability on Data Center.
Record the app listing, hosting option, edition, installed version where applicable, and migration origin on every trial scorecard. This prevents a legacy installation from being compared with a new Cloud trial as if they were identical products.
Use an illustrative checkout release that changes discount eligibility and introduces a payment retry. Give each trial the same requirement, manual test, failed result, and automated result file. Evaluate whether a teammate can recover the evidence without help from the person who configured the trial.
In Xray, Tests, Test Sets, Test Plans, and Test Executions are Jira work-item types. This lets teams apply Jira workflows and issue relationships to those artifacts. The individual Test Run, however, is an internal Xray entity within a Test Execution. Calling every Xray object a Jira issue would misrepresent the data model.
Xray copies the test specification into the run so a later edit does not automatically rewrite the existing run. There is an important boundary: its documentation says general custom fields, summary, and description are not replicated into that snapshot. Put execution-critical instructions in supported specification fields, then verify what the historical execution retains.
Current Zephyr Cloud test cases are app-managed objects rather than the Jira Test issues associated with legacy Squad. Its version-control documentation describes creating and comparing case versions while older versions retain their execution data. The library model can suit teams that want to maintain test assets within the testing app, but they must configure the app's controls rather than assume every case follows a Jira issue workflow.
For the checkout exercise, change the expected discount result after recording a failure. Open the earlier execution and check its steps, expected result, evidence, and defect link. Then prepare a new execution using the revised case. A stored execution snapshot and multiple independently active case versions are different requirements. Xray Enterprise's explicit Test Case Versioning should be evaluated when the latter is essential; basic execution history alone does not answer that need.
Xray groups execution work through Test Execution issues and can organize scope using Test Plans and Test Sets. Zephyr organizes reusable cases into test cycles, assigns testers and environments, and groups cycles into plans for a broader release view. Similar labels do not mean the objects have identical relationships.
For each trial, select the checkout cases needed for the release, assign a tester, identify the environment, and link the requirement. Execute the payment-retry case, record the failure, and link a Jira defect. Ask a release manager to trace the defect back to the requirement and the particular execution, not merely the latest test case.
Next, rerun the failed case after a fix. Zephyr documents starting a new execution that archives the previous result. Do not interpret migration documentation about missing execution audit logs as proof that Zephyr cannot retain previous execution results. A rerun result, an edit-by-edit audit trail, and data transferred from another product are separate evidence questions.
Evaluate reporting with the release and environment fixed. A repository-wide pass rate may combine unrelated work, while requirements coverage measures a different property from code coverage. A useful report should let the reviewer distinguish unexecuted critical cases, unresolved failures, and results from an earlier build. Agree on those acceptance criteria before judging either dashboard.
Separate three tasks in the evaluation: generating test content, running tests against the application, and importing results. Purchasing one capability does not establish the other two.
Xray Standard's current listing includes AI test-case generation, AI prioritization, and automation integrations. Advanced adds AI Test Script Generation. Xray's script documentation requires structured Test work items, and generated output may contain placeholders needing environment-specific replacement. Review the code, complete its configuration, and execute it in the intended runner before treating it as working automation.
Zephyr's edition matrix lists external framework and CI/CD integration across the family. Built-in Record and Play and no-code automation belong to Standard and Advanced, not Essential. The recorder also requires AI features to be enabled, and private or local environments need the documented connectivity setup. Advanced adds capabilities such as cross-browser and parallel execution; confirm the relevant limits in the actual subscription.
Usage is part of the decision. SmartBear's current no-code table lists 200 monthly automated executions for the 11–100 Jira-user band in the middle offering and 800 in Advanced. These limits concern the hosted automation feature, not a universal cap on importing external CI results. Rerunning cases consumes usage, unused executions do not roll over, and reaching the allowance can prevent further hosted execution until reset.
If your team already maintains a reliable external automation suite, compare the integration work before buying additional hosted execution. Import the same representative result file into both trials, rerun the build, and inspect test identity, environment, attachments, and duplicate handling. This exposes mapping and maintenance effort that a list of supported frameworks cannot show.
Marketplace app pricing for Jira generally follows the maximum user count among the Jira apps on the site, rather than the number of testers who open the testing app. A small QA team on a large Jira site should price the site's licensed population. Atlassian also states that a Marketplace app's billing cycle must match the parent app. Its Marketplace billing explanation is the starting point for checking your site's basis.
The following totals were read from the official Marketplace Cloud pricing calculators with 100 users entered. All figures are USD app-subscription totals, excluding Jira itself and tax. Monthly and annual options were checked separately; the annual figures are annual subscriptions, not monthly prices multiplied by twelve. They are a dated comparison point, not a quote for another site, discount, renewal, or deployment.
Cloud Offering | Monthly Total, 100 Users | Annual Total, 100 Users |
|---|---|---|
Xray Standard | USD 728 | USD 7,280 |
Xray Advanced | USD 874 | USD 8,740 |
Xray Enterprise | USD 1,320 | USD 13,200 |
Zephyr Essential | USD 599 | USD 5,990 |
Zephyr Standard | USD 681 | USD 6,810 |
Zephyr Advanced | USD 873 | USD 8,730 |
These rows are commercial offerings, not feature-equivalent bundles. Zephyr Essential's lower price does not include Standard's hosted no-code capabilities. Xray Enterprise addresses requirements such as multiple active test versions that should not be assumed from an Advanced label elsewhere. Match the needed workflow before comparing the subscription totals.
For a defensible purchase request, retain a quote with these fields:
Then separate recurring licensed spend from implementation cost. Your full budget may include Jira, the selected testing app, external runners or browser infrastructure, migration assistance, and administrator time. Existing Jira spend can be shared baseline cost in a replacement decision, but it remains part of the organization's total bill.
Do not infer that an AI-generated script replaces CI infrastructure, or that a no-code allowance covers every planned execution. Recalculate with your actual workload and population, and request deployment-specific quotes for Data Center rather than extrapolating from this Cloud table.
The data model affects who maintains the configuration. With Xray, Jira issue types, screens, workflows, permissions, and the app's own settings all enter the design. With current Zephyr Cloud, test-specific settings and permissions govern app-managed assets alongside Jira access. Neither model guarantees less administration for every team.
Zephyr's permission documentation says its optional permission system is disabled by default, allowing users with Jira-space access to perform test-related actions. Check that setting explicitly during the trial. Test a reviewer and a restricted tester account rather than approving a configuration demonstrated only by an administrator.
Use the same administrative acceptance tests in both products:
Treat these as trial requirements, not promises that each permission boundary is available unchanged in every edition. Record any workaround and its owner. An extra administrative step may be acceptable if it preserves a control that the team needs.
Migration requires a separate rehearsal. Xray's Zephyr Scale-to-Cloud guide warns that the XML export does not carry test-case attachments and identifies custom fields, statuses, preconditions, and organizational structures needing attention. A successful case import therefore cannot establish that cycles, historical evidence, and requirement relationships are complete.
SmartBear's legacy Squad upgrade guidance likewise distinguishes supported primary entities from excluded BDD data and execution audit logs. It warns that rerunning migration can overwrite target data and does not provide delta merging. Do not treat a roadmap date as proof that a missing transfer capability has shipped, or apply a legacy migration procedure to a different generation.
Select a sample containing a parameterized case, attachments, custom fields, a failed execution, a rerun, and linked defects. Compare source and target records field by field. Agree on retained historical access, a write freeze, reconciliation ownership, and rollback conditions before the main move. Preserve evidence until the team has signed off on the new system.
Shortlist Xray when having major test artifacts behave as Jira work items directly supports your workflows, permissions, and relationships. Evaluate Enterprise specifically if independently active test versions or other Enterprise capabilities are purchase requirements. Keep internal run data and deployment-specific APIs in the technical design.
Shortlist current Zephyr Cloud when an app-managed library, case versions, and cycle/plan organization fit the team's work. Consider Essential for the documented core workflow and external integrations, Standard for built-in no-code needs, and Advanced when its execution capacity or browser and parallel capabilities are required. Verify the actual installed generation before applying that recommendation to an existing customer.
For a lean team whose main problem is reviewing AI-proposed coverage changes as requirements evolve, a separate evaluation may be useful. Sherlock Test Management describes a reviewed testing brief, human-approved versioned baselines, and release scope based on Jira Fix Version. In the checkout scenario, the team could review proposed case edits before changing the baseline, then record outcomes in a release-specific run. The Sherlock product workflow explains that process; it does not establish parity with either competitor's hosted automation or CI integrations.
Make the final decision from a completed release exercise, a verified migration sample, and a quote for the required population and capabilities. Select the lowest-cost configuration that meets those acceptance criteria without creating an unowned evidence or maintenance problem.
A green automated test history can help a developer identify a safe-looking change while leaving a QA lead unable to answer a different question:…
A checkout test appears under Payments, again in a Smoke folder, and once more under the latest release. When the payment rules change, someone…
A Jira story is Done, a test case is linked, and the related bug is Closed. Can the release owner conclude that the story's…
Create traceable test cases, prepare release-specific test runs, and keep approved coverage current as the product changes.
