Bitbucket Tests vs Jira Test Management: Compare QA Records and Gaps
A green automated test history can help a developer identify a safe-looking change while leaving a QA lead unable to answer a different question:…
A shared payment flow can appear in several Jira projects, but copying its test into each project creates separate maintenance work. When the declined-card behavior changes, teams can end up executing different expected results under similar case names. A central library avoids that problem only if teams reuse the same Test identity and understand which project controls access, execution, and release context.
Xray Cloud supports this arrangement: a Test Execution can contain Tests from another project, and a Test Execution or Test Set can mix Tests from different projects. These are Xray app capabilities, not a test-library feature supplied by Jira core. Reusing a canonical Test also leaves each execution's results in its own context; a shared case does not mean a shared result.
Start by asking whether separate projects solve an actual ownership or access problem. Xray's Project Organization documentation prefers an all-in-one project because it is self-contained and easier to manage. Keep that arrangement when the same team owns requirements, cases, and executions and has no reason to separate them.
A dedicated library becomes useful when a smaller authoring group maintains reusable tests while other teams select and execute them. The extra project boundary introduces permission checks and, where requirements and executions are separated, release-name coordination. Treat that administration as part of the design.
Consider a fictional commerce setup. LIB holds the canonical declined-card Test, LIB-42. WEB holds the checkout requirement. QA-RUN holds a release Test Execution that includes LIB-42. These keys and responsibilities are illustrative conventions, not built-in Xray roles or fields.
Artifact | Illustrative Home | Responsibility |
|---|---|---|
Test, Precondition, Test Set | LIB | Maintain reusable behavior, prerequisites, and candidate groups |
Requirement | WEB | Define the behavior and its intended release |
Test Execution and Test Plan | QA-RUN | Select testing scope and retain execution context and results |
Defect | The team's agreed defect project | Track investigation and corrective work |
Requirements and executions do not have to be separated merely because cases have a central home. Choose the smallest arrangement that meets the team's needs. The documented cross-project association is not evidence that cases can be shared across different Jira sites.
Before selecting tests, confirm that Xray Cloud is available and configured in the relevant Jira projects. Involve a Jira administrator and the people responsible for the library and execution projects. Have a representative Test, a requirement where applicable, and authorized pilot accounts ready.
The permission-scheme route below concerns company-managed Jira Cloud projects. Current documentation may call projects spaces and issues work items. Team-managed access uses a different role-based setup, so an administrator must map the intended controls to that project type rather than apply this scheme recipe unchanged. This is a documentation-backed procedure, not a configuration verified in your tenant.
For the dedicated-library pattern, Xray documents Test, Precondition, and Test Set types in the library project, with Test Execution and Test Plan types in the execution project. Ask the administrator to make the required types available through the relevant issue or work-item type scheme. The organization guide describes the project-settings Summary shortcut and Jira administration's Issue Type Schemes route; verify the equivalent labels in your tenant before changing a scheme.
Use that example to identify the types you need, not as an instruction to remove unrelated types from an existing project. Review a scheme's existing use before modifying it. Keep defects where the team already manages them unless a separate change is justified.
In the requirement project's Xray Test Coverage settings, have its administrator configure the types that should be testable or coverable, such as Story. The current page is named Space Settings: Test Coverage. Making a type coverable and associating a Test with a particular requirement are separate tasks. Adding a Test to an execution does not establish the missing requirement relationship.
Check permissions against the work item being acted on. Editing the canonical Test is different from editing the list of Tests inside a Test Execution. Recording a Test Run result is another action again. The following checks describe the intended separation, not a complete permission bundle that is sufficient in every tenant.
Person or Action | Target to Check | Documented Control to Validate |
|---|---|---|
Library reader | Canonical Test and related library items | Browse access, subject to work-item security |
Test author | Test Details in the library project | Edit Work Items for the canonical Test |
Execution creator | Destination execution project | Create Work Items for the Test Execution |
Scope coordinator | Tests list of the destination Test Execution | Edit Work Items to add existing Tests |
Test executor | Destination Test Execution and its Test Runs | Browse and Resolve permissions, plus permitted workflow states |
Xray's current permissions guide uses Browse Spaces and Resolve Work Items; its Execute Tests page uses the shorter Browse and Resolve wording for the Test Execution issue. Confirm the corresponding permissions in your tenant. Both the Test and Test Execution must be outside workflow statuses configured to disallow execution. A person who can inspect an already-started run in read-only mode does not necessarily have permission to change its result.
Access does not flow automatically across the association. Xray documents an error when a Test Execution contains a Test that the user cannot access. Check the library Test's visibility and work-item security as well as the execution project's permissions. Do not solve an inaccessible Test by cloning it into a less restricted project.
There is a further Cloud limitation: restricting canonical Test edits does not make the entire repository immutable. The permissions guide documents repository folder operations available with Browse access and says Cloud does not provide repository-specific permission control like Data Center. Validate folder and library behavior separately if the intended boundary requires more than preventing Test Details edits.
Before reuse, review the Test's preconditions, actions, and expected results against the requirement. For fictional LIB-42, the expected outcome might be that a declined card leaves the order unconfirmed. Check that this behavior applies to each consuming product; similar wording alone does not establish that two teams need the same test.
Give the reusable Test a stable product-area home and a named maintenance owner. Keep that responsibility separate from whoever executes it for the next release. The owner and reviewer should know how another team requests a correction and who approves a changed expectation. Record those responsibilities using your configured fields or an ownership map, rather than assuming Maintenance Owner is a universal Jira field. The related Jira test repository organization guide explains how stable homes and cross-cutting selections support that ownership model.
The acceptance check for reuse is the original Test key. LIB-42 should remain LIB-42 when selected into another project's execution. Copying its steps into a newly created Test produces another case to maintain. A Test Run's execution-specific copy of the definition, discussed below, serves a different purpose and does not require another canonical Jira Test.
Once the administrator has checked the destination types and permissions, a coordinator can use the documented Test Execution workflow. The control names below come from Xray's Cloud documentation; available fields and labels can differ with Jira configuration. If the expected project or action is unavailable, resolve that configuration or access issue before continuing.
A Test Set can be a useful source of candidate cases, but its behavior needs a separate check. Xray's Tests from Test Sets operation adds the member Tests to the execution. It does not retain a live association to the Test Set, and later changes to that set do not update the execution's selection. Reconcile the selected keys whenever the release scope changes.
This check applies when requirements and Test Executions live in different projects and you want version-based requirements analysis. Xray explicitly requires the requirement project's version names to match those in the Test Execution's project.
In the fictional setup, suppose the WEB requirement targets release-2026.10. Check that the relevant QA-RUN execution targets a version with the same name. A version called October Release in QA-RUN is not the same name merely because the team considers it the same delivery. Agree on the naming convention and have the appropriate administrators maintain it in both projects.
The important pair is the requirement project and execution project. Setting a Fix Version on LIB-42 in the library does not substitute for that alignment. Nor is a Jira release version the revision number of the Test's steps.
Matching names does not automatically create versions, establish requirement-to-Test relationships, or make all reports include the intended evidence. Confirm the selected requirement, linked Test, execution release, and environment together. If no version-based analysis is used, do not present this conditional reporting requirement as a prerequisite for every cross-project association.
A Test Execution groups contextual Test Runs. Xray describes a Test Run as an instance of a Test containing a copy of its definition and that execution's result. The library owner maintains the canonical case; the executor records what happened during the particular attempt.
For a manual test, perform the check against the intended build and environment before recording its outcome and supporting evidence. Setting a status in Xray records a result; it does not prove that Xray launched a browser or exercised an API. Automated execution and importing a runner's results require their own configured workflow, outside this library-organization procedure.
A later edit to LIB-42 does not automatically update its existing Test Runs. This preserves the distinction between what was specified for an earlier execution and the current canonical case. Do not assume every field displayed on an execution screen is an immutable snapshot, however; inspect the particular definition and evidence involved.
Xray offers explicit Reset and Merge operations when the definition changes. Reset erases previously saved execution information, including status, evidence, and defects. Merge can preserve results for unchanged steps, load definition changes, and recalculate the Test Run status. Neither should be an automatic response to a library edit. Review the effect on recorded evidence, and plan a new execution when a new testing attempt is needed.
Check reporting associations as well. If you use a Test Plan, associating a Test Execution with it does not by itself add that execution's Tests to the plan. Xray documents a warning for Tests missing from an associated plan. Reconcile both the Test membership and execution association before treating the plan summary as the intended release picture.
Begin with a representative case and the people who will maintain, select, execute, and inspect it. Ask the administrator to validate both allowed and denied actions with authorized accounts. Use the following outcomes as local acceptance criteria, not as a claim that documentation alone has proved your configuration:
If the Test is missing, check its key, visibility, and security before changing the library structure. If execution is unavailable, inspect the execution permissions and both entities' workflow restrictions. If release analysis looks wrong, check the requirement and execution projects' version names and the actual report scope before assuming that case reuse failed.
Expand only when the team can demonstrate those boundaries. The useful result is one maintained Test identity, deliberate release selections, and execution evidence that remains understandable. If separating projects adds administration without a needed ownership or access boundary, the all-in-one arrangement remains the simpler choice.
A green automated test history can help a developer identify a safe-looking change while leaving a QA lead unable to answer a different question:…
A checkout test appears under Payments, again in a Smoke folder, and once more under the latest release. When the payment rules change, someone…
A Jira story is Done, a test case is linked, and the related bug is Closed. Can the release owner conclude that the story's…
Create traceable test cases, prepare release-specific test runs, and keep approved coverage current as the product changes.