AI in QA

Rovo Dev Acceptance Criteria Checks: Link Jira Stories to Code Reviews

Rovo Dev Acceptance Criteria Checks: Link Jira Stories to Code Reviews

A pull request can appear on a Jira story while its acceptance criteria never reach the code reviewer. The branch may lack the story key, the description may be empty, or the author may have the wrong Rovo Dev entitlement for the connected site. Before treating an absent finding as good news, establish that the review could find the intended requirements.

Rovo Dev acceptance criteria checks compare pull-request code with requirements in linked Jira work items. Findings can say that criteria are met, missing from the code, or need manual checking. A met finding is a code-review assessment. It does not establish that someone executed the application, observed the expected behavior, or approved the release.

Table of Contents

Confirm the Author, Site, and Repository Prerequisites

The workflow depends on several separate layers: the author's plan, the Jira connection, repository activation, and the acceptance-check setting. A repository administrator can enable a feature without satisfying the author's entitlement. A Jira development link can exist without Rovo Dev reviews being enabled.

Match the PR Author to the Connected Jira Site

Atlassian's current acceptance-check guide requires the pull-request author to have Rovo Dev Standard on the Jira site connected to the Bitbucket workspace or GitHub organization where the PR is created. Check the author, rather than assuming that a licensed reviewer or administrator supplies this requirement.

Code-review usage draws on the author's Rovo Dev credit allocation on the selected site. Confirm the allocation and selected site before troubleshooting the story text. Having Jira access, or a Rovo feature elsewhere, does not establish this entitlement.

This procedure follows the documented Jira Cloud connections to Bitbucket Cloud and GitHub Cloud. The linked GitHub connection guide also covers Enterprise Cloud. Do not infer that these instructions establish the same Rovo Dev workflow for Jira Data Center or GitHub Enterprise Server merely because those products have development integrations.

Activate Both Repository Layers

Use the path for your repository provider and keep initial configuration scoped to one repository. These are setup instructions for an authorized administrator, not steps a PR reviewer can necessarily perform.

  1. Confirm the Jira connection. Bitbucket uses the DVCS connection documented by Atlassian. The Rovo activation guide calls the GitHub connector GitHub for Jira; its linked connection guide now calls it GitHub for Atlassian. Check the installed connector and repository access rather than installing two apps based on the naming difference.
  2. Activate the workspace or organization. In Bitbucket, a workspace administrator uses Workspace settings, Atlassian integrations, then Rovo Dev to select the Jira site under Rovo Dev credits and enable Use Rovo Dev in this workspace. For GitHub, an app admin uses Rovo Dev settings and Manage GitHub settings to enable the organization and select the billing site.
  3. Enable the individual repository. Current Bitbucket activation instructions require Rovo Dev Admin permissions and describe the AI code reviews in this repository toggle under repository settings. GitHub repository enablement requires repository admin permissions; configure the repository from the organization screen.
  4. Check the acceptance-criteria setting and review frequency. Code review activation and acceptance checks are separate controls. Bitbucket acceptance checks are enabled by default; the guide requires repository admin permissions to turn them off. GitHub acceptance-check controls require app admin permissions to turn them off.

Connection administration has its own prerequisites. The Bitbucket connection guide requires a Jira administrator, not space-admin rights alone. The GitHub connection guide requires a Jira site administrator and a GitHub organization owner, potentially working together. Confirm the app can access the intended repository, especially when installation grants access only to selected repositories.

Make the Jira Story Discoverable

For the Rovo Dev Jira story link, prefer one clear work item key in the branch name. Copy the key from Jira rather than relying on a shortened or differently formatted reference. A fictional branch such as feature/SHOP-418-invoice-download makes the intended story explicit.

The acceptance-check guide says Rovo Dev looks in the branch name first. If it finds no work item key there, it checks commit messages. If commit messages contain more than one work item key, it does not use them. Keep one PR focused on one well-described story where practical. The current guide does not specify a rule for choosing among multiple keys in a branch name, so avoid creating that ambiguity.

A Jira development-panel association is not a substitute for this lookup rule. General connector guidance may tell teams to use a key in a PR title, but the acceptance-check guide describes branch and commit lookup. Verify the branch rather than assuming that a title-only link is sufficient. Copy the key exactly as Jira displays it; this recommendation avoids ambiguity without assuming undocumented case normalization or key-rewriting behavior.

Give the story a nonempty description. Rovo Dev will not search for acceptance criteria when that description is empty, even if the summary or a custom field contains requirements. This prerequisite is easy to miss when a team maintains its criteria exclusively in a dedicated field.

The guide lists these custom-field labels:

  • Acceptance Criteria, AC, or ACs.
  • Business Requirements, Functional Requirements, or Requirements.
  • Definition of Done or DoD.

Use the documented labels without inventing a matching rule. Write short criteria focused on a specific observable outcome. For an invoice download, “the account owner can download an invoice for their account” gives reviewers a clearer target than “invoice feature works.” Resolve unclear permission rules with the product owner before asking code review to assess them.

Choose When the Review Runs

Repository policies can run reviews when a PR is created, when it is created and on every new commit, or only when started manually. The trigger guide identifies PR creation as the default. If that policy is selected, a later corrective commit does not automatically receive another review. Choose frequency according to how the team intends to inspect revisions and manage credit usage.

Only the PR author can manually trigger a review. Confirm the actual provider control before documenting a team-specific manual procedure: Atlassian's current activation and dedicated trigger guides disagree on the Bitbucket and GitHub manual controls. The trigger guide describes a Bitbucket Review code action and a GitHub review comment, while the activation guide gives different provider instructions. Do not copy a command across providers or assume a screenshot from an older rollout still applies. Have an administrator resolve the applicable control in the current tenant or with Atlassian support.

The trigger guide also says each Bitbucket commit can be reviewed only once and manual triggering is unavailable when reviews run on PR creation and every commit. Repeated requests are therefore not a general way to force a new assessment. First identify the selected policy and the commit the existing review covered.

No comments does not mean all criteria passed. Check whether the review ran, whether the author's account and credits are available, and whether the correct story could be found. Large changes or repositories can also affect feedback. Atlassian's activation and troubleshooting pages differ on the changed-line cutoff, so use current troubleshooting guidance rather than treating a single numeric threshold as universal.

Interpret Findings With a Fictional Invoice Story

Consider an illustrative story, SHOP-418, on feature/SHOP-418-invoice-download. Its description is populated and states the requirements below. The story, branch, findings, and follow-ups are fictional; they are not results from an observed repository or a claim about what Rovo Dev will return.

The team uses the findings to decide what work remains. It does not translate them directly into test results. In this example, the tester plans to exercise both an authorized account and a different account against the same candidate, so the apparently met download path does not hide the missing access-control question.

Story Criterion

Hypothetical Review Finding

Human Follow-Up

An account owner can download their invoice.

Met in the code review.

Tester checks a deployed candidate and verifies the returned invoice belongs to the account.

A user cannot download another account's invoice.

Missing from the code.

Developer investigates the authorization path and reviewer checks the explanation before QA tests denied access.

An archive-provider timeout produces a recoverable error.

Needs manual checking.

QA owner arranges an approved timeout condition and records the observed result on the candidate.

Product owner approves the customer help text.

Needs manual checking.

Product owner records the approval or requested revision separately from application test results.

For the missing authorization finding, ask the developer to inspect both implementation and context. The protection might be absent, or the reviewer may need to understand an existing shared permission check. The finding deserves investigation; it is not automatically a confirmed product defect. If an existing control covers the criterion, retain the human explanation and the relevant code reference instead of changing correct code merely to satisfy a label. If the protection is absent, correct it and assess the affected regression scope. Either disposition still leaves the planned access checks to execute.

The timeout criterion requires behavior that a code comparison alone cannot demonstrate. Assign an evidence owner and define the environment and controlled failure condition. The help-text approval is a non-code decision with a different owner. Complex behavior and non-code criteria need evidence appropriate to the requirement, even when code review offers a useful starting point.

For every unresolved row, identify the next action. “Manual” without an owner leaves an untested obligation easy to lose. If the dependency needed for the timeout check is unavailable, preserve that limitation and its recovery action instead of recording a pass.

Carry Unresolved Criteria Into QA and Release Evidence

A useful review ends with an accountable handover. Keep implementation questions, test-definition changes, actual execution results, and business decisions distinct so the next person can tell which evidence exists and what remains open.

Keep a Review-to-Test Handover Record

Retain the story and criterion, PR and reviewed commit, finding, human disposition, and follow-up owner. When testing begins, add the candidate build, environment, setup, expected result, actual result, and supporting evidence. These are proposed record contents, not a claim that Rovo creates native fields or assigns testing tasks automatically. Record two separate states when needed: the review concern is resolved, but the application check is still pending. Closing the former must not silently close the latter.

Give the receiving tester current criteria and a testable candidate, rather than forwarding a met label alone. A changed story or replacement build requires a decision about which earlier checks remain applicable. This practical developer-to-QA handover workflow develops the readiness, context, and ownership agreement needed at that point.

Keep the Approved Baseline Separate From Run Results

Sherlock Test Management provides a relevant app-level workflow for the testing that follows. Its current product description supports source-referenced AI case proposals, human approval before baseline changes, and a versioned reusable baseline. A team can review invoice access and timeout cases, then select release-specific scope using Jira Fix Version context and record Pass, Fail, or Blocked in Sherlock Runner or supported Jira panels. Those results belong to the test run and executed case version. This is a human-coordinated workflow, not a claimed Rovo-to-Sherlock integration or autonomous application execution.

For the next story, begin with one repository and one unambiguous Jira key. Confirm the author and site, make the requirements discoverable, and agree who resolves each finding. Before a release decision, ask the named owner to show the evidence for each remaining criterion or state the unresolved risk and required approval. A met code-review assessment can inform that decision; it cannot supply a missing execution result or business sign-off.

Related

More on AI in QA

Create traceable test cases, prepare release-specific test runs, and keep approved coverage current as the product changes.