Manage Test Cases and Test Runs in Jira With AI
Sherlock. turns Jira issues, Confluence pages, and release changes into reviewed test cases and release-specific test runs. Create, refine, execute, and maintain testing through one AI-assisted workflow.
Nothing enters the baseline without a human approval
Available on the Atlassian Marketplace
- In scope
- 28
- Executed
- 24
- Pass rate
- 86% +4
- Defects
- 3
Overview
A Complete AI Test Management Workflow in Jira
Sherlock. connects the product context that defines what should be tested with the test cases, release runs, execution results, and updates required to keep coverage current.
01
Product Context and Testing Request
Jira issues, Confluence pages, files, and the goal in plain language.
02
Reviewed Testing Brief
Product flows, risks, and approach, editable before anything is created.
03
Approved Test Baseline
Versioned cases with steps, priorities, and their sources attached.
04
Release Test Run
A scope built from the Fix Version, with reasons and assignments.
05
Results and Baseline Updates
Results, defects, the published report, and proposed coverage changes.
AI Test Design
Create Reviewed Test Cases From Product Context
Describe the testing goal in plain language and select the Jira issues, Fix Version context, Confluence pages, and files Sherlock. should use. Sherlock. prepares an editable testing brief before creating risk-based test cases.
Feature 01
Testing Brief
Review the product flows, risks, and testing approach before test cases are generated. The brief is the point where the team corrects a misunderstanding cheaply, while it is still one paragraph and not forty cases.
Feature 02
AI Test Case Creation
Generate grouped cases with steps, expected results, priority, and references to the sources used. Every case can be traced back to the issue or page it came from, so a reviewer can check the reasoning rather than trust it.
Feature 03
AI Chat and Review
Ask for new scenarios or edits in plain language and accept or reject every proposed change as a visible diff. Nothing the AI writes reaches the approved baseline on its own.
Test Case Management
Build a Reusable Test Baseline That Stays Current
Approved test cases become a reusable baseline for future releases. Each case retains its source references, and each completed test run remains linked to the baseline version it executed.
Draft and approved states
A case is a proposal until someone approves it. The baseline holds only what a person signed off.
Baseline version history
Every approval creates a version. A finished run stays attached to the version it actually executed.
Coverage grouped by area and source
Read the baseline by product area or by the requirement that produced it, not as one flat list.
Stale-case detection
When a linked requirement changes, the cases it produced are flagged instead of quietly going wrong.
Proposed updates, human approval
Changed requirements and execution results produce suggested edits. The baseline changes only when a user accepts one.
Test Runs and Execution
Plan and Execute Testing for Every Jira Release
Select a Jira Fix Version and Sherlock. compares the release changes with the approved test baseline. It proposes a focused test scope with reasons, priorities, and assignments for the team to review before creating the run.
Feature 01
Release Test Planning
Review included and excluded coverage, edit the scope, and confirm priorities and assignees. Every exclusion carries the reason it was excluded, so the scope is an argument the team can check rather than a number to accept.
Feature 02
Flexible Run Creation
Create a Jira run task with assigned sub-tasks, or keep the run inside Sherlock. Teams that manage capacity in Jira get Jira issues; teams that do not are not forced to create them.
Feature 03
Test Execution
Follow the same test steps and record Pass, Fail, or Blocked in the Sherlock. Runner or supported Jira panels. Wherever a tester works, the run and its results are the same object.
Results and Reporting
Connect Test Results With Jira Delivery
Sherlock. keeps failed tests, Jira defects, release results, and the executed baseline version connected. Teams can review one release in detail and monitor testing across Jira projects.
Feature 01
Linked Defects
Create a Jira defect from a failed test and retain links to the run, executed case, requirement, and source context. The defect arrives in Jira already knowing where it came from.
Feature 02
Release Reports
Review totals, pass rate, results by product area, assignments, defects, and release details. Publish the report to Confluence where supported, so the release record lives where the rest of the documentation does.
- Executed
- 28
- Pass rate
- 86% +4
- Defects
- 3
- Baseline
- v3
Feature 03
Portfolio Visibility
Give QA leads a read-only view of approved baselines, active runs, pass trends, stale cases, and open defects across projects. Read-only on purpose: the view is for noticing, not for editing someone else’s project.
FAQ
Questions About Sherlock.
What is Sherlock
Sherlock. is an AI test management app for Jira that helps teams create, review, execute, and maintain test cases and release-specific test runs.
How does Sherlock work
Sherlock. uses selected Jira issues, Confluence pages, files, and user instructions to prepare a testing brief and test cases. The team approves a reusable baseline, creates a test run for a Jira release, executes it, records defects and results, and reviews proposed coverage updates.
Can Sherlock create test cases from Jira issues and Confluence pages
Yes. Users select the Jira issues, Confluence pages, files, and other context Sherlock. should use. Generated test cases retain references to their sources.
Is Sherlock only an AI test case generator
No. Sherlock. supports the complete test management workflow, including test case review, versioned baselines, release planning, test execution, defects, reports, and ongoing test maintenance.
What is the difference between a test baseline and a test run
The baseline is the approved, reusable set of test cases. A test run is created for a specific release and contains the selected scope, assignments, execution results, and linked defects.
Can teams execute tests directly in Jira
Yes. Testers can execute the same test run through the Sherlock. Runner or supported Jira sub-task panels.
Does Sherlock change approved test cases automatically
No. AI-generated additions and edits are shown as proposed changes. A user must accept or reject them before the approved baseline changes.
Who is Sherlock for
Sherlock. is designed for Jira-based product and QA teams, especially lean product teams, embedded QA teams, and QA leads managing testing across projects.
Manage Test Cases and Release Testing in Jira With Sherlock.
Create a reviewed test baseline, plan each release from the changes it contains, and keep approved coverage current as the product moves.