AI Test Failure Analysis: Trace Logs to a Root-Cause Hypothesis
A checkout test times out while waiting for an order confirmation. The runner reports an assertion failure, the application records a failed request, and…
Two reports can describe the same symptom while disagreeing about where the failure occurs. If a reviewer accepts a duplicate suggestion because the titles look alike, the report containing the distinguishing evidence can disappear from the working backlog. A useful review therefore ends with a reasoned decision about the pair, plus a separate decision about what anyone is allowed to change.
AI duplicate bug detection can help locate reports worth comparing. It does not remove the need to inspect their written reproduction context, preserve contradictions and protect unique information. The procedure below is a document-review exercise: no model request, application test, reproduction run or issue-tracker operation is performed.
Begin with an approved report packet and a named reviewer who understands the affected product area. Keep the original report identity, version and access classification with every excerpt. Before any future use of an AI service, confirm that its use is approved, that the reviewer can access the candidate material, and that redaction preserves the details needed for comparison. Do not upload private reports merely because a tool accepts text.
Decide who may propose a duplicate match, who owns the canonical record and who may authorize changes. These can be different people. Adopt a local human-first policy: suggestions remain proposals, and any available automatic record-changing actions stay outside this review until their owners approve the applicable workflow. A warning or recommendation in an interface is not permission enforcement.
Linear’s current Triage Intelligence documentation describes relationship suggestions when an issue in Triage has strong semantic similarity to an existing issue. It says a reviewer can inspect why a suggestion appears, accept or dismiss it, or follow the referenced issue for further analysis. The feature is documented for Business and Enterprise workspaces; enabling it in Settings > AI requires an administrator and enables it for every team, with individual team controls and an Include suggestions from scope setting.
The same documentation separately allows issue property suggestions to appear, be hidden or be auto-applied. Do not describe all of its AI behavior as necessarily manual. The human gate recommended here is a local review policy, not a claim that the product enforces every downstream approval. Nothing in this exercise requires enabling the feature, using its Find Suggestions command or changing a tenant. The documentation example also does not establish that accepting a relation merges records or preserves every attachment.
Semantic bug matching can surface descriptions that use different words for a related problem. A ranked candidate list answers which records are worth reading next. It does not, by itself, establish identical failure conditions, a shared root cause or a calibrated probability that a report is a duplicate. A score must not replace the evidence comparison, and an apparently low-ranked candidate still deserves consideration when its context fits.
An AST 2024 industrial case study of the Bugle research tool examined language-model-based duplicate recommendations with testers confirming or rejecting suggestions and reformulating queries. Its qualitative findings included disagreements caused by semantic differences, insufficient information and overlooked recommendations. Those findings support careful review, not a universal accuracy claim. The study used a limited participant group, one consultancy’s setting and a small set of observation tasks; it is not evidence of current Linear performance.
Keep retrieval evidence separate from the match decision. Save which description and source versions were considered, the candidate identifiers and any known search scope. An explanation generated by a model is another proposal to check against those records, not an additional independent witness.
Read the incoming report against each candidate independently. Do not make a chain of assumptions in which accepting one pair causes every related report to be treated as equivalent. For each pair, follow the same review sequence:
A difference is not automatically disqualifying. Two browser names may be incidental when the packet explicitly supplies matching failure evidence. Conversely, matching titles do not outweigh a contradiction about whether the new value was persisted. Ask what evidence would change the decision, and preserve the answer alongside the current recommendation.
The following raw packet fixes the comparison inputs before any expected decisions are derived. Read every clause, including its permissions paragraph. The candidate labels identify authored alternatives; they do not imply a retrieval order.
DUP-17 v1 is a fictional, unexecuted report-review packet. No AI retrieval, model request, application test, reproduction run, link, merge or closure has occurred. IN-17 and candidates A-17, B-17 and C-17 are a bounded authored set, not ranked vendor results or a complete index. Every attachment below is an invented evidence description, not a captured customer artifact. Product Ledgerboard, its components and builds are invented.
IN-17, incoming report v1: The title is “Card order resets after reload.” The component is Board Ordering on web build 4.2, in Firefox on a desktop browser. The authored preconditions are a QA-only board in Manual sort with two existing cards and edit permission. The written steps are to move the second card above the first, save, then reload. Expected behavior is that the saved manual order remains; the reported actual behavior is that the old order returns. IN-17 evidence note E1 describes a save response containing the old rank sequence before reload; the attached authored request summary IN-17-a records a single move and its board identifier. IN-17 evidence note E2 adds a Firefox-only focus warning absent from the other reports, whose relationship to ordering is unknown. No timing measurements or root-cause analysis are supplied. IN-17-a and E2 are unique incoming evidence, not replaceable by A-17’s attachment.
A-17, candidate report v2: Its title is “Moving a card does not survive refresh.” Board Ordering on web build 4.2 uses Chromium on desktop with the same fictional board configuration, Manual sort, two existing cards and edit permission. Its written steps move the second card above the first, save and reload, expecting the saved manual order to remain but reporting the old order returning. A-17 evidence note E1 describes the save response already containing the old rank sequence before reload. A-17 evidence note E2 explicitly states that the single-move request and the stale-rank response match IN-17 E1 and IN-17-a for the same fictional configuration; the browser difference is not a contradiction in this packet. Its distinct attachment A-17-a contains an authored Chromium screenshot description. A-17 is open and its history contains no fixed-version claim; it has no confirmed root cause. No other conflicting evidence is supplied.
B-17, candidate report v1: Its title is also “Card order resets after reload.” It concerns Board Ordering on web build 4.2, Chromium desktop, two existing cards and edit permission. B-17 evidence note E1 says that the save response and subsequent stored order retain the new rank sequence, but the board is configured for Alphabetical sort so the display follows titles. Its written steps and expected behavior request manual order while Alphabetical sort remains selected. B-17 evidence note E2 expressly rules out a stale-rank response in this packet. This is an evidence-level contradiction with IN-17 E1, not merely a browser difference or a similar title. No shared root cause is established.
C-17, candidate report v1: The title is “Reordering disappears.” Its raw description says only that cards appear back in their original positions after a refresh. Build, browser, sort mode, edit permission, move/save steps, save response and stored order are all unknown. No attachment or analysis supplies those missing facts. Whether it is open, fixed, reopened or a regression is also unknown; do not infer its timeline from the title.
Evidence and permissions v1: All fictional source records and their attachment descriptions are QA-only. A-17 is the proposed canonical record and is also QA-only; no broader copy destination is approved. The hypothetical reviewer may propose match decisions but neither canonical field ownership nor approval to change records has been confirmed. Original record identities, versions, visibility, attachments, written steps, context and contradictory evidence must remain attributable. An accepted match must not erase IN-17 E2, infer a cause for it, transfer ownership, delete a source, or authorize copying, linking, status changes or closure. Any actual copy or workflow operation remains on hold until the appropriate data owner approves it and its behavior and access boundary are verified.
A-17 has more than a similar symptom. Its supplied evidence explicitly aligns the Manual sort configuration, written sequence and stale-rank response with IN-17. That supports accepting it as a proposed duplicate in this bounded review. It does not confirm a shared root cause or resolve the unique focus warning.
B-17 should be rejected as this duplicate suggestion because its save and stored order retain the new ranks, contradicting the incoming stale-response evidence. Keep that difference visible instead of rewriting the reports to fit the suggestion. This decision concerns the supplied pair; it does not prove that the two reports could never have any relationship.
C-17 needs a deferred decision. Ask for its build, browser, sort mode, permissions, move/save sequence, existing response or stored-order evidence, and relevant status history. Missing context is not evidence either for acceptance or for rejection. These are authored expectations for a future review of this packet, not observed model outputs.
Candidate | Proposed Match Decision | Evidence Basis | Required Next Step |
|---|---|---|---|
A-17 | Accept as a proposed duplicate | Matching manual-sort steps and stale-rank response | Preserve both records and unique evidence; obtain owner approval |
B-17 | Reject this duplicate suggestion | New ranks persist; Alphabetical sort changes the display | Retain the contradictory evidence in its separate report |
C-17 | Defer the match decision | Essential reproduction and persistence context is unknown | Ask for build, sort mode, steps, saved ranks and status history |
The table records proposed match judgments, not completed workflow actions. In particular, A-17’s proposed acceptance leaves copying and record changes on hold because the packet does not establish owner authority. C-17’s unknowns must remain visible until evidence answers the targeted questions.
Treat “merging reports” as a controlled consolidation decision, not as the name of a universally available operation. Marking a duplicate, adding a relationship, copying selected information, merging records and closing a report may be separate actions with different consequences. Verify the actual tool’s behavior and preservation rules before proposing a specific operation; do not assume a menu action coalesces everything losslessly.
Prepare a preservation map before seeking approval. For this packet, retain IN-17 v1 and A-17 v2 as original sources, IN-17-a as the single-move request summary, IN-17 E2 as an unresolved focus warning, and A-17-a as its distinct Chromium attachment description. Keep their written steps, build, environments and access labels attributable. Do not replace incoming evidence with the candidate’s screenshot or turn the warning into a confirmed cause.
A-17 is only the proposed canonical destination. Its QA-only visibility is compatible with the packet’s source boundary, but field ownership and change authority are still unknown. Do not widen access, transfer ownership or copy material into a broader record while resolving them. If a real source has stricter restrictions than the proposed destination, an accepted match does not authorize disclosure; the data owner must approve an appropriate restricted destination or another safe representation.
Any later authorized operation needs its own preservation check and readback. Confirm that source identities and evidence remain available with the intended permissions, and that no distinct report was closed or deleted by implication. A match-review note alone cannot establish that this happened.
Keep a compact decision record containing the incoming and candidate identities and versions, reviewer, decision time, accept/reject/defer recommendation, evidence references, contradictions, unknowns and the next owner or question. Separate the match recommendation from operation approval and from any later execution receipt. Accepted match does not mean authorized merge.
If a tool shows no duplicate, record the scope of that observation. Access permissions, search boundaries, index state and recall may be unknown unless verified; an empty suggestion panel is not an exhaustive search of every relevant report. Retain the incoming report and use an appropriately authorized follow-up rather than claiming uniqueness.
Start with one bounded pair review and require both an evidence rationale and an authority check before changing records. When a contradiction remains unresolved, keep it. When essential context is missing, defer with a precise question. The useful output is an explainable recommendation that preserves what the next reviewer still needs to know.
A checkout test times out while waiting for an order confirmation. The runner reports an assertion failure, the application records a failed request, and…
A recipient-preview rule changes, but the regression suite still checks the earlier behavior. An AI agent may choose a different route through the editor…
A refund story has a linked test called “Refund Permissions.” Its steps check the user's role before the first submission. The current acceptance criteria…
Create traceable test cases, prepare release-specific test runs, and keep approved coverage current as the product changes.