Flaky Test Quarantine: Owners, Expiry, and Nonblocking CI Results
A confirmed flaky test can block unrelated changes without providing a dependable release signal. Removing that obstruction is reasonable only if someone still sees…
A checkout can display translated labels while corrupting a customer's name, hiding the confirmation button, or interpreting the delivery date incorrectly. Reviewing the translation alone will miss some of these failures. Testing only the underlying language support will miss others, including a perfectly visible message that tells customers the wrong thing.
Internationalization testing asks whether the product can support its intended languages and regional conventions. Localization testing asks whether a particular localized version works correctly for its target users. The same checkout needs both, but each layer requires different inputs, expected outcomes, and review skills.
Internationalization, often shortened to i18n, concerns the design and development that makes localization possible. Its testing examines shared foundations such as translatable resources, multilingual text handling, flexible layouts, and locale-aware formatting. Much of this work can begin before translated content exists. Microsoft recommends early code review, source-language review, pseudolocalization, and a small set of pilot languages to expose problems before they spread across localized releases.
Localization, or l10n, adapts a product to a particular language and market. Testing that adaptation includes functional behavior, visual presentation, linguistic accuracy, and relevant regional requirements. W3C's definition extends well beyond translation: dates, currency, sorting, input conventions, imagery, and cultural expectations can all be involved.
The practical distinction is the claim being tested: shared readiness for the required locales, or correctness of a specific target version. Both need an explicit supported scope; neither establishes that the product works in every language.
Criterion | Internationalization Testing | Localization Testing |
|---|---|---|
Main question | Can the shared product support the required locales? | Does this target-language and market version work correctly? |
Typical inputs | Source resources, multilingual data, locale settings, and pseudo text | Translated resources, target-market data, and the localized build |
Example evidence | A name survives saving and reloading; expanded labels remain usable | The real translation is accurate in context and its controls remain usable |
Usual correction | Shared code, resource design, formatting, or layout behavior | Translation, locale configuration, content, or an underlying shared defect |
Remaining limitation | Readiness evidence does not approve a real translation | A correct target version does not prove every other locale is supported |
These categories overlap in practice. A clipped German label may reveal a shared fixed-width component, an unusually long translation, or both. Localization testing can discover an internationalization defect. Route the issue according to its cause and affected scope, rather than assuming that every defect found in a translated build belongs to the translation supplier.
A useful comparison holds the workflow constant and changes the question asked of it. In the examples below, the shared implementation and the real localized build are two layers of evidence for the same customer journey. The requirements are deliberately bounded, so unspecified behavior does not become an invented defect.
These paired examples describe a fictional checkout and have not been executed against an application. The product team has specified a customer name field that must preserve the exact text Zoë 東京 when a customer saves it, reloads the order, and changes the interface language. Search matching and Unicode normalization policy are outside this example.
The internationalization check follows that name through input, submission, storage, retrieval, and display. A field that accepts the text initially but reloads it as question marks has not met the requirement. Merely checking that the page declares a Unicode encoding would leave the save-and-reload path unverified. Microsoft explicitly includes reading, writing, displaying, and preserving data across app languages in its internationalization guidance.
The localization check repeats the journey in the actual target build. Check whether the relevant input method works, the selected fonts display the characters, and any validation message is understandable in context. The interface language does not determine which scripts customers may use in their names. A German interface can still need to preserve Japanese customer data.
Keep the oracle tied to the requirement. This example requires exact preservation; it does not specify whether differently encoded but visually similar names should compare as equal, or how search should match them. Those questions need separate product decisions and tests.
The checkout must keep its complete confirmation action visible and usable at the team's supported narrow viewport. A pseudolocalized build is used to challenge text expansion and direction assumptions before translation. The actual German and Arabic builds still require visual and native-language review; no translation or layout result is supplied here.
For internationalization testing, replace localizable source messages with expanded pseudo text and inspect the confirmation area. Verify that the complete label remains available and the action can still be reached and activated. A screenshot showing an intact button cannot alone establish usability. Include the interaction rather than treating visibility as the entire expected result.
For localization testing, inspect the real German wording at the same supported viewport and review the Arabic layout with the intended direction settings. Check the actual label, adjacent explanations, alignment, and the behavior after activation. A native-language reviewer must judge whether the wording expresses the intended commitment. Fitting inside a button does not establish that a translation is accurate, and a simulated right-to-left build does not approve the real Arabic experience.
A pseudo failure can justify fixing the shared component before translators finish. A pseudo pass still leaves the actual translated strings and interactions to be reviewed. Record the build, viewport, resource version, and language so a later retest exercises the same conditions.
The order's delivery date is 2026-04-03, meaning April 3, 2026. This fictional product explicitly requires 04/03/2026 in its en-US short-date view and 03/04/2026 in its en-GB short-date view. Switching the display locale must not change the stored calendar date. Currency conversion and time-zone conversion are not part of this date-only example.
The internationalization check asks whether one stored date can be formatted according to the selected locale without changing its meaning. The localization check verifies the agreed display in each real target build, including the order confirmation and any other surface covered by the requirement. Seeing different strings is expected here; seeing different delivery days is not.
The exact formats above are this fictional product's requirements. Do not infer them from a language label alone or hardcode a universal country rule into a generic test. Record the locale, formatting options, and relevant user preferences. Microsoft notes that regional settings influence dates, numbers, and currency presentation, and that a locale can include language, country or region, and script.
The internationalization checks challenge the shared ability to preserve multilingual data, accommodate expansion, and format the same date according to an explicit locale. The localization checks validate the real target build, its translated messages, its usable controls, and the agreed regional display. Every application outcome remains NOT RUN; a consistent expected date is not evidence that checkout saved or rendered it correctly.
Pseudolocalization transforms source resources to simulate characteristics that can cause translation-related problems. Depending on the tool, this can include accents, extra text, delimiters around messages, or a simulated text direction. It provides useful test material before real translations are available.
The transformation also makes some implementation problems easier to see. A hardcoded source-language label can stand out because surrounding messages have changed. Delimiters can reveal a truncated message or show that developers assembled a sentence from separately translated fragments. Fragment concatenation is risky because a target language may need a different word order; translators cannot reliably repair that structure by translating each fragment independently.
Android's documented pseudolocales offer a concrete platform example: an accented and expanded English variant and a right-to-left variant. These are development aids, not real-language acceptance tests. Other platforms and tools have their own mechanisms, so verify the applicable setup rather than assuming that one platform's identifiers or build settings apply everywhere.
A pseudo pass has a limited meaning. It can show that the tested pseudo inputs did not expose the checked defects under the recorded conditions. It cannot certify every possible translation length, grammatical construction, font interaction, regional feature, or cultural interpretation. Microsoft's localization guidance specifically notes that real translations can exceed pseudo's configured growth and still be truncated.
Use pseudo to find structural problems early, then retain tests for the real localized product. A readable pseudo message does not tell you whether a French error explains the correct recovery action, whether a Japanese confirmation uses appropriate terminology, or whether a market-specific contact is valid.
Functional localization testing checks the behavior of the actual target version. Reuse suitable existing scenarios to compare common behavior across languages, while documenting intentional market-specific differences. If a local payment method or address flow is part of the approved scope, give it its own expected outcomes instead of calling every difference from the source-language build a defect.
Visual review examines the real resources in their presentation context. Look for clipping, incomplete localization, inappropriate line breaks, font problems, and incorrect placement. Include error states, confirmation screens, help content, and other reachable surfaces that matter to the journey. Sampling can be necessary, but retain what was inspected and what remains outside the sample.
Linguistic review requires different expertise. Microsoft recommends native speakers and strongly recommends reviewers familiar with the target country or region. Give them the product glossary, style guide, source meaning, and access to representative workflows. A translated string can be grammatically valid yet wrong for the action it labels. Reviewing it inside the running product helps expose that mismatch.
Automation can check repeatable expectations such as saved values, selected resources, and known formatting rules when the test infrastructure supports them. It does not establish that every translation conveys the intended meaning. Keep automated functional results, visual findings, and linguistic approval separate so one green report cannot conceal a missing review.
A release decision needs evidence for the shared foundations and the target versions being shipped. Start with the combinations the product actually supports, then use risk to choose depth rather than assuming one representative language covers every script and convention.
For a Jira-based team, Sherlock Test Management can support the evidence workflow through reviewed test baselines, release-specific runs, and recorded Pass, Fail, or Blocked results, as described on its product page. A team could retain the shared name-preservation case alongside cases for each agreed target build, with human review of proposed changes before baseline approval. That organizes the testing record; it does not perform native-language judgment or prove that an application was exercised automatically.
Choose readiness checks to expose shared barriers before translation, then require evidence from the localized versions you intend to ship. If a real translation, supported environment, or qualified reviewer is missing, name that gap in the release decision. Pseudolocalization can bring defects forward, but it cannot supply the missing localized-product evidence.
A confirmed flaky test can block unrelated changes without providing a dependable release signal. Removing that obstruction is reasonable only if someone still sees…
A green automated test history can help a developer identify a safe-looking change while leaving a QA lead unable to answer a different question:…
A sprint can contain carefully estimated testing work and still overload the only person who can perform it. The missing input may be a…
Create traceable test cases, prepare release-specific test runs, and keep approved coverage current as the product changes.