Contact us
Multilingual AI testing

7

Languages tested by native speakers

294

Test cases executed across Android and web

47

Defects reported with reproduction steps and severity

Overview

XpressAble is an AI-powered communication assistant for children with autism, speech delays, and social anxiety — children who can speak, but who sometimes cannot find the words in the moment. When someone asks the child a question, the app listens, transcribes it, generates a warm first-person reply shaped by the child’s own profile, and speaks it privately through their headphones. It was built by a parent for his own son, and it is positioned as a nudge rather than a replacement for the child’s voice.

Testvox delivered a crowd testing engagement to validate the app before a wider release, covering the Android application and the translated feature pages on the web across seven Indian languages: Hindi, Malayalam, Bengali, Urdu, Tamil, Kannada, and Gujarati. Testers were native speakers of each language, working from a structured scenario set covering profile setup, listening modes, response generation, spoken playback, chat history, and the translated web content.

Challenges

  • Quality that only a native speaker can judge A transcription can use the correct script and still be the wrong sentence. A generated reply can be grammatically valid and still sound unnatural coming from a child. In seven Indian languages, no amount of in-house review by a non-speaker could tell the difference — this needed people who speak the language every day.
  • Non-deterministic output with no fixed expected result The app's replies come from an AI model, so the same question does not produce the same answer twice. Pass criteria had to be written around relevance, naturalness, safety, and language correctness rather than exact-match comparison.
  • Speech input as the primary interface. Everything depends on the app correctly hearing a spoken question — in regional accents, at natural speed, in a room that is not silent. That is a hard surface to test from a desk, and it varies enormously by language.
  • Two surfaces that have to agree with each other Families learn the app from the translated feature page and then use the app itself. If the page calls a setting one thing and the app calls it another, the instructions stop working — a real barrier for the caregivers this product depends on.
  • Users for whom a wrong answer is not a cosmetic bug The app speaks on behalf of a child. A response that is irrelevant, unnatural, or ignores the profile's "topics to avoid" is a failure with a social cost for the child, not just a defect in a log.

Testvox Solution

  1. Native-Speaker Crowd Testers, One Language at a Time

    2. Testvox assigned native-speaker testers per language and ran a language-driven schedule across a 14-day window, coordinated by a dedicated project coordinator who allocated tasks, tracked daily progress, and handled client queries. Each language was executed against the same scenario set, so results stayed comparable across all seven rather than becoming seven unrelated test reports.

  2. A Shared Scenario Set With Stable Test IDs

    3. The same numbered scenarios ran in every language: app language switching and persistence, profile creation and editing, topics to avoid, communication style, attention span, short and long response modes, sentence starters, listening durations, manual and auto trigger-word modes, speech recognition in quiet rooms and with regional accents, transcription display, AI response relevance, spoken playback and replay, and chat history with the feedback loop. Keeping test IDs identical across languages is what later made the cross-language patterns visible.

  3. Acceptance Criteria Written for AI Output

    4. Because responses are generated rather than fixed, each AI scenario was judged against defined criteria — is the reply relevant to the question, natural to a native ear, grammatically correct, entirely in the selected language, age-appropriate, and consistent with the child's profile including any restricted topics. This let testers evaluate variable output consistently instead of arguing about one sample answer.

  4. App and Translated Web Content Tested Together

    5. Alongside the Android app, testers validated the translated feature pages for translation accuracy and completeness, natural wording, terminology consistency against the app's own labels, text expansion and layout, and navigation. One scenario had a tester who had never used the app set it up using only the translated page — testing whether the documentation actually works as documentation.

  5. Defect Reports Built for a Small Team to Act On

    6. Every defect was logged with the originating test ID, language, platform, reproduction steps, expected and actual result, severity and priority, and supporting evidence, in a single shared tracker. Daily status updates covered languages completed, features tested, defects found, and open issues.

Result

A Complete, Comparable Picture Across Seven Languages

294 test cases were executed — 210 on Android and 84 on the web — with an overall pass rate of 82%. Because every language ran the same scenarios, the client could see exactly where quality varied: Kannada passed 29 of 30 Android cases, while the Gujarati web experience passed only 4 of 12, making it the clearest priority in the whole report.

47 Defects, Triaged by Severity

47 defects were reported and classified: 17 high, 24 medium, and 6 low. They covered speech recognition accuracy, auto-mode trigger reliability, AI response relevance, response-mode behaviour, microphone and recording controls, chat history persistence, and web translation and layout — each traceable to the scenario and language that found it.

Cross-Language Patterns That Pointed at Root Causes

The most valuable finding was not any single defect but the repetition. The same scenarios failed in language after language: auto-mode trigger recognition and transcription display each failed in four of the seven languages, and AI response relevance failed in three. That pattern told the client these were systemic weaknesses in speech handling for Indic scripts rather than isolated translation errors — a different and far cheaper thing to fix than 47 separate bugs.

Confirmation Where It Mattered Most

The safety-critical behaviour held. The "topics to avoid" scenario, which verifies that restricted subjects stay out of generated responses, passed in all seven languages. Profile creation and persistence, language selection, and sentence-starter behaviour also passed consistently — giving the client evidence that the core design was sound while the speech and localisation layers needed work.

Conclusion

An AI assistant that speaks for a child has to be right in a way that ordinary software does not. It has to hear the question correctly, answer it in the child's language, sound like a child rather than a machine, and never say something it was told to avoid. Testing that requires people who can hear the difference, a scenario set consistent enough to compare across languages, and acceptance criteria built for output that is never the same twice. By running structured crowd testing with native speakers across seven Indian languages, Testvox gave XpressAble a release-readiness picture it could not have produced internally: a ranked defect list, a clear view of which languages were weakest, and — most usefully — the cross-language patterns showing which problems were systemic and worth fixing once.

Related Resources