Designing trust in a black box
Overview
In late 2023, AI products were rapidly emerging, but there were few established patterns for how professionals could safely work alongside them.
BeneDocs aimed to simplify clinical documentation using AI, but translating the technology into a usable product required balancing patient privacy, technical complexity and clinician control.
I led product design from discovery through to beta, shaping the end-to-end experience and aligning founders and engineering around decisions that balanced clinical needs with technical feasibility.
My role: Product Designer, leading discovery-to-beta experience design
Collaborators: Founders (Doctor & Nurse), Design, Engineering
Scope: Workflow design, interaction design, systems thinking, stakeholder alignment and design validation
Translating AI architecture into a clinical workflow
The challenge was determining which parts of the system needed to be visible for doctors to use it confidently, and which could remain behind the scenes.
BeneDocs relied on a multi-stage process involving de-identification, prompt chaining, document generation and re-identification.
Early discussions explored exposing more of this process to give doctors visibility into how the AI worked. But greater technical transparency introduced complexity without necessarily improving their understanding.
I structured the experience into five steps: create or select a template, paste patient notes, de-identify sensitive information, generate and re-identify the document, then edit and export the final output.
This shifted the experience away from the underlying AI architecture and towards the tasks doctors needed to complete.
The five-step workflow became a shared product framework for design, engineering and beta validation, connecting technical dependencies with a clear, end-to-end clinical experience.
A five-step workflow translated the underlying AI architecture into familiar clinical tasks, making complex processing easier to understand and navigate.
Designing for uncertainty without misleading users
Document generation relied on a twelve-step prompt chain with variable processing times. The system could not reliably report its position within that sequence, making conventional progress indicators potentially misleading.
A percentage-based progress bar suggested a level of precision the technology couldn’t support, while an indefinite spinner provided little reassurance that the system was functioning.
I chose to communicate meaningful system status rather than simulate progress the technology couldn’t measure.
I designed dynamic status updates around three priorities:
- Progress: Reassuring doctors that document generation was still underway.
- Security: Communicating that patient information had been de-identified before generation.
- Control: Reinforcing that doctors would review and edit the final document.
This gave users visibility into what mattered without exposing unnecessary technical complexity or creating false expectations.
The decision prioritised accurate feedback over the appearance of certainty, establishing a more dependable interaction with an unpredictable AI system.
Dynamic status messages communicated meaningful system activity without relying on unreliable completion percentages.
Prioritising output quality over onboarding speed
One of the most consequential product decisions was how BeneDocs would establish a doctor’s preferred documentation style.
The original approach used a doctor’s first set of notes as the default template. While technically straightforward and requiring less setup, it risked carrying formatting inconsistencies, incomplete structure or an unsuitable tone into subsequent documents.
The trade-off was between reducing upfront effort and improving the quality of the first AI-generated output.
I challenged the initial approach and aligned the team around a more intentional setup, where doctors provided an example document before generating their first output.
This introduced an additional onboarding step, but gave doctors greater control over the structure and style of their documents from the outset.
For an AI product, the first output was an important point of evaluation. An inaccurate or poorly structured document could undermine confidence before doctors experienced the product’s broader value.
Prioritising output quality over the shortest onboarding flow gave the product a stronger foundation for initial adoption and future support for multiple templates.
An example document established the doctor’s preferred structure and style before generation, prioritising output quality over minimal setup.
The outcome
BeneDocs progressed to a beta-ready product direction, with an end-to-end workflow designed around the realities of clinical documentation and the limitations of the underlying AI.
My work established three important foundations: a structured workflow that simplified technical complexity, system feedback that communicated uncertainty without misleading users, and a template approach that prioritised the quality of the first generated document.
Together, these decisions gave the team a coherent experience to take into beta, with clearer expectations around data handling, document generation and clinician control.
I left the project before launch, so I don’t have post-release adoption or usage metrics. The outcome I can directly speak to is the product direction and design foundation established for beta validation.
Designing around the limits of technology
BeneDocs reinforced that translating emerging technology into a usable product requires more than making complex systems appear simple.
The important decisions were about where to introduce structure, how to communicate uncertainty and when additional user effort was justified by a better outcome.
It shaped how I approach AI product design: understand the capabilities and limitations of the technology, then make deliberate decisions about what users need to see, control and trust.