Artificial Intelligence, in context: The Collaboration for trANslational Artificial Intelligence tRIals (CANAIRI) and the value of local evaluation of AI
Abstract
The literature and governance frameworks relating to the clinical translation of artificial intelligence (AI) products are coalescing around best practices. A major knowledge gap concerns demonstration of local, on-the-ground evidence of on-the-ground performance, equity, and safety. The ‘silent trial’ (also known as ‘silent evaluation’, ‘shadow trial’, ‘shadow evaluation’) refers to the integration of an AI system into the intended operational systems without affecting patient care. A silent trial enables collection of on-the-ground evidence of performance while integrating data security considerations, assessing operational feasibility, validating deployment strategy, selecting risk category thresholds, and testing workflow integration. While increasingly recognized for their value, silent trials remain under-appreciated and under-utilized both in the breadth of their uptake and in the scope of considerations they can address - for example, local fairness assessments and patient engagement, among others. This commentary introduces the Collaboration for trANslational Artificial Intelligence tRIals (CANAIRI), an international consortium initiative that aims to develop guidance around the application of holistic silent evaluations for healthcare AI tools.
