David M. Douglas

An emerging use of machine learning (ML) is creating products optimised using computational design for individual users and produced using 3D printing. One potential application is bespoke surgical tools optimised for specific patients. While optimised tool designs benefit patients and surgeons, there is the risk that computational design may also create unexpected designs that are unsuitable for use with potentially harmful consequences. We interviewed potential stakeholders to identify both established and unique technical risks associated with the use of computational design for surgical tool design and applied ethical risk analysis (eRA) to identify how stakeholders might be exposed to ethical risk within this process. The main findings of this research are twofold. First, distinguishing between unique and established risks for new medical technologies helps identify where existing methods of risk mitigation may be applicable to a surgical innovation, and where new means of mitigating risks may be needed. Second, the value of distinguishing between technical and ethical risks in such a system is that it identifies the key responsibilities for managing these risks and allows for any potential interdependencies between stakeholders in managing these risks to be made explicit. The approach demonstrated in this paper may be applied to understanding the implications of new AI and ML applications in healthcare and other high consequence domains.

As consensus across the various published AI ethics principles is approached, a gap remains between high-level principles and practical techniques that can be readily adopted to design and develop responsible AI systems. We examine the practices and experiences of researchers and engineers from Australia’s national scientific research agency (CSIRO), who are involved in designing and developing AI systems for many application areas. Semi-structured interviews were used to examine how the practices of the participants relate to and align with a set of high-level AI ethics principles proposed by the Australian Government. The principles comprise: (1) privacy protection and security, (2) reliability and safety, (3) transparency and explainability, (4) fairness, (5) contestability, (6) accountability, (7) human-centred values, (8) human, social and environmental well-being. Discussions on the gained insights from the interviews include various tensions and trade-offs between the principles, and provide suggestions for implementing each high-level principle. We also present suggestions aiming to enhance associated support mechanisms.

We present an overview of the literature on trust in AI and AI trustworthiness and argue for the need to distinguish these concepts more clearly and to gather more empirically evidence on what contributes to people’s trusting behaviours. We discuss that trust in AI involves not only reliance on the system itself, but also trust in the developers of the AI system. AI ethics principles such as explainability and transparency are often assumed to promote user trust, but empirical evidence of how such features actually affect how users perceive the system’s trustworthiness is not as abundance or not that clear. AI systems should be recognised as socio-technical systems, where the people involved in designing, developing, deploying, and using the system are as important as the system for determining whether it is trustworthy. Without recognising these nuances, ’trust in AI’ and ’trustworthy AI’ risk becoming nebulous terms for any desirable feature for AI systems.

The term ’ethical risk’ often appears in discussions about the responsible development and deployment of artificial intelligence (AI). However, ethical risk remains inconsistently defined in this context, obscuring what distinguishes it from other forms of risk, such as social, reputational or legal risk, for example. In this paper we present a definition of ethical risk for AI as being any risk associated with an AI that may cause stakeholders to fail one or more of their ethical responsibilities towards other stakeholders. To support our definition, we describe how stakeholders have role responsibilities that follow from their relationship with the AI, and that these responsibilities are towards other stakeholders associated with the AI. We discuss how stakeholders may differ in their ability to make decisions about an AI, their exposure to risk, and whether they or others may benefit from these risks. Stakeholders without the ability to make decisions about the risks associated with an AI and how it is used are dependent on other stakeholders with this ability. This relationship places those who depend on decision-making stakeholders at ethical risk of being dominated by them. The decision-making stakeholder is ethically responsible for the risks their decisions about the AI impose on those affected by them. We illustrate our account of ethical risk for AI with two examples: AI-designed attachments for surgical robots that are optimised for treating specific patients, and self-driving ’robotaxis’ that carry passengers on public roads.