What Is Trustworthy AI & Why Is It Important?

Artificial intelligence (AI) has become essential to the ways we function as a society. Interacting with systems that leverage AI is exciting when you receive a great song or movie recommendation, but not so much when it comes to determining whether you qualify for a financial loan, get interviewed for a job or receive specific medical treatment with systemic and algorithmic bias.

You might have come across these or similar headlines over the past several years, as an alarming number of AI-biased incidents have become more frequent and widespread:

• Amazon scraps secret AI recruiting tool that showed bias against women
• Wrongfully arrested man sues Detroit police over false facial recognition match
• Apple Card algorithm sparks gender bias allegations against Goldman Sachs
• New York regulator probes UnitedHealth algorithm for racial bias

These situations–which are detrimental to society and businesses alike–are a result of AI applications or systems not being "trustworthy," meaning they’ve failed to be designed and operated in a lawful, ethical and robust manner according to the definition from the European Committee (EC).

As AI Complexity Increases, So Does Risk Assessment Complexity

To provide guidance to organizations developing and running AI systems, the EC outlines several key principles that are required in order to be considered trustworthy. While this guidance is applicable to companies with business in the EU, many regulatory bodies around the world are following suit. Therefore, every organization that touches any part of AI should implement this (or a similar) system of principles and related techniques that covers the following: human agency and oversight; technical robustness and safety; privacy and data governance; transparency; diversity, non-discrimination and fairness; societal and environmental wellbeing; and accountability.

These principles should be used to inform the development and responsible use of AI technologies. With the increasing complexity of AI however, the risks related to violating any of these principles are growing. We’re not just talking about the evolution of more advanced and sophisticated data processing and modeling techniques, but also about emerging regulatory guidance, standards and policies. It’s becoming  more challenging to improve, assess and audit AI solutions for the variety of risk scenarios, as well as evaluate potential adverse impacts on your business and customers, as shown in the graphic below:

We all know how notoriously hard it is to plan and manage organizational change around AI, let alone ensuring ethics in how AI systems and applications are designed, used and maintained in the face of mounting challenges. So, it’s not surprising to see many organizations downplaying risks and deprioritizing efforts to mitigate them. For example, respondents in a recent global AI survey shared that they would evaluate risks associated with compromised equity and fairness or violating individual privacy substantially lower than the risks of being out of regulatory compliance. That’s why the EC is just one of the many lawmaking bodies and agencies expressing concerns, issuing calls for action and increasing regulatory pressure to enforce trustworthy AI. 

Setting Your Organization Up for Ethical Success

So, what should organizations do to manage the emerging risks of incompliance? While regulatory guidance and best practices are still in the very early stages of development, there’s one thing that seems to be quite certain: Mitigating risks associated with AI incidents by preventing them in a timely and effective manner is as much about establishing cross-functional ownership and governance and building an organization-wide awareness and culture, as it is about the methods and techniques used in research and development for ethical AI.

The practical, managerial, operational and engineering implications of this proposition for an AI-ethical organization should include:

• Having the ability to look inside the "black box" to understand how AI automates the decision-making process across industries and ensures bias-free and fair outcomes
• Establishing an AI model governance aligned with regulatory guidance, allowing both public and private sectors to work toward laws, standards and best practices
• Implementing product lifecycle and risk management frameworks tailored specifically to AI products, with a focus on data quality management aspects
• Adopting early prevention, performance monitoring and risk mitigation strategies embedded into the workflows for development and releasing products

This may sound like a lot to add to your organization’s AI department, which is often perceived as an expensive cost-center by management. However, the downside of doing little to nothing shouldn’t be underappreciated. Along with regulatory audits and fines, businesses can lose their market share and competitive edge when they misuse AI. When AI products do not perform well for a group of consumers, completely ignore them or discriminate against them, consumers will shift to a more responsible alternative. More importantly, these issues cause irrevocable damage to brand value.

As the AI market continues to evolve, your organization cannot afford to wait until regulatory measures are signed, sealed and delivered. The time is now to consider how you should be properly structured and organized to effectively design and operate trustworthy AI.

Stay tuned for our next post where we will dive into the first topic listed above, also known as “Explainable AI.”