Opting for open source when implementing explainable AI

From a government perspective, Australia must trust the foundation that AI technologies are built on. Could the solution lie in open source software?

When the federal government revealed its $124 million Artificial Intelligence (AI) Action Plan in 2021, it marked the beginning of a new era of public sector confidence in the technology. It was clear the government had been thinking about this for some time. Prior to announcing its overarching AI Action Plan, the government released an AI Ethics Framework, a collection of voluntary guidelines for businesses and government entities to responsibly design, develop and implement AI.

Since the AI Action Plan was released, the macro and micro environment in Australia and globally has changed dramatically, sparking new debates and conversations about changing government policies to fit the new normal. One of the standout transitions is the request for transparency and access to information around how the government funds and operates citizen services. Combined with a heightened cybersecurity landscape, the question is: how does the government provide open access in this complex operating environment whilst maintaining secure platforms that build trust?

From the government’s perspective, if Australia is to realise the immense potential of AI, it needs to be able to trust the foundation these technologies are being built on and be confident that a particular AI technology has come to the correct conclusions within the context of the data that it is being fed. By gaining better visibility into the AI process, government entities using the technology can achieve ‘explainable AI’ — the ability for machines to clearly demonstrate and explain the rationale behind their conclusions. Understanding the process that went into the AI — how specifically it came to a calculation — can help users feel more confident that the solutions they are being presented with are the right solutions. This capability can offer greater trust that an ethical framework can be applied and maintained when using AI technology.

However, not all AI software offers the kind of visibility and transparency needed to adhere to such an ethical framework. For example, proprietary software tends to be very opaque by design as the developers, and the organisations that employ them, want to protect their work and intellectual property. As such, it is exceedingly difficult to tell how a piece of software actually works — the paths that it takes to process and analyse data, from ingestion to recommendation. Without understanding the process, it can be difficult to achieve trusted, explainable AI.

The solution might lie in the adoption of open source software to develop these technologies. Open source software, along with the core cultural tenets of the open source community, can help the public sector achieve its AI objectives. With the right combination of technology and development methodology, agencies can build more transparent AI solutions faster, resulting in greater efficiencies and more accurate and trusted decisions.

Opting for open source

One of the best ways governments can gain the visibility needed to ensure that AI technology can adhere to an ethical framework is by powering it with highly transparent, standards-based software. This is where open source software can help. Open source software is built on the concept of transparency and open collaboration.

To provide some context, open source software is code that is designed to be publicly accessible for anyone to see, modify and distribute as they see fit. Because it is developed in an open and collaborative way, open source software is often less expensive and more flexible than proprietary software.

But most importantly, it is eminently transparent. Open source draws upon a decentralised production model that often involves many contributors working together to find new ways to solve problems in their communities and industry. This requires absolute transparency to work. As a bonus, with so many eyes on the source code, there are fewer chances of security issues slipping through the cracks during the development process.

Indeed, 84% of government IT leaders surveyed for the State of Enterprise Open Source: Highlights from the government sector report said they believe that enterprise open source is as secure as or more secure than proprietary software.

Governments can draw upon the transparency of open source software to build AI systems that are able to incorporate and meet ethical guideline requirements. Open source software allows visibility into how a particular conclusion is derived from a specific dataset.

This kind of visibility not only has the potential to help government agencies better understand how their AI processes analyse data and reach their conclusions, it also offers an unfiltered view of the processes behind the otherwise opaque ‘black box’ technology that sometimes powers AI. At the same time, the security offered by open source has the potential to create safer AI systems.

Opening up to innovation

The government is increasingly embracing innovation. This is a core theme of the AI Action Plan, which aims to see Australia become a global leader in the development and adoption of trusted, secure and responsible AI.

The government’s efforts to support and drive AI technology development mirror efforts in the private sector. Organisations in the private sector are well aware of the value of open source software, and AI is one of the areas seeing surging usage of such software.

For example, 80% of organisations surveyed in our latest State of Enterprise Open Source report indicated they were planning to increase their use of enterprise open source software in areas such as AI, machine learning (ML), edge computing and the Internet of Things (IoT).

Why? A big part of it is that the IT leaders surveyed continued to perceive enterprise open source in an increasingly favourable light, with 77% indicating they had a more positive perception of enterprise open source than they did just a year earlier. This trend could have something to do with the cost, flexibility and security aspects that open source software can offer different use case implementations, including AI.

At the same time, 78% of the government IT leaders endorsed enterprise open source solutions at a time when flexibility and innovation were needed most. Meanwhile, open source software’s lower total cost of ownership ranked as its most favourable benefit among government sector survey respondents.

This is important. Public sector entities in Australia, as in many other countries, are typically bound by procurement rules that compel agencies to find and engage suppliers that represent the best value for money. With the lower total cost of ownership offered by open source software, it represents a logical choice for many government entities.

All of these factors contribute to the rising role that open source can play in enabling the federal government to achieve the ‘explainable AI’ capability that can underpin an ethical framework for the use of AI systems in Australia, ultimately helping to drive the country’s AI ambitions.

Australia has the chance to reap enormous benefits from AI and open source software can provide a pathway to making this a reality.

Image credit: iStock.com/natasaadzic