The benefits and risks of AI usage in the public sector

In little more than 12 months, generative AI has evolved from being a technical novelty into a powerful business tool. However, senior IT managers in the public sector believe the technology brings with it risks as well as benefits.

According to the Immuta 2024 State of Data Security Report, 88% of senior managers say their staff are already using AI tools. This is regardless of whether their organisation has a firm policy of adoption.

Asked to nominate the key IT security benefits offered by AI, respondents to the Immuta survey pointed to improved phishing attack identification and threat simulation as two of the biggest. Others included anomaly detection and better audits and reporting.

When it came to identifying AI-related risks, inadvertent exposure of sensitive information by employees and unauthorised use of purpose-built models out of context were nominated by respondents. Additional named risks included the inadvertent exposure of sensitive data by large language models (LLMs) and the poisoning of training data.

These risks are significant when you consider the large volumes of sensitive and personally identifiable data held by departments and agencies across the sector. Theft or exposure of this data could have significant implications for citizens.

Continuing growth

Despite these concerns, uptake of AI by public-sector agencies appears likely to remain brisk. Analyst firm Gartner predicts that IT spending will increase more than 70% during the next year, and a significant portion will be invested in AI-related technologies and tools. Organisations will need to continue to embrace this new technology to remain competitive and relevant in today’s economic landscape.

It’s likely that 2024 will also become the year of the AI control system. Aside from the hype surrounding generative AI, there is a broader issue around developing a control system for the technology. This is because AI brings an entirely new paradigm where there is little or no human control. AI initiatives therefore won’t get into full-scale production without a new form of control system in place.

At the same time, public-sector organisations will come to realise that, as AI usage increases, they need to focus even more attention on data security. As we have seen with governments around the world, there has also been an urgent need to enact news laws and regulations to ensure that data privacy and data security concerns with generative AI are addressed.

As the technology evolves, it will become clear that the key to harnessing the power of large-language model (LLM)-based AI lies in having a robust data governance framework. Such a framework is essential not only for guiding the ethical and secure use of LLMs, but also for establishing standards for measuring their outputs and ensuring integrity.

The evolution of LLMs will open new avenues for applications in data analysis, customer service and decision-making processes, further embedding LLMs into the fabric of data-driven industries.

The biggest winners when it comes to AI usage will be the departments and agencies that create real value from better data engineering processes that are used to leverage models using their own data and business context. The key impact for these companies will be better knowledge management.

An ongoing reprioritisation and reassignment of resources

With the pace of change in technology and data usage likely to continue to increase, departments and agencies will be forced to redirect resources into new data-related areas that will become priorities. Examples include data governance and compliance, data quality and data integration.

Despite ongoing pressure to do more with less, organisations can’t and won’t halt investment in IT. These investments will be focused on the critical building blocks that form the foundation of a modern data stack that is required to support AI initiatives.

Also, the traditional demarcation between data and application layers in an IT infrastructure will be replaced by a more integrated approach focused on data products. Rather than a few dozen apps there will be hundreds of data products. Dubbed a “data-centric architecture”, this approach will allow organisations to extract greater value from their data resources and better support their operations.

By working closer to the data, data teams can reduce latency and improve performance, opening up new possibilities for real-time reporting and analytics. This, in turn, supports better decision-making and more efficient business processes. The coming year will see some fundamental changes in the way the public sector manages and works with AI and data. Those agencies that take time to experiment with the technology and determine its best use cases will be best placed to extract maximum value and achieve optimal results for citizens.

Top image credit: iStock.com/wildpixel