AI project aims to deliver archive reform

Getting machines to understand the context of government records will be key to automating the archiving of petabytes of data.

The federal government has announced millions of dollars in grants to Australian tech companies, to help spur innovation that will solve some of the trickiest technology challenges it currently faces.

Under the Business Research and Innovation Initiative (BRII), six businesses have been granted $1 million each to develop their proposed solutions. That $6 million is in addition to $1,465,597 allocated in feasibility study funding.

“Some businesses are working to use intelligent data to keep our tourism industry at the leading edge, while others are ensuring we manage risks of hitchhiking pests on shipping containers,” said the federal Minister for Industry, Science and Technology, Karen Andrews.

“This funding will help businesses which have completed feasibility studies further develop their innovative solutions.”

The BRII program challenged tech companies to solve specific problems within the following topic areas:

1. Providing fast and secure digital identity verification for people experiencing family and domestic violence.
2. Using intelligent data to transform tourism service delivery.
3. Upgrading government’s capability to help deliver world-leading digital services.
4. Managing the biosecurity of hitchhiking pests and contaminants on shipping containers.
5. Automating complex determinations for Australian Government information.

Amongst the $1 million recipients working to protect Australia’s flora and fauna from pests, diseases and contaminants that can arrive on sea containers is Industry Spec Drones, which proposes to use unmanned flight technology to manage biosecurity risks, and Trellis Data, which will use detection technologies such as microwave and infrared to manage potential biosecurity threats.

Another firm, WEJUGO, will use its $1 million grant to “develop a visitation and tourism analytics platform that combines data from transactional, telecommunications, social media and other digitally sourced data into a 360° view of tourism impacts across economic, environmental and cultural performance metrics”, according to BRII program documents.

And Surround Australia will leverage existing tech platforms to build a solution that “identifies the cultural and heritage dimensions of records”.

Automating data determinations

The overall aim of the BRII’s ‘Automating complex determinations for Australian Government information’ theme is to “develop an accurate and scalable way to decide the value of government digital information and data and to determine whether it should be preserved or destroyed” using “artificial intelligence, machine learning, automation, data management and analytics, data science, archiving, business process management” technologies, according to the BRII program.

This challenge is not just a theoretical construct. The National Archives of Australia (NAA) has a real problem on its hands with dealing with the volume of data generated by government, and deciding which records need to be kept and for how long, or whether they can be destroyed.

At the moment, this process is done largely manually, which imposes a huge cost burden on the NAA and government in general.

According to BRII program guidance, “The National Archives is looking for an automated, innovative, accurate and reliable solution to create and manage complex decisions about the value of information and data. Humans can then redirect their efforts towards exceptional and complex decision points. This product would be attractive to governments at all levels, as well as any private sector or not-for-profit organisation that manages information and data.”

One company aiming to tackle this challenge is Canberra-based Lenticular, one of the $1 million grant recipients. Lenticular aims to “develop a system that aims to help government make informed decisions about record keeping by developing and crafting contextual knowledge, and accessing this knowledge through user-configurable rules”.

Lenticular was co-founded by Trevor Christie-Taylor and Luan Nguyen, two members of the team behind the Parliamentary Document Management System (PDMS). The PDMS connects Australian Government agencies and parliament under the whole-of-government Parliamentary Workflow Solution system. It is used by more than 50 agencies, has 26,500 registered users and processes an average of 302,000 records every year.

Lenticular’s NAA-challenge inspired BRII project is partly a response to its experience in developing the PDMS, and will rely heavily on artificial intelligence.

“Currently, AI is like a butler, which comes with a timely suggestion and helps answer a question that’s quite narrowly defined. And were trying to open that up a bit. We’re trying to create a more personal relationship between the AI and the users of the organisation,” Christie-Taylor said.

“We’re working on a system where the user is in charge of what is being learned… so the user’s actually steering it like a car. They know they’re steering it. They know they’re guiding the AI towards what needs to be done.

“So instead of asking a narrow question like, ‘What is the weather like in Canberra?’ we’re asking a potentially very difficult question, like ‘Is this document important?’”.

Christie-Taylor points out that it’s hard enough for people to answer such questions, because nobody knows everything. It’s even more difficult for machines to do it, because the question becomes ‘Important to whom and why?’

“But if we put those two in combination, the people and the machine, the people continuously teaching the machine ‘this sort of thing is important to us’ and continually prodding that ‘this particular controversy means a lot to us’,” it will teach the AI which documents are important and why, Christie-Taylor said.

“We’re developing petabytes of data in the form of text every year, and what are we going to do with that stuff?” he asks. “Because some of it is going to be really important to tell the story of Australia. And some of it is just text; it cannot possibly be important to anybody.

“The machine cannot possibly know, just by looking at the words, that ‘this is important’. It has to understand the context, and that’s really where we’re coming from.”

Image credit: ©stock.adobe.com/au/peshkov