Graffiti, garbage and fires — AI tackles city problems

Consolidating numerous data streams into a city’s data lake will enable AI-powered applications running across multiple departments.

The garbage truck rumbles down the street, and its cameras pour video into the city’s data lake. An AI-powered application mines that image data looking for graffiti — and advises whether to dispatch a fully equipped paint crew or a crew with just soap and brushes.

Meanwhile, cameras on other city vehicles could feed the same data lake so another application detects piles of garbage that should be collected. That information is used by an application to send the right clean-up crew. Citizens, too, can get into the act, by sending mobile phone pictures of graffiti or litter to the city for AI-driven processing.

Applications such as these provide the vision for the Intelligent Internet of Things Integration Consortium (I3), a new initiative launched by the University of Southern California (USC), the City of Los Angeles and a number of stakeholders including researchers and industry. At USC, I3 is jointly managed by three institutes: the Institute for Communication Technology Management (CTM), the Center for Cyber-Physical Systems and the Internet of Things (CCI) and the Integrated Media Systems Center (IMSC).

“We’re trying to make the I3 Consortium a big tent,” said assistant professor at the USC Marshall School of Business’s Institute for Communication Technology Management (CTM) Jerry Power, who serves as executive director of the consortium. “Los Angeles is a founding member, but we’re talking to other cities and vendors. We want lots of people to participate in the process, whether a start-up or a super-large corporation.”

As of now, there are 24 members of the consortium, including USC’s Viterbi School of Engineering and Marshall School of Business. And companies are contributing resources. Oracle’s Startup for Higher Education program, for example, is providing US$75,000 per year in cloud infrastructure services to support the I3 Consortium’s first three years of development work.

The I3 Consortium needs a lot of computing power. The consortium allows the cities to move beyond data silos where information is confined to individual departments, such as transportation and sanitation, to one where data flows among departments, can be more easily managed and also lets cities use data contributions from residents or even other governmental or commercial data providers. That information is consolidated into a city’s data lake that can be accessed by AI-powered applications across departments.

The I3 Consortium will provide a vehicle to manage the data flow into the data lake, explained Cyrus Shahabi, a professor at USC’s Viterbi School of Engineering and director of its Integrated Media Systems Center (IMSC). IMSC is using Oracle Cloud credits to create advanced computation applications that apply vast amounts of processing needed to train AI-based, deep learning neural networks and use real-time I3-driven data lakes to recognise issues, such as graffiti or garbage, that drive action.

IMSC is developing a suite of open source software that can run in the cloud or in a city’s own data centres. The code and documentation will provide the base needed for cities, states or even large university campuses to create data lakes and invent new applications of IoT technology. Initially, IMSC is focusing on collecting and analysing images from still and video cameras.

IMSC’s software will be ported to I3 as a data broker, which will ingest streams of camera information, using deep learning to label the data (such as ‘area with graffiti’) in the cloud and make that data available to other applications through I3. The data will also contain valuable metadata, such as the GPS location of the camera, the direction the camera was facing and the image’s environment, such as urban, suburban, rural, parkland or wilderness — all of which are possible in Los Angeles’ 1300-plus square kilometres.

“Labelling is the part that’s very computational-intensive,” he said, “and in a streaming environment, it’s difficult to do this labelling in real time. That’s where the Oracle Cloud credit is very useful. It significantly improved the speed at which we can train the machine-learning systems to efficiently perform the data-intensive classifications so we can do labelling very quickly.”

The I3 research is important because it is “taking us closer to the reality of an IoT-enabled smart city ecosystem”, said Oracle Group Vice President Patrick Mungovan. And the research could be applied in the future well beyond municipalities to other connected community efforts, Mungovan said.

Power sees many future applications of the I3 Consortium’s work, some of which are in development with the City of Los Angeles. One system in the works could integrate video feeds from city-owned and private car parks and parking structures, and work with a mobile app to guide drivers directly to open spaces. Another project would rapidly spot and classify fires burning within the city.

The fire use case is very complex, as it requires accurately understanding whether a fire poses a real danger: The Los Angeles Fire Department doesn’t want to miss uncontrolled fires, but it doesn’t want to deploy trucks and firefighters for false alarms or controlled burns. That’s where the AI comes in, plus USC’s own computer scientists, who are always on the lookout for really tough problems to solve. “We may put in our own resources because fire classification is a challenging research project,” Shahabi said.

The I3 Consortium, announced in November, is still in its formative stages, raising money, signing up partners, conducting feasibility studies and working on intellectual property agreements.

Once the foundational work is complete, the real work begins: Getting data into the data lake, processing it to add value and then using that information to drive action. “If you’re not driving action, there’s no value,” Power said. “We want lots of IoT data to go into the lake, and we want to make that data actionable. That’s our vision.”

Alan Zeichick is a senior technology editor for Oracle.

Please follow us and share on Twitter and Facebook. You can also subscribe for FREE to our weekly newsletter and quarterly magazine.