AI Platform targets crowd-sourced data for military intelligence

The outcome

University of Melbourne researchers in collaboration with Australia’s Defence Science and Technology Group (DSTG) have developed RAPID (Real-time Analytics Platform for Interactive Data-mining), an artificial intelligence-led platform that delivers immediate, high-quality knowledge for analysts from a range of fast-moving data streams, including social media.

“RAPID is a big-data processing platform that uses certain techniques, paradigms and algorithms to analyse a large amount of data in real time,” says Professor Shanika Karunasekera, of the School of Computing and Information Systems in the University’s Faculty of Engineering and Information Technology.

RAPID tracks topics by extracting keywords and hashtags from millions of user posts and discussions, discerning genuine sources from malicious actors, spam or fake accounts.

The platform can expand the connections between the authors of posts and followers, to quickly zero in on significant data, creating visual summaries of posts, users and topics, and the interactions between them.

“Whether it’s information about an emergency like a bushfire, a political protest or military, important information is typically shared via the internet by people who are in the area at the time,” says Prof Karunasekera.

This can be key information for defence intelligence, she adds.

The collaboration between DSTG and the University of Melbourne taps into expertise that can apply high-level analysis of information networks.

Users currently log into the cloud-based RAPID platform to access data and analysis, but the platform also has the ability to push data to downstream systems for more secure analysis.

“The technology we have developed allows different kinds of decision-support scenarios where events can be compared and time-sequenced,” says Prof Karunasekera.

“In military scenarios, where effective decisions rely on comprehensive and trustworthy information, the RAPID platform can deliver timely analysis in a fast-moving situation.”

Learn more about Defence research

The need

According to Prof Karunasekera: “Warfare is no longer focused purely on military hardware – democracies are increasingly being threatened by a misinformation war, where fear campaigns are used to incite discord and to radicalise individuals.”

In times of disaster, mass uprising and political unrest, social media sites such as Twitter, Facebook and even Reddit publish vital information from on-ground witnesses that can improve responders’ decision making, reduce time and potentially save lives.

Crowdsourcing information now plays a vital role in military intelligence, as agencies recognise that the more people who contribute data about global security threats, the more accurate this data will be.

But it can be hugely challenging for people to quickly uncover reliable and useful information from the many gigabytes of crowdsourced data now generated via social media.

Dr Lucia Falzon is a mathematician with expertise in social network analysis and a former research analyst who spent several decades at DSTG, and helped develop the RAPID platform.

Following 9/11, there was growing recognition that tracking and monitoring criminal and terrorist networks could both deter and prevent attacks, she says, with agencies tasked with quickly decoding the secretive connections between these network members.

The research

DSTG worked with social scientists at the University of Melbourne, tapping into their extensive expertise in mapping physical, real-world social networks to build mathematical models that could better analyse and predict these relationships.

Once a social system can be recognised mathematically, it can be analysed at scale, and patterns recognised.

“By treating complex social networks as mathematical objects, we can use graph theory to assess which people are most or least connected, and which person is at the intersection of many paths and might therefore control information,” says Dr Falzon.

By 2015, social media sites were generating vast amounts of data hour by hour, and this team developed mathematical models of social networks which, when populated with data, could reveal broad patterns.

“The work we did with Professor Karunasekera involved filtering the essential information from a big, noisy dataset so we could formally construct networks in a way that would help us to analyse them,” Dr Falzon says.

The RAPID platform was developed to simultaneously soak up social site data while characterising each interaction (from a ‘like’ on a social media post to long discussion threads), clustering users, topics and keywords and showing the relationships between different people within the network.

Applying the technology

The research group tested the platform in 2018 during the Yellow Vest social movement in France.

During the protests, more than 200,000 people responded to fuel price hikes with peaceful protests – but a small group of rioters from both far right and far left groups, vandalised public monuments, looted shops and attacked police.

“This test showed that RAPID was very effective … and as streams of traffic were analysed in real time, we could even get a real sense of the mood of the crowds in different locations,” Dr Falzon says.

The platform has since been trialled in a range of different scenarios, including by DSTG and by researchers from the US Army Research Laboratory.