Twitter's Information Operations - An OSINT AnalysisReturn Home

Key Takeaways

In our last OSINT analysis of Facebook’s Coordinated Inauthentic Behavior we highlighted the pitfalls of Facebook’s data-sharing policies and the lack of transparency when it comes to processes and awareness of influence campaigns on the platform. Although, previous work has been done on some of the Twitter datasets - in this analysis, we extend our work to examine Twitter’s Information Operations (IO) and the measures they are taking (or neglecting) to combat the rampant growth of disinformation, misinformation, and influence campaigns. All the data used in this analysis was downloaded from Twitter’s archives of suspended accounts. The data from Twitter can be accessed on their transparency report, whereas our aggregated data for this analysis is available through GitHub, including the script used to generate the datasets – feel free to send us pull requests.

Platforms like Facebook and Twitter allow ordinary people, civic groups, and journalists to reach a vast and global audience. Controversially, they have also provided an extremely efficient and inexpensive platform for malign influence operations by foreign and domestic actors alike. It’s been well documented how those platforms are being used to construct people’s digital DNA, steer public debate, set the agenda of what journalists are covering, recruit terrorists, reshape warfare itself, and even “change reality”.

The intensification of election meddling, the widespread false information dissemination, and rise of populism and extremism coincide with the growth of online mobs that include both authentic users and automated spam accounts. They intend to build large audiences around similar interests.

Digital Tribes

Influence campaigns thrive on basic crowd psychology tactics that are being mobilized and manipulated by both domestic and foreign actors alike. Instead of building relationships and groups to push a meticulous and strategic message, Twitter is used strategically to join conversations and amplify the dominant narrative. This instigates a psychological bias based on tribal affiliations, creating an ecosystem enmeshed in distrust, paranoia, cognitive blind spots, and one dimensional critical thinking –operating in a very similar manner to cults.

Although there are many ways to tackle this from a psychological perspective, one broad way to look at Twitter’s Information Operations is through the lens of Cultural Cognition, which exploits the basic processes of identity formation in humans. Once we identify with a group (joining a cause, following a trendy narrative, or contributing online to the public debate and discourse) we shape our opinions to conform to the views of the groups with which we most strongly identify with. Leading to two outcomes: it creates solidarity in the group, which increases the chances that our group’s views will prevail online (or even in society at large), and it strengthens the group’s acceptance of us as members in good standing.

Once the threat of the “other” is created (whether bona fide or totally fabricated threats), the more we circle the wagons of our opinions to keep the tribe together and keep our identities intact. This creates an inflexible war of polarities that impede compromise and progress.

Social media platforms, like Twitter, offer fertile grounds to not only create echo chambers that circulate and amplify narratives, but to amass a receptive audience. In this environment, confirmation bias is algorithmically propagated, on a mass scale.

Identifying Malign Behavior

There is still ambiguity concerning how Twitter identifies Information Operations on its platform. A lot of details are yet to be unearthed.

So, how do they identify IO and how do they link accounts together to assume that they are operating together? According to Vijaya Gadde, Legal, Policy and Trust & Safety Lead at Twitter, metadata is used to link accounts’ phone numbers, or email addresses, and in some cases IP addresses. They also rely on online reporting, and tips from external firms.

According to Renee DiResta, from the Stanford Internet Observatory, there are three criteria commonly used to assess whether a given page, account cluster, or channel is manipulative.

Perils of Censorship

Tech giants are responsible for public discourse on a scale unprecedented in human history. Given that centralized global policies at scale are almost impossible to draft and apply, some exceptions include the case of communities on Reddit that have their own moderators that enforce policies.

Although the question of censorship, and free speech vs free reach, go far beyond this analysis it is stills important, however, to bring attention to the protection that those companies are relying on: Section 230 of the Communications Decency Act. Part of the Telecommunications Act of 1996, this piece of legislation, which has been established well before Twitter and other platfroms, gives social media companies broad immunity from being sued for user behavior. Necessary but urgently needing a timely upgrade, this legislation has come under increasing scrutiny, with many critics arguing that tech firms need more accountability.

The road to more transparency and accountability is long, albeit sluggish:

“We are not done. We are not finished.”
- Jack Dorsey, CEO of Twitter, on the Joe Rogan podcast 

Data

In this analysis, we focus on compiling and presenting the released datasets by Twitter pertaining to Operation Information takedowns. This analysis, similar to our previous work on Facebook’s Coordinated Inauthentic Behavior, is an ongoing and open-source project. Contributions, suggestions, and feedback are all encouraged! – Access the full dataset on Github. Moreover, the Python script that was used to compile the data is also available in the Twitter folder.

According to Twitter’s transparency report, Information Operations specifically pertain to alleged state-backed foreign influence campaigns. This leaves room for speculation and (mis)interpretation of the published datasets. Twitter’s definition of IO, in and of itself, excludes organized campaigns operating domestically, or sophisticated campaigns operating on behalf of foreign actors in a fragmented fashion. To tackle parts of this shortcoming, we wanted to make use of the Georeverse code tagging, but it was only possible in the Iranian dataset. All other geolocation data was removed by Twitter from their published dataset, and the columns longitude and latitude were just displaying “present” in those other datasets. This is unfortunate as we were hoping to be be able to draw additional conclusions from the geographical contexts. Tweet languages, account languages, hashtags, and urls (which we used to extract unique domains), have enabled us to draw a wider context to conduct this analysis. Unlike Facebook, which only discloses numbers without additional information around contexts.

Twitter’s published datasets seem to exhibit another pitfall. Although defined under another category (i.e., Platform Manipulation), spam behavior is included as part of Information Operations, this is most notable when examining published datasets pertaining to the Saudi Arabia takedowns, although, an analysis by the Stanford Internet Observatory revealed that some of the spam accounts appeared to attempt to conceal their commercial and political activity by mass-tweeting of religious, sports, and poetry content. According to the same report, approximately 7% of tweets came from client apps that automatically tweeted religious invocations, Dua’. This article on the Emojitracker captured an interesting trend of using the emoji “♻️” as part of the religious bots posting Dua’ tweets on behalf of authentic users.

Given that we are also tracking Facebook’s efforts to combat influence campaigns (termed Coordinated Inauthentic Behavior) on their platform, we couldn’t help but compare and contrast the difference in their strategies and processes in relation to Twitter. A deeper look at the differences between Facebook’s CIB takedowns and Twitter’s Information Operation datasets reveals the discrepancy in the shared information:

FacebookTwitter
Partial vs Full DisclosureNo DataGood
Data DiscrepancyNo DataMedium
Verified Accounts InformationNo DataNot Present
Attribution BlindspotsHighHigh
Domestic Information OperationsLowNone
Users NotificationLowLow

Attribution Bias vs (mis)Attribution


Although it is important to acknowledge the role that foreign actors play in public discourse, it is more important to remain critical of attribution patterns. The Russian tactics are being closely studied and replicated by other groups, such as political parties during Senate elections in the US. Twitter also admitted to misidentifying and falsely attributing around 230 accounts originally thought to be linked to the Russian Internet Research Agency. The accounts were later found (by an independent researcher) to be associated with a Venezuelan operation.

We argue that the lack of transparency and inconsistency by Twitter restricts independent researchers from conducting thorough investigations of claims and attributions. This feeds into the popular narrative of collective fear/paranoia and constructed political foes.

Another important point to raise is: correlation does not equal causation. While foreign operations have definitely seeped into the public life, we ought to question whether they actually act as instigators or whether they simply jump on the trending hashtag bandwagon and magnify their presence.

Conclusion

Disinformation, misinformation, and influence campaigns have become a normal part of the digital public sphere. Employed tactics are continuously evolving at a pace no longer containable by social media platforms. Unless Facebook, Twitter, and others, increase their transparency about shared information, and encourage more open-source investigations, those companies will continue to play a non-stop game of whack-a-mole with influence campaigns.

As tactics and Information Operations are evolving, the modus operandi is also evolving: influence campaigns are also using other platforms such as Reddit, YouTube, or even Tiktok. Last year, Google’s Threat Analysis Group closed 210 YouTube accounts “to combat coordinated influence operations” but there is very little information on the other accounts that they have closed. It would also be nice if Google’s Threat Analysis Group released Twitter-like datasets about those accounts.

Unfortunately, very few companies provide comprehensive datasets and we can only ask ourselves what new and creative ways domestic and foreign actors are utilizing to upgrade their influence campaigns. Until then, those campaigns will keep seeping into the collective conciousness… For instance, Netflix docuseries are growing in popularity, could they also be subconsciously used against us?