Six years ago we started on the code that has evolved into the Netwar System, named in a nod to
Briefly, the system permits groups of Twitter users to add accounts to multiple lists for the sake of observation, it's possible to capture internal discussions by connecting to a Slack server, and content is stored in Elasticsearch. We released the full code base about a month ago:
The system evolved some batch collection abilities over the last year and we are releasing data on the 31.8M unique followers of legislators for Australia, Canada, New Zealand, the United Kingdom, the United States, and the members of the European Parliament.
This content contains just the names of the legislator accounts and the numeric IDs of followers - which adheres to Twitter's requirements for publishing 'dehydrated' datasets. Our Elasticsearch cluster contains this data in a more accessible form but we have a bit more work to do with Search Guard before we can let others access it.
The next features for the system will be the ingestion of Telegram data and RSS feeds, which should happen over the next month or so.
If you have any questions about the system or the data being released on Figshare I would be happy to answer them.
--
Wire:nrauhauser
sms:202-642-1717
