Ted Cruz’s trollbot army, looking conclusive edition

With deeper data analysis, we’re seeing still more worrisome signs that Ted Cruz’s recent Twitter surge has been driven by bots or other manipulative accounts.

The upshot here? In a comparison between the 200 most active posters in the #TXSen race and the 200 most active posters in all other competitive races, the #TXSen posters (almost all Cruz supporters) were about 40% more likely to be deemed “trollbots.” More than three-quarters of top #TXSen posters are rated “Problematic” or “Alarming.” 

Last week we looked at the top 20 most frequent users of Cruz’s #KeepTexasRed hashtag. Bot Sentinel’s “Trollbot”-spotting service rated nine of these top 20 accounts as “Alarming,” with an overall average (mean) Bot Sentinel score of 71.3% — thus, at the high end of that service’s “Problematic” category.

It seemed striking. And was. But there was a problem with this observation: it lacked context.

Understanding the scope of a problem requires a basis for comparison. Without a benchmark, data that seems relevant may simply lack sufficient context. For instance, given the slightly-less-than-transparent way that machine-learning tools function, it could be that Bot Sentinel’s ratings simply skew very high, and that 71% isn’t as terrible as it sounds.

Alternately, the service’s ratings could be very heavily biased against Tweets that support the GOP. This bias would be invisible if we only looked at a single group of similar posters, as we did. You see the point; without perspective, the numbers appeared alarming, but were in fact suspect.

For all these reasons, we’re expanding our analysis to include a comparison group this time, while also looking at a much larger sample of Cruz-supporting Tweets.

COMPARISON TO: EVERYTHING ELSE

This time around, we’re examining the top 200 posters in the entire TXSen (Texas Senate) district, over a one-week period from Sept. 8 – Sept. 15 (with about half of Sept. 16 thrown in for good measure). As a benchmark comparison, we’re using the top 200 posters in all other 125 competitive districts contained in the Pollchatter database, in the same time period.

Before going farther, we should note that this is not a random sample from either population. Thus, we are not arguing that our results can be extrapolated beyond these top-200 lists. For example, it’s quite conceivable (though we haven’t researched this yet) that bots or motivated trolls may post more often than regular civilian-type folks, and thus be over-represented at the high end of lists such as these.

What this comparison does do is examine the accounts that the average Twitter viewer interested in #TXSen was most likely to run across as she read or posted during the course of that week. That is, an account that posts 40 times (or 142 times) in one week is likely to make a larger splash, and be seen by more people, than an account that posts once or twice. Thus, this top-200 measure is relevant in terms of potential impact within the Twitter community.

But enough caveats. Here are the results.

#TXSEN POSTERS ARE HIGHLY SKEWED TOWARD “PROBLEMATIC” OR “ALARMING”

Within the sample week’s top-200 posters, the vast majority were Cruz supporters. Based on an examination of tweet content and hashtag use, just three were Beto O’Rourke supporters, while one was a reporter for the Texas Tribune.

The average (mean) Bot Sentinel score for the top 200 posters was 62.9%. That’s somewhat lower than the average within last week’s top 20 #KeepTexasRed posters, but still at the midrange of Bot Sentinel’s “Problematic” category (Definition: “Our analysis has concluded XXXX exhibits problematic tweet activity and patterns similar to a trollbot account.”). The median, which can attenuate the influence of high and low outliers, was a score of 66%.

Within this list, 68 of the 200 were deemed “Alarming” (Definition: “Our analysis has concluded XXXX exhibits alarming tweet activity and patterns that match a trollbot account. Please report this account to Twitter and avoid.”). A total of 156 of the 200 were either Problematic or Alarming – thus, showing a Bot Sentinel score of above 50%. Every one of these 156 were Cruz supporters.

BENCHMARK POPULATION INSTEAD LOOKS “NORMAL”

The stats look very different within the top 200 list of accounts posting in all other competitive districts within the same time frame.

The average (mean) score in this population is just 23.22%, at the high end of Bot Sentinel’s “Normal” category (Definition: “Our analysis has concluded XXXXX exhibits normal tweet activity and is not a trollbot account.”). The median in this group was just 18%.

Within this comparison group, 10 of the accounts were deemed “Alarming”, and another 18 were “Problematic”. The remainder were either “Tolerable” or “Normal.”

This population comparison also helps address, if not wholly eliminate, some of the possible suspicions about bias within the Bot Sentinel model. For example, @WontMarchforSoros, a high-frequency pro-GOP poster, falls into the “Tolerable” category, while @MissouriGOP (the state Republic Party’s official account) is viewed as “Normal.” By contrast, the Democrat-supporting @DemocraticBlue is deemed “Alarming.”

So: It now seems very clear that Cruz’s surge of Twitter support over the last week, at least, has been strongly driven by suspicious accounts, in patterns that fail to resemble the norms set by other districts. Who’s responsible? We can’t say.

However, it’s certainly a concern. The very significant differences between the highly active Cruz-supporting population and the highly active population at large seem to make it evident that the Texas Senate race is currently the target of social-media manipulation, whatever its source. We’re in an era where this ought to be the subject of electoral-law investigations.

A final note: As any number-cruncher will tell you, this is a pretty rough-and-ready statistical analysis. For this reason, and because Pollchatter’s Twitter feed is not complete (see “How to use this site“), these conclusions should be treated as a slightly dirty window rather than as scientific-level analysis.

However, the conclusions are all replicable. The charts below contain the screen names that are members of each top-200 group, and our figures can be checked against Bot Sentinel (though that service’s ratings may change over time, as new information accrues). The Pollchatter source code is also available for inspection on GitHub. If anyone has any objections or concerns, we’d love to hear them.

Table 1: Top 200 posts within #TXSen, 9/8-9/16

Rank	Screen name	# of Pollchatter appearances, 9/8-9/16	Bot Sentinel score (out of 100)	Bot Sentinel category
1	Croatansound	142	52	Problematic
2	maniseku	92	52	Problematic
3	Mariatweets3	90	97	Alarming
4	smartgirls4gop	81	74	Problematic
5	raybae689	79	0	Normal
6	KayQuirk1	78	50	Problematic
7	MikeDiaz285	64	72	Problematic
8	BeverlyConserv1	61	76	Alarming
9	pmbasse	58	67	Problematic
10	GernertLynn	57	39	Tolerable
11	ChgocadChic	57	82	Alarming
12	Dan33494483	54	83	Alarming
13	deeg25	47	69	Problematic
14	Texan2A	46	48	Tolerable
15	JoeBiala	44	60	Problematic
16	PatrioticCindy	44	66	Problematic
17	MalsLAW	43	79	Alarming
18	BolindoMiss1975	42	89	Alarming
19	Momcat2111	42	55	Problematic
20	BearingTruth	40	56	Problematic
21	R98250729	40	42	Tolerable
22	BrandonJLandry	40	79	Alarming
23	thathad2hurt	40	66	Problematic
24	bcarter76	40	65	Problematic
25	BevConservative	39	64.02	Imputed score
26	Epluribusunimus	39	62	Problematic
27	AnilNS425	39	67	Problematic
28	keramirez	39	76	Alarming
29	Jojoinguette	39	91	Alarming
30	TCruzCrew	38	64.02	Imputed score
31	BoiseMarineMom	37	77	Alarming
32	Dlw20161950	37	26	Tolerable
33	bestnaniever	36	52	Problematic
34	skyjones55	36	0	Normal
35	SueAyoub	36	80	Alarming
36	Lisa22545183	35	75	Alarming
37	NatalieTexas	34	70	Problematic
38	AmericaFirst150	34	50	Problematic
39	RhondaHurley2	34	86	Alarming
40	Saint_Obama	34	67	Problematic
41	bonniemurphy	34	81	Alarming
42	RobertS30289919	33	42	Tolerable
43	D_Ericson	33	26	Tolerable
44	Gretche66694001	33	79	Alarming
45	OrlandoandresM	32	66	Problematic
46	TRUMP4ISRAEL	32	50	Problematic
47	PaulRog52896828	32	93	Alarming
48	rratkinson	32	50	Problematic
49	Patriot_Girl_TX	31	70	Problematic
50	neff630	31	74	Problematic
51	fitzie63	31	59	Problematic
52	Craybo23	31	30	Tolerable
53	PhillipLaird	31	64	Problematic
54	Ray_Nolen	30	81	Alarming
55	VetWithACause	30	90	Alarming
56	TrumpCat04	30	78	Alarming
57	SBtx17	30	92	Alarming
58	DebraMMason1	30	71	Problematic
59	lisamichelle20	30	85	Alarming
60	zopitedaniel	30	33	Tolerable
61	VNTEXAS	30	31	Tolerable
62	BubbaJoeBaker	29	88	Alarming
63	JTS_1957	29	57	Problematic
64	mastrolyn	29	41	Tolerable
65	dianne1h	29	78	Alarming
66	DrSueDVM	29	54	Problematic
67	KTyrky1	29	50	Problematic
68	avoiceofliberty	29	40	Tolerable
69	Sgroberts121657	28	61	Problematic
70	WantBigHammer	28	61	Problematic
71	Beaudancer1	28	77	Alarming
72	Iloveperfumeso	28	61	Problematic
73	AggieDad2015	28	48	Tolerable
74	SonsMary	28	87	Alarming
75	cinrui	28	79	Alarming
76	Dr_Nahil	28	41	Tolerable
77	lynn_weiser	28	80	Alarming
78	aranom12	28	67	Problematic
79	DRADAIV	27	69	Problematic
80	JewelsJones1	27	65	Problematic
81	cmdorsey	27	62	Problematic
82	texasaunt	27	58	Problematic
83	tracy_texas6	27	78	Alarming
84	kasmouse	27	80	Alarming
85	ScribbledPages1	27	72	Problematic
86	pamela_pmlpatt	27	86	Alarming
87	pattygilli	26	71	Problematic
88	tedcruz	26	8	Normal
89	patriotmom2911	26	62	Problematic
90	VeldaMcclung	26	90	Alarming
91	CraigLisathomas	26	59	Problematic
92	RLiberalskiddin	26	69	Problematic
93	HorseShort	26	90	Alarming
94	MBOKSR_MAGA	26	70	Problematic
95	pononzu	26	71	Problematic
96	Sethro_tull_556	26	39	Tolerable
97	oidcop	25	87	Alarming
98	CaprockDemocrat	25	0	Normal
99	braggingdragon1	25	0	Normal
100	jearuiz	25	96	Alarming
101	stringfellowfr4	25	71	Problematic
102	lisamichelle12	25	90	Alarming
103	TeamTrumpTeX	25	43	Tolerable
104	Woke2008	25	69	Problematic
105	cs0058sc	25	64.02	Imputed score
106	BeeNewsDaily	25	17	Normal
107	RightAsRain7	25	80	Alarming
108	Drbob444	25	61	Problematic
109	SnowLover814	25	65	Problematic
110	DestinyandBruce	25	71	Problematic
111	Sp9825Micah	25	64.02	Imputed score
112	DarleneHBrook	25	41	Tolerable
113	TXSVIKING	25	40	Tolerable
114	88_Charlotte_88	24	91	Alarming
115	daliafierro	24	87	Alarming
116	Stargazer2020	24	82	Alarming
117	Dondorey1	24	75	Alarming
118	TriathleteUSA	24	75	Alarming
119	THETXEMBASSY	24	73	Problematic
120	usajustice4all	24	70	Problematic
121	RebelReedRight	24	64.02	Imputed score
122	StillKicking62	24	61	Problematic
123	TexianRising	24	45	Tolerable
124	Angelemichelle	24	29	Tolerable
125	PatrickSvitek	24	3	Normal
126	DelarioDeborah	23	92	Alarming
127	aktweet123	23	92	Alarming
128	SharNeal	23	88	Alarming
129	roses12411	23	84	Alarming
130	tcarmistead	23	82	Alarming
131	SandraTXAS	23	77	Alarming
132	Vfigi1	23	73	Problematic
133	let736	23	73	Problematic
134	AmericaFirst46	23	72	Problematic
135	OhioWomen4Trump	23	64	Problematic
136	JasonKManners	23	64.02	Imputed score
137	FreedomFrayed	23	61	Problematic
138	KST2424	23	50	Problematic
139	Tsand88	23	44	Tolerable
140	BennytheKite	23	20	Normal
141	BlogUSA2018	22	88	Alarming
142	Blackdi51264299	22	86	Alarming
143	KeathGifts	22	85	Alarming
144	lopezginnyyaho1	22	82	Alarming
145	saiwarlord	22	82	Alarming
146	HeyJudeOregon	22	78	Alarming
147	robinlecronesco	22	75	Alarming
148	Ronin1021	22	63	Problematic
149	DrSue19380	22	60	Problematic
150	210GOPTEJANA	22	56	Problematic
151	AdamlCitizen	22	51	Problematic
152	LuzAyala2	22	49	Tolerable
153	marjo219	22	48	Tolerable
154	ruhoward4	22	47	Tolerable
155	NancyLRogers1	22	22	Normal
156	Len21878832	21	86	Alarming
157	grainofsand2020	21	82	Alarming
158	LibertysArmyJr	21	82	Alarming
159	stefilassiter	21	80	Alarming
160	Mr_Arizona92	21	79	Alarming
161	usvetram	21	78	Alarming
162	CASuperrunner	21	77	Alarming
163	harleygrl3465	21	76	Alarming
164	djdun3	21	76	Alarming
165	LawsEris	21	72	Problematic
166	Childlikefaaith	21	72	Problematic
167	mimiinftworth	21	64	Problematic
168	TexasTrumpGirl	21	63	Problematic
169	Trumptrooper2	21	62	Problematic
170	FuriousFarmBoy	21	61	Problematic
171	pwirish	21	60	Problematic
172	SusanABriggs6	21	58	Problematic
173	treva21448150	21	55	Problematic
174	SharonEmily	21	44	Tolerable
175	tonywesolick	21	43	Tolerable
176	voiceofstentor	21	27	Tolerable
177	CruzNewsLive	21	18	Normal
178	sheacamm	21	0	Normal
179	Evenstar888	20	93	Alarming
180	trader_sam	20	88	Alarming
181	AndrewsHarley	20	81	Alarming
182	DonnaCastel	20	80	Alarming
183	mountainman84	20	78	Alarming
184	ruskaj	20	73	Problematic
185	thomasj17431826	20	73	Problematic
186	Lissening2u	20	70	Problematic
187	MsJonsie2you	20	69	Problematic
188	nancy73gg	20	66	Problematic
189	Freedomtoroam1	20	64	Problematic
190	lorylovesanimal	20	57	Problematic
191	RachelSuperEli1	20	54	Problematic
192	NanaOxford	20	53	Problematic
193	silverbird458	20	50	Problematic
194	deb4liberty	20	49	Tolerable
195	sisterwash65	20	47	Tolerable
196	NascarChuck336	20	45	Tolerable
197	DavidNe99469223	20	44	Tolerable
198	davealvord164	20	43	Tolerable
199	b_roelf	20	25	Tolerable
200	TexOnTheLege	20	5	Normal
	Mean	62.9
	Median	66

 

Table 2: Top 200 posts not in #TXSen (125 districts), 9/8-9/16

Rank	Screen name	# of Pollchatter appearances, 9/8-9/16	Bot Sentinel score (out of 100)	Bot Sentinel category
1	votepledges	1226	0	Normal
2	SuMoh7	744	15	Normal
3	jckthesword	650	0	Normal
4	MD_386	629	0	Normal
5	ybbkaren	492	32	Tolerable
6	CallMcclintock	492	4	Normal
7	OperationOppo	460	61	Problematic
8	IndivisibleIL14	331	5	Normal
9	RavenRothisPink	311	23.22	Imputed score
10	smithbarbv	301	10	Normal
11	GailWaldby	294	1	Normal
12	StormResist	280	13	Normal
13	GoldieAZ	272	18	Normal
14	Susana_Tupper	270	18	Normal
15	Fawkes_2017_	249	45	Tolerable
16	InSpiteOfTrump	246	4	Normal
17	waydelsew	240	37	Tolerable
18	PhilAmmann	220	19	Normal
19	PillsMom18	219	11	Normal
20	trudygonzales	217	24	Normal
21	WontMarch4Soros	204	38	Tolerable
22	tsteph92	202	4	Normal
23	Mean_Adam	201	9	Normal
24	VAPolitical	191	4	Normal
25	mersmom8	190	0	Normal
26	MagaGoldHat	187	85	Alarming
27	renaitre8	187	16	Normal
28	davealvord164	184	43	Tolerable
29	BearmanShannon	179	25	Tolerable
30	pml_tray	178	0	Normal
31	Sisterwriter	176	5	Normal
32	Croatansound	173	52	Problematic
33	rieth_damon	171	8	Normal
34	jgcOCANADA	171	6	Normal
35	Unconquerable	169	8	Normal
36	EllisBelfer	167	39	Tolerable
37	AUdebm	167	3	Normal
38	RuneK_15	159	19	Normal
39	R98250729	155	42	Tolerable
40	MentalNote2018	154	3	Normal
41	EzraBridger2018	153	23.22	Imputed score
42	RdrgzKr	149	2	Normal
43	dpmax100	145	25	Tolerable
44	JakeWilkns	144	4	Normal
45	CongressRTBot	143	0	Normal
46	Milvio_Diaz	141	76	Alarming
47	JOEROIREY	132	24	Normal
48	ArtistsPo	132	15	Normal
49	Joni_Looking	129	22	Normal
50	jwyattronan	129	3	Normal
51	HMeemaw	126	34	Tolerable
52	ellievan65	125	5	Normal
53	MMScholes	125	4	Normal
54	mydogtristan	124	19	Normal
55	DemocraticBlue	123	79	Alarming
56	Tony_Glorioso	122	23.22	Imputed score
57	4CommonSense1	119	56	Problematic
58	Str8DonLemon	114	59	Problematic
59	TeamPappasNH	114	0	Normal
60	podod12	112	40	Tolerable
61	prettyangeljac1	111	21	Normal
62	Indivisible19NY	111	2	Normal
63	MissouriGOP	110	12	Normal
64	realdebfarmer	109	15	Normal
65	mcy2000	108	3	Normal
66	HDowning113	107	76	Alarming
67	muellerew	105	42	Tolerable
68	ObamasHope	105	23.22	Imputed score
69	GarrenShipley	105	3	Normal
70	ssmtx6	104	15	Normal
71	SassyGma6	104	5	Normal
72	TheTrumpParty	103	81	Alarming
73	2017Gamora	103	29	Tolerable
74	PaulMic1954hael	101	32	Tolerable
75	MadelyneRants	101	16	Normal
76	misen__	101	1	Normal
77	w55unicorn	100	40	Tolerable
78	stlajn1	100	13	Normal
79	gerfingerpoken2	99	96	Alarming
80	KimTompkins14	99	20	Normal
81	joncprevo	99	2	Normal
82	CaroleChatelai1	98	17	Normal
83	ProgressiveFor	98	13	Normal
84	Eugene_Begay45	97	33	Tolerable
85	myserenity69	96	34	Tolerable
86	HH201707	96	21	Normal
87	jacksnowknows	96	16	Normal
88	gerfingerpoken	95	97	Alarming
89	SallyDeal4	95	53	Problematic
90	Feminizzzm	95	4	Normal
91	ActorAaronBooth	95	0	Normal
92	elitmik	94	45	Tolerable
93	BookerBenjaminA	92	55	Problematic
94	ChrisPappasNH	92	0	Normal
95	TroyMGood1	91	66	Problematic
96	igggie1	91	25	Normal
97	Lacus09	91	20	Normal
98	NashvilleResist	91	5	Normal
99	JimMcNichols1	91	0	Normal
100	knsh55a	90	26	Tolerable
101	timeindawater1	90	22	Normal
102	CindyMundell	90	13	Normal
103	Vote_Faso_NY19	89	9	Normal
104	jennobenno	88	29	Tolerable
105	N1RMFAN	87	49	Tolerable
106	Cassie_o_peia	87	30	Tolerable
107	freeandclear1	87	13	Normal
108	FulcrumOp	87	11	Normal
109	johninphx	86	56	Problematic
110	NotJayWebber	86	44	Tolerable
111	MyVoteMyKids	86	25	Tolerable
112	5141quid	86	25	Tolerable
113	prairielive	86	13	Normal
114	henricoblue	86	10	Normal
115	ElliotMalin	86	3	Normal
116	MagaCandidates2	85	59	Problematic
117	Solutioneer72	85	54	Problematic
118	Pie_SocialMedia	85	18	Normal
119	dadkind	85	15	Normal
120	immelza	85	12	Normal
121	JolynnMarie612	84	59	Problematic
122	DannyHortonMO	84	32	Tolerable
123	haine_xxx	84	0	Normal
124	Bread_Maker18	83	34	Tolerable
125	lbweaver	83	33	Tolerable
126	CaptainsLog2O18	83	7	Normal
127	BlondeMamaUSA_	82	71	Problematic
128	gayleasher1	82	42	Tolerable
129	smcenturff	82	35	Tolerable
130	RGreggKeller	82	10	Normal
131	CATargetBot	82	1	Normal
132	511nyLongIsland	82	0	Normal
133	get_out_andVOTE	81	60	Problematic
134	CarmoSteph	81	31	Tolerable
135	SueMayerNH	81	12	Normal
136	ShannonRitenour	81	12	Normal
137	dino4dino	80	42	Tolerable
138	guardingblue	80	31	Tolerable
139	MatthewWolfff	80	29	Tolerable
140	cmcdonald048	80	9	Normal
141	carlyinNJ	79	33	Tolerable
142	ElliotKumpe	79	10	Normal
143	investtoelectil	79	4	Normal
144	dearjudge	79	0	Normal
145	BSmarty17	78	25	Tolerable
146	RodneyClaeys	77	33	Tolerable
147	BigSkyDemocrats	77	1	Normal
148	barricko_bama	76	26	Tolerable
149	BrianCollinsMO	75	54	Problematic
150	laureldavilacpa	75	18	Normal
151	Intl_Resistance	74	68	Problematic
152	Die__Trying_	74	34	Tolerable
153	kals55	74	13	Normal
154	archeology_girl	74	13	Normal
155	progresivetrend	74	10	Normal
156	jistBhappy	74	9	Normal
157	TeamHawley	74	6	Normal
158	UniteBlueMN	74	0	Normal
159	MisterAndyWong	74	0	Normal
160	jimstev03759375	73	50	Problematic
161	ehartman1975	73	27	Tolerable
162	RichLeonard4NH	73	18	Normal
163	NDGOP	73	5	Normal
164	45xiatai	73	3	Normal
165	Kopser4Congress	73	0	Normal
166	BeckyStroud50	72	77	Alarming
167	bannerite	72	23	Normal
168	VoteHealth	72	21	Normal
169	Beverly21811568	72	0	Normal
170	Susanv12345	71	62	Problematic
171	DonaldsNeckVag	71	32	Tolerable
172	TonyContino	71	22	Normal
173	AndersonDrew	71	2	Normal
174	Bellatrixx2018	70	25	Tolerable
175	Nov2018election	70	23.22	Imputed score
176	AgitatorsNeeded	70	22	Normal
177	BlueWaveCS	70	6	Normal
178	Jojoinguette	69	91	Alarming
179	ElizabethGrisc1	69	66	Problematic
180	KikiAdine	69	22	Normal
181	WebOften	69	18	Normal
182	ChrisJZullo	69	17	Normal
183	GretchenPerki14	69	13	Normal
184	pleasesaveour	69	9	Normal
185	Its_Bhagyashree	69	3	Normal
186	WMUR9	69	1	Normal
187	TrumpWatchNews	69	0	Normal
188	ReneeDahl5	68	68	Problematic
189	GailHorn	68	47	Tolerable
190	Paulsen_Trump	68	31	Tolerable
191	cateady53	68	10	Normal
192	AVoiceForAllGA	68	8	Normal
193	emyryly	68	4	Normal
194	TeamDean	68	3	Normal
195	NaphiSoc	67	16	Normal
196	LambYoga	67	7	Normal
197	tankcat	67	3	Normal
198	DestructiveChem	66	21	Normal
199	NomdeB	66	10	Normal
200	PlyMNIndivisibl	66	2	Normal
	Mean		23.2255
	Median		18

NOTE: Within both populations, a small number of accounts could not be assigned scores by Bot Sentinel’s service. Within the benchmark population, we have given these accounts a score equivalent to the overall average (mean), a technique known as “imputation” in statistics. Within the Texas Senate population, each of the missing values was a Cruz supporter – thus, rather than assigning them to the overall mean, we have assigned them a value equivalent to the mean among all Cruz supporters.