Semantics and homophily on Twitter

I took about 600 recent tweets with the hashtag #NHSbill and analysed them for word clusters using Latent Dirichlet Allocation (a hierarchical Bayesian technique). I've been thinking of building a tool that could (1) help identify discussion themes, (2) find related tweets, and (3) suggest tweeps that have similar or different views to your own.

This is a stab at step (1). The following collections of words are summaries of the discussion themes and, given that it's Twitter, associated tweeps (hope you don't mind the associations) and links.

#nhs @silv24 proud
disaster healthcare standing
talking professionals Lansley

@martinmckee @safcbob
#nhsreform doctors
week Andrew @ijgreener
@ingridjohanna66 RCPsych

@drgrumble
500 GPs [involved in] commissioning groups
want clinical withdrawn
amended http://www.bbc.co.uk/news/health-16771304

@rcplondon @djnicholl oppose
extraordinary mtg
@welsh_gas_doc saying
gen care pensions

support royal
alternative Lansley
paper Colleges
@welsh_gas_doc
r4 says Times

@cpeedell @clarercgp
fundamentally
@marcuschown @leicesterliz
wrong  modification
enough thrown out

What was frustrating was that @profchrisham kept popping in and out of the model as I was fiddling with the stop word list ('a', 'the' ,'and' etc.). Supportive voices for the bill were uncommon in the sample of tweets so I might go for a bigger sample. Like I said before, maybe I need to find a debate that is less, eh, one-sided, on Twitter.

Just finding discussion themes, step (1), can be a way of following the various threads in a Twitter hashtag live meeting - if could be done live.