On Awards and Algorithms

I am very proud to be listed as No. 8 on MindTouch’s list of Tech Comm Influencers for 2011. Nevertheless, and despite that fact that I don’t have a good success record at questioning awards I receive (or don’t receive), I do have a few questions for MindTouch about their list. I suppose that’s just because I’ve always been part of the awkward squad.

I am on record as expressing some doubts about the value of assessments of importance based largely on robotic algorithms. It wouldn’t be right for me to change my mind and say that algorithms are brilliant just because this algorithm has given me a high ranking. In the past I have been told by Klout, the most well known self-appointed arbiter of online influence (whatever that may be) that I am “influential” about Beer and Airports. While I and my Twitter followers may find that sort of thing mildly amusing, there are some people who have come to dislike Klout enough to delete their profiles.

So, as a bit of a sceptic when it comes to algorithm generated lists, I’d like to ask MindTouch about the “phenomenal social media analysis tool that hasn’t launched yet” that they used to collate their list of the 400 most influential people in Tech Comm and Content Strategy.

I have noticed a number of anomalies in the list. Many of the people ranked below me on the MindTouch list have wider circles of Twitter followers than I do, present at conferences more than I do, and blog more frequently. These include many people who influence me far more than I influence them.

The MindTouch list doesn’t distinguish between individuals and companies or organisations. I know this might be difficult to do, as many consultants blog and tweet under company names rather than under their own names.  I wasn’t surprised to see that the STC is listed as no. 5 overall, as it is the largest English-speaking organisation for technical communicators in the world. But I was surprised to see that the Twitter feed for the STC’s UK and Ireland Chapter is ranked at no. 107, when it hasn’t posted a single tweet since September 2010.

The MindTouch list also seems to be limited to people and organisations that tweet in English. Germany’s organisation for technical writers, Tekom, gets more visitors and more exhibitors at its TC World conference than the STC gets at its Summit, but acccording to this list it has no influence. Why is that?

My final question for Mind Touch is why, if they claim their compilation represents influencers in both TechComm and Content Strategy, does the list completely ignore all the leading experts on content strategy on Twitter today? No Kristina Halvorson, no Gerry McGovern, no Erin Kissane, not even Rahel Bailie, to name just a few.

I do know that if MindTouch revise their algorithm I may well lose my Top 10 ranking. But I really do want to know more. I think we all do.

This entry was posted in content strategy, technical communication and tagged , , , , . Bookmark the permalink.

19 Responses to On Awards and Algorithms

  1. Ray Gallon says:

    David, I’m also on the list – a lot further down than you 😉 but I share your concerns and would also like to see more. I especially agree about the lack of real CS gurus on this list.

  2. David?

    You?

    On the awkward squad?

  3. David,

    The social media and influencer analysis tool that we used will be announced in the next couple months. The company has only allowed a few dozen people access as it’s still in stealth. Anyway, you make a great point about Rahel, Christina, etc. I believe I’ve figured out our error. We should have never combined #Techcomm and #ContentStrategy.

    The tool output, among other things, a list of 700+ individuals ranked by a variety of factors, but a primary factor that determines rank is the number of followers by identified topic insiders. Realizing this should answer all of your questions. Moreover, this, for me, seems to be a no BS metric for determining influence. If one is followed by influencers they too must be an influencer.

    So how did this impact misranked Content Strategists? If an individual is identified as an influencer in #ContentStrategy they will be ranked in accordance with the number of other #ContentStrategy insiders that follow them. You can problem figure out why we should have not combined the lists based on this. There are just far fewer people that align themselves with “Content Strategy” than there are with those that align with “Techcomm”. Hence, by combining these lists we dramatically under-ranked Content Strategists. Oooops.

    As for some of you other points, the list could have surely benefited from human curation and the aforementioned tool has an interface for doing this. For example, we could have gone through and removed entries like the STC chapter that haven’t tweeted in months. We discussed this internally and ultimately decided that if we hand pruned the list it would quickly lose it’s cold machine objectivity, which we preferred to a slightly improved list. Since we couldn’t agree upon an algorithm for consistently pruning we decided objective with some anomalies were preferable to something that was subjective.

    Perhaps we should run a separate content strategy report. What do you think? Also, thanks for your critique David.

  4. CJ Walker says:

    Very well articulated, David.

    I question anyone coming at a new field and “ranking” it, while not even mentioning most of its thought leaders and important players. Is this because they don’t know them? Hmmm.

  5. Karen Mardahl says:

    You wrote the blog post i chickened out of doing. 🙂

  6. David Farbey says:

    Thanks to everyone who has left a comment, and particularly thanks to Aaron Fulkerson for contributing to the discussion in such a frank and open way. I look forward to reading more about your analysis tool in due course. In the meantime, a separate Content Strategy list sounds like a good idea.

  7. david jones says:

    Like many things probably best to take with a pinch of salt

  8. Rhonda says:

    Well put, David.

    A question for Aaron — does the use of #techcomm in your response mean that this is the main criteria you look for in Tweets? If so, that excludes a lot of tech comm people who Tweet about tech comm but who don’t always include that hash tag, often because of the 140-character limit, but sometimes because we just plain forget to do so.

    Another question — are blog posts on tech comm included in the analysis? I checked your process page, and it looks like it’s just Tweets. So, if an influential tech comm person (influential in other ways such as blogs, email discussion lists, forums, conference presentations and attendance, etc.) doesn’t Tweet, are they even considered?

    –Rhonda

  9. @Rhonda,

    This tool is much more intelligent than simply evaluating a hash tag and Twitter isn’t the only consideration. This said, no it’s not magic. 🙂

  10. Noz Urbina says:

    Hi David,

    Great post. My feelings are generally similar. The list is ok for a Techcomm list and their heart was in the right place combining it with Content Strategy, but the algorithm wasn’t up to making a human-meaningful result.

    Nice try. Hopefully next time around will work better. There are many that say that even bad publicity is still publicity.

    And yes, I thought I got boned being at 100th place, but I beat out the Rockley Group by 31 spots, and Ann Rockley herself didn’t even make the list. So, I am glad just to be on it!


    Noz – http://lessworkmoreflow.blogspot.com // @nozurbina // @congility

  11. I agree with your points, David. I would also make a point about—if you are going to assess Twitter influence—examining the quality of the tweets. That is, what are people saying about the technical communication profession? Personally I only like to follow people on Twitter (from a professional standpoint) who are tweeting often enough about the techcomm.

  12. Kai says:

    Thanks, David, for a more rational, well-argued approach vis-à-vis “the list” than my own more impressionistic reactions at http://kaiweber.wordpress.com/2012/01/16/tech-comm-influencer/

    I second your point about language, esp. seeing that at least one account tweeting in Spanish made the list (and another one often tweeting in Danish – Hi, Karen!). If that

    I second Jason’s point about tweet content. I’ve had a lot of fun with personal and/or plain silly tweets, but like him, I try to keep it professionally relevant for the most part and like to follow those who do (unless I know them personally).

    I understand and respect that Aaron cannot say more about an unreleased tool, but the situation is a bit awkward and makes me disagree that “objective with some anomalies [is] preferable to something that was subjective.”

    On a positive note, I’ve discovered more than 20 twitterers I wouldn’t know about without the list, so I’m fine with the list and my position on it, I just wish it was a little more reliable in terms of who’s ranked and on which position.

  13. Karen Mulholland says:

    Well stated, David.
    I’ve had similar concerns – while it’s flattering to be on the list, I’m not at all certain that I belong on it. But the list has been valuable to me because it’s acquainted me with people I should follow; Kai is right about that. Sometimes things turn out to be useful for reasons entirely other than those originally envisioned by their creators. 🙂

  14. Sarah Maddox says:

    Hallo David and all
    Nice post, David, and a good discussion in the comments. Thanks especially to Aaron for the information about the tool.

    When I look at a list of influencers such as the one we’re discussing, I see it as just one source of information about who are the people to follow. I guess the aim of the tool is to provide a definitive list by aggregating information from various sources. But I also know that I have my own priorities and requirements when deciding who is influential. Indeed, I may compile a number of different lists based on my reason for wanting the list in the first place.

    So, while it’s great to be on a list, it’s even greater to be part of the wide, messy community that is the web. And I’m hoping that anyone whose looking for influential people will spread their search wider than just one list.

    Kudos to MindTouch for jumping in and getting us all talking. I love the work you guys do. And I love the community of tech writers that we all are.

    Cheers, Sarah

  15. As someone who is 12th on the list I echo everyone’s concern. I viewed this exercise as fun but one on which you shouldn’t place too much importance. It will certainly help some people keep in touch, but it should never be used as the definitive list of techcomm or content strategy influencers. I think everyone on the list knows this but the danger is that those less in tune with the industry may not.

  16. I agree with those who have made the point about the content of the tweets.
    My tweets are not about influencing people, my role in the community is providing information via my site about using RoboHelp and that does not make me an influencer. I should not be on the list, anywhere.
    I see others ranked out of order compared with any human assessed rating.
    I think the problems come from labeling the list Influencers and the strong link with tweeting.

  17. Gordon says:

    I think it’s a bit of fun, that no-one is (or should) assign any huge value to other than “ohhh didn’t know about [that person]”. That’s not to say I don’t think it has any value, but that ANY list is going to be subjective regardless of the mechanism of production.

    As a conversation piece it is valuable, of course, but as Sarah suggests this is just one of many ways of interacting with this big messy community/profession we share.

    I like it. Not because I’m on it but because it gets people talking, and exploring, our profession.

  18. Rahel Bailie says:

    I don’t dismiss lists and rankings quite so lightly. I don’t put huge stock in them as a quality barometer, but I do use them (with a rather large grain of salt) to identify who are industry leaders to follow, who have similar interests, and so on. For example, I notice who Twitter recommends I follow. (I don’t always take Twitter’s advice, but I consider it.) It has led to some interesting finds. I look at my Klout score to see who else is an influencer and whether they’re posting interesting articles that can contribute to my ongoing professional development.

    It didn’t surprise me that I didn’t rank on the list. The amount of techcomm-related publishing I’ve done (blog posts, articles elsewhere, tweets, LinkedIn posts, FB posts) have been minimal, unless it’s an overlap with content strategy. And then, as someone mentioned, the hash tag lengths restrict my choice. I’m more likely to use #contentstrategy and hope that if it’s of interest to the techcomm community, someone will retweet and change the hash tag. And I suppose that’s why I made the content strategy list that Mindtouch published a while back.

    And that’s the thing about lists. Being a full-fledged, lifetime member of the awkward squad, I never expect to be part of the cool crowd. But algorithms don’t care whether you’re cute and bubbly or a hipster. Algorithmic hiccups aside, the list is based on substance, not personal politics. What’s more important is how you assess and then use the list. I may not hire someone based on how far up the list their name is, but the list may serve some other purpose that ends in a serendipitous find.

  19. I think you’re pretty damn cool Rahel.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.