Two post-SpotOn thoughts

by Eva Amsen

Following the SpotOn London conference, I have been thinking about a few things that came up at the meeting.

1. A thought about stories told by scientists

I had this whole long bit typed here about all the things that could hold people back from talking or writing about their life and how that affects their research, but the longer I thought about it, the more I realized that it was all nonsense.

For every excuse I could come up with on behalf of respectable scientists, I was able to name at least one scientist who broke the trend. I am not even going to bother linking to all the departmental rock bands and personal blogs and art projects by both prominent and established researchers as well as by people who have built their science career from the ground up while writing blogs.

So stop worrying – your writing or speaking is not going to affect your career. (If it does, I’m willing to bet that the problem was the content of what you wrote or said, not the actual act of it. Basically, don’t be an offensive jerk.)

 

2. A thought about altmetrics

Altmetrics are everywhere. This is an actual thing that’s happening now, and it’s not going away. However, as I brought up in one of the many altmetric-themed sessions at SpotOn,  the difference between alternative and conventional metrics is, well, lack of convention.

Even though measuring success of a scientist by the impact factor of the journal their work is published makes little sense once you start thinking about what that actually means, and even the h-index is not perfect, citations themselves are rather easy to count, and also timeless. Your work can be cited twenty years later in the same way as it will be cited next year. But what does a tweet mean? If your paper from 2007 has no tweets and your paper from 2011 has twenty tweets, is the second one really better, or merely published at just the right time to catch a wave of conversation? And what will we be using in 2018? Still Twitter, or have we moved to NewThingr by then? And how does 1 NewThingr in 2018 relate to 1 Tweet in 2011? What if besides NewThingr people are also regularly using FancyNewTool, ModernTwit, WebTalky and Clickamajig to link to and discuss papers? [I just made all these up on the spot.] And how will we ever find the old tweets talking about that 2011 paper, when you can’t even find tweets from two weeks ago? Citations in other papers, for all their flaws, are at least relatively timeless.

I told Euan at SpotOn that I’d also think about a way to solve this, but haven’t come up with anything clever yet. Clearly, some sort of standard MeasuringThingForScience would be useful, where a 2011 Tweet can be compared with a 2018 NewThingr. But at the moment, I don’t even know if you can compare a 2010 Tweet with a 2012 Tweet. You need to normalize for total users, I suppose, but not linearly, because if you get many tweets from a service with very few people on it, there’s also a good chance that those tweets are *yours*.

I’m not a numbers person, but I know some of you are, so I’ll gladly off-load this second thought.

Related Articles

2 comments

Matt Hodgkinson November 16, 2012 - 10:35 PM

Good questions about comparability and permanence of metrics.

PLOS has archived tweets linking to papers since earlier this year. Internet Archive or WebCite could archive blog posts and other websites.

If metrics can be put into context of field and time since publication, that makes comparability easier even when what is being counted changes. That’s being worked on.

I’m not sure the “worth” of different metrics can be easily quantified – how many Facebook likes equal one Tweet? Even with the gold standard of citations, not all are equal. A self-cite should count for less, or a critical citation. Metrics, including citations, are best seen as a guide, I think.

Martin Fenner November 19, 2012 - 2:16 PM

Eva, thanks for your post. I completely agree with #1. And I don’t think we should worry too much about FancyNewTool, ModernTwit, WebTalky and Clickamajig. Altmetrics includes citations, and they will continue to be an important metric. Usage stats will also have a similar meaning 5 years from now. Some of the newer metrics might not be around in 2017, or will mean something completely different, in particular if tied to a particular tool such as Twitter, Facebook or Mendeley. But Twitter, Facebook and Mendeley are useful in 2012 and most likely also in 2013.

Comments are closed.