Some time ago, I set Google Scholar to alert me if anyone cited one of the papers I’ve authored. I recommend that academic authors should do this on Scopus and Web of Science too. I forgot all about it until yesterday, when an alert duly popped into my e-mail.
It is gratifying to see that someone has cited you (& perhaps an occasional reminder to update the h-index on your CV), but more importantly, it alerts you to papers in your area of interest. This is the paper I was alerted to:
José Luis Ortega (2015) Relationship between altmetric and bibliometric indicators across academic social sites: The case of CSIC’s members. Journal of Informetrics, Volume 9, Issue 1, Pages 39-49 doi: 10.1016/j.joi.2014.11.004
I don’t have a subscription to ScienceDirect so I couldn’t read it in full there, but it does tell me the research highlights:
Usage and social indicators depend on their own social sites.
Bibliometric indices are independent and therefore more stable across services.
Correlations between social and usage metrics regarding bibliometric ones are poor.
Altmetrics could not be a proxy for research evaluation, but for social impact of science.
and of course, the abstract:
This study explores the connections between social and usage metrics (altmetrics) and bibliometric indicators at the author level. It studies to what extent these indicators, gained from academic sites, can provide a proxy for research impact. Close to 10,000 author profiles belonging to the Spanish National Research Council were extracted from the principal scholarly social sites: ResearchGate, Academia.edu and Mendeley and academic search engines: Microsoft Academic Search and Google Scholar Citations. Results describe little overlapping between sites because most of the researchers only manage one profile (72%). Correlations point out that there is scant relationship between altmetric and bibliometric indicators at author level. This is due to the almetric ones are site-dependent, while the bibliometric ones are more stable across web sites. It is concluded that altmetrics could reflect an alternative dimension of the research performance, close, perhaps, to science popularization and networking abilities, but far from citation impact.
I found a fuller version of the paper on Academia.edu and it is indeed an interesting read. I’ve read other papers that look specifically at altmetric and bibliometric scores for one particular journal’s articles, or articles from within one discipline. I like the larger scale of this study, and the conclusions make sense to me.
And my paper that it cites? A co-authored one that Brian Kelly presented at the Open Repositories 2012 conference.
It is also a paper that is on Academia.edu. I wonder if that’s partly why it was discovered and cited? The alt- and biblio-metrics for that paper are not likely to be high (I think of it as embryonic work, for others to build on), but participation in an online community is still a way to spread the word about what you’ve investigated and found, just like attending a conference.
Hence the title of this blog post. I find the alert useful to keep my knowledge up to date, and the citation gives me a sense of being part of the academic community, which is why I find metrics so interesting. What they tell the authors themselves is of value, beyond any performance measurement or quality signalling aspects.
When choosing where to publish a journal article, what signs do you look out for? Here are some questions to ask or aspects to investigate, for clues.
1 – Is it peer reviewed? (Y/N and every nuance in between) See the journal’s website.
2- Who is involved in it? The editor & publisher? Are they well known & well thought of? Who has published articles there already: are these big players in your field? Read the journal!
3- Is it abstracted/indexed by one of the big sources in your field? (The journal’s website should tell you this. Big publishers also offer their own databases of house journals)
4- What happens when you search on Google for an article from the journal? Do you get the article in the top few results? And on GScholar?
5- Does it appear in Web of Science or Scopus journal rankings?
6- Take a look on COPAC: which big research libraries subscribe?
7- have a look at the UK’s published RAE2008 / forthcoming REF2014 data and see if articles from that journal were a part of the evidence submitted, and rated as 4*
8- Do the journal articles have DOIs? This is a really useful feature for promotion of your article, and it will mean that altmetric tools can provide you with evidence of engagement with your article.
9- Is there an open access option? (See SherpaRomeo) This is a requirement of many research funders, but it is also useful for you, when you want to promote your article.
10- Is it on the list of predatory OA journals? You might want to avoid those, although check for yourself. Note that some journals on the list are disputed/defended against the accusation of predation!
12- If you have access through a library subscription, is it listed on Ulrich’s periodicals director? What does this tell you about it? Note the “peer review” symbol of a striped referee’s shirt: if the shirt is not there, it doesn’t necessarily mean that the journal is not peer reviewed: you may have to investigate further.
- What type of peer review is used? Is it rigorous? Is it useful to you, even if you get rejected?
- Time to rejection/acceptance: how soon do you need to be published?
- Acceptance/rejection rate (see sources)
- Journal Impact Factor/ SJR score(s) /quartile for the field
Here are three ways you can invest a little time to save it in the long run, based on things I found were wasting my time:
1) Wake up your computer faster by clearing your desktop! Apparently it helps your computer to boot up faster if you have no documents & only short-cuts on your desktop. Wikihow has lots of tips on how to make your computer run faster, but just keeping files in order and off the desktop is relatively quick and easy to do. In any case, it gives me a lovely sense of order to have very little on my desktop, and tidiness generally saves me time looking for stuff. Or should I really say “organised messiness”?!
2) Back-up phone contacts and only leave the people on your phone who you know you’re going to call. (A quick Google search will reveal how to do this, for your phone.) I used to have at least 20 people under each letter of the alphabet, regularly causing me to waste time scrolling through all those names when searching for the people who I wanted to call! I can always look up old contacts in my back-up, if I really need to.
3) Be efficient with passwords: find a system that works for you! I lose count of the times I’ve had to fill out those “lost password” boxes and then wait for the password to arrive in my inbox. Sometimes, whilst waiting, I got distracted and the link in my inbox timed out and I had to start all over again… now I have a system that works (please forgive me for not sharing it!), it saves me from frustration, as well as time!
What could you do, to save yourself time in the long run?
I’m interested in tools that measure/monitor social media activity, partly because they can potentially be used by the authors of such activity (ie I can learn something useful for myself!) but partly also because of the prevailing wind of performance measurement by numbers. What do the numbers tell me?
I’ve previously looked at Twitter & tools for measuring that. But now I’m looking at my blog, and I’m starting with what WordPress can tell me about it, what I think is worth measuring and how it can direct my future social media activity. You can get a flavour of my discoveries by just reading the stuff in bold!
Number of views
The WordPress dashboard features a graph of recent activity, but also a “Site stats” link on the left-hand menu. Clicking on this presents me with a nice clear blue bar-chart with a snapshot recent view numbers, on a daily basis. I’m much more interested in the monthly view: this is where I can check trends and see how consistently my blog is doing. In general, because there is more content over time, I should be accruing more views over time. I ask myself, what if it doesn’t? Would I stop blogging? Yes. Am I happy with the number of views I have, and the growth rate? Well, I don’t know what to compare it to, but perhaps I could compare it to my most viewed month.
I can look on the bar charts for anomalies and WordPress also tells me my “best ever” day, in terms of views, which has clearly influenced one particular month. Such a spike seems worth investigating: why did my blog suddenly accrue more views on that day? I think I know why: it was a great blog post title and it accrued a lot of Twitter activity in terms of re-tweeting, including by influencers on Twitter. This was partly because I was blogging about a presentation by one of those influencers, and also because I made a point of tweeting at him to tell him, so that he re-tweeted! This sort of context could tell me how to accrue more views in future.
I’m also aware though, that views can be anything from a long sit-down and read with a cup of tea, to a glance and click away…
Referrers to my blog
Below the graph and headline numbers, I can see referrers to my blog: I clicked on “summaries” to investigate these, and once again I can choose the time period to view. I chose “all time” and I would estimate from the numbers that nearly 50% of views come from search engines (I looked into this and the vast majority of these were from Google), and about another 40% come from Twitter. Another significant referrer is my old blog, and I can see the URL of an event I presented at, too. I got 4 views each from comments I made on influencers blogs. These stats tell me less about the success of my blog, and more about the influence of my other activities.
Given how many blog views come from Twitter, it seems to me to be worthwhile continuing my presence in Twitter. I’m sure that I could build on Twitter’s effectiveness in driving traffic to my blog, by more direct tweeting at influencers. However, I’m not only seeking views of my blog as pure numbers!
I’m particularly pleased to see the number of views from the event I spoke at, because I know the audience from that event: these are people who I know I want to reach! So I can consider further speaking engagements as a wise investment of my time, in terms of driving traffic to my blog (especially if, as with that event, it is widely publicised and links are made to my blog and/or online profile).
What the stats don’t tell me is why people followed the link and what they thought when they did: is my blog reflecting well on me? I think so, because after that event I had a number of enquiries, but I don’t know for sure. All I know is that the event was effective as a way of raising my online profile.
Commenting on influencers’ blogs is considerably less effort than speaking at an event, and could also extend my reach to those with shared interests. What if I were to try writing less on my blog, but commenting more on others’ blogs, as a part of my mini-strategy? In this way, I could get more views for content already written, but would people lose interest in my blog if I post less often? This is perhaps worth trying!
For me, this was a fairly disappointing area, because although I can see that lots of views come through search engines, most of the search terms are apparently “unknown”. And indeed, glancing through those that are known, many of them involve a search for my name or my blog name. I can take from this that my blog name is worth hanging on to!
This is interesting, because it suggests a level of engagement with my blog that goes beyond viewing it: presumably those who share it have at least scanned through the content! Their sharing might also bring more views. My most shared posts appear to be the ones that I consider to be most academic. Looking at the service that was used to share my post is also quite telling: Facebook is significant, with twice as many shares as Twitter, and I’m not a big user of Facebook. Perhaps I should be? I could investigate whether being active on Facebook makes a difference! But I confess: I’m not sure I have time to do that in the near future.
Followers & Comments
I can also see numbers of these on the main Site stats page, and I know that commenting isn’t a big deal on my blog. Perhaps it would be if I commented more on others’ blogs? Looking at top commenters is also not especially useful to me, since there aren’t so many, although I like that I can see at a glance if I am following the commenters’ own blogs. This is a space to watchif I do choose to comment more.
The summaries of these interest me, because a click also seems to be an interaction level deeper than a mere view. What are my blog readers clicking on? This might signify what they are interested in, and thus indicate what I could blog about in future. Lots of people have clicked on the link to my LinkedIn profile, which supports what I found in the search terms. There are clicks to Twitter: to a picture tweet of my leaving cake from Warwick. I already know that some of my blog readers are former colleagues! This makes me think though, are picture tweets more effective at attracting attention? Not one I can investigate in the near future, but definitely food for thought.
Reflecting on my goals, on impact and other sources beyond WordPress
My goal when blogging is partly to raise my profile, so that potential employers and customers know who I am and what expertise I have. Beyond that, others might be interested to read what I have learnt or benefit from my experiences, and I’m happy to share. I know I’m successful in that when I meet people who have either read my blog, or know of its existence, they tell me so. I don’t get so many comments on my blog but I do get them in person. Perhaps I could gather such anecdotes if I were going to report on my blogging activity to others.
In judging my own success at this, I ought to reflect on how much time I spend on my blog, and consider the return on my investment, in comparison with other profile-raising activities. That’s why I’ve started using some time management tools, so that I can add that dimension into my reflections.
What if I was explicitly trying to achieve “impact” through my blog? I would like to simplify “impact” into three varieties:
A highly significant interaction with a small number of people
Bringing information to a target audience, who engage with it in a moderately significant way
Outreach to a wide-ranging, various and large population
Perhaps the first two varieties belong together, since they are essentially about things that are measured in a more qualitative way. WordPress’ stats report gives me clues about where to look for more qualitative information.
What is “highly significant” about an interaction will of course be open to interpretation and vary widely from one field to another, but I feel it’s beyond the scope of my blog. It is perhaps something that I could achieve through the sum total of my activity over a long period of time, or indeed through interactions with an individual.
What I mean by “moderately significant” is that the information given is actually read and interpreted or used by others, in some way. Perhaps my blog content gets re-purposed into some other librarian’s guide for students, or it at least prompts such a guide to be written. The only way I’d know about this is if someone were to tell me, or possibly if they linked back to my blog and people clicked on that link, and I watched referrals. In the meantime, I know that I re-purpose my content myself, so that’s a good start!
Which brings me to the “Outreach” notion, which is what I think most of these stats really indicate. It as a possible foundation for my second flavour of impact, and in any case, my profile does have a significant impact on my own life and career! What the WordPress stats do for me is they indicate the success of my blogging in achieving outreach, and they pointout where I should look for qualitative clues about deeper impact.
I’ve been blogging for years, but this is a feature of WordPress that wasn’t available on Warwick Uni’s own blogging platform. I like it as a way of engaging with other bloggers whose content I like (like re-tweeting and blog commenting all rolled into one!), plus it’s a way of providing content to anyone following my blog, when I find something of interest from somewhere else and don’t have time to write a lot myself.
It feels a bit like cheating, to me, because of the lack of effort, but if someone re-blogged my content with proper attribution (which WordPress does) and a friendly introductory comment, then I’d be happy. I note that in order to read the full post that I’ve re-blogged, you have to visit the source blog in any case, so it ought to drive traffic to the blogs of people who I’ve re-blogged.
I noticed that my re-blogged content did not appear on my LinkedIn updates (Aside: Who sees what in my Linkedin updates is increasingly a mystery to me!), even though my fresh blog content does seem appear there. But a re-blogged post does get tweeted and it appears to anyone subscribing to my blog through WordPress, of course.
Note to self: think about tweet appearance when commenting as I re-blog!
The elements displayed in that tweet are:
Title of blog post: my twitter handle: beginning of my comment: shortened link
All in all, a re-blog is a simple way to engage with social media.
I’ve tried a couple of tools out lately.The time-tracking tool I’m using right now is from Yast, and it’s free, unless you want to pay for additional features, of course. I also installed an app on my smartphone, called Gleeo a while ago: it’s a freebie and it works well, but I don’t always have my phone handy and I didn’t like the way it displayed how I had spent my time.
I’m also a list-maker and when it comes to time management, I rather like this recent blog post on “Ask a Freelancer“, but my lists are just pieces of scrap paper that float around my desk looking untidy when they are actually crucially important! I’ve always had a list for things to do in the immediate future (today, or this week at least: really urgent stuff is asterisked), and one for longer term aims (in the next month or two). Sometimes I have one for very long term aims, too, and once a week or so, these get updated. I’ve also learnt, when taking my own notes at a meeting, to use asterisks to denote things that I need to do: these can get transferred onto my usual lists, or just get done in one go so that they never appear on the main lists.
However, I much prefer looking at lists of things that I’ve done, as opposed to lists of things “to do”. Reflecting like this helps to keep me focussed on priorities because if I know I’ll look back at how I spent my time then I’ll make an effort to impress myself! That’s why although my Yast account has my professional projects listed, it also has projects called “housework” and “knitting”: these things are important to me and I like to allow them into my day since I work from home, but I also record my time, to keep them under control.
By recording the time I spent on my priority activities, I can have the satisfaction of looking at a day well spent. I like the way Yast displays my days’ and weeks’ activity back to me. I don’t beat myself up when I have empty days (sometimes I just forgot to set Yast recording), but I do congratulate myself for days that are full of the colours of projects that I believe I should be working on!
This tweet was of particular interest as it not only provided news of the new service and a link to a post in which the service was reviewed but also a brief summary of findings from the analysis of the posters’ use of Twitter with suggestions for best practice: “Images get more engagement“. The longer version explained how:
Finally, what Twitter Media and News staff had already told people who are listening is backed up by what they’re showing me: including pictures, maps and graphics in your tweets will raises your “engagement” numbers, at least as measured by people resharing tweets, favoriting them, @mentioning or @replying…
I heard a great story from a researcher who tweeted event information directly at 72 of her contacts, and then they re-tweeted her message to a potential audience of around 50,000.
Note that she used Twitter to direct-message people who could help her to promote her work. The “potential audience” of 50,000 were all the followers of the people who tweeted or re-tweeted about her work. I really like this story, as a way to impress line managers with your effective use of social media. It’s simple, it’s got numbers (line managers like those!) and it demonstrates that you go beyond just tweeting into the void at whoever is following you. It’s using your network contacts properly!
2. Monitor your twitter high-hitters & report on media attention
I also noticed that the JISC headlines which land in my inbox feature a section for “Our media coverage” and a section on “Our social media activity”. It’s a very nice example of an e-mail newsletter altogether, but the “Our social media activity” section attracted my attention, because of the way it presents tweets. They look something like this:
Retweeted by 94 people with a potential reach of 84.7k
‘Forget 24-hour drinking; students want 24-hour libraries’ http://bbc.in/1knWz0M (via @BBCNews) pic.twitter.com/qnXLEJvnCT
(from JISC’s May 2014 headlines)
I like this because these measures are easy to find in Twitter analytics, so that any researcher can see his/her tweets with the widest reach. JISC presumably tell folks this in their newsletter because there may be others who also find them interesting, and they are using Twitter as a filter & highlighter for you.
By monitoring your own high-hitting tweets in this way you will soon learn what your audience is interested in re-tweeting. Have you got the right audience? (If not, start following people who you would like to have follow you!) Can you tailor your message to attract their attention? (If indeed, attention is one of your goals on Twitter.)
You could also look out for “faves” and replies on Twitter, but I note that JISC is not doing so in this context.
Such a record of high-hitting tweets & of media attention might be something of interest to other team members, to line managers & possibly even research project funders if it’s part of your impact strategy to reach a broad audience.
Of course, I follow a lot of twitterers and only see a fraction of what they tweet so I know that the “potential reach” is just that, and the actual reach is likely to be considerably lower. Still, with a wider potential reach, you ought to have a wider actual reach, and those who have re-tweeted have considered your tweet to some degree. Although it is very easy to re-tweet without investigating, so I still wouldn’t claim too much without more context.
How do you get more context?
In the story of the researcher with her direct message tweets, these were about an event. So she will have a lot more information about the success of that event, I imagine.
The number of hits on the link(s) in your tweets could also indicate a more participative Twitter audience, but if it’s your website you’re promoting, then I’d rather look at the number of visitors to that site in total, as a success story to report to line managers (bigger numbers!). You could check how many visitors came there via Twitter, to see if your efforts on Twitter are paying off, but just those who click on the links you tweeted will be a smaller number than that figure, since people might also “MT” or “via” your tweet with their own shortcut links.
A journal article that you’re promoting will have altmetrics: if your publisher doesn’t collate these, your institutional repository might, or you can use ORCID and ImpactStory to do it yourself.
You could possibly do some kind of calculation that for x tweets in the course of a year, your ROI (return on investment) has been x visitors from twitter to your website/blog/article(s), although this is less simple, and it’s the simplicity of these examples that I like.