Thursday, 3 March 2016

Repost: Why publishing negative findings is hard

From retraction watch, the story of a professor who tried to submit his reanalysis of two papers to journals and the obstacles he faced.


Abstract of Paper: Job Insecurity in Academic Research Employment: An Agent-Based Model


This paper presents an agent-based model of fixed-term academic employment in a competitive research funding environment.  The goal of the model is to investigate the effects of job insecurity on research productivity.  Agents may be either established academics who may apply for grants, or postdoctoral researchers who are unable to apply for grants and experience hardship when reaching the end of their fixed-term contracts.  Results show that in general adding fixed-term postdocs to the system produces less total research output than adding half as many permanent academics.  An in-depth sensitivity analysis is performed across postdoc scenarios, and indicates that promoting more postdocs into permanent positions produces significant increases in research output.
More details at:

Monday, 15 February 2016

Network analysis paper: Weaving the fabric of science: Dynamic network models of science's unfolding structure

Weaving the fabric of science: Dynamic network models of science's unfolding structure

By Feng Shi, Jacob G. Foster, and James A. Evans


• Our hypergraph framework captures the multi-mode, higher-order complexity of science.
• Our random walk model powerfully predicts how science evolves.
• Our approach reveals intriguing modal dispositions behind the advance of science.
• We find that entities of one type typically connect through entities of another type.
• We find a special bridging role for methods and chemicals in the fabric of science.
• We find that adding more node types leads to superlinear improvements in prediction.


Science is a complex system. Building on Latour's actor network theory, we model published science as a dynamic hypergraph and explore how this fabric provides a substrate for future scientific discovery. Using millions of abstracts from MEDLINE, we show that the network distance between biomedical things (i.e., people, methods, diseases, chemicals) is surprisingly small. We then show how science moves from questions answered in one year to problems investigated in the next through a weighted random walk model. Our analysis reveals intriguing modal dispositions in the way biomedical science evolves: methods play a bridging role and things of one type connect through things of another. This has the methodological implication that adding more node types to network models of science and other creative domains will likely lead to a superlinear increase in prediction and understanding.


Link prediction; Hypergraphs; Random walks; Multi-mode networks; Science studies; Metaknowledge

Paper (open access) available at:

Friday, 8 January 2016

Modelling academic research funding as a resource allocation problem

OK this is 5 years old but I have only just come accross it. From 2010 but modified 2014.

Geard, Nicholas and Noble, Jason (2010) Modelling academic research funding as a resource allocation problem. In, 3rd World Congress on Social Simulation, University of Kassel, Germany, 06 - 09 Sep 2010. (Submitted). 

Academic research funding is allocated through a competitive bidding process that may lead to inefficiency as excessive time is spent on proposal writing. We develop a simple agent-based model of the process and find that current systems are indeed likely to be inefficient. Alternative allocation schemes involving either a cap on individual effort or appraisal from the centre are indicated as improvements.

For code and paper see:

Monday, 9 November 2015

New publications on peer reivew

Casnici N., Grimaldo F., Gilbert N. and Squazzoni F. (2015) Attitudes of referees in a multidisciplinary journal: An empirical analysis, JASIST (Journal of the Association for Information Science and Technology), forthcoming

Bianchi F. and Squazzoni F. (2015) Is Three Better Than One? Simulating the Effect of Multiple Reviewer Selection on the Quality and Efficiency of Peer Review, Ylmaz et al. (Eds), Proceedings of the 2015 Winter Simulation Conference

Cowley S. (2015) How peer review constrains cognition: on the frontline in the knowledge sector, Frontiers in Psychology

Nedić O. and Dekanski A. (2015) A survey on publishing policies of the Journal of the Serbian Chemical Society – On the occasion of the 80th volume, The Journal of the Serbian Chemical Society, 959-969, 2015

Caram L.F., Caiafa C. F., Ausloos,M. and Proto A. N. (2015) Cooperative peer-to-peer multiagent-based systems, Physical Review E, E 92, 022805

Huutoniemi K. (2015) Interdisciplinarity as Academic Accountability: Prospects for Quality Control Across Disciplinary Boundaries, Social Epistemology A Journal of Knowledge, Culture and Policy (DOI:10.1080/02691728.2015.
As collected by the PEERE COST action.

Thursday, 5 November 2015

Paper: Reviewer Fatigue? Why Scholars Decline to Review their Peers’ Work

This paper reports the results of a survey of academics about their attitudes and experiences of peer review.

Reviewer Fatigue? Why Scholars Decline to Review their Peers’ Work

Marijke Breuninga1, Jeremy Backstroma2, Jeremy Brannona1, Benjamin Isaak Grossa1 and Michael Widmeiera1

a1 University of North Texas
a2 National Consortium for the Study of Terrorism and the Responses to Terrorism (START)


As new academic journals have emerged in political science and existing journals experience increasing submission rates, editors are concerned that scholars experience “reviewer fatigue.” Editors often assume that an overload of requests to review makes scholars less willing to perform the anonymous yet time-consuming tasks associated with reviewing manuscripts. To date, there has not been a systematic investigation of the reasons why scholars decline to review. We empirically investigated the rate at which scholars accept or decline to review, as well as the reasons they gave for declining. We found that reviewer fatigue is only one of several reasons why scholars decline to review. The evidence suggests that scholars are willing to review but that they also lead busy professional and personal lives.

The paper is at:

Wednesday, 4 November 2015

Paper modelling the impact of possible EU research funding policies

Ahrweiler, Petra, Schilperoord, Michel, Pyka, Andreas and Gilbert, Nigel (2015) 'Modelling Research Policy:  Ex-Ante Evaluation of Complex Policy Instruments' Journal of Artificial Societies and Social Simulation 18 (4) 5 <>. doi: 10.18564/jasss.2927


This paper presents the agent-based model INFSO-SKIN, which provides ex-ante evaluation of possible funding policies in Horizon 2020 for the European Commission’s DG Information Society and Media (DG INFSO). Informed by a large dataset recording the details of funded projects, the simulation model is set up to reproduce and assess the funding strategies, the funded organisations and projects, and the resulting network structures of the Commission’s Framework 7 (FP7) programme. To address the evaluative questions of DG INFSO, this model, extrapolated into the future without any policy changes, is taken as an evidence-based benchmark for further experiments. Against this baseline scenario the following example policy changes are tested: (i) What if there were changes to the thematic scope of the programme? (ii) What if there were changes to the instruments of funding? (iii) What if there were changes to the overall amount of programme funding? (iv) What if there were changes to increase Small and Medium Enterprise (SME) participation? The results of these simulation experiments reveal some likely scenarios as policy options for Horizon 2020. The paper thus demonstrates that realistic modelling with a close data-to-model link can directly provide policy advice.