New publications on peer reivew

Casnici N., Grimaldo F., Gilbert N. and Squazzoni F. (2015) Attitudes of referees in a multidisciplinary journal: An empirical analysis, JASIST (Journal of the Association for Information Science and Technology), forthcoming

Bianchi F. and Squazzoni F. (2015) Is Three Better Than One? Simulating the Effect of Multiple Reviewer Selection on the Quality and Efficiency of Peer Review, Ylmaz et al. (Eds), Proceedings of the 2015 Winter Simulation Conference

Cowley S. (2015) How peer review constrains cognition: on the frontline in the knowledge sector, Frontiers in Psychology http://www.peere.org/wp-content/uploads/2015/11/Cowley2015Frontiers.pdf)

Nedić O. and Dekanski A. (2015) A survey on publishing policies of the Journal of the Serbian Chemical Society – On the occasion of the 80th volume, The Journal of the Serbian Chemical Society, 959-969, 2015

Caram L.F., Caiafa C. F., Ausloos,M. and Proto A. N. (2015) Cooperative peer-to-peer multiagent-based systems, Physical Review E, E 92, 022805 http://journals.aps.org/pre/abstract/10.1103/PhysRevE.92.022805

Huutoniemi K. (2015) Interdisciplinarity as Academic Accountability: Prospects for Quality Control Across Disciplinary Boundaries, Social Epistemology A Journal of Knowledge, Culture and Policy (DOI:10.1080/02691728.2015.

1015061) http://www.tandfonline.com/doi/full/10.1080/02691728.2015.1015061

As collected by the PEERE COST action.

Paper: Reviewer Fatigue? Why Scholars Decline to Review their Peers’ Work

This paper reports the results of a survey of academics about their attitudes and experiences of peer review.

Reviewer Fatigue? Why Scholars Decline to Review their Peers’ Work

Marijke Breuning^a1, Jeremy Backstrom^a2, Jeremy Brannon^a1, Benjamin Isaak Gross^a1 and Michael Widmeier^a1

^a1 University of North Texas
^a2 National Consortium for the Study of Terrorism and the Responses to Terrorism (START)

ABSTRACT

As new academic journals have emerged in political science and existing journals experience increasing submission rates, editors are concerned that scholars experience “reviewer fatigue.” Editors often assume that an overload of requests to review makes scholars less willing to perform the anonymous yet time-consuming tasks associated with reviewing manuscripts. To date, there has not been a systematic investigation of the reasons why scholars decline to review. We empirically investigated the rate at which scholars accept or decline to review, as well as the reasons they gave for declining. We found that reviewer fatigue is only one of several reasons why scholars decline to review. The evidence suggests that scholars are willing to review but that they also lead busy professional and personal lives.

The paper is at: http://dx.doi.org/10.1017/S1049096515000827

Paper modelling the impact of possible EU research funding policies

Ahrweiler, Petra, Schilperoord, Michel, Pyka, Andreas and Gilbert, Nigel (2015) 'Modelling Research Policy:  Ex-Ante Evaluation of Complex Policy Instruments' Journal of Artificial Societies and Social Simulation 18 (4) 5 <http://jasss.soc.surrey.ac.uk/18/4/5.html>. doi: 10.18564/jasss.2927

Abstract

This paper presents the agent-based model INFSO-SKIN, which provides ex-ante evaluation of possible funding policies in Horizon 2020 for the European Commission’s DG Information Society and Media (DG INFSO). Informed by a large dataset recording the details of funded projects, the simulation model is set up to reproduce and assess the funding strategies, the funded organisations and projects, and the resulting network structures of the Commission’s Framework 7 (FP7) programme. To address the evaluative questions of DG INFSO, this model, extrapolated into the future without any policy changes, is taken as an evidence-based benchmark for further experiments. Against this baseline scenario the following example policy changes are tested: (i) What if there were changes to the thematic scope of the programme? (ii) What if there were changes to the instruments of funding? (iii) What if there were changes to the overall amount of programme funding? (iv) What if there were changes to increase Small and Medium Enterprise (SME) participation? The results of these simulation experiments reveal some likely scenarios as policy options for Horizon 2020. The paper thus demonstrates that realistic modelling with a close data-to-model link can directly provide policy advice.

Session of 3 papers on Simulating the Social Processes of Science @SocSim 2015

Social Simulation 2015
Tuesday September 15
location: Het Kasteel, Melkweg 1, Groningen  

14:00 – 15:30 
Simulating the Social Processes of Science
116: Flaminio Squazzoni and Federico Bianchi. Is three better than one? An agent-based model of the reviewer selection in peer review
121: Matthias Meyer, Iris Lorscheid and Jonas Hauke. The Recent Development of Social Simulation as Reflected in JASSS from 2008-2014: A Citation and Co-Citation Analysis.
7: Georg Mueller. Simulating Thomas Kuhn‘s Scientific Revolutions: The Example of the Paradigm Change from System Dynamics to Agent Based Modelling 

New paper by Loet Leydesdorff "Can intellectual processes in the sciences also be simulated? The anticipation and visualization of possible future states"

Can intellectual processes in the sciences also be simulated? The anticipation and visualization of possible future states

• Loet Leydesdorff

Abstract

Socio-cognitive action reproduces and changes both social and cognitive structures. The analytical distinction between these dimensions of structure provides us with richer models of scientific development. In this study, I assume that (1) social structures organize expectations into belief structures that can be attributed to individuals and communities; (2) expectations are specified in scholarly literature; and (3) intellectually the sciences (disciplines, specialties) tend to self-organize as systems of rationalized expectations. Whereas social organizations remain localized, academic writings can circulate, and expectations can be stabilized and globalized using symbolically generalized codes of communication. The intellectual restructuring, however, remains latent as a second-order dynamics that can be accessed by participants only reflexively. Yet, the emerging “horizons of meaning” provide feedback to the historically developing organizations by constraining the possible future states as boundary conditions. I propose to model these possible future states using incursive and hyper-incursive equations from the computation of anticipatory systems. Simulations of these equations enable us to visualize the couplings among the historical—i.e., recursive—progression of social structures along trajectories, the evolutionary—i.e., hyper-incursive—development of systems of expectations at the regime level, and the incursive instantiations of expectations in actions, organizations, and texts.

In Scientometrics, online first, open access:

http://link.springer.com/article/10.1007/s11192-015-1630-6

Submission dealing for special session extended to 27th April

The deadline for all Social Simulation submissions, including for those for the special session on simulating the social processes of science, have the deadline extended to 27th April.

Paper: "Alternatives to peer review: novel approaches for research evaluation"

A review article in Frontiers in Computational Neuroscience, describing experiments in a project on opening peer review (see http://peerevaluation.org/) coming out of the LiquidPub project.

Aliaksandr Birukou, Joseph Rushton Wakeling, Claudio Bartolin3, Fabio Casati1, Maurizio Marchese, Katsiaryna Mirylenka, Nardine Osman, Azzurra Ragone, Carles Sierra and Aalam Wassef (2011) Alternatives to peer review: novel approaches for research evaluation. Front. Comput. Neurosci., 14 December 2011. doi: 10.3389/fncom.2011.00056

In this paper we review several novel approaches for research evaluation. We start with a brief overview of the peer review, its controversies, and metrics for assessing efficiency and overall quality of the peer review. We then discuss five approaches, including reputation-based ones, that come out of the research carried out by the LiquidPub project and research groups collaborated with LiquidPub. Those approaches are alternative or complementary to traditional peer review. We discuss pros and cons of the proposed approaches and conclude with a vision for the future of the research evaluation, arguing that no single system can suit all stakeholders in various communities.

Scientific Reports introducing a fast-track reviewing process for authors who choose to pay to get their paper reviewed in three weeks

In an interesting, but controversial (see peerreviewneutrality.org), move Scientific Report (already an offshoot of nature with a lighter-touch but pay-to-publish open publication model) us offering authors who pay an additional fee a faster touch in terms of reviewing (paying reviewers in turn for timely reviews). This involves the Nature group sub-contracting the organisation of these reviews to a commercial third party, Rubriq (rubriq.com).

Rubriq offers any author a reviewing service that can be used to get feedback prior to publication for $650. Also see an article about this company at: http://www.economist.com/news/science-and-technology/21578987-portable-reviews-look-set-speed-up-publication-papers-peer-peer

Submission instructions for Special Issue of Scientometrics on “Simulating the Social Processes of Science”

Details about the CfP here:
http://simsocsci.blogspot.co.uk/2014/10/cfp-special-issue-of-scietometrics-on.html
Submission is by the end of April 30th 2015 in pdf format, with no particular formatting regulations for the review stage.

Papers should be submitted to: https://easychair.org/conferences/?conf=sispos2015

Depending on the number of submissions by the 23rd April I will consider requests for extensions, but not if I already have notice of a good number of papers - so do NOT depend on this :-)

As ever, any queries or comments email me, bruce@edmonds.name

Bruce Edmonds

3 relevant papers in the latest issue of JASSS

http://jasss.soc.surrey.ac.uk/JASSS.html

• Innovation Suppression and Clique Evolution in Peer-Review-Based, Competitive Research Funding Systems: An Agent-Based Model
  by Pawel Sobkowicz
  http://jasss.soc.surrey.ac.uk/18/2/13.html
  
  
• Does Longer Copyright Protection Help or Hurt Scientific Knowledge Creation?
  by Shahram Haydari and Rory Smead
  http://jasss.soc.surrey.ac.uk/18/2/23.html
  
  
• Modelling Academics as Agents: An Implementation of an Agent-Based Strategic Publication Model
  by Xin Gu, Karen Blackmore, David Cornforth and Keith Nesbitt
  http://jasss.soc.surrey.ac.uk/18/2/10.html
  

CfP: Special Session on "Simulating the Social Processes of Science"

At Social Simlation 2015 - The Eleventh Conference of the European Social Simulation Association, Groningen (NL), 14-18 September 2015.

DeadLine: 13th April 2015
Details, registration, submission at: http://essa2015.org

This session is for presenting and discussing simulations that represent part of that set of phenomena we call 'science'. It is part of the activities of the ESSA SIG on the same subject (http://simsocsci.blogspot.co.uk). This follows on from the special issue of JASSS (http://jasss.soc.surrey.ac.uk/14/4), the Lorentz workshop at Leiden in 2014 and previous special issues at the ESSA/SocSim series of conferences. It also connects to the PEERE (http://peere.org) and Knowescape (http://knowescape.org) COST actions. This session includes (but is not limited to) simulations about: scientific creativity/innovation, the peer review system, how research grants are distributed, co-authorship and academic collaboration, the emergence and impact of academic fields, and how researchers utilise the knowledge from previous researchers.

For any queries about this session and its scope contact me, Bruce Edmonds <bruce@edmonds.name>.

PNAS paper: "Measuring the effectiveness of scientific gatekeeping" by Siler, Lee and Bero

Measuring the effectiveness of scientific gatekeeping

by

1. Kyle Siler
2. Kirby Lee
3. Lisa Bero

December 22, 2014, doi: 10.1073/pnas.1418218112
PNAS December 22, 2014  

Abstract and paper at: http://www.pnas.org/content/early/2014/12/17/1418218112 

"We found that editors and reviewers generally made good decisions regarding which manuscripts to promote and reject. However, many highly cited articles were surprisingly rejected. Our research suggests that evaluative strategies that increase the mean quality of published science may also increase the risk of rejecting unconventional or outstanding work."