New Paper: Opening the Black-Box of Peer Review: An Agent-Based Model of Scientist Behaviour

Squazzoni, Flaminio and Gandelli, Claudio (2013) 'Opening the Black-Box of Peer Review: An Agent-Based Model of Scientist Behaviour' Journal of Artificial Societies and Social Simulation 16 (2) 3 <http://jasss.soc.surrey.ac.uk/16/2/3.html>.

Abstract

This paper investigates the impact of referee behaviour on the quality and efficiency of peer review. We focused on the importance of reciprocity motives in ensuring cooperation between all involved parties. We modelled peer review as a process based on knowledge asymmetries and subject to evaluation bias. We built various simulation scenarios in which we tested different interaction conditions and author and referee behaviour. We found that reciprocity cannot always have per se a positive effect on the quality of peer review, as it may tend to increase evaluation bias. It can have a positive effect only when reciprocity motives are inspired by disinterested standards of fairness

 

Keywords:

Peer Review, Referees, Referee Behaviour, Reciprocity, Fairness

 

• Available at: http://jasss.soc.surrey.ac.uk/16/2/3.html

Conference track on "Social Simulation of Science Processes" @ESSA 2013

There will be a special track of the following conference on the social simulation of science, chaired by Flaminio Squazzoni:

9th European Social Simulation Association Conference
Warsaw School of Economics, Warsaw, Poland
September 16-20, 2013
http://www.essa2013.org

The European Social Simulation Association and Warsaw School of
Economics, Division of Decision Analysis and Support, invite
researchers and scholars interested in applications and theoretical
foundations of simulation modeling in social sciences to participate
in the 9th European Social Simulation Association Conference.

The conference aims to provide an interdisciplinary forum for social
scientists, theorists, applied researchers and simulation modelers to
cooperate and exchange ideas concerning state of the art in methods
and applications of computational social sciences.

Submission

        - Full paper - in the length of 10 to 12 pages, which should be
comprehensive and consists of detailed presentation of theory,
methodology and simulation results
        - Extended abstract - in the length of 3 to 4 pages, which presents
the current topic in progress with enough detail to ensure proper
evaluation
        - Poster - should present the current work in progress

* Important dates *
- Registration opens:           1 January 2013
- Paper submission closes:      1 March 2013
- Notification of acceptance:   15 April 2013
- Final manuscript due:         15 May 2013

Paper publication

* Accepted full papers will be published by Springer in the conference
proceedings series "Advances in Intelligent Systems and Computing",
http://www.springer.com/series/11156 (indexed by ISI Proceedings,
DBLP, Ulrich's, EI-Compendex, SCOPUS, Zentralblatt Math, MetaPress,
Springerlink).
* Authors of selected full papers or extended abstracts will be
invited to submit their extended version for special issue of Central
European Journal of Economic Modelling and Econometrics,
http://cejeme.org/ (indexed by IndexCopernicus, IC Value in 2011: 4.85
and RePEc).
* Full paper abstracts, extended abstracts and poster summaries will
be presented on conference website.

* Local organizing committee contact: contact@essa2013.org
* Conference website: http://www.essa2013.org

On behalf of the organizing committee:
Mateusz Zawisza
Warsaw School of Economics

A special session on "Social simulation of science processes" (chaired by Flaminio Squazzoni) has been scheduled at the 9th ESSA conference in Warsaw, Sept 16-20 2013. For info on submission, visit: http://www.essa2013.org

 

Paper: How the Scientific Community Reacts to Newly Submitted Preprints: Article Downloads, Twitter Mentions, and Citations

How the Scientific Community Reacts to Newly Submitted Preprints: Article Downloads, Twitter Mentions, and Citations

Xin Shuai¹, Alberto Pepe^2*, Johan Bollen¹

1 School of Informatics and Computing, Indiana University, Bloomington, Indiana, United States of America, 2 Center for Astrophysics, Harvard University, Cambridge, Massachusetts, United States of America

Abstract

We analyze the online response to the preprint publication of a cohort of 4,606 scientific articles submitted to the preprint database arXiv.org between October 2010 and May 2011. We study three forms of responses to these preprints: downloads on the arXiv.org site, mentions on the social media site Twitter, and early citations in the scholarly record. We perform two analyses. First, we analyze the delay and time span of article downloads and Twitter mentions following submission, to understand the temporal configuration of these reactions and whether one precedes or follows the other. Second, we run regression and correlation tests to investigate the relationship between Twitter mentions, arXiv downloads, and article citations. We find that Twitter mentions and arXiv downloads of scholarly articles follow two distinct temporal patterns of activity, with Twitter mentions having shorter delays and narrower time spans than arXiv downloads. We also find that the volume of Twitter mentions is statistically correlated with arXiv downloads and early citations just months after the publication of a preprint, with a possible bias that favors highly mentioned articles.

http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0047523

 

Report: How Readers Discover Content in Scholarly Journals

A report that presents analyses of some survey data on the various means that researchers use to find the academic papers they wish to read:

A set of slides summarising the main findings:
http://www.sic.ox14.com/presentations/How%20Readers%20Discover%20Content%20in%20Scholarly%20Journals%20-%20ALPSP%202012.pdf

A free summary pdf of the report:
http://www.renewtraining.com/How-Readers-Discover-Content-in-Scholarly-Journals-summary-edition.pdf

Other, more detailed, versions (not free):
http://www.renewtraining.com/publications.htm

Call for applications to participate in a workshop on Simulating the Social Processes of Science, 7-11 April 2014, Netherlands

We are planning to submit an application for a Lorentz Centre workshop (http://www.lorentzcenter.nl/) on the topic of the Special Section of JASSS about a year ago on the Topic "Simulating the Social Processes of Science" (http://jasss.soc.surrey.ac.uk/14/4/14.html).  If successful this will happen 7-11 April 2014 in the Netherlands.  This is a very nice set-up with lunch, refreshments and facilities laid on in the centre, leaving lots of time for discussion and collaboration.

We are looking for people who could contribute to this.  In other words people who:

1.  Have done a simulation of a social aspect of science
2.  Are currently working on a simulation of a social aspect of science
3.  Has written about how to simulate a social aspect of science
3.  Is or could be involved in writing computer science tools to help with this
3.  Is a relevant philosopher, complexity science, science studies,
visualisation expert (or similar)
3.  Has studied/observed how scientists interact
4.  Is planning on doing one of the above

If you are interested in participating, please read the details at http://cfpm.org/spos and apply!  Deadline November 10th 2012.

Paper: Opening The Black-Box of Referee Behaviour. An Agent-Based Model of Peer Review

Opening The Black-Box of Referee Behaviour. An Agent-Based Model of Peer Review
by Flaminio Squazzoni and Claudio Gandelli

ABSTRACT. This paper investigates the impact of referee behaviour on the quality and efficiency of peer review. We focused especially on the importance of reciprocity motives to ensure cooperation between everyone involved. We modelled peer review as a process based on knowledge asymmetries and subject to evaluation bias. We built various simulation scenarios where we tested interaction conditions and manipulated author and referee behaviour. We found that reciprocity per se can have a negative effect on peer review as it tends to increase evaluation bias. It can have a positive impact only when purged by self-interest motivation and accompanied by disinterestedness and fairness standard.

KEYWORDS. Peer review; referees; referee behaviour; reciprocity; agent-based model.

AVAILABLE: http://www.scs-europe.net/conf/ecms2012/ecms2012%20accepted%20papers/sdcb_ECMS_0022.pdf