A simulation of disagreement for control of rational cheating in peer review
by Francisco Grimaldo and Mario Paolucci
ABSTRACT. We present an agent-based model of peer review built on three entities - the paper, the scientist and the conference. The systems is implemented on a BDI platform (Jason) that allows us to define a rich model of scoring, evaluating and selecting papers for conferences. Some of the reviewers apply a strategy (called “rational cheating”) aimed to prevent papers better than their own to be accepted. We show how a programme committee update based on disagreement control can remove them.
KEYWORDS. Artificial social systems, Peer Review, Agent-based simulation, Trust reliability and reputation
Available at: http://www.scs-europe.net/conf/ecms2012/ecms2012%20accepted%20papers/sdcb_ECMS_0088.pdf
Thursday, 27 September 2012
Paper: Social Dynamics of Science
Social Dynamics of Science
(Archived on 22 Sep 2012)
The birth and decline of disciplines are critical to science and society. However, no quantitative model to date allows us to validate competing theories of whether the emergence of scientific disciplines drives or follows the formation of social communities of scholars. Here we propose an agent-based model based on a \emph{social dynamics of science,} in which the evolution of disciplines is guided mainly by the social interactions among scientists. We find that such a social theory can account for a number of stylized facts about the relationships between disciplines, authors, and publications. These results provide strong quantitative support for the key role of social interactions in shaping the dynamics of science. A "science of science" must gauge the role of exogenous events, such as scientific discoveries and technological advances, against this purely social baseline.At: http://arxiv.org/abs/1209.4950
Tuesday, 25 September 2012
Paper: Modeling peer review: an agent-based approach
Modeling peer review: an agent-based approach
Stefano Allesina, Chicago UniversityAbstract: The peer review system is under severe strain. Corrections have been
proposed, but experiments to determine effective measures are difficult to
perform. I propose a framework in which alternatives to the current peer
review system can be studied quantitatively using agent-based modeling. I
implement three possible systems. I show how, all other things being equal,
these alternatives produce different results in terms of speed of
publication, quality control, reviewers' effort, and authors' impact. This
modeling framework can be used to test other solutions for peer review,
leading the way to an improvement of how science is disseminated.
Keywords: peer review; agent-based modeling; publishing; editorial rejection;
simulations
http://library.queensu.ca/ojs/
Tuesday, 11 September 2012
Paper: "Positive assortment for peer review" by Aktipis & Thompson-Schill
Positive assortment for peer review
- C Athena Aktipis
- Department of Ecology and Evolutionary Biology, University of Arizona, aktipis@alumni.reed.edu
Abstract
We suggest that the introduction of
positive assortment (the pairing of individuals with similar
characteristics) to the peer
review process would increase the speed of
reviewing, improve the quality of reviews, and decrease the burden on
reviewers.
In assortative reviewing, each reviewer is given a
score based on speed of reviewing, the usefulness of the review, the
rate
of reviewing, or any other priority of the journal
editor. Authors submitting manuscripts are then paired with reviewers
who
have similar scores to themselves. This is a
no-cost solution that aligns reviewers’ incentives by accounting for the
benefits
provided to the scientific community and returning
them in kind. This assortative reviewing system can promote rapid, high
quality, and high volume reviewing at a benefit to
the scientific community at no financial cost.
Tuesday, 10 July 2012
CfP: EOPS V - Epistemological Perspectives on Simulation 2012
=============================
CALL FOR PAPERS OR EXTENDED ABSTRACT
Epistemological Perspectives on Simulation 2012 (EPOS V)
San Antonio, Texas, USA
October 10-12, 2012
trinity.edu/epos
CALL FOR PAPERS OR EXTENDED ABSTRACT
Epistemological Perspectives on Simulation 2012 (EPOS V)
San Antonio, Texas, USA
October 10-12, 2012
trinity.edu/epos
Extended Deadline for Manuscript Submission: August 20, 2012.
==============================
Epistemological Perspectives on Simulation 2012
EPOS V Call for Papers
The Fifth Edition of Epistemological Perspectives on Simulation promises to be a productive meeting of minds. Held of the first time outside of Europe, this meeting aims to bring together researchers working on common issues of Computer Simulation, Methodology, and Philosophy of Science from many different angles, making for a truly interdisciplinary opportunity for discussion on the state of the art in this blooming field.
==============================
Epistemological Perspectives on Simulation 2012
EPOS V Call for Papers
The Fifth Edition of Epistemological Perspectives on Simulation promises to be a productive meeting of minds. Held of the first time outside of Europe, this meeting aims to bring together researchers working on common issues of Computer Simulation, Methodology, and Philosophy of Science from many different angles, making for a truly interdisciplinary opportunity for discussion on the state of the art in this blooming field.
The meeting will take place on the grounds of Trinity
University, in San Antonio, Texas, on October 10 – 13, 2012.
Like its predecessors the workshop wants to provide
a forum
for researchers from various disciplines, such as the social sciences,
economics, computer science, engineering or the natural sciences, who
are interested in discussing epistemological aspects of simulation
across disciplinary boundaries.
Philosophers are highly welcome, too.
Topics to be addressed in the workshop include, but are not restricted to issues such as the epistemology of simulation, credentials for model building, and standards for presenting and analyzing simulation results. EPOS V will bring together simulation experts, modelers, scientists and philosophers of science to share the latest advancements in simulation technology, its applications, as well as potential benefits and drawbacks. Like its predecessors, the workshop wants to provide a forum for researchers from various disciplines, such as the social sciences, computer science, engineering or the natural sciences, who are interested in discussing epistemological aspects of simulation across disciplinary boundaries.
Topics to be addressed in the workshop include, but are not restricted to issues such as the epistemology of simulation, credentials for model building, and standards for presenting and analyzing simulation results. EPOS V will bring together simulation experts, modelers, scientists and philosophers of science to share the latest advancements in simulation technology, its applications, as well as potential benefits and drawbacks. Like its predecessors, the workshop wants to provide a forum for researchers from various disciplines, such as the social sciences, computer science, engineering or the natural sciences, who are interested in discussing epistemological aspects of simulation across disciplinary boundaries.
Following
the tradition of previous EPOS workshops, we will publish a selection of the
accepted papers in a special issue of a journal after a further reviewing
process. This time, the venue will be a special issue of the International Journal of Agent Technologies
and Systems (IJATS).
Important deadlines are as follows:
August 20, 2012: Submission of papers or extended abstracts
(Extended Deadline)
September 10, 2012: Notification of acceptance
September 28, 2012: Receipt of full papers
October 10-12, 2012: Workshop, San Antonio, Texas, USA
October 13, 2012: Complementary field trip for participants
in the beautiful Texas Hill Country
Hosts: Prof. Dr. Dante Suarez and Prof. Dr. Yu Zhang
(Trinity University), Prof. Dr. Klaus G. Troitzsch (University of
Koblenz-Landau) and Prof. Dr. Manuel Castañon-Puga (University of Baja
California).
Tuesday, 3 July 2012
An article on predatory scholarly publishing
"Predatory scholarly publishing" by Moshe Y. Vardi
Original article is at: http://dl.acm.org/citation.cfm?doid=2209249.2209250
The source material is on the website: http://scholarlyoa.com
Original article is at: http://dl.acm.org/citation.cfm?doid=2209249.2209250
The source material is on the website: http://scholarlyoa.com
Friday, 17 February 2012
Paper: Saint Matthew strikes again: An agent-based model of peer review and the scientific community structure
Saint Matthew strikes again: An agent-based model of peer review and the scientific community structure
Flaminio Squazzoni, Claudio Gandelli
University of Brescia, Italy
University of Brescia, Italy
email: squazzon@eco.unibs.it
http://www.sciencedirect.com/science/article/pii/S1751157711001179
This paper investigates the impact of referee reliability on the quality and efficiency of peer review. We modeled peer review as a process based on knowledge asymmetries and subject to evaluation bias. We tested various levels of referee reliability and different mechanisms of reviewing effort distribution among agents. We also tested different scientific community structures (cohesive vs. parochial) and competitive science environments (high vs. low competition). We found that referee behavior drastically affects peer review and an equal distribution of the reviewing effort is beneficial only if the scientific community is homogeneous and referee reliability is the rule. We also found that the Matthew effect in the allocation of resources and credit is inherent to a ‘winner takes all’ well functioning science system, more than a consequence of evaluation bias.
Keywords: Peer review, Referees, Referee reliability, Matthew effect, Agent-based model
Published: Informetrics, 6, 2, 2012, pp. 265-275
Subscribe to:
Posts (Atom)