The State of science, journals, peer-review, thoughts on Open Science, reproducibility, and Science 2.0? Dr. Benjamin Laken University of Oslo @benlaken
Jul 28, 2015
The State of science, journals, peer-review,
thoughts on Open Science, reproducibility,
and Science 2.0?
Dr. Benjamin Laken
University of Oslo
@benlaken
We are now on the brink of an achievable aim: for all science literature to be online, for all of the data to be online and for the
two to be interoperable.
Essentially, the volume of published work is constantly increasing.
A rough estimate of the rate of increase is …
Unsurprisingly, the number of PhDs is also increasing
Nature 472, 276-279 (2011) | doi:10.1038/472276a
Trends in PhDs granted per year for all disciplines,
values x103
The number of science doctorates earned each year grew by nearly 40% between 1998 and 2008, to
some 34,000, in countries that are members of the Organisation for
Economic Co-operation and Development (OECD). The growth
shows no sign of slowing
Systems Research and Behavioral Science Syst. Res (2014) Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/sres.2324
(Sta
ts h
ere
for U
SA b
iom
edic
al s
cien
ces)
However, there is no matching rise in permanent positions
This has created ‘The Postdoc Pile-Up’
Nature 520, 144–147 (09 April 2015) doi:10.1038/520144a
In essence a saturated Postdoc market. This means that postdocs who want to stay in academia are usually destined to travel from one place to another with no
long-term prospects. (One of the reasons you may hear postdocs referring to themselves as ‘hobos with a PhD’.)
In the context of the increasing pressures on individual scientists to compete—largely through publications—enters
the problem of Academic publishers as for-profit entities
The first journal, Proceedings of the Royal Society, was created by Henry Oldenburg (1619--1677), in 1665. The idea was to drive science forward through rapid
communication and sharing to a broad audience.
However, things have gone slightly astray over the years…
The role of journals: have they changed from a force of innovation to a hinderance?
Apple: Exxon:
Profits (% of revenue) in 2011
Elsevier:
Springer:
John Wiley & Sons:
~25%~8.5%
36% (£724m)
33.9% (£866m)
42% ($106m)
Are journals overcharging?
Cushing, 2014, https://www.techdirt.com/articles/20140605/05175627467/academic-publisher-fights-publication-paper-criticizing-publishers-price-increases-profits.shtml
Is a good solution to science communication simply traditional journals but as open-access?
Open access does not mean free, often the authors still pay high prices to publish their research.
The growth of Open Access (OA) journals in the last 10 years opened the door for the creation of hundreds of new Publishing company and new
journals, who engage in behaviour often best described as ‘predatory’…
They claim their articles are peer-reviewed, however independent research and many examples have repeatedly shown that the real barrier
to publication in their journals is simply the publication fee…
The Dark-side of the OA journal explosion
In 2005, following journal submission request emails, Peter Vamplew sent a 10 pg paper (with two figures) to International Journal of Advanced Computer Technology…
…they accepted the paper. An anonymous reviewer rated it as “excellent”, and the journal requested a fee of $150.
Tom Spears submitted a completely incoherent paper featuring a made-up study area (Nepean Desert)
Several famous examples of OA journal ‘stings’
Sent to 18 journals. 8 responded quickly, accepting, asking a $1k - 5$ fee.
More famous sting: John Bohannon
J Bohannon Science 2013;342:60-65
AbstractDozens of open-access journals targeted in an
elaborate Science sting accepted a spoof research article, raising questions about peer-review practices in much of the open-access world.
Peer review reviewed.Few journals did substantial review that identified the paper's flaws.
Another problem: the valuable reviews are usually wasted
It is not a question of if an article will be published, but where.
In this context, the expert reviews, community feedback, and reproducibility are the hallmarks of quality, and a real factor in
the value of the research.
We need to develop systems which focus on these aspects of the peer system, rather than see peer-review as the ‘behind
the scenes’ work.
Explosion in OA journals over last decade (and rapid growth of predatory OA journals)
Published science doubles every ~9 yrs
New PhD’s awarded annually increasing (simultaneous decrease in TT and Permanent positions)
‘Publish or perish’, saturated PhD-level job market, and resulting competitiveness has combined with journal profiteering
…and if a paper gets a hard review, rejection usually means a paper will be submitted (unchanged) to new journal, until accepted
Many Journals = opaque review process, expensive, & behind the scenes politics/motives
End result (*at least for me) = scepticism, cynicism, frustration, time loss, high-blood pressure, and madnessPublication ≠ quality guarantee
Favour systems totally open (inc. review) and communal New paradigm: publication not the end of a funding cycle but the
start of a collaborative communal conversation
Summary
My fantasy model of Science 2.0
Data, code, paper: all in one open repository (like ArXiV)Version control system (like Github)
All researchers have unique ID profile (like RG/ORCID)
ID-tagged comments. Can be general, or specific: specific comments tagged to sections, as a thread, including up-
vote system (like StackOverflow) on paper and comments.Copying, and contributing (pull-requests) allowed
to create rapid-iterations and community refinement (like Github)
Vote-based score system, with scores contributing to different levels of community privileges (like StackOverflow).
Score a measure of a researchers overall contributions (Like RG), > than citation metrics [e.g. as bad papers often cited on purpose]
Needs shared super-computer-scale resources to re-run experiments (Like Google Earth Engine)
(This model already exists for the open software we use every day. The model can be used for research, and for lecture/course/textbook material too.)