In this guest post, Michael Bon, presents “The Self-Journals of Science” (SJS): a new platform for evaluating scientific contributions based on scientists’ collective intelligence and judgment.
SJS breaks with the concept of “academic journals”. In the traditional system, a journal can be described as a seat of power whose input and output are controlled by a handful of co-opted individuals sitting on editorial boards. Journals hand out a limited number of “stamps of approval” for which peers have to compete, and whose value is built on blind trust in its unverifiable internal peer-review processes. The fragmentation and privatization of scientific publishing, and a journal-based evaluation system leads to a number of significant problems, e.g. the “publish-or-perish” culture, lower quality, lack of open access, exorbitant costs, chaotic dissemination.
In contrast, SJS is an open and transparent environment where scientists can come together, discuss science and its standards, and evaluate scientific contributions with appropriate incentives for maximal quality and collaborative input. Papers on SJS are not branded: the value of the platform comes from a straightforward and verifiable application of the very principles of science. The underlying power structure of SJS is horizontal: scientists have equal means to post, review and evaluate so that science can actually be driven by those who actively care for it (including many junior scientists), rather than by self-righteous third parties with private interests, agendas and an unscientific sense of prestige to uphold. This new logic naturally unfolds to offer concrete solutions to all the problems listed in the previous paragraph. Importantly, SJS introduces new metrics with a solid background to capture what is valid and important in the eyes of the scientific community. In strategic terms, an open and community-managed system for research assessment is what we need most to cure ourselves from our dependency on journals. This is certainly the most important – and most unique – thing SJS can offer. Here I explain briefly how the system works and what kind of scientific value it promotes (further information and explanations can be found here, here or here)
1) Scientific value – validity and importance
A scientific paper contains refutable statements that contribute to a body of knowledge. The value of a paper is determined by two issues: the validity of its statements and the importance of its contribution. The scientific community is the only authoritative reference for both notions. An article can be considered as valid when the community debates it and fails to prove it wrong; an article is important if the community regards it as such. Validity and importance are however different notions, which should therefore be measured by different processes.
Validity is established through community-wide open debate (which I term “peer -review”, in contrast to the journals’ gatekeeping process, which I refer to instead as “peer-trial”). Here, we take “open” to mean signed, public, open to all authenticated scientists and open in time. Since the statements made in a scientific paper are refutable, the debate will in principle (given sufficient time and resources to test the statements), eventually converge on a consensus about whether the paper has achieved scientific standards. Debate may also occur around what these standards should be: methodological soundness, clarity of presentation (without ambiguity), satisfaction of various protocols, inclusion of appropriate references, etc. On SJS, the aim of peer-review is not for an individual authority to decide whether to accept or reject the article, but for the community to reach a degree of consensus. The validity of an article is thus measured by the degree of consensus it achieves. It is captured by a non-anonymous and open vote, with two possibilities: “This article has reached scientific standards” or “This article still needs revision”.
The debate takes place using an interface that accurately embeds reviewers’ comments in the article, so that they become part of the paper. The article is not seen as limited to the text initially written by the authors, but also includes the debate it has spawned within the community. Reviewers hold no authority over an article (i.e. no assumption is made about who is right or wrong); however, an author cannot afford to leave a relevant review unanswered since all readers will see the review. Reviews are open and subject to peer scrutiny, just like the paper they critique. Comments can be evaluated and the reviewer is credited. With the traditional goal of acceptance/rejection no longer an issue, open peer-review is no longer an adversarial affair. It is the mutual interest of authors and reviewers to participate in a courteous high-level public debate, as their respective expertise can shine and ongoing in-depth arguments are likely to grab the attention of the rest of the community.
Unlike validity, the notion of importance is unlikely to generate consensus. The perceived importance of an article is highly subjective. It depends on the personal appraisal, understanding or intuition of any given scientist, and this can even evolve over time. The importance of an article is thus socially determined.
To capture this idea of importance, SJS has introduced the concept of a “self-journal”: a personal curation tool that allows every scientist to structure and release a topical selection of articles from any source on any area of interest (ex1, ex2, ex3). Scientists can use their self-journals to freely express their individual viewpoint on their field, by making sense of a range of scientific output. A self-journal is organized into structured and topical “issues”, each featuring an editorial and a selection of web-based scientific items that can be individually commented on by the curator. Self-journals support other uses by curators, e.g. reviewing a field, science popularization, hosting the proceedings of a conference or a journal club. A self-journal is helpful to readers because it provides them with consistent analysis of the scientific output by a peer, which is always a huge need. In turn, it also benefits curators by disseminating their personal vision of the field and can help build their reputation as an expert. In this ecosystem of self-journals, the importance of any one article is captured by the number of scientists who have decided to curate it into an issue of their self-journal.
4) A collaborative and open knowledge economy
In SJS, the value of an article, measured as a combination of validity and importance, is generated by collective peer recognition. In contrast with the traditional vertical system, where value is determined through a stamp of approval from an individual authority, this horizontal approach implies three major differences:
- In SJS, all scientists are free to upload, review and evaluate papers. No separation is assumed between these three different dimensions of scientific activity. The relationships between scientists hence become reciprocal. For instance, authors will always be able to review papers that have been authored by their own reviewers.
- Peer recognition is a near-limitless resource. Recognizing the value of one article does not diminish the value of another. Good articles do not rise at the expense of other papers. There is no artificial structural limitation to the recognition of scientific quality, and the process of evaluation itself is not adversarial.
- Constant community-wide peer pressure is what drives science towards its maximal quality.
This new approach restores symmetry between scientists so that their respective interests can converge. On SJS, the best individual strategy to maximize your value is to “do to others what you would have them do to you”. In other words, if scientists want their peers to pay attention and attribute value to their work, they should themselves pay attention to and recognize the value of their peers’ work. This mutually beneficial mindset is good for scientific knowledge.
A more horizontal paradigm also rewards openness. Since the goal of scientists becomes to convince as many peers as possible, rather than to satisfy the policy of an individual authority, scientists who are open to others will have a clear advantage over those who are not, since their article will have greater outreach. Openness is thus in the interest of every scientist and can be achieved without resorting to legal or bureaucratic methods.
5) Expansion of the platform
The long-term strength of SJS is that science will become community-managed, without intermediaries, in an environment that incentivizes quality. This long-term strength is also its short-term weakness, because it requires part of the community to first come together to kick-start a virtuous shift with sufficient momentum to spread to the whole community. This is not automatic in a world where scientists are under constant pressure to publish and find grants. However, it is clear that if we want this madness to stop, scientists must start to organize themselves at some point. Therefore, I would like to invite any readers who are concerned with the current state of the publishing system to further explore the concepts behind SJS and consider the following arguments:
- Since SJS is a science-friendly environment and not a journal, it is not in conflict with the traditional publishing system and can be used in parallel. Scientists willing to participate in this alternative solution do not need to choose between SJS and the traditional system: they can – and should – use both. Activity on SJS involves no copyright transfer. There is no restriction or disadvantage in discussing material that has been uploaded to SJS or to release an issue of your self-journal with curated hyperlinks. Use of SJS is thus risk-free, and the paradigm shift it will introduce can take place gradually and quietly. Using SJS is simple and enjoyable, and can be useful for building a reputation, especially in combination with a social media strategy.
- The SJS evaluation system gives equal opportunities to all scientists to influence their science. In the traditional system, most scientists (especially junior scientists) will never have that chance and will have to spend their whole life complying with the expectations of a small elite. The fact that everybody can be an author, a reviewer and an evaluator levels out the hierarchical power structure that currently controls science.
- SJS fits what institutions are looking for in a publishing system. It i) is open access ii) has negligible costs (no middle man!) iii) provides metrics which are as easy to use as the impact factor. If scientists provide the initial momentum, there is a real chance that institutions will start to get interested in the concept and support it with gradual recognition.
- SJS is free. There is no intermediary between scholars. Registration requires only an institutional email address so that identities can be checked, in order to offer assurances to other users.
- SJS has a unique model of open governance: again there is no board of insiders wielding power over outsiders.While SJS’ science is openly managed by its users, SJS’ technical aspects are managed by Open Scholar, an international non-profit association that any scholar is free to join. You are not required to trust it, because you can join it in and have your say in the future of SJS.
I firmly believe that SJS is a paradigm that serves the interests of most scientists, and I would welcome input and reviews of the articles in which I have elaborated on these ideas (here or here). Most of all, I would invite you to join in and start feeding this alternative system of value in science, in the interests of a better future and possibly some immediate benefits too!
Follow us on Twitter!
SJS designer and webmaster. Biophysicist, PhD.
Michael Bon spent most of his years of research at the Institut de Physique Théorique – CEA Saclay. He is a specialist of all problems related to RNA secondary structure prediction. As a lover of Science, he was saddened to discover the many flaws of the academic publishing business. He has temporarily quit research to create SJS as a novel and coherent way to solve some of the prominent issues of scholarly communication.
Check his SJS profile.