bioRxiv to help pilot DocMaps for peer reviews

Cold Spring Harbor Laboratory, eLife/Sciety, EMBO and Knowledge Futures to pilot the DocMaps framework in community preprint reviews and evaluations

Depiction of a DocMap from the Early Evidence Base representing the peer review of a bioRxiv preprint by Review Commons.

Beginning in the next few weeks, Cold Spring Harbor Laboratory (CSHL), eLife/Sciety, EMBO, and the Knowledge Futures Group will pilot the new DocMaps framework by applying it to peer reviews and other evaluations of preprints posted on the bioRxiv and medRxiv preprint servers operated by CSHL. These reviews are also aggregated by EMBO’s Early Evidence Base and eLife’s Sciety platforms. DocMaps will provide machine-readable data and context about how community groups and peer review platforms are evaluating preprints. This will facilitate the exchange, aggregation and publishing of peer reviews within a distributed, interoperable infrastructure.

DocMaps are a way to describe the processes used to create documents in a structured manner. The framework is sufficiently flexible to capture a variety of contextual data about a document — from a minimum assertion that a process took place to a detailed history of every edit to the document. This pilot focuses on a specific use case, but the ultimate goal is to take advantage of DocMaps’ extensibility to cover a variety of use cases across different documents generated in scholarly publishing (and beyond). The pilot builds on the work of the DocMaps Technical Committee, a group of leading publishers, technology and infrastructure developers, review services, taxonomy definers, and open science advocates convened in the summer and fall of 2020 by Knowledge Futures, ASAPBio, and TU Graz’s Open and Reproducible Research Group.

The DocMaps pilot is a collaborative effort between four open science infrastructure developers that seeks to support article evaluation models pioneered by groups like the Novel Coronavirus Research Compendium (NCRC), EMBO Press, Review Commons, eLife, and the MIT Press’s Rapid Reviews: COVID-19 (RR:C19) journal. It has adopted an agile approach to rapidly defining an initial use-case, prototyping a solution, testing it against real-world data, and iterating. The result is a working prototype of the framework that will go from idea to real-world execution in less than 6 months.

This collaborative ethos extends to the design of the framework itself. At its core, the DocMaps framework is a set of agreed-upon conventions for aligning editorial processes with the Publishing Status Ontology (PSO) and the Publishing Workflow Ontology (PWO) and for expressing these events in a domain-specific language that can be easily interpreted by machines and humans alike.

By aligning with well defined publishing ontologies, DocMaps makes use of community endorsed, tested, and extensible models rather than trying to invent a new model, which would be less likely to succeed. This also allows developers and researchers who are familiar with RDF and other Semantic Web technologies to model editorial events as graphs and serialize and study them in many formats.

The use of domain-specific language conventions makes it easy for users to follow editorial events without needing to understand the underlying ontologies or semantic web structures. This also allows developers who are less familiar with semantic web technologies to create and consume DocMaps in simple formats like JSON using their existing technology stacks.

The pilot represents a significant step forward in supporting a fully fledged, distributed preprint review ecosystem. In the coming months, members of the working group plan to further develop the framework’s documentation and tooling for developers, build a community around DocMaps to expand its conventions and use cases, and provide multiple easy-to-use infrastructure options for groups who want to participate in transparent preprint review and take advantage of the benefits DocMaps offer.

To learn more about the DocMaps framework, see the draft documentation. If you are interested in implementing DocMaps as a reviewer, aggregator, or consumer, please contact Gabe Stein, Head of Product at Knowledge Futures, at gabe@knowledgefutures.org.