C-Score Suite

Measuring Collaboration and Why it Matters

The C-Score Suite of tools and processes, conceived by Rapid Science in 2013 and evolving ever since, is designed to enable, incentivize, and reward all contributors engaging in open collaborative research.

The current reward system based on publication results in competition to reveal closely held findings in top journals. Most contributors are not credited, and as a result passed over by hiring and tenure committees and funding agencies. Further, collegial feedback and formal peer review occurs when the research project is completed, absent complete data and code, often resulting in work that cannot be replicated. This impedes progress and fresh insights in the scientific establishment.

The C-Score Suite of Solutions

Rapid Science solutions presented here address research initiatives involving multiple institutions and disciplines, where sharing results and interacting outside of one’s team is critical to achieve reliable and valuable outcomes.

To succeed, a collaborative approach must respect researchers’ valuable time and minimize adoption of an unfamiliar workflow. It requires well crafted policies, funding, infrastructure, and ongoing facilitation to ensure meaningful connections and timely exchanges of findings between team members and the wider community. There must also be the means to assess and reward the contributions of every individual.

A Case Study

Such an effort has been led by the groundbreaking initiative Aligning Science Across Parkinson’s (ASAP). Rapid Science, Stratos, and DataSeer have assisted the leadership in strategizing and implementing many of the requirements described above. By tracking, assessing, and reporting researchers’ collaborative behaviors, it serves as a model for hiring and career advancement reform. See ASAP’s policies and their Blueprint for Open Collaborative Science.
ASAP Blueprint for Collaborative Open Science

Components of the C-Score Suite

Lay the Foundation

Collaboration Workspace

This is the infrastructure that enables interaction among team members, including their profiles, research goals, lab resources that can be shared, access to incremental findings and null results, and tools for discussion, review, and revision. Another key component is the Research Output Management System (ROMS), a centralized methodology for tracking and recording the team’s research outputs.

Facilitate and Communicate

Editorial Facilitator and Open Narrative

The Editorial Facilitator prompts team members to share and discuss their early findings, both privately among grantees and more openly. The EF is a peer of the team, bringing both subject and editorial expertise to the project, akin to journal editors in today’s publishing environment. See an exemplary model job description for the role of an EF.

An Open Narrative is conceived as an interactive document where research teams share, discuss, and review their early findings and insights – initially privately and then periodically as a series of snapshots or a continually updated preprint for public viewing and commenting. The Facilitator writes and maintains the Narrative to reflect new findings and feedback, and all team members are credited as co-authors.

See the progression of a project’s findings from lab to workspace to preprint.

Measure and Reward Success

The C-Score: Incentivizing Collaboration

The Collaboration Score reflects researchers’ contributions to a project, such as sharing their research findings early, often, and openly; it can offer evaluators an alternative to publication as a reward metric. An algorithm measures an array of activities (defined by the funder and participating researchers) that occur in meetings, conferences, and the project workspace. Linking the metric and its contribution permits qualitative analysis, which can minimize gaming and allow evaluators to review an individual’s work more deeply.