Enhancing Equity, Diversity and Inclusion in the Funding Landscape

The CES Transformation Fund is an international grant scheme funded by the John Templeton Foundation and led by Prof Rachel Kendal and team, at Durham University, UK, on behalf of the Cultural Evolution Society (CES). Cultural evolution as a growing academic field cannot flourish without diverse perspectives, hence we implemented several measures to enhance equity, diversity and inclusion (EDI) in our funding process. Here we give a brief report of what we have learned.

Since completion of the application process (in September 2022 we announced the 20 projects funded through the CES Transformation Fund) we have been busy analysing our diversity data alongside feedback from applicants and reviewers. We have been evaluating and reflecting on the measures we implemented to assess their efficacy and improve them for further enhancing EDI in funding and Higher Education (HE), as well as share what we have learned with those administering application-based processes more widely.

The diversity objectives

The CES Transformation Fund has four diversity objectives which were at the core of the funding competition:

• Enhance opportunities for early-career researchers and staff (ECR/ECS)
• Extend the geographical reach of the CES beyond North America, Australia, New Zealand, and (Western) Europe (but including indigenous peoples in these regions)
• Extend the disciplinary reach of the CES
• Investigate and support the application of cultural evolution research to societal issues

The EDI- enhanced application process

We developed an EDI-focused framework, in order to achieve our diversity objectives. The main measures we introduced were:

• Sequential anonymized-deanonymized review: reviewers were asked to first score and comment on research proposals written anonymously; only after scores for the anonymous section were submitted, did the grant team release information about the team, enabling reviewers to score and comment on the suitability of the team to achieve the proposed research. Importantly, reviewers could not alter scores for the anonymous section after reading the non-anonymous part.
• 2-stage application process: an outline application stage that prioritised ideas over presentation and, if successful, a full application stage including mentoring opportunities for (i) ECR/ECS, (ii) those new to the cultural evolution field, and (iii) those where reviewer comments implied that advice on cutting-edge methods would be useful.
• Diversity scoring system: in a confidential ‘diversity box’, applicants could outline how their project and their team fulfilled the diversity objectives, along with any other sensitive issues to do with personal disadvantage/EDI they wished considered. The grant team independently considered diversity in order to (i) record where the grant scheme’s diversity objectives were met by a project and (ii) ensure that groups with protected characteristics were not disadvantaged. Diversity was considered anonymously alongside the independent reviewer scores for applications close to the threshold for invitation to full application/funding.
• ‘Ethics box’: applicants were asked to engage with ethical questions regarding their projects (including disciplinary guidelines; fair treatment of team members, participants and communities; open science, etc.) and reviewers considered this in their assessment.
• Diverse review panel: each application was assessed by a panel of reviewers that was as diverse as possible in terms of gender (assumed due to logistics of asking each reviewer to self-identify), geography/nationality, career stage, and discipline. To maintain anonymity, panels were different at outline and full application stage.

Throughout the application process we aimed to maintain the highest standards and embed EDI into our process, including scrutinising reviewer comments and providing advice in applicant feedback.

Did the application process enable the diversity objectives?

Data from the 20 funded projects (representing 17% of applications) indicate that the CES Transformation Fund’s diversity objectives were fulfilled through our open competition that, importantly, did not employ diversity-related eligibility criteria nor quotas. In a nutshell:

• 12 projects have an ECR/ECS in a leadership position (8 with Principal Investigators (PI) and 10 with Co-Investigators (Co-I))
• 10 projects extend the geographical reach of the CES, either by being led by scholars from outside (Western) Europe, Australia, New Zealand, or North America (4 PIs and 10 Co-Is), by being hosted (PI institution) in countries outside those regions, or by employing scholars in those regions (16 Post-Doctoral Research Associates (PDRAs) and many Research Assistants (RAs))
• 15 projects conduct primary data collection, of which 12 do so outside of North America, (Western) Europe, Australia, and New Zealand
  • All of these projects involve team members from the countries where primary data collection occurs [with PIs (2), Co-Is (5), PDRAs/named RAs (5), short-duration RAs/field roles (8)], minimising the risk of parachute, helicopter, or exploitative research
• 19 projects are interdisciplinary and involve a team member from a discipline not generally associated with cultural evolution [likely responsible for the pleasing result, for a field founded in mathematical evolutionary modelling, of over 50% of projects employing mixed (qualitative & quantitative) methods]
• 15 projects directly address a societal issue within the funded period (many involving co-production of research with communities)
• Many projects plan to share their research outcomes with policy-makers/NGOs (19), community members/general public (15), and business (2)

What have we learned and how can we take this forward?

After funding decisions were made, we asked applicants and reviewers to complete anonymous surveys to share their experience/thoughts regarding the application process and the implemented EDI measures. The feedback was positive, with many highlighting that they would like to see the EDI measures (particularly the sequential anonymized-deanonymized review and the use of diversity considerations alongside reviewer scores) employed more widely in funding competitions. The survey feedback, and our own evaluation, have certainly highlighted that there are aspects of the application process that can be improved, EDI factors that we may have missed, or aspects of our implementation that may have inadvertently disadvantaged particular groups. This reflects the complexity of addressing EDI and of changing established processes.

Enhancing EDI (especially across complex international and interdisciplinary contexts) is an iterative learning process requiring honest reporting of outcomes, thorough and continuous reviewing, openness to feedback, and evaluation to understand and act upon any shortcomings. Moreover, controlled experimental studies are required to determine the extent to which each measure (alone) contributes to enhancing EDI in application processes. Nevertheless, we are confident that the framework we implemented led to achieving the fund’s diversity objectives (in an open call without recourse to diversity-related eligibility criteria nor quotas) and in challenging structures and processes which discriminate against diverse applicants.

We will share our application process, data, and evaluation in more detail in a dedicated publication. We are also working towards disseminating lessons learned and enabling good practice for enhancing equity in application-based processes in Higher Education more widely (funding and recruitment), as well as funding bodies in academia and the charity sector. You can now download a how-to-guide for automating an application review procedure that separates anonymous from identifiable content here.  

Stay tuned and do get in touch (ces.transformationfund@durham.ac.uk) if you would like to discuss!