Experimental design in the limelight

We are launching a new series of monthly blog posts by our Chief Executive, Dr Vicky Robinson. In her first blog she focuses on the growing interest in the issues surrounding experimental design, arguably the most important aspect of any scientific study.

The design, analysis and reporting of animal research is an important part of the NC3Rs strategy, particularly since we published a review in PLOS ONE highlighting significant scope for improvement.

Making changes in this area is not a simple undertaking because there are many reasons why many animal studies are poorly designed and reported. These include a lack of access to the right training or expertise, study designs based on the rationale of “we have always used X number of animals”, and a misunderstanding of the definition of ‘reduction’ in the 3Rs. For the record, reduction is about using the minimum number of animals consistent with the study’s objectives - not using the smallest (cheapest) number so as to render the findings meaningless!

Providing scientists with the tools to tackle the problems that we, and subsequently others, have identified has been key to our work on improving experimental design over the past five years. So is our strategy delivering?

Last October we launched our online Experimental Design Assistant (EDA) which provides tailored, study-specific advice and feedback on experimental plans and statistical analysis. It includes tools for randomisation, blinding and sample size calculations which help to address the most common flaws found in in vivo research. The EDA requires researchers to build a visual representation of their experimental plans. This is far more explicit than the simple and often cursory text description traditionally included in grant applications, ethical review submissions or journal publications and our goal is to integrate the EDA into the scientific process to facilitate better peer review of experimental plans.

The EDA was featured in Nature at the end of February and already has over 1200 registered users from around the world. It is early days but feedback on it has been very positive and our programme managers Nathalie Percie du Sert and Katie Lidster, who work together on our experimental design programme, have been kept busy demonstrating the EDA at a wide range of institutions including the Wellcome Trust Sanger Centre and the University of Oxford, as well as to regulators and funding bodies.

The EDA complements our ARRIVE guidelines which we published to tackle the poor reporting of animal studies - an issue which contributes to the irreproducibility of in vivo research and ultimately leads to the unnecessary use of animals. The guidelines also featured in Nature last month, this time in its Trend Watch section, highlighting the growing support from journals for ARRIVE. We now have more than 700 journals signed up to the ARRIVE guidelines, the latest of which is Anaesthesiology which published an accompanying hard-hitting editorial on transparency and enforcement in preclinical research. Our pocket-sized copies of the ARRIVE guidelines have been requested by researchers in 39 countries and 20,000 Mandarin translations were recently sent to China for distribution by the Chinese Association for Laboratory Animal Science.

Although the ARRIVE guidelines were developed for the end-stage of the academic process – publication – they and the evidence supporting them have had other impacts. For example, many of the UK’s major funders have signed up to the guidelines and include them as a condition of grant funding. More recently, the Research Councils led by the MRC have used the ARRIVE guidelines to strengthen and increase the information that they require on animal use in grant applications. We have been working closely with the MRC to facilitate this and hosted our second joint workshop for grant panel members in March, helping to ensure that they are fully equipped to assess whether the proposed design and analysis of animal experiments in grant applications is sound. Dr John Tregoning, from Imperial College London, wrote an interesting blog describing his experience of participating in the workshop which I recommend reading.

As a research funder, we have also introduced an additional step into our grant assessment processes to ensure that where we fund research involving animals (in refinement research or to validate an alternative approach for example) that the plans are optimal: anyone who is successful in the first round of our project grant application process is now required to attend an NC3Rs-led workshop on experimental design before submitting a full proposal.

All of the signs show that our approach is starting to gain traction. The issue of poorly designed animal experiments is now being taken more seriously by scientists and their funders, employers and publishers. However, there is still much more to do in raising awareness about the problem and changing attitudes and practice. This year our focus will be on encouraging the use of the EDA. We will also be starting to look more closely at the effectiveness of the ARRIVE guidelines and what we can learn from the evidence, both published and anecdotal, on their use. You can find further information about our work on our experimental design resource hub.

Subscribe to our newsletter