From this set of studies, the team drew up a list of 10 analytical methods commonly used in Brazil — including the MTT assay to assess cells’ metabolic activity, RT-PCR to amplify and detect specific genetic sequences, and the elevated plus maze to test rodent behaviour. The researchers then randomly chose experiments for replication that use one of these techniques.
Amaral and his team expect to finish the project by 2021, with funding from the Serrapilheira Institute in Rio de Janeiro — Brazil’s first private organization dedicated to supporting basic research in the natural sciences, computer sciences, engineering and mathematics. An initial 145,000-Brazilian-real (US$37,000) grant allowed the researchers to establish the project’s general methodology, select the experiments to analyse and build a countrywide network of collaborators.
Now the project is working with participating labs to establish protocols for replication attempts, with the help of another 1,000,000-real grant from the institute, awarded in January.
Lather, rinse, repeat
The Brazilian project follows in the footsteps of several attempts to replicate scientific outcomes on a large scale. One of the first was the Reproducibility Project: Psychology, which launched in 2011. It gathered 270 scientists to replicate the results of 100 psychology articles in different journals, yielding a reproducibility rate of 36–47%2. Similar initiatives in experimental economics, philosophy and social sciences arrived at replication rates ranging between 57 and 78%.
The effort’s leaders hope that it will reveal ways to predict the reproducibility of scientific studies. “It might be invaluable for future decisions on how to finance and elevate science in Brazil,” says Roger Chammas, an oncologist at University of São Paulo School of Medicine and coordinator of one of the replicating labs.
Daniel Martins-de-Souza, a biochemist at the University of Campinas in Brazil, agrees. “If the project moves forward, it may aid defining which types of studies or methods have more potential to obtain new possibilities of therapy against diseases,” he says. “It could guide the decision-making process of funding agencies.”
Others are more sceptical. Lygia da Veiga Pereira, a geneticist at the University of São Paulo, says it is too early to tell whether the project’s findings will be able to help guide future research. Still, she says, “testing how much of Brazilian science is reproducible will be a good diagnosis for us”.