"Digital tools for analyzing text are powerful and improving dramatically over time," Emily Grubert of Stanford University told environmentalresearchweb. "We demonstrate that digital tools are useful for supporting human reading when searching for patterns at very different scales. This is particularly relevant in environmental sciences because the literature grows so quickly. No one person can read and process it all, at least not on the level that a computer can."

Social sciences and humanities researchers use computational tools to interrogate large amounts of text; the field of digital humanities emerged in the 1990s. But the tools are not yet common in the environmental sciences.

Grubert and Stanford colleague Anne Siders carried out two case studies demonstrating the use of these techniques. One was a broad review of around 8200 abstracts on environmental life-cycle assessment, whilst the other was a more in-depth analysis of 275 full-text articles and reports on adaptive capacity to climate change.

"Digital tools help parse field-scale trends replicably and transparently even when human readers are challenged to keep up with changes," said Grubert. Writing in Environmental Research Letters (ERL) , Grubert and Siders note that even small digitally-aided analyses are close to the upper limit of what can be done manually.

The broad review used topic modelling to examine how the field of life-cycle assessment has evolved. Over the last 20 years, there have been more life-cycle assessments focused on climate change at the expense of those concerned with other impacts, particularly human health, the technique showed. Although researchers had previously suspected this to be the case, they felt uncomfortable making the claim given the thousands of articles involved. The computational analysis provided confirmation.

The in-depth review, meanwhile, used collocation and network analysis to assess empirical field work on the adaptive capacity of social systems to climate change. This meant the researchers could analyse information at a micro-scale not visible to human readers, picking up on which of 165 determinants of adaptive capacity occurred in the same texts and how these texts were linked. The results, Grubert and Siders write, raise questions about assumptions and theories currently held, consciously or unconsciously, by researchers in the field and inform the development of a new mechanism-based model of adaptive capacity.

Grubert and Sider hope they have demonstrated that digital tools are well enough developed to be extremely useful for those in the environmental community asking questions about field-scale trends.

"With digital tools, it’s possible to explore the entire literature from a given field or subfield, which is not generally possible when researchers are restricted to what we can read and process manually," said Grubert. "Digital tools can help researchers ask bigger questions, test for biases, and consider literature gaps holistically because they enable transparent and replicable investigations of large bodies of work. We also hope that digital tools can help literature reviews and meta-analyses become less ad hoc, providing frameworks for careful consideration of the appropriate work."

Meta-analyses are valuable for identifying trends and gaps in the literature, building consensus, and generating field-wide research agendas, according to the researchers. Traditionally these analyses have needed quantitative data. Computational text mining, however, enables quantitative assessment of qualitative and narrative data such as that from work on social-ecological systems.

The team believes that digitally-aided reviews reduce subjectivity and increase transparency by requiring researchers to identify assumptions in their review methods. There’s also still a need for expert judgement in developing selection criteria and interpreting results.

"Our work draws heavily upon the advancements of digital humanities and computational social science," said Grubert. "We happen to be applying these tools to natural science and engineering fields, but our work could not exist without the work of scholars in these other disciplines. As interdisciplinary researchers, we think it’s extremely significant that using digital methods like these put us in a position to learn from and collaborate with people in the humanities and social sciences who are pioneering these methods."

Grubert and Siders became interested in computational text analysis tools when they began to investigate the large literatures from their fields. "We started to see interesting patterns, but providing evidence that those patterns existed was difficult using human reading alone," said Grubert. "We wanted to find tools that would help us test and support those hypotheses in a way that is more transparent and replicable than simply telling our audiences to trust us. Journal articles have so much information that allows a researcher with the right tools to develop and test interesting hypotheses."

Now the team is exploring further ways to use digital tools in environmental research, both within and beyond the review and meta-analysis phase. Grubert is using digital tools to assess social and environmental attitudes amongst the general public by analysing newspaper articles, blogs, and court cases. Siders, meanwhile, is considering how language choices affect goal-setting and decision-making activities and ultimately change our priorities for environmental work.

"As the community gains a facility with digital methods, digitally-aided reviews could advance to the level of novel meta-research, asking questions like how science influences policy; how diversity in the sciences affects the types of questions being addressed through research; or how institutional or disciplinary biases affect results," the pair write in their review. Style analysis could investigate whether major grant calls affect the topics scientists choose to work on or whether certain topics are written more like government versus industry documents. "As important, though, might be the opportunity to engage in truly and deeply transdisciplinary collaboration that sharing a method across fields with dramatically different modes of inquiry could bring," Grubert and Siders conclude.

Related links

Related stories