Of
note is the large proportion of abstracts that state no position on
AGW. This result is expected in consensus situations where scientists '
...generally
focus their discussions on questions that are still disputed or
unanswered rather than on matters about which everyone agrees' (Oreskes
2007, p 72). This explanation is also consistent with a description of consensus as a 'spiral trajectory' in which '
initially intense contestation generates rapid settlement and induces a spiral of new questions' (Shwed and Bearman
2010);
the fundamental science of AGW is no longer controversial among the
publishing science community and the remaining debate in the field has
moved to other topics. This is supported by the fact that more than half
of the self-rated endorsement papers did not express a position on AGW
in their abstracts.
The self-ratings by the papers' authors
provide insight into the nature of the scientific consensus amongst
publishing scientists. For both self-ratings and our abstract ratings,
the percentage of endorsements among papers expressing a position on AGW
marginally increased over time, consistent with Bray (
2010) in finding a strengthening consensus.
The
process of determining the level of consensus in the peer-reviewed
literature contains several sources of uncertainty, including the
representativeness of the sample, lack of clarity in the abstracts and
subjectivity in rating the abstracts.
We address the issue of
representativeness by selecting the largest sample to date for this type
of literature analysis. Nevertheless, 11 944 papers is only a fraction
of the climate literature. A Web of Science search for 'climate change'
over the same period yields 43 548 papers, while a search for 'climate'
yields 128 440 papers. The crowd-sourcing techniques employed in this
analysis could be expanded to include more papers. This could facilitate
an approach approximating the methods of Doran and Zimmerman (
2009),
which measured the level of scientific consensus for varying degrees of
expertise in climate science. A similar approach could analyze the
level of consensus among climate papers depending on their relevance to
the attribution of GW.
Another potential area of uncertainty
involved the text of the abstracts themselves. In some cases, ambiguous
language made it difficult to ascertain the intended meaning of the
authors. Naturally, a short abstract could not be expected to
communicate all the details of the full paper. The implementation of the
author self-rating process allowed us to look beyond the abstract. A
comparison between self-ratings and abstract ratings revealed that
categorization based on the abstract alone underestimates the percentage
of papers taking a position on AGW.
Lastly, some subjectivity is
inherent in the abstract rating process. While criteria for determining
ratings were defined prior to the rating period, some clarifications and
amendments were required as specific situations presented themselves.
Two sources of rating bias can be cited: first, given that the raters
themselves endorsed the scientific consensus on AGW, they may have been
more likely to classify papers as sharing that endorsement. Second,
scientific reticence (Hansen
2007) or 'erring on the side of least drama' (ESLD; Brysse
et al 2012)
may have exerted an opposite effect by biasing raters towards a 'no
position' classification. These sources of bias were partially addressed
by the use of multiple independent raters and by comparing abstract
rating results to author self-ratings. A comparison of author ratings of
the full papers and abstract ratings reveals a bias toward an
under-counting of endorsement papers in the abstract ratings (mean
difference 0.6 in units of endorsement level). This mitigated concerns
about rater subjectivity, but suggests that scientific reticence and
ESLD remain possible biases in the abstract ratings process. The
potential impact of initial rating disagreements was also calculated and
found to have minimal impact on the level of consensus (see
supplemental information, section S1 available at
stacks.iop.org/ERL/8/024024/mmedia).
Our sample encompasses those surveyed by Oreskes (
2004) and Schulte (
2008) and we can therefore directly compare the results. Oreskes (
2004)
analyzed 928 papers from 1993 to 2003. Over the same period, we found
932 papers matching the search phrase 'global climate change' (papers
continue to be added to the ISI database). From that subset we
eliminated 38 papers that were not peer-reviewed, climate-related or had
no abstract. Of the remaining 894, none rejected the consensus,
consistent with Oreskes' result. Oreskes determined that 75% of papers
endorsed the consensus, based on the assumption that mitigation and
impact papers implicitly endorse the consensus. By comparison, we found
that 28% of the 894 abstracts endorsed AGW while 72% expressed no
position. Among the 71 papers that received self-ratings from authors,
69% endorse AGW, comparable to Oreskes' estimate of 75% endorsements.
An
analysis of 539 'global climate change' abstracts from the Web of
Science database over January 2004 to mid-February 2007 found 45%
endorsement and 6% rejection (Schulte
2008).
Our analysis over a similar period (including all of February 2007)
produced 529 papers—the reason for this discrepancy is unclear as
Schulte's exact methodology is not provided. Schulte estimated a higher
percentage of endorsements and rejections, possibly because the strict
methodology we adopted led to a greater number of 'No Position'
abstracts. Schulte also found a significantly greater number of
rejection papers, including 6 explicit rejections compared to our 0
explicit rejections. See the supplementary information (available at
stacks.iop.org/ERL/8/024024/mmedia)
for a tabulated comparison of results. Among 58 self-rated papers, only
one (1.7%) rejected AGW in this sample. Over the period of January 2004
to February 2007, among 'global climate change' papers that state a
position on AGW, we found 97% endorsements.
No comments:
Post a Comment