Visual Turfgrass Quality Revisited.

A rating system on a scale of 1 (worst) to 9 (best) has been used for decades by researchers to describe the visual appearance of turfgrasses. However, such data collection has been criticized for being inconsistent over time and among evaluators, and for lacking reproducibility. A turfgrass trial located at New Mexico State University was used to collect visual turf quality ratings of five different evaluators with varying years of experience ranging from 1 year to 27 years. The trial included 25 different cool-season grasses to which different levels of drought stress were imposed by irrigating at 5 different evapotranspiration replacement levels. Variance components were estimated by means of a random linear model and subsequently used to compute reliabilities for comparing quality ratings. Both evaluators and replications were treated as factors contributing to the error in the comparisons. Such reliabilities are commonly used in the social sciences to assess the dependability or consistency of subjective measurements along a rating scale. Reliability coefficients between 0.7 and 0.9 are usually expected for scientific experiments.

Our results indicate that a reliability of 0.7 could be achieved with a single evaluator and 4 replications per treatment. However, in order to attain a reliability of 0.85 or greater the plots should be replicated between 4 and 6 times and assessed by 3 evaluators. If the ratings show a high variability between evaluators and the group of assessors include barely trained scientists, a greater number of evaluators may be required to attain an adequate reliability.