Highway agencies commonly use automated pavement data collection techniques to collect pavement surface distress data at the network level. Although an immense amount of data is collected at the network level, agencies realize that there is a lack of understanding of the quality of the data collected. Traditionally, either the overall pavement condition rating or individual distress ratings are used to evaluate the quality of the condition data. However, each measure has its own pros and cons, rendering the use of a single measure inadequate. This paper presents a set of performance measures that highway agencies can use to quantify the quality of the pavement condition data collected and processed. The set of measures consists of (a) the pavement condition rating and hypothesis testing for differences, (b) the percentage cumulative difference in the pavement condition rating over its entire range, and (c) kappa statistics for individual distresses. This set of performance measures can be used to assess the effectiveness of an automated method for the collection of pavement condition data and the effect of sampling on the pavement condition ratings obtained by automated techniques. The effectiveness of an automated technique is assessed by comparison with the findings of benchmark manual visual surveys. The performance measures offer a complete assessment of the effect of sampling on the overall pavement condition rating, its variation over the entire range, and the identification of individual surface distresses.