Herbarium records provide a broad spatial and temporal range with which to investigate plant responses to environmental change. Research on plant phenology and its sensitivity to climate has advanced with the increasing availability of digitized herbarium specimens, but limitations of specimen-derived data can undermine the inferences derived from such research. One issue that has received little attention is collection site uncertainty (i.e. error distance), a measure of confidence in the location from which a specimen was collected. We conducted comparative analyses of phenoclimatic models to determine whether spatial deviations of 2, 5, 15 or 25 km between recorded and simulated collection sites, as well as the error distance reported in digitized records, affect estimates of the phenological sensitivity of flowering time to annual temperature and precipitation in a widespread annual California wildflower. In this approach, we considered both spatial and interannual variation in climatic conditions. Simulated site displacements led to increasingly weak estimates of phenological sensitivity to temperature and precipitation anomalies with increasing distances. However, we found no significant effect of reported error distance magnitude on estimates of phenological sensitivity to climate normals or anomalies. These findings suggest that the spatial uncertainty of collection sites among specimens of widely collected plant species may not adversely affect estimates of phenological sensitivity to climate, even though real discrepancies and georeferencing inaccuracy can negatively impact such estimates. Collection site uncertainty merits further attention as a potential source of noise in herbarium data, especially for research on how plant traits respond to spatial and interannual climatic variation.