This article aims to determine the process underlying the subjective impression about the fidelity of reproduced object colors. To this end, we present the concept of the naturalness constraint and a framework for specification of naturalness judgments. We consider several research questions that are essential for this framework and discuss plausible answers supported by experiments. In general, naturalness assessment of reproduced object colors can be (1) defined as similarity to prototypical object colors, and (2) characterized by a probability density function (e.g., Gaussian). Experiments show that (3) there is a considerable amount of consistency in naturalness judgments of locally and globally processed images (although observers are slightly more tolerant of global image processing), and (4) naturalness judgments vary for different object categories; e.g., subjects are more consistent in naturalness judgments of skin, grass, and sky reproductions than shirt reproduction. We suggest that (5) naturalness of a whole picture is determined by the naturalness of the most critical object in that picture. Finally, we introduce a naturalness index predicting perceived naturalness of color reproduction of real-life scenes.