The aim of the present work is to propose methods and tools for classifying sweet pepper into groups according to their degree of maturity based on colour and spectral characteristics extracted from colour images on the surface of the vegetables. The investigated pepper is two varieties of sweet — red Banji and yellow Liri. Five groups were formed, depending on the degree of maturity, and 16 colour and 11 spectral indices were calculated for each of the groups. By successively using the ReliefF and PLSR methods, a selection of informative features and subsequent reduction of the vector formed by them was carried out, thereby aiming to increase the predictive results and minimize the time for data processing. The obtained classification errors between the individual stages of ripening vary according to the type of pepper and depending on which of the two types of maturity the fruits are in — technical or biological. For red sweet pepper, the separation inaccuracy obtained using a discriminant classifier with a quadratic separation function is in the range of 8–19%, while for yellow it is from 5 to 23%. The results obtained in the present work for the classification of pepper into groups according to their degree of maturity would support decision-making in selective harvesting and overall more accurate and efficient management of the harvesting process from the point of view of precision agriculture. The work will continue with studies related to the prediction of various compounds indicating changes in the colour of peppers, including chlorophylls, carotenes and xanthophylls. In this way, it is possible to increase the accuracy in determining the degree of ripeness, since in pepper the colour does not always follow the same pattern of change from green to yellow to orange to red.