Preserving the quality of fish is a challenging task. Several different cooling methods and materials are used during their storage, transportation purpose. These are responsible factors that decide the freshness of a post harvested fish. In this proposed algorithm, a computer vision-based technique is developed to predict the freshness level of fish from its image. Eyes of the fish are considered as the region of interest, as a good correlation has been observed between the colour of the eye and different duration of storage day. It is segmented from the image of a fish sample and then a strategic framework is used for extraction of the discriminatory features. These extracted features show a degradation pattern which acts as an indicative parameter to determine the level of freshness of sample of fish. The proposed method provides a recognition accuracy of 96.67%. The experimental results indicate that this is an efficient and non-destructive methodology for detecting the fish freshness. The high accuracy of freshness detection and low computation time makes this non-destructive methodology efficient for real-world usage in the fish industry and market.