Automated lesion detection in cotton leaf visuals using deep learning

Preprocessing

During the preliminary stage of the proposed framework, collection of an extensive dataset comprising images of cotton crops are performed. Images are obtained from Kaggle datasets. Prioritizing the inclusion of a wide range of healthy and unhealthy cotton leaves plays significant role. For the data collected, 80% of it constituted training set while the other 20% was used for validation. This ensures a good amount of data for training the model as well as validation of its performance. In addition to the competitor (Rastogi et al., 2024), another thing that also has a big effect on how well disease detection models develop is dataset size and diversity. In this stage, several steps are executed. A Gaussian filter is widely used for image smoothness by applying the average on pixels when contrasting with its neighbour, and then the Gaussian function is used for weight calculation (Kaur et al., 2024; Paul Joshua et al., 2024). The Gaussian filter can be expressed by following equation:

(4) $$G(x,y)=\frac{1}{2\pi {\sigma }^2}\exp \left(-\frac{x^2+y^2}{2{\sigma }^2}\right)$$where:

• $G(x,y)$ is the Gaussian function.

• $x$ and $y$ are the distances from the origin in the horizontal and vertical axes, respectively.

• $\sigma$ is the standard deviation of the Gaussian distribution.

Gaussian filter helps in making boundaries more visible also enhance the lesion detection on cotton leaf by improving the variability among the images and smooth the images by removing the high-frequency noise. For detection of edges in image Laplacian filter is applied, when rapid intensity change is observed in specific region Laplacian filter highlights that region (Ramu et al., 2024). The Laplacian filter when applied for cotton disease detection it filters out the uninterested region and make the interested region of image more prominent that create ease for deep learning model for lesion detection in cotton crop images (Lv et al., 2024; Yamada, 2024). The Laplacian filter can be expressed by following equation:

(5) $$\mathrm{\Delta }f(x,y)=\frac{{\mathrm{\partial }}^{2}f}{\mathrm{\partial }{x}^2}+\frac{{\mathrm{\partial }}^{2}f}{\mathrm{\partial }{y}^2}$$

An example of a popular kernel for a 3 × 3 neighborhood Laplacian filter is:

(6) $$\left[\begin{array}{ccc}0& 1& 0\\ 1& -4& 1\\ 0& 1& 0\end{array}\right]$$

A pre-trained generative adversarial network (GAN) was used to produce synthetic images, generated from the dataset for minority classes. The GANs was trained specifically for the underrepresented classes to fix this issue of class imbalance in our real dataset. Figure 4 presents GANs-generated images. In GANs, two networks compete with each other and tries to discriminate between real and synthetic image. GANs play a pivotal role in synthesizing images that not only resemble real images but also increase the dataset size, robust model for lesion detection and improve the training process (Gangadhar et al., 2024; Zekrifa et al., 2024). GANs function can be expressed as following:

(7) $$\underset{G}{min}\underset{D}{max}V(D,G)=E_{x\sim p\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{a}(x)}[\log D(x)]+E_{z\sim p_z(z)}[\log (1-D(G(z)))]$$where:

• $D(x)$ represents the probability that the discriminator correctly identifies a real image, rather than the absolute probability that $x$ is real.

• $z$ is a random noise vector.

• $G(z)$ is the synthetic image generated from $z$.

Feature extraction

The process of feature extraction is one of the main elements, which “translates” raw data to more structured and interpretable form of the feature representation which can be comprehended by deep neural network models. This phase aid in identifying relevant features and pattern for the identification of cotton leaf lesion detection. Three different architectures: VGG16 is used for relevant pattern extraction. VGG16 plays significant role for the extraction of features due their simplicity, depth, hierarchical feature extraction, spatial dimensionality reduction and pre-trained weight enables transfer learning (Mohmmad et al., 2024). The convolution functioning for VGG16 can be expressed using following equation:

(8) $$F_{l+1}(x,y)=\sigma \left(\sum _{i=-k}^k\sum _{j=-k}^j{w}_{ij}{F}_l(x+i,y+j)+b\right)$$where:

• $F_l$ is the feature map at layer $l$,

• $w_{ij}$ are the convolutional weights,

• $b$ is the bias,

• $\sigma$ is the ReLU activation function,

• $k$ is the kernel size.

The Inception V3 module used parallel convolutions for the extraction of features from cotton leaf in order to identify lesion. The Inception V3 module increase computational efficiency, and ensure the reduction of over-fitting due it hierarchical nature of feature extraction and transfer learning capabilities (Meena et al., 2023). The feature extraction process in the Inception V3 module can be represented by following equation:

(9) $$O=[f_{1\times 1}(I),f_{3\times 3\mathrm{\_}reduce}(I),f_{3\times 3}(I),f_{5\times 5\mathrm{\_}reduce}(I),f_{5\times 5}(I),p_{pool}(I)]$$where:

• I is the input feature map,

• $f_{1\times 1}$, $f_{3\times 3\mathrm{\_}reduce}$, $f_{3\times 3}$, $f_{5\times 5\mathrm{\_}reduce}$, and $f_{5\times 5}$ are convolution operations with respective kernel sizes,

• $p_{pool}$ is a pooling operation.

ResNet50 freeze top layers and popular for feature extraction due its residual connections, transfer learning capabilities and generalization ability. When compared with traditional feature extraction techniques, ResNet50 can capture more abstract and complex patterns. The following equation expressed the extraction of patterns using ResNet50.

(10) $$y=F(x,\{W_i\})+x$$where:

• $x$ is the input feature map,

• $y$ is the output feature map,

• F is the residual mapping to be learned,

• $\{W_i\}$ are the weights of the layers in the residual block.

Feature fusion

In this phase features that are extracted using VGG16, Inception V3 and ResNet50 fused together using channel and spatial attention. For adaptability of context, reduction of model complexity and accurate lesions variations detection channel and spatial attention feature fusion technique is used. In channel attention mechanism weight for each feature is computed. These computed weights are then applied to each channel in order to eliminate irrelevant channel. The mathematical representation of channel attention weight calculation is as follows:

(11) $$M_c(F)=\sigma \left(\mathrm{M}\mathrm{L}\mathrm{P}\left(\mathrm{A}\mathrm{v}\mathrm{g}\mathrm{P}\mathrm{o}\mathrm{o}\mathrm{l}(F)\right)+\mathrm{M}\mathrm{L}\mathrm{P}\left(\mathrm{M}\mathrm{a}\mathrm{x}\mathrm{P}\mathrm{o}\mathrm{o}\mathrm{l}(F)\right)\right)$$where:

• $M_c(F)$ is the channel attention map,

• F is the input feature map,

• $\sigma$ is the sigmoid activation function,

• $\mathrm{M}\mathrm{L}\mathrm{P}$ is a multi-layer perceptron,

• $\mathrm{A}\mathrm{v}\mathrm{g}\mathrm{P}\mathrm{o}\mathrm{o}\mathrm{l}$ and $\mathrm{M}\mathrm{a}\mathrm{x}\mathrm{P}\mathrm{o}\mathrm{o}\mathrm{l}$ are average and max pooling operations, respectively.

Spatial attention mechanism is used for understanding of relationship among spatial space and ignore the irrelevant regions. Weight is computed for each spatial region for the selection of informative regions by using following mathematical expression.

(12) $$M_s(F)=\sigma \left(\mathrm{C}\mathrm{o}\mathrm{n}{\mathrm{v}}_{1\times 1}\left([\mathrm{A}\mathrm{v}\mathrm{g}\mathrm{P}\mathrm{o}\mathrm{o}\mathrm{l}(F);\mathrm{M}\mathrm{a}\mathrm{x}\mathrm{P}\mathrm{o}\mathrm{o}\mathrm{l}(F)]\right)\right)$$where:

• $M_s(F)$ is the spatial attention map,

• F is the input feature map,

• $\sigma$ is the sigmoid activation function,

• $\mathrm{C}\mathrm{o}\mathrm{n}{\mathrm{v}}_{1\times 1}$ is a $1\times 1$ convolution operation,

• $\mathrm{A}\mathrm{v}\mathrm{g}\mathrm{P}\mathrm{o}\mathrm{o}\mathrm{l}$ and $\mathrm{M}\mathrm{a}\mathrm{x}\mathrm{P}\mathrm{o}\mathrm{o}\mathrm{l}$ are average and max pooling operations, respectively.

When features are extracted using spatial and channel attention mechanism then these features are fused together by multiplying each element of input feature map. Thus ensure the better performance and reliable results for combining the relevant features from cotton leaves. Following mathematical equation express the feature fusion mechanism.

(13) $$F_{\mathrm{f}\mathrm{u}\mathrm{s}\mathrm{e}\mathrm{d}}=F\odot M_c(F)\odot M_s(F)$$where:

• $F_{\mathrm{f}\mathrm{u}\mathrm{s}\mathrm{e}\mathrm{d}}$ is the fused feature map,

• F is the input feature map,

• $M_c(F)$ is the channel attention map,

• $M_s(F)$ is the spatial attention map,

• $\odot$ denotes element-wise multiplication.

Classification

During the post-processing step, the conclusions made by the multitude of ensembles are aggregated in order to determine the health status of the cotton leaf. Ensemble learning involved the use of three correct models of VGG16, Inception V3 and ResNet50 which ensured accurate classification of cotton lesion. This disease categorization is determined by attracting the highest average number of votes among all the corresponding models. The application of this self-organised decision making method helps to reduce prejudices and improve the overall robustness of the model even despite possible inaccuracy of the individual forecasts. The increase in accuracy of the decision is due to the fact that, various models can be trained either on different sub-routines of training data or different hyper- parameters. Thus, using multiple models promises to achieve a higher level of accuracy due to the Law of Large Numbers and the Central Limit Theorem, although, at the same time, one must admit that the assumption of independent and identically distributed (i.d.d.). Which most likely does not apply to deep learning’s outputs. Due to the variety of the architectures of the models and training strategies the further forming of their outputs is not independent identically distributed (Zhou, 2012). However, more variations in this case help to decline the phenomena of over-fitting and cover more features. Other reasons for environmental potency of the ensemble are the established stability arising from the different angle visions and the option for avoiding biases of individual models. From a theoretical perspective, the convergence of forecasted probability distributions toward the consensus prediction can be described mathematically using the principles of the Law of Large Numbers and the Central Limit Theorem (Lipnik, Madritsch & Tichy, 2024). Let $X_1,X_2,\dots ,X_N$ denote the outputs of these models, which are treated as random variables, and $S_N$ represent the collective response provided by the models. As the number of models increases, the relative sample mean $S_N$ converges to the true sample mean in almost all samples according to the Law of Large Numbers. This convergence is expressed mathematically as:

(14) $$\underset{n\to \mathrm{\infty }}{lim}P(|\frac{{\displaystyle 1}}{n}\sum _{i=1}^n{X}_i-\mu |<\epsilon )=1$$where:

• $\mu$ represents the true underlying disease class,

• $\epsilon$ is a small positive number representing the desired level of precision.

When the number of models increased this lead towards infinity so the variable $S_n$ merge to a normal distribution. Class with highest probability represents the level of dispersion of the decision made by the ensemble. Due to the central limit theorem which claims distribution of the sample mean converges to a normal distribution as the sample size gets bigger (del Barrio, González Sanz & Loubes, 2024).

(15) $$\sqrt{n}\left(\frac{1}{n}\sum _{i=1}^n{X}_i-\mu \right)\stackrel{d}{\to }\mathcal{N}(0,{\sigma }^2)$$where:

• ${\sigma }^2$ is the variance of the individual model predictions, and

• $\stackrel{d}{\to }$ denotes convergence in distribution.

The code of the methodology is available at https://doi.org/10.5281/zenodo.13324708

Dataset

We developed a custom dataset by combining images from different publicly available datasets on Kaggle. The dataset comprises 1,300 images. Eight classes are identified: Aphid, Army Worm, Blight, Healthy, Leaf Curl, Mildew, Leaf Spot, and Wilt. We applied augmentation to balance the dataset and increase the number of images. After augmentation the number of images is 4,793. Table 3 presents the number of images in each class, both before and after augmentation. Table 4 presents the total parameters, Flops (floating-point operations), and training time for VGG16, Inception V3, ResNet50, and our proposed model.

Performance measure

Each model was evaluated using accuracy, sensitivity, specificity, precision, and F1 score. The model’s accuracy is established by mathematical formulae.

(16) $$\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}=\frac{TP+TN}{(TP+FN)+(FP+TN)}$$

(17) $$\mathrm{S}\mathrm{e}\mathrm{n}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{i}\mathrm{t}\mathrm{y}=\frac{TP}{TP+FN}$$

(18) $$\mathrm{S}\mathrm{p}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{f}\mathrm{i}\mathrm{c}\mathrm{i}\mathrm{t}\mathrm{y}=\frac{TN}{TN+FP}$$

(19) $$\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}=\frac{TP}{TP+FP}$$

(20) $$\mathrm{F}1\mathrm{S}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{e}=\frac{2\times TP}{2\times (TP+FN+FP)}$$

Experimental results

Deep convolution networks (DCNs) come across as autonomous learners that are able to derive features from pre-learned images that have passed through discernment of the images via a reliable protocol. This step enhances the model’s capacity to identify distinct patterns and features present alongside healthy and diseased cotton leaves. DCNs present themselves as unsupervised learners which have the ability to extract features from pre-learned images that went through a process of discerning them based on some trusted method. This step helps the model differentiate between different patterns and properties that are observed in healthy vs. diseased cotton leaves.

Many deep neural network methods have been proposed to identify several leaf diseases. These methods include forming custom deep neural networks (DNNs), but also, reusing existing pre-trained nets. In our study, we utilized the Kaggle dataset to conduct an analysis that involved the application of various deep learning models, namely Inception V3, ResNet50, VGG16, and a Transfer Learning approach. This led to the development of a comprehensive ensemble of pre-trained models through training procedures. Datasets used in this study are publicly available at: https://www.kaggle.com/datasets/dhamur/cotton-plant-disease/data and https://www.kaggle.com/datasets/janmejaybhoi/cotton-disease-dataset/data.

By incorporating a diverse range of models, there is a potential to encompass a broader array of leaf attributes compared to relying solely on a singular paradigm. Inception V3, VGG 16 and ResNet 50 results are combined on the bases of voting in order to extract a wide range of leaf features from a comprehensive integrated model. Table 5 summarizes the comparison FLOPs and training time of each model and proposed ensemble based learning model. ResNet50 has the least FlOPs and training, yet successfully achieved highest accuracy when compared with VGG16 and Inception V3. ResNet 50 is beneficial for real time applications due to its residual blocks where computing power is the constrained (Falaschetti et al., 2021; Anand et al., 2024). Table 6 presents the Precision, Recall, and F-score results of the proposed model. Adding more models to the ensemble improves its performance, as seen by higher accuracy and F1 scores. This strategy exhibits a remarkable capacity for making sweeping generalizations.

The model was trained and tested on deep neural network and afterwards fine-tuning was performed by applying hyper-parameters. The optimizer used for ResNet50 was SGD and the rest of models were trained using Adam. We made this decision based on the specific properties and training dynamics of ResNet50, for which SGD with momentum has been proved to be beneficial in terms model convergence and generalization. Different optimizers were employed, so that in each case the strength of models architecture was utilized for a training process.

Table 7 presents hyper parameters for Inception V3, VGG16, ResNet50 and the proposed model. The model was trained in 1 h, 17 min, and 2 s. Five stages of convolution, a dropout layer, and a max-pooling layer are incorporated into the model of deep neural networks to boost performance. The softmax layer has eight classes for diagnosing leaf disease. Proposed model employed a total of 2,629,639 parameters, resulting in an accuracy rating of 98.5% as given in Table 4. Transfer learning is applied on VGG16, Inception V3, and ResNet50 models. It was the best response to an ILSVRC challenge. VGG16 contains a total of 16 layers. We conducted our experiment by freezing the VGG16 pre-trained model’s outermost layers. The model was executed in 54 min and 3 s on a cotton dataset with 18,678,382 parameters, and it had a 91.33% accuracy. In comparison to a Inception V3, the model had an accuracy that was significantly higher while training on fewer parameters. An already trained machine learning model is called ResNet50. On the Image-net dataset, the model was initially trained. To put it into practice, ResNet50’s top layers were frozen. A total of 95.12% of the model’s predictions were accurate. Whereas Inception V3 model got 89.61% accuracy. In ensemble technique, voting was performed and on the basis of majority voting disease was labeled with a bounding box. Figure 5 represents results achieved from three different architectures for lesion detection.

The ratios of true positive (TP) and true negative (TN) effectively distinguished an equivalent proportion of imagery depicting healthy and aberrant leaves, respectively. To describe instances of healthy leaves that are incorrectly labeled as sick, and vice versa, the terms “false positive” (FP) and “false negative” (FN) are used. To determine the confidence range for the accuracy of classification, it is important to calculate the ratio of correct predictions to the total number of predictions. The model’s efficacy is assessed by considering the F1 score, precision, and recall, along with the traditional metrics for assessment. Table 6 presents the Precision, Recall, and F-score results of the proposed model. Based on the data depicted in Fig. 5, When compared to other models on our cotton dataset, our suggested method achieves a remarkable 98.50% accuracy. Figure 6 depicts a heatmap representing each class.

The above-mentioned result demonstrates the model’s capability to successfully navigate the intricate domain of cotton disease detection, indicating encouraging implications for the field of precision agriculture.