Segmentation and Classification of images depending on the

presence of the AMD Illness using a Hybrid Deep Learning

approach
1. Background study

AMD stands for age-related macular degeneration and is a disorder of the eye that primarily
affects people over 60. In addition, the prevalence of AMD is growing due to the increasing
number of older adults in the community. As a result, identification at an earlier stage is
essential to avoid visual loss in senior people. Putting up a thorough eye examination for
geriatric patients to identify AMD is a long and challenging task. In response to this need,
this study designed a Deep Convolutional Neural Network (DCNN) model with fourteen
layers, capable of diagnosing AMD in its earliest stages in an accurate and automated
manner. Results from testing the model's efficacy with ten-fold cross-validation and a
blindfold approach indicated an efficiency of 95.45% and 91.17%, respectively. Early
diagnosis of age-related macular degeneration can be challenging, but this new model may
help in a brief eye exam. The developed approach can potentially serve as a driving force
behind large-scale screening programs. However, it is necessary to examine a population that
is both bigger and more diverse [1,2].

2. Problem formulation

Age-related macular degeneration (AMD) is a common eye disease that affects older
individuals, causing gradual loss of vision in the central area of the retina. Image
segmentation and Classification of retinal images can aid in the early detection and diagnosis
of AMD.

Salient edge and spatial feature extraction are two critical techniques used in image
processing for feature extraction. Salient edges are those edges that contain essential
information about the image content, while spatial features refer to features that describe the
spatial distribution of pixels in an image. In AMD image segmentation and Classification,
salient edge and spatial feature extraction can be used to identify and isolate regions of
interest in the retinal image, such as the macula, drusen, and other features associated with
AMD. In addition to assisting with the diagnosis and monitoring of AMD, it can also enhance
the precision with which images are segmented and classified.
Long Short-Term Memory (LSTM) and Recurrent Convolutional Neural Networks (RCNN)
are popular deep learning techniques for image classification and segmentation. An RCNN is
a deep learning design that pools the strengths of several different types of convolutional
neural networks (CNNs) with region proposal algorithms, allowing it to identify objects in an
image and classify them. LSTM, on the other hand, is a type of neural network that is
particularly good at modeling sequential data, making it useful for analyzing time-series data.
In AMD image segmentation and Classification, RCNN can identify regions of interest in the
retinal image, such as the macula and drusen. In contrast, LSTM can be used to analyze
changes in these regions over time, allowing for the monitoring of disease progression.

Overall, combining salient edge and spatial feature extraction with RCNN and LSTM can
provide a powerful tool for the segmentation and Classification of AMD retinal images,
aiding in the early detection and diagnosis of this debilitating disease.

3. Research Methodology

This section discusses the techniques used in this study's suggested approach for image
segmentation and Classification of AMD disease.

3.1 Feature Set Optimization using Particle Swarm Optimization (PSO)

for Age-Related Macular Degeneration
Particle swarm optimization (PSO) is a metaheuristic optimization technique to optimize
features extracted from age-related macular degeneration (AMD) images. Here is an
example of how this technique could be applied:

o Feature extraction: A set of features can be extracted from the AMD images using
techniques such as salient edge detection, Gabor filters, and spatial feature extraction via
LSTM. The components can include features that capture the AMD lesions' texture,
shape, and color.
o Feature selection: Reduce the dimensionality of the feature space; a feature selection
algorithm can be used to select a subset of the most relevant features can be done using
techniques such as correlation-based feature selection or recursive feature elimination.
 o PSO optimization: PSO can optimize the subset of selected features by minimizing a
cost function that measures the performance of a machine learning model trained on the
selected features. The PSO algorithm iteratively updates a set of candidate feature
subsets by moving particles in the search space, where each particle represents a
candidate feature subset. The cost function is evaluated for each candidate feature
subset, and the particles are updated based on their current location and the best area
found so far.
o Evaluation: The performance of the PSO-optimized feature set can be evaluated on a
held-out validation set of AMD images. The machine learning model can be trained and
tested using the PSO-optimized feature set and compared to the performance of a model
trained on the original feature set.
o Refinement: The PSO algorithm can be run multiple times with different settings and
parameters to find the optimal feature subset. The results can be compared, and the best
feature subset can be selected.

PSO can be a powerful technique for optimizing the feature set used in machine learning
models for AMD diagnosis and monitoring. Selecting the most important features can assist
enhance the model's performance by lowering the dimensionality of the feature space.

3.2 Feature Extraction using SALIENT EDGE Gabor Filter for Age-
Related Macular Degeneration
After applying salient edge detection with Gabor filters to an age-related macular
degeneration (AMD) image, we can extract features from the resulting edge map to
diagnose and monitor the disease. Here are a few examples of features that can be extracted:

o Edge density: The density of edges in the image can be computed by counting the
number of edge pixels and dividing by the total number of pixels in the image. A higher
edge density may indicate the presence of abnormal blood vessels or other features
associated with AMD.
o Edge orientation: The orientation of edges in the image can be analyzed by computing
the dominant orientation of the Gabor filters that produced the most robust responses,
which can help to identify the direction of blood vessel growth or other features
associated with AMD.
 o Edge width: The width of edges in the image can be measured by computing the
distance between the inner and outer edges of the detected edges and can provide
information about the thickness of blood vessels or other features associated with AMD.
o Fractal dimension: The fractal dimension of the edge map can be computed to measure
its complexity. A higher fractal dimension may indicate the presence of irregular blood
vessels or other features associated with AMD.
o Histogram features: Histograms of edge intensities or orientations can be computed and
used as features for classification or segmentation tasks. For example, the distribution of
edge orientations may be different in images with AMD compared to healthy images.

These features can be used alone or in combination to develop machine learning models for
AMD diagnosis or monitoring. For example, a model could be trained to predict the severity
of AMD based on the edge density and orientation features extracted from a patient's fundus
image.

3.3 Feature Extraction using SPATIAL via LSTM for Age-Related

Macular Degeneration
Spatial feature extraction via LSTM is a technique that can be used to capture spatial
dependencies and patterns in sequential data, such as images. Here is an example of how
this technique could be used for feature extraction in age-related macular degeneration
(AMD) images:

o Data preparation: The AMD images can be pre-processed to normalize the size and
intensity values across all images in the dataset. The images can be divided into patches
of equal size, and each patch can be considered a sequence of pixels.
o LSTM model: An LSTM model can be designed to take the sequence of pixel values in
each patch as input and output a feature vector that captures the spatial information in
the patch. It is possible for the LSTM model to include several layers of LSTM cells,
with the last layer being a fully linked one that generates the feature vector.
o Feature extraction: Once the LSTM model is trained on the AMD image dataset, it can
extract features from new images. Each image can be divided into patches, and the
LSTM model can be applied to each patch to extract its corresponding feature vector.
 o Feature analysis: The extracted feature vectors can be analyzed to identify patterns or
spatial dependencies associated with AMD. For example, principal component analysis
(PCA) can identify essential features, or clustering techniques can be used to group
similar patches.
o Machine Learning: This feature extraction process may then be sent into a machine
learning method like a random forest or support vector machine, to classify AMD
images into different categories or predict the disease's severity. The machine learning
algorithm can be trained and tested on a labeled dataset of AMD images.

Overall, spatial feature extraction via LSTM is a powerful technique for capturing spatial
information in sequential data, such as images, and can be used for feature extraction in
AMD images to aid in diagnosing and monitoring the disease.

3.4 RCNN Model via Transfer Learning for Age-Related Macular

Degeneration
RCNN is a deep learning model with great success in object detection and localization
tasks. An approach in deep learning known as "transfer learning" includes assimilating the
knowledge and data from one model into another that is being trained for a different but
related purpose. Here is an example of how the RCNN model via transfer learning could be
used for age-related macular degeneration (AMD) detection and localization:
o Data Preparation: Collect a large dataset of AMD images with different stages of
severity. Label each image with the corresponding stage of AMD.
o Pre-processing: Resize all the images to a uniform size, and normalize the pixel values
to a range of 0-1.
o Splitting the Dataset: Divide the dataset into validation, training, and testing sets. The
training set should be the largest, while the validation and testing sets should be smaller.
o Extract Regions of Interest (ROIs): Use an object proposal algorithm like Selective
Search to extract regions of interest from the images.
o Fine-Tune a Pre-Trained CNN: Fine-tune a pre-trained CNN like VGG16 on the
training set to classify the ROIs into AMD stages.
o Train a Classifier: Train a classifier like SVM or Random Forest on the features
extracted from the fine-tuned CNN to classify the AMD stage of each ROI.
 o Evaluate Performance: Assess the Model's performance on the validation set, and make
adjustments to the Model as needed to enhance its accuracy.
o Test the Model: Test the final model on the testing set to evaluate its performance.
o Deploy the Model: Deploy the trained model to classify new AMD images into their
corresponding stages.
The RCNN model via transfer learning can be a robust AMD detection and localization
technique. It can help to leverage the power of pre-trained models and transfer knowledge
from related tasks to improve the model's performance on the AMD detection task.

3.5 Sequential Model for Age-Related Macular Degeneration

A sequential model is a neural network architecture commonly used in deep learning for
sequential data analysis, such as time series or sequence classification tasks. Here is an
example of how a sequential model could be used for age-related macular degeneration
(AMD) diagnosis:

o Data preparation: The AMD images can be pre-processed to normalize the size and
intensity values across all images in the dataset. The images can be labeled as either
healthy or diseased.
o Model architecture: A sequential model can be designed for AMD diagnosis using
convolutional, pooling, and dense layers. The input to the model can be the pre-
processed AMD images, and the output can be a binary classification of healthy or
diseased.
o Model training: The Model can be trained on a training set of AMD images with
corresponding labels. Stochastic gradient descent and Backpropagation are two training
methods that could be used to reduce the loss function and boost the model's accuracy.
o Hyperparameter tuning: For instance, the model's hyperparameters could include the
number of layers and the number of neurons in each layer, the learning rate, and the
activation functions, can be optimized using techniques such as grid search or Bayesian
optimization.
o Model evaluation: The Model's performance can be evaluated on a held-out validation
set of AMD images. The efficacy of a model could be evaluated using a number of
different standards, including its precision, accuracy, recall, and F1 score.
 o Refinement: The Model can be refined by adjusting the hyperparameters, such as the
number of layers, learning rate, and regularization, and by training the model on larger
datasets or adding more layers to the network.

Overall, a sequential model can be a powerful technique for AMD diagnosis. It can leverage
the power of deep learning to learn complex patterns in AMD images and provide an
accurate and reliable diagnosis.

3.6 VGG-16 Model for Age-Related Macular Degeneration for Image

Classification

VGG16 is a CNN architecture widely utilized in image classification tasks. Loss of central
vision is the end result of age-related macular degeneration (AMD), a degenerative eye
condition. CNNs have been successfully used to diagnose AMD by analyzing retinal
images. Here's how you could use VGG16 for AMD image classification:

1. Data collection: Collect a large dataset of retinal images of patients with AMD and
healthy individuals.
2. Data pre-processing: Data should be preprocessed to get rid of artifacts and noise and
to have the pixel values normalized.
3. Data augmentation: Perform data augmentation techniques such as rotation, scaling,
and flipping to increase the size of the dataset.
4. Model training: Train the VGG16 model on the pre-processed and augmented dataset.
You could use transfer learning by initializing the model weights with pre-trained
weights on a large dataset such as ImageNet.
5. Model evaluation: Evaluate the model's performance on a separate validation dataset.
You could use metrics such as accuracy, precision, and recall to measure the
performance.
6. Model Optimization: Increase the model's efficiency by fine-tuning its hyperparameters.
In this case, techniques such as Bayesian optimization and grid search are useful.
7. Deployment: Deploy the model to classify retinal images of patients with AMD in a
clinical setting. The model could be integrated with a user-friendly interface to enable
doctors to upload and classify retinal images of their patients.
 Overall, VGG16 is a robust architecture that could be used to classify AMD retinal images.
However, it is essential to note that the model's success depends on the quality of the data
and the optimization of the hyperparameters.

3.1 RCNN Model for Age-Related Macular Degeneration for Image

Segmentation
RCNN is a deep learning algorithm commonly used for object detection and image
segmentation. Here's an algorithmic overview of using RCNN for Age-Related Macular
Degeneration (AMD) image segmentation:
1. Data Preparation: Collect a dataset of retinal images with annotations of AMD
regions. Separate the data into a set to be used for training and one to be used for
testing.
2. Feature Extraction: Extract features from the images using a pre-trained CNN
model such as VGG, ResNet, or Inception.
3. Region Proposal: Select regions of interest (ROIs) within the image using a
region proposal algorithm such as Selective Search or Edge Boxes.
4. Region Classification: Apply a classifier (e.g., SVM) to each proposed region to
determine whether it contains AMD.
5. Refinement: Use a bounding box regression algorithm to refine the location of
the ROIs.
6. Segmentation: Use a segmentation algorithm such as Mask R-CNN to generate a
mask for the AMD regions within each ROI.
7. Evaluation: Evaluate the performance of the model using metrics such as
Intersection over Union (IoU), Precision, and Recall.
8. Fine-tuning: Fine-tune the model by adjusting hyperparameters and/or adding
more training data.
9. Inference: Use the trained model to segment AMD regions in new retinal images.
It's worth noting that there are many variations of the RCNN algorithm, and the specifics
of the implementation may vary depending on the requirements of the AMD
segmentation task.
3.7 DataSet Description: ADAM
ADAM contains 400 JPEG retinal fundus photos with 8 bits per color channel. Canon CR-2
cameras with 1444×1444 pixels captured the fundus photos (400 images). The fundus
cameras' operational protocols were followed. Photographs were centered on the optic disc,
macula, or half of the two. The exams were done on 53.19 ± 15.59-year-old individuals (47%
female). Manually picking high-quality left and right-eye photos created the final dataset.
Table 1 shows the features of the 400-image online dataset. Pixel-wise segmentation masks
of the optic disc and lesions and binary AMD/not-AMD labels. ADAM has the most labels
for AMD analysis, to our knowledge. Clinical diagnostic data, including fundus pictures and
optical coherence tomography, were used to determine AMD categorization (OCT). The
proposed collection includes samples with early, intermediate, or severe AMD and those
without AMD but with other retinal diseases.

Early/intermediate/advanced-dry/advanced-wet AMD samples in this dataset are 18/7/3/61.

(see Table 1). The proposed dataset has more AMD samples, notably wet AMD, than the
actual world owing to retrospectively acquiring scans from a hospital that treats critical
ocular illness. While training a model, patients boosted the size of the AMD class to help
even out the distribution of classes. Samples taken from people with and without AMD have
the potential to detect conditions such myopia, diabetic retinopathy, and glaucoma. At first,
seven separate ophthalmologists annotated the whole optic disc manually, pixel by pixel. The
five lesions –exudate, drusen, scar, hemorrhage, and a different category for 'other lesions' are
carefully inspected and outlined as the targets in all pictures. These ophthalmologists
annotated the drusen, exudate, hemorrhage, scar, and an additional category for 'other
lesions.' To prevent bias, they had no patient information or illness prevalence data. After
creating a single disc or lesion segmentation mask for each picture based on the vote results
of all seven annotations, a senior expert was brought in to assess the image's quality.

When annotation mistakes were found, this extra expert examined each of the seven
segmentations, eliminated incorrect ones, and restarted the majority voting procedure. The
original coordinates of the fovea were gathered by seven separate specialists, with the final
coordinate being determined by averaging the seven annotations and a senior specialist doing
a quality check. Errors in final coordinates were corrected using the segmentation mask
approach. Inter-rater agreement was evaluated in dataset creation.
Table 1: ADAM dataset labels and annotations.
 Intermediate

Late-dry

Late-wet
Early
Optic Drus Exudat Hemor Other
Images Fovea AMD Scar
Disc en e rhage Lesions

with 270 396 89 35 34 17 13 4

2
8
7

5
0
400
With
130 4 311 26 4 2 0 13
out

4. Proposed methodology

This section proposes an approach for segmenting and classifying AMD disease using hybrid
deep learning techniques. In this approach, RC.N.N. architecture is used to improve the
overall performance of the suggested model. The methodology depicted in Figure 3 provides
a detailed overview of the work of the proposed approach.

Figure 3. Proposed methodology

Algorithm: Drusen detection using Neural Network.

Start

Numpy, cv2, SkImage, random, TensorFlow,

IMPORT-MODULE : → Keras, OpenCV, VGG16, Dense, Flatten, Input,
SVC, hog, pyramid_gaussian

Phase I – DataSet Collection from Steps 1 – 3.

Step 1: Multiple hospitals and other medical facilities contribute to the data pool.
Step 2: The Retinal Picture is mined for information and used as input.
Step 3: Images are collected from various institutions, so an image consists of noise. Image deNoising
will be used as a data pre-processing technique in conjunction with CNN's Autoencoder.

Phase II – Image deNoising on collected DataSet from Steps 4 – 7.

Step 4: Keras will be used as module creation where Dense, Conv2D, Maxpooling2D, and
UpSampling2D are used.
Step 5: Convert all the images in numeric distribution for data normalization, where each image should
represent from 0 – 1 in the matrix form.
Step 6: Sequential model will be created with three layers:
o encoder network with Conv2D(32, 3, activation function relu, padding will be same for
all cases, where input_shape will be (28, 28, 1)), MaxPooling2D(2), Conv2D(16, 3,
activation function relu, MaxPooling2D(2, padding='same'),
o decoder network layer with Conv2D(16, 3, activation function relu), Conv2D(32, 3,
activation function relu), UpSampling2D(2), and
o output networklayer Conv2D(1, 3, activation function sigmoid padding)

Step 7: Now compile the model with adamoptimizer with loss of binary_crossentropy.

Phase III – Image Augmentation using GAN in Step 8.

Step 8: Image Augmentation with image, NOISESIGMA ← 0, RESIZESCALE, ROTATIONANGLE ← 0,
FLIPHORIZONTAL ← False, FLIPVERTICAL ← False
ADD GAUSSIAN NOISE
if noise_sigma > 0 do
noise ← normalize the random factor with Increment IMAGE with
NOISE
NOISESIGMA as well as IMAGESHAPE

RESIZE THE IMAGE

if resize_scale != 1.0 do
image ← resize the image with cv2 module where user-defined
RESIZESCALE in x and y axes, INTERCUBIC interpolation.

ROTATE THE IMAGE

if rotation_angle != 0 do
uniform angle with RANDOMIZE STANDARIZATION from [-90 →
90]
skimage module will transform rotatation with RESIZED_IMAGE &
ANGLE)
else
ROTATEIMAGE with it's RESIZEDIMAGE version

FLIP IMAGE HORIZONTALLY & VERTICALLY

if flip_horizontal do
image ← cv2.flip(image, 1)
if flip_vertical do
image ← cv2.flip(image, 0)

Phase IV – Feature Selection / Set Optimization using PSO from Steps 9 – 10.
Step 9: OPTIMIZE FEATURES PSO with FEATURESNAME, NUMofITERATIONS, NUMofPARTICLES, c1,
c2, TARGET,
Initialize PARTICLES, their VELOCITIES & best PARTICLE and its fitness score

particles ←

INPUT: ⟦ random. sample(features , len(features))for¿ range (Num of Particles)⟧

→ velocities ← ⟦ ⟦ 0 ⟧∗len(features) for¿ range (Num of Particles) ⟧

best_particle ← particles ⟦ 0 ⟧
best_fitness ← get_fitness_score(best_particle, target)

for j → NUMofITERATIONS do

for i → NUMofPARTICLES do
particle ← particles[i] ; velocity ← velocities[i]
fitness ← get_fitness_score(particle, target)
r1, r2 ← random.random()

VELOCITIE ← [v + c{1→2} * r{1→2} * (best_particle[j] - particle[j]) for j, v in

 ENUMERATE(VELOCITY)]
PARTICLE ← [p + v for p, v in ZIP(PARTICLE, VELOCITY)]

if fitness > best_fitness do

best_particle = particle
best_fitness = fitness

Step 10: Calculate FITNESS SCORE with PARTICLE & TARGET

fitness ← 0
for p,t in zip(particle, target) do
fitness +← (p – t)2
fitness ← fitness0.5

Phase VI – Feature Extraction using Gabor Filter from DataSet from Steps 11 – 20.
Step 11: Use cv2.imshow with RGB or RGBA pictures (e.g., not Black and White images)
Step 12: Although matplotlib most likely uses RGB order, OpenCV uses BGR order.
Step 13: Display the image using a cmap of grey and bicubic interpolation while concealing the tick
values along the X and Y axes.
Step 14: A uniform distribution of theta angles in pi radians/180o.
Step 15: Gabor filters have a directional component (Theta). That requires many GaborFilters to cover
edges in all directions. Since this was the case, make a set of filters whose angles sum180o,
which breezes with OpenCV's in-built getGaborKernel feature.
Step 16: Filters may be applied to our image using this generic function.
Step 17: Make a NumPy array of the same size as the image we're working with and recursively apply
Gabor Filter to each photo, beginning with a blank one.
Step 18: Iteratively take the maximum value across all filters by "superimposing" the most significant
deals at the end of each cycle. The completed picture is returned.
Step 19: To keep the same depth as the source picture. Iterate through the GaborFilter's kernels. Then,
take the more extensive Numpy maximum comparisons between our filter and cumulative
image.
Step 20: The Gabor function in the spatial domain is represented as:
g ( x , y )=s ( x , y ) × wr (x , y ).
where
s ( x , y ) → represents the complex sine wave,
w r (x , y) → represents the 2D Gaussian.

Phase V.II – Feature Extraction using SPATIAL via LSTM in Step 21.
Step 21: TRANSFER_FEATURE_EXTRACTION using IMAGES, MODELWEIGHTS
LOAD pre-trained WEIGHTS into UNET.
FREEZE UNET: UNET.trainable ← FALSE

UNET model image features: ← UNet.predict (images)

TIME-DIMENSION extracted features: ← np.expand_dims(features, axis=0)

Define an LSTM model : features.shape⟦ 1 ⟧ : features.shape⟦ 3 ⟧

LSTM = tf.keras.Sequential()
LSTM.add(tf.keras.layers.LSTM(256, input_shape=input_shape, return_sequences=True))
LSTM.add(tf.keras.layers.LSTM(256))
LSTM.add(tf.keras.layers.Dense(128, activation='relu'))
LSTM.add(tf.keras.layers.Dense(64, activation='relu'))
LSTM.add(tf.keras.layers.Dense(32, activation='relu'))
LSTM.add(tf.keras.layers.Dense(16, activation='relu'))
LSTM.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

LSTM-based picture feature extraction: ← LSTM.predict (features)

Phase VI – CONSTRUCT & TRAIN OPTIMAL RCNN model via TRANSFER LEARNING from Steps 22 – 30.
Step 22: DataSet will be divided into train and test segments with a test size of 30%.
Step 23: Now the model will be trained with a noisy image with an actual picture with an epoch of 100.
Step 24: Now the image is deNoised for further processing.

Step 25: R-CNN extracts spatial characteristics from successive pictures. LSTM recurrent sequence
learning models recognize temporal properties in picture sequences using the outputs.
Step 26: Finally, a fully connected layer predicts the whole input sequence categorization using the
combined characteristics.
Step 27: Compute loss function with accuracy and compare with threshold value.

Step 28: Adversarial training is a framework for employing two or more networks with different loss
functions as a search algorithm and for adversarial attacks. In an adversarial attack, a rival
network learns how to add things to pictures that confuse its classification network.
Step 29: Adversarial network may learn to create misclassifications by confining its collection of
augmentations and distortions, producing an effective search algorithm. These augmentations
improve classification model weaknesses.
Step 30: Thus, adversarial training can enhance data search. Adversarial augmentations may not be test-
set instances but can strengthen the learned decision boundary.

Phase VII – CONSTRUCT & TRAIN OPTIMAL RCNN model for Image Classification from Steps 31 – 45.
Step 31: Define image size for pre-processing
Step 32: Load the dataset ← load_data()
Step 33: Split dataset into training, validation, and testing sets
→ train_data, val_data, test_data ← split_data(data)

Step 34: Load pre-trained CNN model : VGG16(include_top← False, weights← 'imagenet',
input_shape← (img_size, img_size, 3))
Step 35: remove the final layer : base_model.output, Flatten()(x)

Step 36: Add new fully connected layer : fc1 ← Dense(1024, activation← 'relu')(x)
Step 37: final classification layer :← Dense(num_classes, activation←'softmax')(fc1)
Step 38: Define new Model ← Model(inputs=base_model.input, outputs=output)

Step 39: Freeze the pre-trained layers

for i → base_model.layers do
 i.trainable = False

Step 40: Compile Model : model.compile(loss← 'categorical_crossentropy', optimizer←'adam',

metrics←['accuracy'])
Step 41: Train Model : model.fit_generator(train_data, epochs←num_epochs, validation_data)

Step 42: Extract features from the training set using fine-tuned CNN
for i, j → train_data do
 train_features.append(base_model.predict(preprocess_image(i)))
 train_labels.append(j)

Step 43: Train SVM classifier on the extracted features

→ svm ← SVC(kernel='linear', probability=True)
→ svm.fit(train_features, train_labels)

Step 44: Evaluate performance on testing set

for i, j → test_data do
 test_features.append(base_model.predict(preprocess_image(i)))
 test_labels.append(j)

Step 45: accuracy ← svm.score(test_features, test_labels)

Phase VIII – CONSTRUCT & TRAIN OPTIMAL RCNN model for Image Segmentation from Steps 46 – .
Data Preparation
Step 46: load dataset and split into training and validation sets :← load_dataset()

Feature Extraction
 Step 47: load pre-trained CNN model :← load_pretrained_cnn()
Step 48: extract features from training set :← cnn_model.extract_features(train_set)
Step 49: extract features from validation set :← cnn_model.extract_features(val_set)

Region Proposal
Step 50: generate for training set :← region_proposal(train_features)
Step 51: generate interest for validation set :← region_proposal(val_features)

Region Classification
Step 52: train SVM classifier on proposed regions :← train_svm(train_rois)
Step 53: predict labels for training set regions :← svm_model.predict(train_rois)
Step 54: predict labels for validation set regions :← svm_model.predict(val_rois)

Refinement
Step 55: refine location of ROI’s using bounding box regression :← refine_rois(train_rois)
Step 56: val_rois :← refine_rois(val_rois)

Segmentation
Step 57: masks for AMD regions in training set ROI’s :← segment_amd(train_rois)
Step 58: masks for AMD regions in validation set ROI’s :← segment_amd(val_rois)
Model Evaluation
Step 59: training set :← evaluate(train_masks, train_set)
Step 60: validation set :← evaluate(val_masks, val_set)

Fine-tuning
Step 61: fine-tune model by adjusting hyperparameters training data :← fine_tune_model()

Inference
Step 62: load new retinal image :← load_new_image()
Step 63: extract features from the new image :← cnn_model.extract_features(new_image)
Step 64: generate proposed regions of interest for new image :← region_proposal(new_features)
Step 65: refine the location of the ROIs :← refine_rois(new_rois)
Step 66: generate masks for AMD regions in new image ROIs :← segment_amd(new_rois)

Phase IX – Performance Evaluation from Steps 32 – 34.

Step 67: Provide classification report.
Step 68: Calculate the best Accuracy, Precision, F1 – Score, and Recall.
Step 69: CNN and RNN together on retinal images resulted in the Extraction of both spatial and temporal
characteristics, which led to a considerable improvement in the diagnosis of AMD illnesses.

End

References
[1] Tan, Jen Hong, Sulatha V. Bhandary, Sobha Sivaprasad, Yuki Hagiwara, Akanksha
Bagchi, U. Raghavendra, A. Krishna Rao et al. "Age-related macular degeneration
detection using deep convolutional neural network." Future Generation Computer
Systems 87 (2018): 127-135.
[2] Tan, Jen Hong, Hamido Fujita, Sobha Sivaprasad, Sulatha V. Bhandary, A. Krishna Rao,
Kuang Chua Chua, and U. Rajendra Acharya. "Automated segmentation of exudates,
haemorrhages, microaneurysms using single convolutional neural network." Information
sciences 420 (2017): 66-76.