CS419 Project Presentation
Group – 35
Work Distribution (Group-4)
Name Roll Number Contribution
Ashok Nayak 22B0455 Code Ideation and Testing
Ayush Jha 22B0051 Report and Presentation
Abhisek Hazra 22B2424 ODIN testing
ODIN (Out of Distribution Detection for NN)
• Odin is an anomaly detection technique for neural networks. It uses the
perturbation of input data combined with temperature scaling to amplify the
models confidence for in-distribution samples while reducing it for out of
distribution samples
• 𝑂𝑑𝑖𝑛 𝑆𝑐𝑜𝑟𝑒 = max 𝑠𝑜𝑓𝑡𝑚𝑎𝑥 𝑝𝑒𝑟𝑡𝑢𝑟𝑏𝑒𝑑 𝑙𝑜𝑔𝑖𝑡𝑠 − 𝑠𝑜𝑓𝑡𝑚𝑎𝑥 𝑜𝑟𝑖𝑔𝑖𝑛𝑎𝑙 𝑙𝑜𝑔𝑖𝑡𝑠
• Perturbed logits are the logits obtained by adding perturbations to the input data.
• Original logits are the logits obtained from the original data
• Softmax is a function applied to the logits to obtain probabilities.
Pytorch OOD
• PyTorch OOD (Out-of-Distribution) is a collection of techniques and tools developed within
the PyTorch ecosystem to address the challenge of detecting out-of-distribution samples in
machine learning models.
• PyTorch OOD provides various methods and algorithms to help detect out-of-distribution
samples, including:
• ODIN
• MaxSoftmax
• Energy-based Models
Model-1(Standard model without any OOD detection)
• we train a simple convolutional 2neural network (CNN) using CIFAR-10 dataset and evaluate
its performance using the area under the receiver operating characteristic curve (AUC)
• Loss = cross entropy
• Optimizer = SGD
• Running over 20 epochs
• This model is a simple CNN architecture with two convolutional layers followed by three fully
connected layers is defined, along with the forward pass method.
Model-2(ODIN)
• We’re using pre-trained WideResNet model for CIFAR-10 to process a test image through series of
transformations, performs a forward pass through the model to obtain predicted probabilities for each class,
and applies out-of-distributon (OOD) detection technique called ODIN to calculate ODIN score
• 𝑂𝐷𝐼𝑁 𝑠𝑐𝑜𝑟𝑒 = max(𝑠𝑜𝑓𝑡𝑚𝑎𝑥𝑐 𝑝𝑒𝑟𝑡𝑢𝑟𝑏𝑒𝑑 𝑙𝑜𝑔𝑖𝑡𝑠 − 𝑠𝑜𝑓𝑡𝑚𝑎𝑥 𝑜𝑟𝑖𝑔𝑖𝑛𝑎𝑙 𝑙𝑜𝑔𝑖𝑡𝑠 )
• Steps: 1. Loading Pretrained WideResNet model for CIFAR-10 dataset
2. Model Prediction using preprocessed images.
3. Probability Calculation using sofrmax function to convert logits into probabilities for each class.
4. Apply ODIN to calculate OOD score for the input image. ODIN adjusts the input image and
measures the change in output probabilities to detect anomalies
Model-2(ODIN) Validation
• Test Image • Output
Model-3(MaxSoftmax)
• In this iteration we’ll use a pre-trained WideResNet model to predict the class probabilities of a test image
and computes its OOD score using MaxSoftmax detector (explained in pytorch_ood functionalities).
• Steps: 1. Loading Pretrained WideResNet model for CIFAR-10 dataset
2. Model Prediction using preprocessed images.
3. Probability Calculation using sofrmax function to convert logits into probabilities for each class.
4. Create MaxSoftmax Detector using the pre-trained model
5. Compute OOD score using the MaxSoftmax detector for the input image.
Model-3(MaxSoftmax) Validation
• Test Image • Output
Model-4(Energy-based)
• This model evaluates the out-of-distribution (OOD) scores using an energy-basesd method for WideResNet
model trained on CIFAR-10 dataset. It computes the negative maximum logit as an energy function and
calculates the OOD score using the sigmoid function
• Steps:
• 1. Define Energy Function to compute energy of logits.
• 2. Load the CIFAR-10 train and test datasets using torchvision
• 3. Calculate OOD score by iterating over the train and test loaders, moving inputs to the specified device
(CPU), computing logits using the model, calculating energy scores using the defined function, applying the
sigmoid function to obtain OOD scores, and appending the scores to a list.
Model-4 (Energy based) Validation
• Test Image • Output
Model-5 (Energy-based with Fine tunning)
• This Model fine-tunes a pre-trained WideResNet model on the CIFAR-10 dataset using energy-bounded learning. It
defines an energy function to compute logits' energy, optimizes the model parameters using both logits and energy
losses
• Steps:
• 1. Define Energy Function to compute energy of logits.
• 2. Load the CIFAR-10 train and test datasets using torchvision
• 3. Define parameters such as minimum and maximum margins for energy, the energy coefficient lambda, and
optimizer settings.
• 4. Train the model for 3 epochs using energy-bounded learning. Iterate over the training data, calculate the energy
loss and logits loss, and update the model parameters using backpropagation.
• 5. Threshold for Defining out-of-distribution
Model-5 (Energy-based with Fine tunning) Validation
• Test Image for In-distribution • Test image for our-of-distribution
Thank You
