ML Paper
ML Paper
Camouflaged Lifeform
A comprehensive detailing of identifying camouflaged Lifeforms
Lalitha S.∗ Aarushi Singh Ahlawat Namratha B
Department of Electronics and Department of Electronics and Department of Electronics and
Communication Engineering, Amrita Communication Engineering, Amrita Communication Engineering, Amrita
School of Engineering, Bengaluru, School of Engineering, Bengaluru, School of Engineering, Bengaluru,
Amrita Vishwa Vidyapeetham, India Amrita Vishwa Vidyapeetham, India Amrita Vishwa Vidyapeetham, India
s_lalitha@blr.amrita.edu BL.EN.U4EAC21001@bl.students.amrita.edu
BL.EN.U4EAC21046@bl.students.amrita.edu
The general biological (human) approach to visual processing the algorithm finds relevance in security and military defense. The
and analysis for camouflaged scenes involves a multistage process capability to identify camouflaged objects in surveillance scenarios
[7]. Initially, a global scene is analyzed using a knowledge base and contributes to enhanced reconnaissance and threat detection. It is
low-level information to identify potential areas of interest. The important to note that while the proposed model focuses on the
attention is then refined using higher-level feature information, and direct application of detecting camouflaged lifeforms, the potential
this process continues recursively until the desired level of detec- applications extend far beyond, specifically with enhanced perfor-
tion is achieved. Inspired from human vision, a multilevel approach mance to ensure low type-I and type-II errors, it can be used in
has the potential to deliver computationally efficient and accurate critical applications as well e.g., medical image analysis, medical
detection and identification of highly camouflaged lifeform without condition detection and identification.
compromising performance for general CODnI task that has less
complexity. At the first level, a limited number of features, such as
RGB images, are used for detection and identification. If the uncer- 2 LITERATURE REVIEW
tainty of detection is high, a refined model incorporating additional Though the design and use of camouflage and COD is as old as
features, e.g., IR band, multispectral information and segmented human history. Many tricks of the magic show are based on well-
focused regions of the image, is employed. This combination of designed camouflaging techniques. COD techniques are also exist-
multilevel approaches provides better control over detection un- ing for a long time in the form of effective ways to solve the visual
certainty and computational efficiency. By starting with a lighter puzzles, e.g., “find the difference” and “find hidden creature/object
model and progressively refining it based on prediction results, the in the image”. Investigation of these approaches is important to
approach reduces model bias and improves generalization. systematically explore the opportunity for development of effec-
Hence, the primary objective of the presented work is to develop tive techniques, here development of computer/machine vision
a human vision inspired multilevel approach for camouflaged object techniques for COD that are relevant for present study have been
detection, specifically for lifeforms, to address the challenges and reviewed.
limitation of the present state-of-the-art specifically measured with Considering the critical applications and importance of COD, sig-
respect to the following performance parameters: nificant efforts have been made over past several decades. One set
of efforts are made on compilation and standardization of dataset for
• Miss rates: Percentage of cases where model is not able to
COD study. C.H.A.M.E.L.E.ON (Cryptic Hidden Animals Masked in
detect the lifeform present in the data
Environment, Labelled and Evaluated (-ON), 2017) is a systematic
• Type-1 errors or false positive: detecting and identifying a
and comprehensive database of images to provide a common eval-
lifeform even when it is not present in the data
uation dataset for COD techniques [8]. Though it is challenging
• Type-II errors or false negative: either not detecting or in-
enough for OD techniques, it is having only 76 images. To over-
correctly identifying a target life form when it is actually
come the limitation of C.H.A.M.E.L.E.ON, a bigger, comprehensive
present in the data
and systematic dataset called COD10K is available [9]. This is the
• Confidence of detection: a measure of certainty of detection,
most challenging and high-quality COD dataset containing 10000
i.e. at what level “detected” outcome is significantly different
images (5,066 camouflaged, 3,000 background, 1,934 noncamou-
from “not detected” outcome.
flaged). Dataset is divided into 10 super-classes and 78 sub-classes
The present approach is to achieve the enhanced performance on (69 camouflaged, nine non-camouflaged). All the samples are cate-
above parameters without a significant increase in computational gory annotated and provided with bounding box, object-level and
complexity. To enhance the computational efficiency, present ap- instance-level labels for systematic evaluation of performance of
proach is augmented with physics that is used in the nature for COD techniques. This dataset [9] is not freely hosted but available
camouflaging a lifeform to hide from predators and to detect the only on request.
lifeform by the predators. A comprehensive explanation of use of Another set of contributions are on performance evaluation cri-
different wavelength by the chameleon to camouflage itself in the teria for COD approaches. Some of these are: systematic evaluation
environment and emission of infrared (thermal) radiations by the criteria and metric and loss function for COD by Fan, D.P.; Ji, G.P.;
lifeforms, specifically by animals has been presented in [2]. An Qin, X.; Cheng, M.M. [9] which is further rationalized by Jiang,
additional feature, i.e. IR band data in the image in addition to X.; Cai, W.; Ding, Y.; Wang, X.; Yang, Z.; Di, X.; Gao, W. [10] to
red, green and blue (RGB) bands provides computationally efficient evaluate a ternary cascaded technique for the COD10K dataset [7].
image segmentation to detect the lifeform. Hence, the approach for A number of COD approaches developed and evaluated for above
CODnI is augmented with domain knowledge of physics of COD&I. mentioned datasets and various evaluation criterion. A ternary cas-
Well researched and documented public domain dataset, caded technique is presented by Jiang, X.; Cai, W.; Ding, Y.; Wang,
C.H.A.M.E.L.E.ON [8] has been used to demonstrate and evalu- X.; Yang, Z.; Di, X.; Gao, W. [10]. This approach provides high
ate the performance of the presented approach. accuracy but computationally expensive even for less challenging
Though the presented approach for CODnI demonstrated for samples. Budiman, I., et.al. presented a KNN based classification
application in uncovering lifeforms concealed within dense nat- approach specifically targeted for classification of bird species [11].
ural habitats, such as shrubbery and rainforests. This ability to Different positions and action pose of bird species are covered but
detect camouflaged species, often invisible to the human eye even not in significantly camouflaging environment. Li, X.M.; Huang,
at close proximity, holds significant implications for ecological stud- Q.C. [12] investigated performance of COD for desert battle field
ies and biodiversity monitoring. Beyond ecological applications, setting using additional IR band data and polarization. Using fusion
Multi-level Approach for Detection and Identification of Camouflaged Lifeform IC3 2024, August 08–10, 2024, Noida, India
of IR band data and polarized image enhanced detection accuracy Considering the critical applications and importance of COD, sig-
of object in highly challenging camouflaged backdrop of desert [12]. nificant efforts have been made over past several decades. One set
This has been tested and tuned for very specific type of COD. Jiang, of efforts are made on compilation and standardization of dataset for
Bin, et al. [13] and Sheik Mohammed., S, et al. [14] demonstrated COD study. C.H.A.M.E.L.E.ON (Cryptic Hidden Animals Masked in
application of light Convolutional Network Neural Network (CNN) Environment, Labelled and Evaluated (-ON), 2017) is a systematic
and modified CNN, respectively for animal classification and de- and comprehensive database of images to provide a common eval-
tection. This machine learning and adaptation approach found to uation dataset for COD techniques [8]. Though it is challenging
be successful in classification and identification of animal lifeforms enough for OD techniques, it is having only 76 images. To over-
with significantly improved performance. Y. Lv, J. Zhang, Y. Dai, A. come the limitation of C.H.A.M.E.L.E.ON, a bigger, comprehensive
Li, N. Barnes and D. -P. Fan, presented a comprehensive analysis of and systematic dataset called COD10K is available [9]. This is the
physical and optical features of camouflaged environment leading most challenging and high-quality COD dataset containing 10000
to better understanding physics of camouflaging and detection in images (5,066 camouflaged, 3,000 background, 1,934 noncamou-
nature leading to effective use of tools and techniques for COD flaged). Dataset is divided into 10 super-classes and 78 sub-classes
[15]. Based on the understanding a framework that simultaneously (69 camouflaged, nine non-camouflaged). All the samples are cate-
localize, segment, and rank camouflaged objects presented and gory annotated and provided with bounding box, object-level and
evaluated not only for dataset but for few real-world samples as instance-level labels for systematic evaluation of performance of
well. COD techniques. This dataset [9] is not freely hosted but available
Techniques using additional features such as IR band, multispec- only on request.
tral data, sweeping hyperspectral system, etc. help in overcoming The proposed model utilizes the integration of an Infrared (IR)
the limitation of visible spectrum data by providing additional in- sensor into the camera system. By doing so, we tap into the intrinsic
formation content but also add additional features leading to a ability of living beings to emit IR radiation. This feature becomes
computationally expensive COD solution. particularly potent in detecting camouflaged animals or beings.
Even the most advanced state-of-the-art deep COD techniques When combined with the capabilities of the Google Lens API, the
often produce missed detections and low confidence for COD pri- IR sensor enhances the detection process, allowing for a more
marily because of following reasons: (i) the nature has optimized nuanced and accurate identification of camouflaged entities (Figure
the camouflaged system of lifeforms to such extent that the color 1.).
and texture seamlessly merge in the surrounding background leav- Thus, utilization of IR sensing not only adds a layer of physics
ing very little features to discriminate and (ii) the camouflaged based deterministic feature to the detection process but also aligns
lifeforms have dynamic irregular shapes restricting use of spatial with the natural mechanisms of living organisms. This approach
information features for COD, specifically for lifeforms. goes beyond traditional visual cues, introducing a thermal dimen-
Hence, for camouflaged lifeforms in their natural environment sion to the detection model. A high-level schematics and flowchart
use of highly representative key point information results in poor of the proposed approach is shown in Figure 2.
generalization of the camouflaged lifeform leading to either missed
detections or false detection.
Human vision inspired multilevel approach, specifically for life
form, provide potential opportunity to overcome the limitation and 4 DATASET UTILIZED
challenges of the available techniques for COD. Potential solution The proposed approach has been demonstrated using libraries and
approach starts with limited features and progressively move for trained models available through Google Lens API [6] for the pre-
higher level features in case of low confidence and high uncertainty processed data sourced from the C.H.A.M.E.L.E.ON [8] dataset.
in detection to address the high miss rates, high type-1 and type-II While this dataset lacked Infrared (IR) band, synthetic IR band data
errors, low confidence and high computational efforts. is available in literature has been used along with RGB bands to
evaluate the performance of the proposed solution.
3 PROPOSED APPROACH
The development of proposed approach is primarily based on a
multilevel model, i.e., starting with fewer features and based on 5 METHODOLOGY
detection and identification uncertainty in results, progressively The methodology for the proposed model is design image segmen-
adding features. Further, the proposed model has been augmented tation, feature ranking and selection model to preprocess the data
with underlying physics of detection of pray by some of predators to present to comprehensively trained Google Lens or other CODnI
using thermal radiation emission. Objects with a temperature above models using their API. As the objectives of the present study are
absolute zero emit thermal radiation, predominantly in the form of to develop a multilevel model that is computationally efficient and
infrared electromagnetic waves. Living organisms, encompassing augmented by physics of lifeform detection using IR band data, de-
both humans and animals, are no exception to this natural law. veloped model is trained and coupled with readily available trained
Their metabolic processes continually generate heat, leading to a ML models of Google Lens. Google Lens API accepts data only in
constant emission of infrared radiation. This thermal emission is a the format of RGB image, therefore IR data and features with high
direct reflection of the body’s temperature, typically different than sensitivity (ranking) need to be embedded in the RGB compatible
the surrounding environment. image format.
IC3 2024, August 08–10, 2024, Noida, India Lalitha S et al.
5.1 Infrared (IR) Band Data Integration MATLAB whereas evaluation of the results has been carried from
Since flux of IR emission from the body of lifeform (animal) is in the output of Google Lens API using evaluation criteria selected
general different than that of surrounding environment, this data for the study and dataset.
provides opportunity of image segmentation and focus zone in the
RGB image. Therefore, a model with tunable weights has been 5.4 Fine Tuning
developed in MATLAB [16] for image segmentation of RGB data The proposed CODnI approach, incorporating additional features,
using IR band data. This part of the solution is not computationally is applied and fine-tuned to address challenges identified in the
expensive and hence, this can be implemented as Edge computing marked samples. Subsequently, a combined multilevel approach is
solution on the RGB + IR imaging device. tested on the entire dataset to evaluate model bias, generalization,
and accuracy in CODnI. Weights for image segmentation using
5.2 Feature Ranking and Selection IR band data are tuned to achieve optimized performance for the
Computational complexity increases with number of features from selected dataset.
the input data used for CODnI processing. A model for ranking the
features and a progressive selection of features to generate input 5.5 Evaluation Criteria
RGB compatible image for CODnI of lifeform using Google Lens has Three performance evaluation criteria [17] have been used for train-
been developed using MATLAB [16] AI/ML toolbox. However, it is ing and optimization of the proposed model.
found that with optimal segmentation using IR band data, results 1. Miss rates: Percentage of cases where model is not able to
are good with fewer high rank features like edge and texture of detect the lifeform w.r.t. total cases where life form was
focused segment. present in the ground truth data
bullet· Type-1 and Type-II error-based criteria or Confusion metrics-
5.3 Integration with Existing CODnI Models in based criteria
Cloud (Google Lens APIs) Key values of confusion matrix are:
Two critical aspect for seamless integration with Google Lens API TN = True Negative
are input format and evaluation of results. Input format compatibil- FP = False Positive
ity is ensured in the output of the model that was developed in the FN = False Negative
Multi-level Approach for Detection and Identification of Camouflaged Lifeform IC3 2024, August 08–10, 2024, Noida, India
S. No Image RGB Image IR Google API detection Google API detection Ground Truth
without IR with IR
Figure 3: Google Lens API test results for COD and Identification (subset of results to cover all possible outcomes).
• Confidence Level: a 95% confidence level (p value of 5%) detection is poor then addition of outline feature around the re-
has been used to make decision to qualify the detected and gion of interest) provides very high performance for CODnI using
Identified lifeform. Google API provides the results with present approach. In the results it may be seen that without a
different p-values. physics augmented model, i.e. using RGB image alone detection
and identification is very poor with very high miss rate of 68.5% F1
score of less than 42% (0.42).
6 RESULTS AND ANALYSIS Present approach results in a very low miss rate 2.63% and F1
Images with significant detection and identification challenges are score of 0.97. However, there are instances where present as well as
shown in Figure 3. Google Lens native RGB based approach failed to correctly detect
It is clear that one approach alone that is using only RGB cam- and Identify the lifeform.
ouflaged images for detection and identification is having poor It may be noted that for images at s.no 1 in Figure 3 is not
detection and identification accuracy even with high performance identified by the Google Lens but a with two features in addition
computational resources and comprehensively trained ML model to IR based segmentation resulted in correct identification.
available on Google Cloud through Google Lens APIs. However, If the model had detected an inanimate object instead of the
using a IR data based segmentation and two-level approach (initial living being (e.g., refer to s.no 7 in Figure 3.), then the code should
level with texture feature over segmented focused region and if
Multi-level Approach for Detection and Identification of Camouflaged Lifeform IC3 2024, August 08–10, 2024, Noida, India
be modified such that the threshold set for the detection is reversed ACKNOWLEDGMENTS
and this way even if the IR radiated by the inanimate object becomes Help and support from teachers, parents and friends in the de-
more, the proposed model will accurately identify the camouflaged velopment of this approach, solution and this article is gratefully
lifeform. acknowledged.
7 CONCLUSIONS REFERENCES
A human vision inspired multilevel CODnI approach has been [1] Gnaneshwar Bandari, L. Nirmala Devi and P. Srividya. 2022. Wild Animal De-
presented to improve both the detection/identification accuracy and tection using a Machine Learning Approach and Alerting using LoRa Commu-
nication. In Proceedings of the International Conference on Smart Generation
the computational efficiency. Presented model has been augmented Computing, Communication and Networking (SMART GENCON), Bangalore,
by understanding of the physics of camouflage of pray and detection India, 2022, 1-5. https://doi.org/10.1109/SMARTGENCON56628.2022.10083577
[2] Stuart-Fox Devi, Moussalli Adnan and Whiting Martin J. 2008. Predator-specific
by predator in the nature. Primary objective of the presented work camouflage in chameleons. Biol. Lett. 4326–329. https://doi.org/10.1098/rsbl.2008.
is to provide high performance CODnI specifically to address the 0173
[3] Yufan He, Dong Yang, Holger Roth, Can Zhao and Daguang Xu. 2021. DiNTS:
challenges of high miss rates, high type-1 and type-II errors and Differentiable Neural Network Topology Search for 3D Medical Image Segmen-
low confidence. Performance has been evaluated for a camouflaged tation. In Proceedings of the IEEE/CVF Conference on Computer Vision and
natural environment image dataset that has been used extensively Pattern Recognition (CVPR), Nashville, TN, USA, 5837-5846. https://doi.org/10.
1109/CVPR46437.2021.00578
over the last few years. Presented technique found to achieve [4] J. G. Fennell, L. Talas, R. J. Baddeley, I. C. Cuthill, and N. E. Scott-Samuel. 2019.
significantly high detection rate, accuracy and confidence. However, Optimizing colour for camouflage and visibility using deep learning: the effects
using single level approach even with IR data having significant of the environment and the observer’s visual system. J. R. Soc. Interface 16:
20190183. https://doi.org/10.1098/rsif.2019.0183
miss on Google Lens API whereas using multilevel approach results [5] Xinhao Jiang., Wei Cai., Zhiyong Yang, Peiwei Xu and Bo Jiang. 2022. IARet: A
in nearly 99.9 % correct detection and identification. IR band data Lightweight Multiscale Infrared Aerocraft Recognition Algorithm. Arab J Sci Eng
47, 2289–2303. https://doi.org/10.1007/s13369-021-06181-7
augmented segmentation results in significant improvement in [6] Google Lens, 2023, Albhabet Inc., Mountain View, CA, USA (Google), Accessed
performance of CODnI at much lower computational complexity. on October 09, 2023 from https://lens.google/
Further optimized algorithm with more than two level approach is [7] Deng-Ping Fan, Ge-Peng Ji, Xuebin Qin, and Ming-Ming Cheng. 2021. Cognitive
vision inspired object segmentation metric and loss function. SCIENTIA SINICA
expected to provide even higher computational efficiency. For basic Informationis. https://doi.org/10.1360/SSI-2020-0370
deployment in field with online real-time CODnI for lifeform, IR [8] Skurowski P. 2017. C.H.A.M.E.L.E.ON – Cryptic Hidden Animals Masked in
augmented image segmentation and automated zoom/focus region Environment, Labelled and Evaluated (-ON), Retrieved on October 9, 2023 from
https://www.polsl.pl/rau6/chameleon-database-animal-camouflage-analysis/
control can be implemented as Edge computing on ‘RGB + IR’ [9] Deng-Ping Fan, Ge-Peng Ji, Guolei Sun, Ming-Ming Cheng, Jianbing Shen, and
imaging device whereas detection and identification can run on the Ling Shao. 2020. Camouflaged object detection. In Proceedings of the IEEE/CVF
Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA,
Cloud. USA, 2774-2784. https://doi.org/10.1109/CVPR42600.2020.00285
[10] Xinhao Jiang, Wei Cai, Yao Ding, Xin Wang, Zhiyong Yang, Xingyu Di and Weijie
8 RECOMMENDATIONS AND FUTURE Gao. 2023. Camouflaged Object Detection Based on Ternary Cascade Perception.
Remote Sens. 15, 5, 1188. https://doi.org/10.3390/rs15051188
DIRECTIONS [11] Ichsan Budiman, Diena Rauda Ramdania, Yana Aditia Gerhana, Alif Rakasha
Pratama Putra, Nisairrizqy Nabilah Faizah and Maisevli Harika. 2022. Classifica-
C.H.A.M.E.L.E.ON [8] data set is having only 76 images. For compre- tion of Bird Species using K-Nearest Neighbor Algorithm. In Proceedings of the
hensive evaluation of the performance of the presented approach 10th International Conference on Cyber and IT Service Management (CITSM),
a bigger and challenging dataset such as COD10K [9]., be used as Yogyakarta, Indonesia 1-5. https://doi.org/10.1109/CITSM56380.2022.9936012
[12] Xiaoming Li and Qinchao Huang. 2017. Target detection for infrared polarization
soon as actual of synthetic IR band data is available. The present image in the background of desert. In Proceedings of the IEEE 9th International
approach can also be implemented for use with wildlife camera and Conference on Communication Software and Networks (ICCSN), Guangzhou,
cellphone sensors having both RGB and IR (night vision) cameras. China, 1147-1151. https://doi.org/10.1109/ICCSN.2017.8230290
[13] Bin Jiang, Wei Huang, Wenxuan Tu and Chao Yang. 2019. An Animal Classifica-
As presented in configuration of the system in Figure 1. a feedback tion based on Light Convolutional Network Neural Network. In Proceedings of
to automatically zoom into region of interest for search, detection the International Conference on Intelligent Computing and its Emerging Applica-
tions (ICEA), Tainan, Taiwan, 45-50. https://doi.org/10.1109/ICEA.2019.8858309
and identification of lifeforms in camouflaged environment will [14] Sheik Mohammed. S, T. Sheela and T. Muthumanickam. 2022. Development
result in a powerful tool for ecological and environmental studies. of Animal-Detection System using Modified CNN Algorithm. In Proceedings
As performance of CODnI improves, design of camouflage for of the International Conference on Augmented Intelligence and Sustainable
Systems (ICAISS), Trichy, India. 105-109. https://doi.org/10.1109/ICAISS55157.
security and defence may be tested and improved for enhancement 2022.10011014
of performance of the camouflage so that it remains undetected [15] Yunqiu Lv, Jing Zhang, Yuchao Dai, Aixuan Li, Nick Barnes and Deng-Ping
even with this enhanced capability. Fan. 2023. Toward Deeper Understanding of Camouflaged Object Detection.
IEEE Transactions on Circuits and Systems for Video Technology, vol. 33, no. 7.
3462-3476. https://doi.org/10.1109/TCSVT.2023.3234578
[16] MATLAB version: (R2023a), The MathWorks Inc. Natick, Massachusetts, USA.
https://www.mathworks.com
[17] Željko Ð. Vujovic. 2021. Classification Model Evaluation Metrics. International
Journal of Advanced Computer Science and Applications (IJACSA), 12, 6. http:
//dx.doi.org/10.14569/IJACSA.2021.0120670