0% found this document useful (0 votes)

33 views148 pages

AI Report

The document outlines the course structure for the Artificial Intelligence Lab (CSCL-4101) at SZABIST, detailing the experiments, objectives, and grading policy. It includes a comprehensive list of topics such as Python programming, machine learning algorithms, and optimization techniques. Additionally, it specifies course learning outcomes and their mapping to program learning outcomes, emphasizing the importance of attendance and participation.

Uploaded by

bsai23108187

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

33 views148 pages

AI Report

Uploaded by

bsai23108187

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

You are on page 1/ 148

Artificial Intelligence BS (AI)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology

(SZABIST)

Artificial Intelligence
CSCL-4101
Semester 3rd

Name: _______________________________________________

Roll Number: _________________________________________

Faculty of Computing and Engineering Sciences

Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

List of Experiments

Lab No Topics Remarks Mapping

Introduction to Python Programming and to get familiar with
Integrated Development Environment (IDE), Google Colab online
1. CLO_1
laboratory, Jupyter notebook. Python fundamentals and Introduction
to AI and Algorithms.
Basic Programming in Python: Data Types and Variables, Types of
2. Operators and Operator Precedence, Advance data types, CLO_1
List/tuple/dictionaries/slicing.
Python: Conditional statements (If-Else, Nested if-else and ifelif-else
ladder), Understanding of Control Structure: For Loop, While Loop),
3. CLO_2
Concept of Functions (built-in, user defined functions, Recursion),
Concept of Arrays, Structures.
Familiarization with Python packages for AI: Pandas, Numpy, Sci-
kit learn, Mat-plot library.
4. CLO_2
Machine Learning: Artificial Neural Network (Single Layer
Perceptron)
Machine Learning: Artificial Neural Network (Multi-Layer
5. CLO_2
Perceptron)
6. Machine Learning: K-Nearest Neighbor KNN. CLO_2
7. Machine Learning: Correlation, Regression CLO_2
8. Machine Learning: K-mean clustering CLO_2
9. Machine Learning: Decision Tree CLO_2
10. Machine Learning: Naïve Bayes CLO_2
11. Machine Learning: SVM Apriori Algorithm CLO_2
12. Optimization: Genetic Algorithm GA CLO_2
Solution search strategies :Breath-first search, Depth first search,
13. CLO_2
Best First search
14. Project CLO_3
15. Final Lab CLO_3

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

CSCL 4101 Lab Artificial Intelligence

COURSE INFORMATION SHEET

PROGRAMME: Artificial DEGREE: BS

Intelligence
COURSE: Lab Artificial SEMESTER: 3 CREDITS: 1
Intelligence
COURSECODE: CSCL 4101 COURSE TYPE: CORE
COURSE AREA/DOMAIN: CONTACT HOURS: 3 hours/Week.
Computer Science
CORRESPONDING LAB LAB COURSENAME: Nil
COURSE CODE (IFANY): Nil

Instructor:

Prerequisites:

Objectives:
The objective of the course is to present an overview of artificial intelligence (AI) principles and
approaches. Develop a basic understanding of the building blocks of AI as presented in terms of
intelligent agents: Search, Knowledge representation, inference, logic, and learning

Contents:
Introduction to Python Programming, Integrated Development Environment (IDE), Google Colab
online laboratory, Jupyter notebook. Introduction to AI and Algorithms, Basic Programming in
Python: Data Types and Variables, Types of Operators and Operator
Precedence, Advance data types, List/tuple/dictionaries/slicing, Conditional statements (If-Else,
Nested if-else and if-elif-else ladder), Understanding of Control Structure: For Loop, While Loop),
Concept of Functions (built-in, user defined functions, Recursion), Concept of Arrays, Structures,
Familiarization with Python packages for AI: Pandas, Numpy, Sci-kit learn, Mat-plot library,
Machine Learning: Artificial Neural Network (Single Layer Perceptron, Multi Layer Perceptron,
KNN, K-mean clustering, Decision Tree, Regression, Naïve Bayes, SVM Apriori Algorithm ,
market basket analysis, Principal Component Analysis PCA. Optimization: Genetic Algorithm
GA, Particle Swarm Optimization PSO. Solution search strategies :Breath-first search, Depth first
search, Best First search, Beam search, Bi-directional search, Uniform cost search, Iterative

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)
Deepening Depth First search, Informed search, Heuristic Search. Deep Learning: Deep Network
Designing, Transfer Learning.
Recommended Book:
 Artificial Intelligence. A Modern Approach, Stuart Russell and Peter Norvig, null Edition,
Prentice Hall
 Understanding Machine Learning: From Theory to Algorithms, Shai Shalev-Shwartz, Shai
Ben-David, null Edition, Cambridge University Press,2020
 Prentice Hall. Prolog Programming for Artificial Intelligence, Bratko, I., (2012).
 Addison-Wesley. Machine Learning, Mitchell, T., (1997), McGraw-Hill.
Administrative Instructions:
 Title and Group members name for Lab/Course project should be submitted by 9th week of
lab.
 According to institute policy, 75% attendance is mandatory to appear in the final
examination but 100% will be expected. Approved leaves will not be considered towards
attendance.
 Every student should bring manual and notebook in each lab.
 Every student is expected to be in lab before schedule starting time.
 In any case there will be no rescheduling and makeup of labs.

COURSE LEARNING OUTCOMES (CLOs):

BT
CLO Description Domain
Level*
Understand key components in the field of artificial
1 C 2
intelligence
2 Implement classical artificial intelligence techniques C 3
Analyze artificial intelligence techniques for practical
3 C 4
problem solving
*BT=Bloom’s Taxonomy, C=Cognitive domain, P=Psychomotor domain, A=Affective domain

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

MAPPING COURSE LEARNING OUTCOMES (CLOs) AND PROGRAM LEARNING

OUTCOMES (PLOs):

Mapping of CLOs and PLOs

S. No. Course Learning Outcomes PLOs Bloom’s Taxonomy

Be able to practice measuring variables of

different physical systems
C2
CLO_1 PLO_1
By the end of this lesson, the student will be able (Understand)
to describe and understand the basics of AI and
python programming interface.
C3
Be able to implement the algorithms and
CLO_2 PLO_2 (Apply)
techniques in artificial intelligence
Be able to experiment and analyze the C4
CLO_3 PLO_3
algorithms and techniques in artificial intelligence (Analyze)

PROGRAM LEARNING OUTCOMES (PLOs):

PLO DESCRIPTION
Ability to apply knowledge of mathematics, science, computing fundamentals andany of
a
its specializations to solve complex problems.
Ability to identify, formulate, research literature, and analyze complex problems reaching
b substantiated conclusions using basic principles of mathematics, natural sciences and
computer science and Artificial intelligence

Ability to design solutions for complex problems and design software systems,
c components or processes that meet specified needs with appropriate considerationfor
public health and safety, cultural, societal, and environmental considerations.

Ability to investigate methodically complex problems including literature survey,design

d and conduct of experiments, analysis and interpretation of experimental data, and
synthesis of information to derive valid conclusions
Ability to create, select and apply appropriate techniques, resources, and modernIT
e tools, including prediction and modelling, to complex activities, with an understanding
of the limitations
Ability to recognize importance of, and pursue lifelong learning in the broadercontext
k
of innovation and technological developments

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Overall Grading Policy

Assessment Items Percentage
Lab Performance/Manual 14%
Mid Term Exam 20%
Theory Teacher Viva and Lab Project 30%
Final Examination 30%
Quiz 6%

Rubric for mark of each experiment:

Below Average Satisfactory Excellent

n
(0% of n) (50% of n) (100% of n)
Class  Present but not  Present as well as
0.3  Absent
Participation actively involved actively involved
 Performs
 Performs
experiment in
experiment in
 Does not casual manner.
Experiment professional way.
0.5 perform  Rules and
Performance  Rules and
experiment regulations are
regulations are
partially
followed.
followed.
 Absent on
experiment  Absent on
 Present and also
day experiment day
perform
Experiment  Also did not  Perform
0.2 experiment at his /
Reporting perform experiment with
her time slot
experiment other section
 Reporting done
with other  Reporting done
section
Total Marks 1
Marks
Obtained

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Experiment Performance

Performan
ce between Performan
Below ce between
Below Satisfactor
m Average Satisfactor Excellent
Average y
and y and
Satisfactor Excellent
y
(50% of (75% of
(0% of m) (25% of m) (100% of m)
m) m)
Equipment  Improper  Casual  Proper
4
Handling handling handling handling

Procedur
e of
 Get correct
taking
 Not able to readings
Measureme readings
6 get through
nts is proper
readings proper
but the
procedure
readings
are
incorrect
 Does not  Knows
know about  Knows
about basic basic about basic
theory theory theory
regarding regarding regarding
Viva 10
experiment experime experiment
 Does not nt  Replies
completely  Does not sufficient
reply all reply all questions
questions questions
Total Marks 20
Marks
Obtained

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Theory Teacher Viva* and Project

Performan
Performan
ce between
ce
Below
Below Satisfactor between
m Average Excellent
Average y Satisfacto
and
ry and
Satisfactor
Excellent
y
(50% of (75% of
(0% of m) (25% of m) (100% of m)
m) m)
 Knows
about
 Does not basic
 Knows
know about theory
about basic
basic theory regardin
theory
regarding g
regarding
respective respectiv
respective
course (and e course
Viva 15 course (and
project, if (and
project, if
applicable) project,
applicable)
 Does not if
 Replies
completely applicabl
sufficient
reply all e)
questions
questions  Does not
reply all
questions
Total
15
Marks
Marks
Obtained

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Communicati Hardware

d
Report Viva Heads

Total
on skills Presentation

Marks
Marks

Obtaine
Marks

3
6
3
3

15
(m)
Performanc
e below
Poor
 Similarity  Does not know  Listen but  Bad packaging (0% of m)
index >19% about basic do not  Unacceptable Poor
 Do not contain theory regarding understand Hardware (No use (20% of m)
theoretical project along with of aesthetic sense,
background  Does not non- if applicable)
 Diagrams are completely reply effective  Hardware not
all hardware and reply Performanc
copy-paste working
theory related e between
questions Poor and
(SZABIST)

Satisfactory
(40% of m)
 Similarity  Knows about  Listen and  Average
Artificial Intelligence BS (AI)

index <=19% basic theory understand packaging Satisfactory

 Contain regarding project but reply  Presentable (60% of m)
Lab Semester Project Rubrics

theoretical  Does not reply non- Hardware (Use of

background all hardware and effectively aesthetic sense, if
 Diagrams are theory related applicable) Performanc
not in good questions  Hardware not e between
order. working Satisfactory
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology

 Similarity  Knows about  Listen,  Optimized and

Excellent

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

index <10% basic theory understand packaging Excellent
(100%
(80% of of
m)
 Contain regarding project and reply  Presentable m)
theoretical  Replies all effectively Hardware (Use of

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
background hardware and aesthetic sense, if
 Diagrams are theory related applicable)
in good order. questions  Hardware
working
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Marks Evaluation
Marks
Experiment No. Class Experiment Experiment
Total
Participation Performance Reporting
(1)
(0.3) (0.5) (0.2)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
Total

Instructor’s signature

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Performance Performance
Performance between Poor between
Marks Poor Satisfactory Excellent
below Poor and Satisfactory
Heads
Satisfactory and Excellent

(0% of m) (20% (100% of

(m) (40% of m) (60% of m) (80% of m)
of m) m)

Understanding
6
of code

Communication
3
skills
Viva 3

Report 3
Total Marks 15
Marks
Obtained

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)
General Laboratory Procedure
While there is no specific document to be submitted at the beginning of the Lab –unless your
instructor advises you otherwise-, you are expected to read the experiment fully before you come
to the laboratory? Interestingly, you can even try parts of the experiment at home. Here is a list of
programs that will equip you with a virtual lab at your home:
Troubleshooting
Things will not always go as expected; this is the nature of the learning process. While conducting
the Experiment think before you do anything. If you do so you will avoid wasting time going
down dead-end streets. Be logical and systematic. First, look for obvious errors that are easy to
fix. Is your measuring device correctly set and connected? Are you looking at the proper scale? Is
the power supply set for the correct voltage? Is the signal generator correctly set and connected?
How are the variables in the code set? Is there a syntax error? And so on. Next, check for obvious
misconnections or broken connections, at least in simple circuits.
As you work through your circuit, use your Lab Manual record tests and changes that you make
as you go along; don't rely on your memory for what you have tried. Identify some test points in
the system at which you know what the signal should be and work your way backwards from the
output through the test points until you find a good signal.
Neatness
When you have finished for the day, return all modules to their proper storage bins, return all test
leads and probes to their storage racks, return all equipment to its correct location, and clean up
the lab station. If appropriate switch off the unneeded equipment. Save your files in the Computer
and on any USB device for your records because you might not get the same PC System again for
the next experiment. Also email your file contents to your email address as a backup.
Laboratory Safety

Always pay attention to what you are doing and you’re surrounding during the experiments and
notify the Instructor for any unlikely event or mishap and leave the Laboratory with the permission
of Instructor immediately.
All students must read and understand the information in this document with regard to laboratory
safety and emergency procedures prior to the first laboratory session.
Your personal laboratory safety depends mostly on YOU. Effort has been made to address
situations that may pose a hazard in the lab but the information and instructions provided cannot
be considered all-inclusive.
Students must adhere to written and verbal safety instructions throughout the academic term. Since
additional instructions may be given at the beginning of laboratory sessions, it is important that all

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)
students arrive at each session on time. With good judgment, the chance of an accident in this
course is very small. Nevertheless, research and teaching workplaces (labs, shops, etc.) are full of
potential hazards that can cause serious injury and or damage to the equipment. Working alone
and unsupervised in laboratories is forbidden if you are working with hazardous substances or
equipment. With prior approval, at least two people should be present so that one can shut down
equipment and call for help in the event of an emergency. Safety training and/or information should
be provided by a faculty member, teaching assistant, lab safety contact, or staff member at the
beginning of a new assignment or when a new hazard is introduced into the workplace.
Emergency Response
1. It is your responsibility to read safety and fire alarm posters and follow the instructions during
an emergency
2. Know the location of the fire extinguisher, eye wash, and safety shower in your lab and know
how to use them.
3. Notify your instructor immediately after any injury, fire or explosion, or spill.
4. Know the building evacuation procedures.
Common Sense
Good common sense is needed for safety in a laboratory. It is expected that each student will work
in a responsible manner and exercise good judgment and common sense. If at any time you are not
sure how to handle a particular situation, ask your Teaching Assistant or Instructor for advice DO
NOT TOUCH ANYTHING WITH WHICH YOU ARE NOT COMPLETELY
FAMILIAR!!! It is always better to ask questions than to risk harm to yourself or damage to the
equipment.
Personal and General laboratory safety
1. Never eat, drink, or smoke while working in the laboratory.
2. Read labels carefully.
3. Do not use any equipment unless you are trained and approved as a user by your supervisor.
4. Wear safety glasses or face shields when working with hazardous materials and/or equipment.
5. Wear gloves when using any hazardous or toxic agent.
6. Clothing: When handling dangerous substances, wear gloves, laboratory coats, and safety shield
or glasses. Shorts and sandals should not be worn in the lab at any time. Shoes are required
when working in the machine shops.
7. If you have long hair or loose clothes, make sure it is tied back or confined.
8. Keep the work area clear of all materials except those needed for your work. Coats should be
hung in the hall or placed in a locker. Extra books, purses, etc. should be kept away from
equipment that requires air flow or ventilation to prevent overheating.
9. Disposal - Students are responsible for the proper disposal of used material if any in appropriate
containers.
10. Equipment Failure - If a piece of equipment fails while being used, report it immediately to
your lab assistant or tutor. Never try to fix the problem yourself because you could harm
yourself and others.
11. If leaving a lab unattended, turn off all ignition sources and lock the doors.
12. Never pipette anything by mouth.
13. Clean up your work area before leaving.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)
14. Wash hands before leaving the lab and before eating.
15. Unauthorized person(s) shall not be allowed in a laboratory for any reason
Electrical safety
1. Obtain permission before operating any high voltage equipment.
2. Maintain an unobstructed access to all electrical panels.
3. Wiring or other electrical modifications must be referred to the Electronics Shop or the Building
Coordinator.
4. Avoid using extension cords whenever possible. If you must use one, obtain a heavy- duty one
that is electrically grounded, with its own fuse, and install it safely. Extension cords should not
go under doors, across aisles, be hung from the ceiling, or plugged into other extension cords.
5. Never, ever modify, attach or otherwise change any high voltage equipment.
6. Always make sure all capacitors are discharged (using a grounded cable with an insulating
handle) before touching high voltage leads or the "inside" of any equipment even after it has
been turned off. Capacitors can hold charge for many hours after the equipment has been turned
off.
7. When you are adjusting any high voltage equipment or a laser which is powered with a high
voltage supply, USE ONLY ONE HAND. Your other hand is best placed in a pocket or behind
your back. This procedure eliminates the possibility of an accident where high voltage current
flows up one arm, through your chest, and down the other arm.
8. Discard damaged cords, cords that become hot, or cords with exposed wiring.
9. Before equipment is energized ensure, (1) circuit connections and layout have been checked by
a Teaching Assistant (TA) and (2) all colleagues in your group give their assent.
10.Know the correct handling, storage and disposal procedures for batteries, cells, capacitors,
inductors and other high energy-storage devices.
11.Experiments left unattended should be isolated from the power supplies. If for a special reason,
it must be left on, a barrier and a warning notice are required.
12.Equipment found to be faulty in any way should be reported to the Lab Engineer immediately
and taken out of service until inspected and declared safe.
13.Voltages above 50 V rms AC and 120 V DC are always dangerous. Extra precautions should
be considered as voltage levels are increased.
14.Never make any changes to circuits or mechanical layout without first isolating the circuit by
switching off and removing connections to power supplies.
15.Know what you must do in an emergency.
16.Emergency Power Off: Every lab is equipped with and Emergency Power Off System.
17. Only authorized personnel are permitted to reset power once the Emergency Power Off system
has been engaged.
Electrical Emergency Response
The following instructions provide guidelines for handling two types of electrical emergencies:
1. When someone suffers serious electrical shock, he or she may be knocked unconscious. If the
victim is still in contact with the electrical current, immediately turn off the electrical power
source. If you cannot disconnect the power source, depress the Emergency Power Off switch.
2. Do not touch a victim that is still in contact with a live power source; you could be electrocuted.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)
3. Have someone call for emergency medical assistance immediately. Administer first-aid, as
appropriate.
4. If an electrical fire occurs, try to disconnect the electrical power source, if possible. If the fire
is small and you are not in immediate danger; and you have been properly trained in fighting
fires, use the correct type of fire extinguisher to extinguish the fire. When in doubt, push in the
Emergency Power Off button.
5. NEVER use water to extinguish an electrical fire.
Mechanical safety
1. When using compressed air, use only approved nozzles and never direct the air towards any
person.
2. Guards on machinery must be in place during operation.
3. Exercise care when working with or near hydraulically- or pneumatically-driven equipment.
Sudden or unexpected motion can inflict serious injury.
Additional Safety Guidelines
1. Never do unauthorized experiments.
2. Never work alone in laboratory.
3. Keep your lab space clean and organized.
4. Do not leave an on-going experiment unattended.
5. Always inform your instructor if you break a thermometer.
Do not clean mercury yourself!!
6. Never taste anything. Never pipette by mouth; use a bulb.
7. Never use open flames in laboratory unless instructed by TA.
8. Check your glassware for cracks and chips each time you use it. Cracks could cause the
glassware to fail during use and cause serious injury to you or lab mates.
9. Maintain unobstructed access to all exits, fire extinguishers, electrical panels, emergency
showers, and eye washes.
10. Do not use corridors for storage or work areas.
11. Do not store heavy items above table height. Any overhead storage of supplies on top of
cabinets should be limited to lightweight items only. Also, remember that a 36" diameter area
around all fire sprinkler heads must be kept clear at all times.
12. Areas containing lasers, biohazards, radioisotopes, and carcinogens should be posted
accordingly. However, do not post areas unnecessarily and be sure that the labels are removed
when the hazards are no longer present.
13. Be careful when lifting heavy objects. Only shop staff may operate forklifts or cranes.
14. Clean your lab bench and equipment, and lock the door before you leave the laboratory.
Clothing
1. Dress properly during a laboratory activity.
2. Long hair, dangling jewelry, and loose or baggy clothing are a hazard in the laboratory.
3. Long hair must be tied back, and dangling jewelry and baggy clothing must be secured.
4. Shoes must completely cover the foot.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)
5. No sandals allowed on lab days.
6.A lab coat or smock should be worn during laboratory experiments.
Accidents and Injuries
1.Do not panic.
2.Report any accident (spill, breakage, etc.) or injury (cut, burn, etc.) to
the teacher immediately, no matter how trivial it seems.
3.If you or your lab partner is hurt, immediately (and loudly) yell out the
teacher's name to get the teacher's attention.
General Warning Signs

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

LAB 01: Introduction to Python, Artificial Intelligence and to get

familiar with Integrated Development Environment (IDE)

Objective:
To discuss and elaborate AI and algorithms, python programming fundamentals and get
familiarization with Integrated Development Environment (IDE), Google Colab online laboratory,
Jupyter notebook.
Software and Tools:
● 1.6 GHz or faster processor
● 4 GB of RAM
● 20 GB of available hard disk space
● Windows 7 or later
● Spyder (Python 3.9)
● Google Colab

Background and Procedure:

Introduction to Artificial Intelligence

Artificial intelligence (AI) is a wide-ranging branch of computer science concerned with building
smart machines capable of performing tasks that typically require human intelligence.
"Artificial intelligence is a set of algorithms and intelligence to try to mimic human intelligence.
Machine learning is one of them, and deep learning is one of those machine learning techniques."

Types of Artificial Intelligence

1. Reactive Machines
2. Limited Memory
3. Theory of Mind
4. Self-Awareness

Examples of AI Applications
• Siri, Alexa and other smart assistants
• Self-driving cars (Driver-less Cars)
• Robo-advisors
• Conversational bots (Chat Bots)
• Email spam filters
• Netflix's recommendations (Recommendation system)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)
The major limitation in defining AI as simply "building machines that are intelligent" is that it
doesn't actually explain what artificial intelligence is? What makes a machine intelligent? AI is an
interdisciplinary science with multiple approaches, but advancements in machine learning and
deep learning are creating a paradigm shift in virtually every sector of the tech industry.

Categories of Artificial Intelligence

Artificial intelligence generally falls under two broad categories:

1. Narrow AI:
Sometimes referred to as "Weak AI," this kind of artificial intelligence operates within a limited
context and is a simulation of human intelligence. Narrow AI is often focused on performing a
single task extremely well and while these machines may seem intelligent, they are operating under
far more constraints and limitations than even the most basic human intelligence.

Example:
• Google search
• Image recognition software
• Siri, Alexa and other personal assistants
• Self-driving cars
• IBM's Watson

2. Artificial General Intelligence (AGI):

AGI, sometimes referred to as "Strong AI," is the kind of artificial intelligence we see in the
movies, like the robots from Westworld or Data from Star Trek: The Next Generation. AGI is a
machine with general intelligence and, much like a human being, it can apply that intelligence to
solve any problem.

Reactive Machines
A reactive machine follows the most basic of AI principles and, as its name implies, is capable of
only using its intelligence to perceive and react to the world in front of it. A reactive machine
cannot store a memory and as a result cannot rely on past experiences to inform decision making
in real-time. Perceiving the world directly means that reactive machines are designed to complete
only a limited number of specialized duties.
A famous example of a reactive machine is Deep Blue, which was designed by IBM in the 1990‘s
as a chess-playing supercomputer and defeated international grandmaster Gary Kasparov in a
game.
Another example of a game-playing reactive machine is Google‘s AlphaGo. AlphaGo is also
incapable of evaluating future moves but relies on its own neural network to evaluate developments
of the present game, giving it an edge over Deep Blue in a more complex game. AlphaGo also
bested world-class competitors of the game, defeating champion Go player Lee Sedol in 2016.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)
Limited Memory
Limited memory artificial intelligence has the ability to store previous data and predictions when
gathering information and weighing potential decisions — essentially looking into the past for
clues on what may come next. Limited memory artificial intelligence is more complex and presents
greater possibilities than reactive machines.
Limited memory AI is created when a team continuously trains a model in how to analyze and
utilize new data or an AI environment is built so models can be automatically trained and renewed.
When utilizing limited memory AI in machine learning, six steps must be followed: Training data
must be created, the machine learning model must be created, the model must be able to make
predictions, the model must be able to receive human or environmental feedback, that feedback
must be stored as data, and these steps must be reiterated as a cycle.

There are three major machine learning models that utilize limited memory artificial intelligence:

 Reinforcement learning - Which learns to make better predictions through

repeated trial-and-error.

 Long Short Term Memory (LSTM) - Which utilizes past data to help predict the
next item in a sequence. LTSMs view more recent information as most important when
making predictions and discounts data from further in the past, though still utilizing it to
form conclusions.

 Evolutionary Generative Adversarial Networks (E-GAN) - Which evolves over

time, growing to explore slightly modified paths based off of previous experiences with
every new decision. This model is constantly in pursuit of a better path and utilizes
simulations and statistics, or chance, to predict outcomes throughout its evolutionary
mutation cycle.

Theory of Mind:

Theory of Mind is just that theoretical. We have not yet achieved the technological and scientific
capabilities necessary to reach this next level of artificial intelligence.
The concept is based on the psychological premise of understanding that other living things have
thoughts and emotions that affect the behavior of one‘s self. In terms of AI machines, this would
mean that AI could comprehend how humans, animals and other machines feel and make decisions
through self-reflection and determination, and then will utilize that information to make decisions
of their own. Essentially, machines would have to be able to grasp and process the concept
of ―mind,‖ the fluctuations of emotions in decision making and a litany of other psychological
concepts in real time, creating a two-way relationship between people and artificial intelligence.

Self-Awareness:

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)
Once Theory of Mind can be established in artificial intelligence, sometime well into the future,
the final step will be for AI to become self-aware. This kind of artificial intelligence possesses
human-level consciousness and understands its own existence in the world, as well as the presence
and emotional state of others. It would be able to understand what others may need based on not
just what they communicate to them but how they communicate it.
Self-awareness in artificial intelligence relies both on human researchers understanding the
premise of consciousness and then learning how to replicate that so it can be built into machines.

Introduction to Python
Python is an open source and cross-platform programming language that has become increasingly
popular over the last ten years. It was released in 1991. It is an interpreted, objectoriented, high-
level programming language with dynamic semantics developed by Guido van Rossum. Latest
version is 3.7.0. Python is a multi-purpose programming languages (due to its many extensions).
Python is an interpreted programming language, this means that as a developer you write Python
(.py) files in a text editor and then put those files into the python interpreter to be executed.
Depending on the Editor you are using, this is either done automatically, or you need to do it
manually.

Examples: scientific computing and calculations, simulations, web development (Django Web
framework), etc.

Introduction to Anaconda:
Anaconda offers the easiest way to perform Python/R data science and machine learning on a
single machine. You can work with thousands of open-source packages and libraries with it.
Anaconda is a distribution package, where you get Python compiler, Python packages and the
Spyder editor. Anaconda includes Python, Jupyter Notebook, and other commonly used packages
for scientific computing and data science.

Jupyter Notebook:
The Jupyter Notebook is an open-source web application that allows you to create and share
documents that contain live code, equations, visualizations and text.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Lets open your Python Editor and type the following:

Print(“Hello World”)

If you get an error message like this:

'python' is not recognized as an internal or external command, operable program or batch
file. Then you need to add Python to your path. See instructions below.
Add Python to Path:
In the Windows menu, search for \advanced system settings" and select View advanced
system settings.
In the window that appears, click Environment Variables . . . near the bottom right.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

In the next window, select the user variable named Path and click Edit. . . to change its
value. Select "New" and add the path where "python.exe" is located. See Figure 3.6.
The Default Location is:

Click Save and open the Command Prompt once more and enter "python" to verify it works.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Get Help:
An extremely useful command is help(), which enters a help functionality to explore all the
students python lets you do, right from the interpreter. Press q to close the help window and
return to the Python prompt.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)
Introduction to Spyder:
There are various IDEs in the market to select from such as Spyder, Atom, Pycharm, Pydev
etc. Data scientists prefer Spyder among all the different IDEs available and the driving fact
behind this is that Spyder was built specifically to be used for data science. Its interface
allows the user to scroll through various data variables and also ready to use online help
option. The output of the code can be viewed in the python console on the same screen. You
can work on different scripts at a moment and then try them out one by one in the same
console or different as per your choice all the variables used will be stored in the variable
explorer tab. It also provides an option to view graphs and visualizations in the plot window.
You can also cover the basics concepts by taking up free Syder python and also check out
Python Libraries for Machine Learning from Great Learning Academy.

Download and installation:

Go to Anaconda Website: https://www.anaconda.com/. Download anaconda for your
respective operating system. Download and install Anaconda. You can launch Spyder with
anaconda navigator.

Google Colab

Colaboratory, or ―Colab‖ for short, is a product from Google Research. Colab allows
anybody to write and execute arbitrary python code through the browser, and is especially
well suited to machine learning, data analysis and education. More technically, Colab is a
hosted Jupyter notebook service that requires no setup to use, while providing access free
of charge to computing resources including GPUs.

Colab Interface

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Resources in Colab are prioritized for interactive use cases. We prohibit actions associated
with bulk compute, actions that negatively impact others, as well as actions associated with
bypassing our policies. The following are disallowed from Colab runtimes:

• file hosting, media serving, or other web service offerings not related to
interactive compute with Colab
• downloading torrents or engaging in peer-to-peer file-sharing
• using a remote desktop or SSH
• connecting to remote proxies
• mining cryptocurrency
• running denial-of-service attacks
• password cracking
• using multiple accounts to work around access or resource usage restrictions
creating deep fakes.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Objective:
To perform the basic python programming using Data Types and Variables, Types of
Operators and Operator Precedence, Advance data types, List/tuple/dictionaries/slicing.

Background and Procedure:

Python Data Types

Standard data types
With Python you don't get so much out of the box. Instead of having all of its functionality
built into its core, you need to install different packages for different topics.
The data stored in memory can be of many types. For example, a person's age is stored as a
numeric value and his or her address is stored as alphanumeric characters. Python has
various standard data types that are used to define the operations possible on them and the
storage method for each of them.
• Numeric data types: int, float, complex
• String data types: str
• Sequence types: list, tuple, range
• Binary types: bytes, bytearray, memoryview
• Mapping data type: dict
• Boolean type: bool
• Set data types: set, frozenset

Python Numeric Data types

var1 =
10 var2
= 2.55
Python supports three different numerical types −
• int (signed integers)
• float (floating point real values)
• complex (complex numbers)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Python Strings
Strings in Python are identified as a contiguous set of characters represented in the quotation
marks. Python allows either pair of single or double quotes. Subsets of strings can be taken
using the slice operator ([ ] and [:] ) with indexes starting at 0 in the beginning of the string
and working their way from -1 to the end. The plus (+) sign is the string concatenation
operator and the asterisk (*) is the repetition operator.
String Input:
Python allows for command line input. That means we are able to ask the user for input.
The following example asks for the user's name, then, by using the input() method, the
program prints the name to the screen:

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Python Lists
Lists are the most versatile of Python's compound data types. A list contains items separated
by commas and enclosed within square brackets ([]). To some extent, lists are similar to
arrays in C. One of the differences between them is that all the items belonging to a list can
be of different data type.
The values stored in a list can be accessed using the slice operator ([ ] and [:]) with indexes
starting at 0 in the beginning of the list and working their way to end -1. The plus (+) sign
is the list concatenation operator, and the asterisk (*) is the repetition operator.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Python Tuples
A tuple is another sequence data type that is similar to the list. A tuple consists of a number
of values separated by commas. Unlike lists, however, tuples are enclosed within
parenthesis. The main difference between lists and tuples is- Lists are enclosed in brackets
( [ ] ) and their elements and size can be changed, while tuples are enclosed in parentheses
( ( ) ) and cannot be updated. Tuples can be thought of as read-only lists.

The following code is invalid with tuple, because we attempted to update a tuple,
which is not allowed. Similar case is possible with lists: tuple = ( 'abcd', 786 , 2.23,
'john', 70.2 ) list = [ 'abcd', 786 , 2.23, 'john', 70.2 ] tuple[2] = 1000 # Invalid syntax
with tuple list[2] = 1000 # Valid syntax with list

Python Dictionary

Python's dictionaries are kind of hash-table type. They work like associative arrays or hashes
found in Perl and consist of key-value pairs. A dictionary key can be almost any
Python type, but are usually numbers or strings. Values, on the other hand, can be any
arbitrary Python object.
Dictionaries are enclosed by curly braces ({ }) and values can be assigned and accessed
using square braces ([]).
Dictionaries have no concept of order among the elements. It is incorrect to say that the
elements are "out of order"; they are simply unordered.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Python Range() function

We can generate a sequence of numbers using range() function. range(10) will generate
numbers from 0 to 9 (10 numbers).

We can also define the start, stop and step size as range(start, stop,step_size). step_size
defaults to 1 if not provided.

This function does not store all the values in memory; it would be inefficient. So it
remembers the start, stop, step size and generates the next number on the go.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

A slice object is used to specify how to slice a sequence. You can specify where to
start the slicing, and where to end. You can also specify the step, which allows you to
e.g. slice only every other item. Following is the syntax:

• slice(start, end, step)

• slice (stop)
• slice (start, stop[, step])

Following are the Parameters:

• start: Starting index where the slicing of object starts.
• stop: Ending index where the slicing of object stops.
• step: It is an optional argument that determines the increment between each
index for slicing.
Return Type: Returns a sliced object containing elements in the given range only.

A sequence of objects of any type (string, bytes, tuple, list or range) or the object which
implements __getitem__() and __len__() method then this object can be sliced using slice()
method.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Python Variables
Variables are nothing but reserved memory locations to store values. It means that when
you create a variable, you reserve some space in the memory.
Based on the data type of a variable, the interpreter allocates memory and decides what can
be stored in the reserved memory. Therefore, by assigning different data types to the
variables, you can store integers, floats or characters in these variables.
Variables are defined with the assignment operator, \=". Python is dynamically typed,
meaning that variables can be assigned without declaring their type, and that their type can
change. Values can come from constants, from computation involving values of other
variables, or from the output of a function.
Basic rules for Python variables:
• A variable name must start with a letter or the underscore character
• A variable name cannot start with a number
• A variable name can only contain alpha-numeric characters (A-z, 0-9) and
underscores
• Variable names are case-sensitive, e.g., amount, Amount and AMOUNT are
three different variables.

Assigning Values to Variables

Python variables do not need explicit declaration to reserve memory space. The declaration
happens automatically when you assign a value to a variable. The equal sign (=) is used to
assign values to variables.
The operand to the left of the = operator is the name of the variable and the operand to the
right of the = operator is the value stored in the variable.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Multiple Assignments

Python allows you to assign a single value to several variables simultaneously.

For
exampl
ea=b
=c=1
Here, an integer object is created with the value 1, and all the three variables are assigned to
the same memory location. You can also assign multiple objects to multiple variables.

For example
a, b, c = 1, 2, "john"
Here, two integer objects with values 1 and 2 are assigned to the variables a and b
respectively, and one string object with the value "john" is assigned to the variable c.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Types of Operators
Python language supports the following types of operators:
• Arithmetic Operators
• Comparison (Relational) Operators
• Assignment Operators
• Logical Operators, Bitwise Operators
• Membership Operators
• Identity Operators

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Example Program:
Assume variable a holds 50 and variable b holds 23:

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Python Comparison Operators

These operators compare the values on either side of them and decide the relation among
them. They are also called Relational operators.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Example Program:

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Python Membership Operators

Python‘s membership operators test for membership in a sequence, such as strings, lists, or
tuples. There are two membership operators as explained below

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Python Identity Operators

Identity operators compare the memory locations of two objects. There are two Identity
operators as explained below:

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Operator precedence affects the evaluation of an expression.

For example, x = 7 + 3 * 2; here, x is assigned 13, not 20 because the operator * has higher
precedence than +, so it first multiplies 3*2 and then is added to 7.
Here, the operators with the highest precedence appear at the top of the table, those with the
lowest appear at the bottom.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Example Program:

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Objective:
To perform python programming using Conditional statements (If-Else, Nested if-else and
ifelif-else ladder), Understanding of Control Structure: For Loop, While Loop), Concept of
Functions (built-in, user defined functions, Recursion), Concept of Arrays, Structures.

Background and Procedure:

Conditional Statements
The „if‟ Statement
The ability to control the flow of your program, letting it make decisions on what code to execute,
is valuable to the programmer. The ―if statement‖ allows you to control if a program enters a
section of code or not based on whether a given condition is true or false. Decision-making is the
anticipation of conditions occurring during the execution of a program and specified actions taken
according to the conditions.
Decision structures evaluate multiple expressions, which produce TRUE or FALSE as the
outcome. You need to determine which action to take and which statements to execute if the
outcome is TRUE or FALSE otherwise.
The if statement contains a logical expression using which the data is compared and a decision is
made based on the result of the comparison,
If the Boolean expression evaluates to TRUE, then the block of statement(s) inside the if statement
is executed. In Python, statements in a block are uniformly indented after the : symbol. If Boolean
expression evaluates to FALSE, then the first set of code after the end of block is executed.

Syntax:
if (CONDITION):
Execute the next statement

Program

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

The „else‟ Statement

An else statement can be combined with an if statement. An else statement contains a block of
code that executes if the conditional expression in the if statement resolves to 0 or a FALSE value.
The else statement is an optional statement and there could be at the most only one else statement
following if.

Syntax:
if
expression
:
statement(
s) else:
statement(
s)

Program

The „if...elif...else‟ statements

The elif statement allows you to check multiple expressions for TRUE and execute a block of code
as soon as one of the conditions evaluates to TRUE.
Like the else, the elif statement is optional. However, unlike else, for which there can be at the
most one statement, there can be an arbitrary number of elif statements following an if.
Core Python does not provide switch or case statements as in other languages, but
we can use if..elif...statements to simulate switch case as follows

Syntax:

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Program
In the above example, discount is calculated on the input amount. Rate of discount is 5%, if the
amount is less than 1000, and 10% if it is above 5000 and else takes care of all values by 15% rate
of discount.

The if-else Statements in One Line (Ternary Operator)

Syntax:

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)
if expression: set of statement to execute if condition is
true else: statements to execute if condition is false
OR
a if condition else b

Program

The Nested if-else Statements

if statement can also be checked inside other if statement. This conditional statement is called a
nested if statement. This means that inner if condition will be checked only if outer if condition is
true and by this, we can see multiple conditions to be satisfied.

Syntax
if (condition1):
# Executes when condition1 is
true if (condition2):
# Executes when condition2 is true
# if Block is end here
# if Block is end here

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

The „if-elif-else‟ Ladder

There may be a situation when you want to check for another condition after a condition resolves
to true. In such a situation, you can use the nested if construct. In a nested if construct, you can
have an if-.elif-else construct inside another If-elif-else construct. Syntax if expression1:
statement(s) if expression2: statement(s) elif expression3:

statement
(s) else
statement
(s) elif
expressio
n4:
statement
(s) else:
statement
(s)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Loop
In general, statements are executed sequentially: The first statement is executed first, followed by
the second, and so on. There may be a situation when you need to execute a block of code several
number of times. A loop statement allows us to execute a statement or group of statements multiple
times. The following diagram illustrates a loop statement.

For Loop
In Python, for keyword provides a more comprehensive mechanism to constitute a loop. The for
loop is used with sequence types such as list, tuple, set, range, etc. The body of the for loop is
executed for each member element in the sequence. Syntax for x in sequence:
statement1
statement2
...
statementN

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

While Loop
In most computer programming languages, a while loop is a control flow statement that allows
code to be executed repeatedly based on a given Boolean condition. The while loop can be thought
of as a repeating if statement.
The ‗for‘ loop does something a fixed number of times. But in a case where you don‘t know how
many times you want to do something before you start the loop, you use a different loop: the while
loop. Here, statement(s) may be a single statement or a block of statements. The condition may be
any expression, and true is any non-zero value. The loop iterates while the condition is true. When
the condition becomes false, program control passes to the line immediately following the loop.
In Python, all the statements indented by the same number of character spaces after a programming
construct are considered to be part of a single block of code. Python uses indentation as its method
of grouping statements

Syntax
while
expression
:
statement(
s)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Concept of Functions
A function can be called as a section of a program that is written once and can be executed
whenever required in the program, thus making code reusability.
• Keyword def that marks the start of the function header.
• A function name to uniquely identify the function. Function naming follows the
same rules of writing identifiers in Python.
• Parameters (arguments) through which we pass values to a function. They are
optional.
• A colon (:) to mark the end of the function header.
• Optional documentation string (docstring) to describe what the function does.
• One or more valid python statements that make up the function body. Statements
must have the same indentation level (usually 4 spaces).
• An optional return statement to return a value from the function.
• Once we have defined a function, we can call it from another function, program, or
even the Python prompt. To call a function we simply type the function name with
appropriate parameters.

Syntax def function_name(argument1,

argument2, ...) :
statement_1
statement_2
....

Scope and Lifetime of function variables:

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)
• Scope of a variable is the portion of a program where the variable is recognized.
Parameters and variables defined inside a function are not visible from outside the function.
Hence, they have a local scope.
• The lifetime of a variable is the period throughout which the variable exists in the
memory. The lifetime of variables inside a function is as long as the function executes.
• They are destroyed once we return from the function. Hence, a function does not
remember the value of a variable from its previous calls.

Python User-defined Functions

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Python Built-in Functions

The abs() method takes a single argument:

Concept of Arrays
An array is a collection of items stored at contiguous memory locations. The idea is to store
multiple items of the same type together. This makes it easier to calculate the position of each
element by simply adding an offset to a base value, i.e., the memory location of the first element
of the array (generally denoted by the name of the array)
Array in Python can be created by importing array module. array(data_type, value_list) is used to
create an array with data type and value list specified in its arguments.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Objective:
To perform the basic machine learning artificial neural network (Single Layer Perceptron)
algorithm and determine the outcome with familiarization with python packages Pandas, Numpy,
Sci-kit learn, and Mat-plot library.

Background and Procedure:

Python Packages and Plotting

With Python you don't get so much out of the box. Instead of having all of its functionality built
into its core, you need to install different packages for different topics.

Python Packages for Science and Numerical Computations:

Some important Python Packages for Science and Numerical Computations are:

NumPy:
NumPy is the fundamental package for scientific computing with Python SciPy:
SciPy is a free and open-source Python library used for scientific computing and technical
computing. SciPy contains modules for optimization, linear algebra, integration, interpolation,
special functions, FFT, signal and image processing, ODE solvers and other tasks common in
science and engineering.
Matplotlib:
Matplotlib is a Python 2D plotting library.
Pandas:
Pandas Python Data Analysis Library

Built-in Functions:
Python consists of lots of built-in functions. Some examples are the print (function that we already
have used (perhaps without noticing it is actually a Built-in function).
Python also consists of different Modules, Libraries or Packages. These Modules, Libraries or
Packages consists of lots of predefined functions for different topics or areas, such as mathematics,
plotting, handling database systems, etc.
More functions we will explore later

Python Standard Library:

Python allows you to split your program into modules that can be reused in other Python programs.
It comes with a large collection of standard modules that you can use as the basis of your programs.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)
The Python Standard Library consists of different modules for handling file I/O, basic
mathematics, etc. You don't need to install these separately, but you need to important them when
you want to use some of these modules or some of the functions within these modules. The math
module has all the basic math functions you need, such as: Trigonometric functions: sin(x), cos(x),
etc. Logarithmic functions: log(), log10(), etc. Constants like pi, e, inf, nan, etc.

Plotting in Python:
Typically you need to create some plots or charts. In order to make plots or charts in
Python you will need an external library. The most used library is Matplotlib. Matplotlib
is a Python 2D plotting library.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Subplots:
The subplot command enables you to display multiple plots in the same window.
Typing "subplot(m,n,p)" partitions the figure window into an m-by-n matrix of small
subplots and selects the subplot for the current plot. The plots are numbered along the first
row of the figure window, then the second row, and soon.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Single-Layer Perceptron

Single Layer Perceptron Mathematical Model

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

a) Round 1

We will apply 1st instance to the perceptron. x1 = 0 and x2 = 0.

Sum unit will be 0 as calculated below

Σ = x1 * w1 + x2 * w2 = 0 * 0.9 + 0 * 0.9 = 0

Activation unit checks sum unit is greater than a threshold. If this rule is satisfied, then it is
fired and the unit will return 1, otherwise it will return 0. BTW, modern neural networks
architectures do not use this kind of a step function as activation.

Activation threshold would be 0.5.

Sum unit was 0 for the 1st instance. So, activation unit would return 0 because it is less than
0.5. Similarly, its output should be 0 as well. We will not update weights because there is
no error in this case.

Let‘s focus on the 2nd instance. x1 = 0 and x2 = 1.

Sum unit: Σ = x1 * w1 + x2 * w2 = 0 * 0.9 + 1 * 0.9 = 0.9

Errors:

Activation unit will return 1 because sum unit is greater than 0.5. However, output of this
instance should be 0. This instance is not predicted correctly. That‘s why, we will update
weights based on the error.

ε = actual – prediction = 0 – 1 = -1

We will add error times learning rate value to the weights. Learning rate would be 0.5. BTW,
we mostly set learning rate value between 0 and 1.

w1 = w1 + α * ε = 0.9 + 0.5 * (-1) = 0.9 – 0.5 = 0.4

w2 = w2 + α * ε = 0.9 + 0.5 * (-1) = 0.9 – 0.5 = 0.4

Focus on the 3rd instance. x1 = 1 and x2 = 0.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Activation unit will return 0 this time because output of the sum unit is 0.5 and it is less than 0.5.
We will not update weights.

Mention the 4rd instance. x1 = 1 and x2 = 1.

Sum unit: Σ = x1 * w1 + x2 * w2 = 1 * 0.4 + 1 * 0.4 = 0.8

Activation unit will return 1 because output of the sum unit is 0.8 and it is greater than the threshold
value 0.5. Its actual value should 1 as well. This means that 4th instance is predicted correctly. We
will not update anything.

b) Round 2

In previous round, we‘ve used previous weight values for the 1st instance and it was classified
correctly. Let‘s apply feed forward for the new weight values.

Remember the 1st instance. x1 = 0 and x2 = 0.

Sum unit: Σ = x1 * w1 + x2 * w2 = 0 * 0.4 + 0 * 0.4 = 0.4

Activation unit will return 0 because sum unit is 0.4 and it is less than the threshold value 0.5. The
output of the 1st instance should be 0 as well. This means that the instance is classified correctly.
We will not update weights.

Feed forward for the 2nd instance. x1 = 0 and x2 = 1.

Sum unit: Σ = x1 * w1 + x2 * w2 = 0 * 0.4 + 1 * 0.4 = 0.4

Activation unit will return 0 because sum unit is less than the threshold 0.5. Its output should be 0
as well. This means that it is classified correctly and we will not update weights.

We‘ve applied feed forward calculation for 3rd and 4th instances already for the current weight
values in the previous round. They were classified correctly.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Learning Term:

We should continue this procedure until learning completed. We can terminate the learning
procedure here. Luckily, we can find the best weights in 2 rounds.

Updating weights means learning in the perceptron. We set weights to 0.9 initially but it
causes some errors. Then, we update the weight values to 0.4. In this way, we can predict
all instances correctly.

Experiment
Implementation of Single Layer Perceptron

We can create perceptron that act like gates: they take 2 binary inputs and produce a single
binary output.
1. Defining inputs size, the learning rate and number of epochs as inputs and
weights vector for the network. One is added to the input size to include the bias in
the weight vector.
2. Implementing activation function which returns 1 if the input is greater than
or equal to 0 and 0 otherwise.
3. Creating prediction function to run an input through the perceptron and
return an output. The bias is added into the input vector then we can simply compute
the inner product and apply the activation function.
4. Perceptron Learning Algorithm Code: Create a function to keep applying this
update rule until our perceptron can correctly classify all of our inputs. We need to
keep iterating through our training data until this happens; one epoch is when our
perceptron has seen all of the training data once. Usually, we run our learning
algorithm for multiple epochs. We can create a function, given inputs and desired
outputs, run our perceptron learning algorithm. We keep updating the weights for a

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)
number of epochs, and iterate through the entire training set. We insert the bias into
the input when performing the weight update. Then we can create our prediction,
compute our error, and perform our update rule.

Basic perceptron can generalize any kind of linear problem. The both AND and OR Gate problems
are linearly separable problems. On the other hand, this form cannot generalize nonlinear problems
such as XOR Gate. Perceptron evolved to multilayer perceptron to solve nonlinear problems and
deep neural networks were born.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Lab Exercises:

1. To reinforce the perceptron, apply learning procedure for OR Gate. The gate returns 0 if
and only if both inputs are 0. Do not hesitate to change the initial weights and learning rate
values.

2. Can this form generalized the same algorithm for nonlinear problem XOR gate , why
or why not?
________________________________________________________________________
___
________________________________________________________________________
___
________________________________________________________________________
___
______________________________________________________________________
_____ _______________________________________________________________
Conclusion:
________________________________________________________________________
______
________________________________________________________________________
______
________________________________________________________________________
______
________________________________________________________________________
______
________________________________________________________________________
______

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Artificial Neural Network (Multi-Layer Perceptron)

LAB 05: Biological Binary Model

Objective:
To perform the basic machine learning artificial neural network (Multi Layer Perceptron)
algorithm and determine the outcome with familiarization with python packages Pandas,
Numpy, Sci-kit learn, and Mat-plot library.

Background and Procedure:

Multi Layer Perceptron Mathematical Model

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Multi-Layer Perceptron Mathematical Model and Algorithm

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Multi-Layer Perceptron Code (XOR Gate)

Introduction:
In this implementation of Multi-Layer Perceptron neural network, a single hidden layer
neural network is used, with sigmoid activation function in hidden layer units and sigmoid
activation function for output layer too, since the output of XOR logic is binary i.e. 0 or 1
only one neuron is in the output layer. This time we are not adding bias with inputs as SLP.
However, the algorithm will run fine without adding bias but can‘t without weight updates.
The mathematics and basic logic behind the neural network is also explained.
[Hints]
X = inputs
Y= desired output
No_x = number of inputs
No_y = number of neurons in output layer
No_h = Number of neurons in hidden layer
Tot = Total training examples
Lr = learning rate
W1, w2 = hidden layers weights
Losses = error or loss calculation
frwd_prop = defining forward propagation
method sigmoid = defining sigmoid method
z = pre-activation /weights vector sum
z1,a1,z2,a2 = weight updates and activation variables (forward propagation)
dz2,dw2,dz1,dw1 = derivative of weights and weights updates (w.r.t. weight vector and
activation function)
mlp_test = testing multilayer perceptron learning and network prediction

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Rewrite the Coding as Define Below and Place Output Screen shot:

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Lab Exercises:

1. To reinforce the perceptron, apply learning procedure for XOR Gate. The gate returns 0
if and only if both inputs are 0. Do not hesitate to change the initial weights and learning
rate values.

3. Can this form generalized the same algorithm for nonlinear problem XNOR gate , why
or why not?
________________________________________________________________________
___
________________________________________________________________________
___
________________________________________________________________________
___
______________________________________________________________________
_____ _______________________________________________________________
Conclusion:

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

LAB 06: K-Nearest Neighbors (KNN)

Objective:
To perform the K-Nearest Neighbor algorithm and determine the outcome of the algorithm
performance.

Background and Procedure:

K-Nearest Neighbor

The k-nearest neighbors algorithm is a simple easy to implement supervised machine

learning algorithm that can be used to solve both classifications and regression problems.

Implementation of KNN Algorithm using IRIS dataset About IRIS Data Set
This dataset contains 3 classes of 50 instances each, which refers to a type of iris plant.
Attribute Information:
1. Sepal length in cm
2. Sepal width in cm
3. Petal length in cm
4. Petal width in cm

5. Species: ―Iris Setosa‖, ―Iris Versicolour‖, ―Iris Virginica‖

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Lab Exercises:

1. Perform the given KNN algorithm and analyze result prediction.

4. Can this learning generalized the same algorithm for unsupervised problem, why or
why not?
________________________________________________________________________
___
________________________________________________________________________
___
________________________________________________________________________
___
______________________________________________________________________
_____ _______________________________________________________________
Conclusion:

________________________________________________________________________
______
________________________________________________________________________
______
________________________________________________________________________
______
________________________________________________________________________
______
________________________________________________________________________
______
________________________________________________________________________
______

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

LAB 07: Correlation and Regression:

Objective:
To perform the correlation and regression algorithms and determine the machine learning
mechanism of the algorithm performance.

Background and Procedure:

Statistics and data science are often concerned about the relationships between two or more
variables (or features) of a dataset. Each data point in the dataset is an observation, and the
features are the properties or attributes of those observations.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Regression
Regression searches for relationships among variables. Linear regression is used to identify
the relationship between a dependent variable and one or more independent variables and is
typically leveraged to make predictions about future outcomes.
When there is only one independent variable and one dependent variable, it is known as
simple linear regression. As the number of independent variables increases, it is referred to
as multiple linear regression. For each type of linear regression, it seeks to plot a line of best
fit, which is calculated through the method of least squares.
However, unlike other regression models, this line is straight when plotted on a graph.
Linear regression is leveraged when dependent variables are continuous, whereas logistical
regression is selected when the dependent variable is categorical, meaning they have binary
outputs, such as

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)
"true" and "false" or "yes" and "no."
While both regression models seek to understand relationships between data inputs, logistic
regression is mainly used to solve binary classification problems, such as spam
identification.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Creating DataFrame file for prediction.csv

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Lab Exercises:

1. Perform correlation problem define in the manual.

2. Implement linear regression technique for a sample dataset for house price
prediction.

Conclusion:

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)
LAB 08: K-Means Clustering
Objective:
To perform the K-Means clustering algorithm and determine the outcome of the algorithm
performance.

Background and Procedure:

Clustering

Clustering is a set of techniques used to partition data into groups, or clusters. Clusters are loosely
defined as groups of data objects that are more similar to other objects in their cluster than they
are to data objects in other clusters. In practice, clustering helps identify two qualities of data:
• Meaningfulness
• Usefulness

Meaningful clusters expand domain knowledge. For example, in the medical field, researchers
applied clustering to gene expression experiments. The clustering results identified groups of
patients who respond differently to medical treatments.

Useful clusters, on the other hand, serve as an intermediate step in a data pipeline. For example,
businesses use clustering for customer segmentation. The clustering results segment customers
into groups with similar purchase histories, which businesses can then use to create targeted
advertising campaigns.

When we are working with huge volumes of data, it makes sense to partition the data into logical
groups and doing the analysis. We can use Clustering to make the data into groups with the help
of several algorithms like K-Means.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

K MEANS CLUSTERING ALGORITHM STEPS

• Choose a random number of centroids in the data. i.e k=3.
• Choose the same number of random points on the 2D canvas as centroids.
• Calculate the distance of each data point from the centroids.
• Allocate the data point to a cluster where its distance from the centroid is
minimum.
• Recalculate the new centroids.
• Recalculate the distance from each data point to new centroids.
• Repeat the steps from point 3, till no data point change its cluster.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Lab Exercises:

1. Implement cluster algorithm Kmeans for income dataset of salaried employees.

2. What main difference does Kmeans has when compared to KNN?

________________________________________________________________________
___
________________________________________________________________________
___
________________________________________________________________________
___
______________________________________________________________________
_____ _______________________________________________________________
Conclusion:

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

LAB 09: Decision Tree

Objective:
To perform decision tree algorithm techniques in AI and observe results of the algorithm.

Background and Procedure:

Decision Tree

A Decision Tree is just a Flow Chart, and can help you make decisions based on previous
experience.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Lab Exercises:

2. Implement decision tree algorithm on salaries data.

Conclusion:

______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Background and Procedure:

Naïve Bayes
It is a classification technique based on Bayes‘ Theorem with an assumption of independence
among predictors. In simple terms, a Naive Bayes classifier assumes that the presence of a
particular feature in a class is unrelated to the presence of any other feature.

For example, a fruit may be considered to be an apple if it is red, round, and about 3 inches in
diameter. Even if these features depend on each other or upon the existence of the other features,
all of these properties independently contribute to the probability that this fruit is an apple and that
is why it is known as ‗Naive‘.

Naive Bayes model is easy to build and particularly useful for very large data sets. Along with
simplicity, Naive Bayes is known to outperform even highly sophisticated classification methods.

Bayes theorem provides a way of calculating posterior probability P(c|x) from P(c), P(x) and
P(x|c). Look at the equation below:

Above,

P(c|x) is the posterior probability of class (c, target) given predictor (x, attributes).
P(c) is the prior probability of class.
P(x|c) is the likelihood which is the probability of predictor given class. P(x) is the prior probability
of predictor.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

1. Implement naïve bayes algorithm technique to detect spam email.

Conclusion:

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Objective:
 Learn about SVM and SVC
 Learn about how SVM can be used with PCA
 MINIST using SVM

Introduction:

Support Vector Machine

Support Vector Machine (SVM) is a supervised machine learning algorithm used for both
classification and regression. Though we say regression problems as well it’s best suited for
classification. The main objective of the SVM algorithm is to find the optimal hyperplane in an N-
dimensional space that can separate the data points in different classes in the feature space. The
hyperplane tries that the margin between the closest points of different classes should be as
maximum as possible. The dimension of the hyperplane depends upon the number of features. If
the number of input features is two, then the hyperplane is just a line. If the number of input
features is three, then the hyperplane becomes a 2-D plane. It becomes difficult to imagine when
the number of features exceeds three.

Let’s consider two independent variables x1, x2, and one dependent variable which is either a blue
circle or a red circle.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

One reasonable choice as the best hyperplane is the one that represents the largest separation or
margin between the two classes.

Multiple hyperplanes separate the data from two classes

So we choose the hyperplane whose distance from it to the nearest data point on each side is
maximized. If such a hyperplane exists it is known as the maximum-margin hyperplane/hard
margin. So from the above figure, we choose L2. Let’s consider a scenario like shown below

Selecting hyperplane for data with outlier

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)
Here we have one blue ball in the boundary of the red ball. So how does SVM classify the data?
It’s simple! The blue ball in the boundary of red ones is an outlier of blue balls. The SVM algorithm
has the characteristics to ignore the outlier and finds the best hyperplane that maximizes the
margin. SVM is robust to outliers.

Hyperplane which is the most optimized one

So in this type of data point what SVM does is, finds the maximum margin as done with previous
data sets along with that it adds a penalty each time a point crosses the margin. So the margins in
these types of cases are called soft margins. When there is a soft margin to the data set, the SVM
tries to minimize (1/margin+∧(∑penalty)). Hinge loss is a commonly used penalty. If no violations
no hinge loss.If violations hinge loss proportional to the distance of violation. Till now, we were
talking about linearly separable data(the group of blue balls and red balls are separable by a straight
line/linear line). What to do if data are not linearly separable?

Original 1D dataset for classification

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)
Say, our data is shown in the figure above. SVM solves this by creating a new variable using a
kernel. We call a point xi on the line and we create a new variable yi as a function of distance from
origin o.so if we plot this we get something like as shown below

Mapping 1D data to 2D to become able to separate the two classes

In this case, the new variable y is created as a function of distance from the origin. A non-linear
function that creates a new variable is referred to as a kernel.

In this lab, we'll use the SVC module from the sklearn.svm

package to demonstrate the support vector classifier and the SVM:

from sklearn.svm import SVC

1 Support Vector Classifier

The SVC() function can be used to fit a support vector classifier when the argument
kernel="linear" is used. This function uses a slightly different formulation of the equations we
saw in lecture to build the support vector classifier. The c argument allows us to specify the cost
of a violation to the margin. When the c argument is small, then the margins will be wide and
many support vectors will be on the margin or will violate the margin. When the c argument is
large, then the margins will be narrow and there will be few support vectors on the margin or
violating the margin. We can use the SVC() function to fit the support vector classifier for a given
value of the cost parameter. Here we demonstrate the use of this function on a two-dimensional

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)
example so that we can plot the resulting decision boundary. Let's start by generating a set of
observations, which belong to two classes:

# Generating random data: 20 observations of 2 features and divide into two

classes.
np.random.seed(5)
X = np.random.randn(20,2)
y = np.repeat([1,-1], 10)

X[y == -1] = X[y == -1]+1

Let's plot the data to see whether the classes are linearly separable:

plt.scatter(X[:,0], X[:,1], s=70, c=y, cmap=mpl.cm.Paired)

plt.xlabel('X1')
plt.ylabel('X2')

Nope; not linear. Next, we fit the support vector classifier:

svc = SVC(C=1, kernel='linear')

svc.fit(X, y)

We can now plot the support vector classifier by calling the plot_svc()

function on the output of the call to SVC(), as well as the data used in the call to SVC()

plot_svc(svc, X, y)

The region of feature space that will be assigned to the −1 class is shown in light blue, and the
region that will be assigned to the +1 class is shown in brown. The decision boundary between the
two classes is linear (because we used the argument kernel="linear"

The support vectors are plotted with crosses and the remaining observations are plotted as circles;
we see here that there are 13 support vectors. We can determine their identities as follows:

svc.support_

What if we instead used a smaller value of the cost parameter?

svc2 = SVC(C=0.1, kernel='linear')

svc2.fit(X, y)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Now that a smaller value of the c parameter is being used, we obtain a larger number of support
vectors, because the margin is now wider.

The sklearn.grid_search module includes a a function GridSearchCV() to perform cross-

validation. In order to use this function, we pass in relevant information about the set of models
that are under consideration. The following command indicates that we want perform 10-fold
cross-validation to compare SVMs with a linear kernel, using a range of values of the cost
parameter:

from sklearn.model_selection import GridSearchCV

# Select the optimal C parameter by cross-validation

tuned_parameters = [{'C': [0.001, 0.01, 0.1, 1, 5, 10, 100]}]
clf = GridSearchCV(SVC(kernel='linear'), tuned_parameters, cv=10,
scoring='accuracy')
clf.fit(X, y)

We can easily access the cross-validation errors for each of these models:

clf.cv_results_

The GridSearchCV() function stores the best parameters obtained, which can be accessed as
follows:

clf.best_params_

c=0.001 is best according to GridSearchCV.

As usual, the predict() function can be used to predict the class label on a set of test observations,
at any given value of the cost parameter. Let's generate a test data set:

np.random.seed(1)
X_test = np.random.randn(20,2)
y_test = np.random.choice([-1,1], 20)
X_test[y_test == 1] = X_test[y_test == 1]-1

Now we predict the class labels of these test observations. Here we use the best model obtained
through cross-validation in order to make predictions:

svc2 = SVC(C=0.001, kernel='linear')

svc2.fit(X, y)
y_pred = svc2.predict(X_test)
pd.DataFrame(confusion_matrix(y_test, y_pred), index=svc2.classes_,
columns=svc2.classes_)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Now consider a situation in which the two classes are linearly separable. Then we can find a
separating hyperplane using the svm() function. First we'll give our simulated data a little nudge
so that they are linearly separable:

X_test[y_test == 1] = X_test[y_test == 1] -1
plt.scatter(X_test[:,0], X_test[:,1], s=70, c=y_test, cmap=mpl.cm.Paired)
plt.xlabel('X1')
plt.ylabel('X2')

Now the observations are just barely linearly separable. We fit the support vector classifier and
plot the resulting hyperplane, using a very large value of cost so that no observations are
misclassified.

svc3 = SVC(C=1e5, kernel='linear')

svc3.fit(X_test, y_test)
plot_svc(svc3, X_test, y_test)

No training errors were made and only three support vectors were used. However, we can see from
the figure that the margin is very narrow (because the observations that are not support vectors,
indicated as circles, are very close to the decision boundary). It seems likely that this model will
perform poorly on test data. Let's try a smaller value of cost

svc4 = SVC(C=1, kernel='linear')

svc4.fit(X_test, y_test)
plot_svc(svc4, X_test, y_test)

Using cost=1

, we misclassify a training observation, but we also obtain a much wider margin and make use of
five support vectors. It seems likely that this model will perform better on test data than the model
with cost=1e5.

2 Support Vector Machine

In order to fit an SVM using a non-linear kernel, we once again use the SVC()

function. However, now we use a different value of the parameter kernel. To fit an SVM with a
polynomial kernel we use kernel="poly", and to fit an SVM with a radial kernel we use

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)
kernel="rbf". In the former case we also use the degree argument to specify a degree for the
polynomial kernel, and in the latter case we use gamma to specify a value of γ

for the radial basis kernel.

Let's generate some data with a non-linear class boundary:

from sklearn.model_selection import train_test_split

np.random.seed(8)
X = np.random.randn(200,2)
X[:100] = X[:100] +2
X[101:150] = X[101:150] -2
y = np.concatenate([np.repeat(-1, 150), np.repeat(1,50)])

X_train, X_test, y_train, y_test = train_test_split(X, y, train_size=0.5,

random_state=2)

plt.scatter(X[:,0], X[:,1], s=70, c=y, cmap=mpl.cm.Paired)

plt.xlabel('X1')
plt.ylabel('X2')

See how one class is kind of stuck in the middle of another class? This suggests that we might
want to use a radial kernel in our SVM. Now let's fit the training data using the SVC()

function with a radial kernel and γ=1

svm = SVC(C=1.0, kernel='rbf', gamma=1)

svm.fit(X_train, y_train)
plot_svc(svm, X_test, y_test)

Not too shabby! The plot shows that the resulting SVM has a decidedly non-linear boundary. We
can see from the figure that there are a fair number of training errors in this SVM fit. If we increase
the value of cost, we can reduce the number of training errors:

# Increasing C parameter, allowing more flexibility

svm2 = SVC(C=100, kernel='rbf', gamma=1.0)
svm2.fit(X_train, y_train)
plot_svc(svm2, X_test, y_test)

However, this comes at the price of a more irregular decision boundary that seems to be at risk of
overfitting the data. We can perform cross-validation using GridSearchCV()

to select the best choice of γ

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

tuned_parameters = [{'C': [0.01, 0.1, 1, 10, 100],

'gamma': [0.5, 1,2,3,4]}]
clf = GridSearchCV(SVC(kernel='rbf'), tuned_parameters, cv=10,
scoring='accuracy')
clf.fit(X_train, y_train)
clf.best_params_

Therefore, the best choice of parameters involves cost=1

and gamma=0.5. We can plot the resulting fit using the plot_svc() function, and view the test
set predictions for this model by applying the predict()

function to the test data:

plot_svc(clf.best_estimator_, X_test, y_test)

print(confusion_matrix(y_test, clf.best_estimator_.predict(X_test)))
print(clf.best_estimator_.score(X_test, y_test))

85% of test observations are correctly classified by this SVM. Not bad!

3. ROC Curves

The auc()

function from the sklearn.metrics

package can be used to produce ROC curves such as those we saw in lecture:

from sklearn.metrics import auc

from sklearn.metrics import roc_curve

Let's start by fitting two models, one more flexible than the other:

# More constrained model

svm3 = SVC(C=1, kernel='rbf', gamma=1)
svm3.fit(X_train, y_train)
# More flexible model
svm4 = SVC(C=1, kernel='rbf', gamma=50)
svm4.fit(X_train, y_train)

SVMs and support vector classifiers output class labels for each observation. However, it is also
possible to obtain fitted values for each observation, which are the numerical scores used to obtain

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)
the class labels. For instance, in the case of a support vector classifier, the fitted value for an
observation X=(X1,X2,...,Xp)T

takes the form β^0+β^1X1+β^2X2+...+β^pXp

For an SVM with a non-linear kernel, the equation that yields the fitted value is given in (9.23) on
p. 352 of the ISLR book. In essence, the sign of the fitted value determines on which side of the
decision boundary the observation lies. Therefore, the relationship between the fitted value and the
class prediction for a given observation is simple: if the fitted value exceeds zero then the
observation is assigned to one class, and if it is less than zero than it is assigned to the other.

In order to obtain the fitted values for a given SVM model fit, we use the .decision_function()

method of the SVC:

y_train_score3 = svm3.decision_function(X_train)
y_train_score4 = svm4.decision_function(X_train)

Now we can produce the ROC plot to see how the models perform on both the training and the
test data:

y_train_score3 = svm3.decision_function(X_train)
y_train_score4 = svm4.decision_function(X_train)

false_pos_rate3, true_pos_rate3, _ = roc_curve(y_train, y_train_score3)

roc_auc3 = auc(false_pos_rate3, true_pos_rate3)

false_pos_rate4, true_pos_rate4, _ = roc_curve(y_train, y_train_score4)

roc_auc4 = auc(false_pos_rate4, true_pos_rate4)

fig, (ax1,ax2) = plt.subplots(1, 2, figsize=(14,6))

ax1.plot(false_pos_rate3, true_pos_rate3, label='SVM $\gamma = 1$ ROC curve
(area = %0.2f)' % roc_auc3, color='b')
ax1.plot(false_pos_rate4, true_pos_rate4, label='SVM $\gamma = 50$ ROC curve
(area = %0.2f)' % roc_auc4, color='r')
ax1.set_title('Training Data')

y_test_score3 = svm3.decision_function(X_test)
y_test_score4 = svm4.decision_function(X_test)

false_pos_rate3, true_pos_rate3, _ = roc_curve(y_test, y_test_score3)

roc_auc3 = auc(false_pos_rate3, true_pos_rate3)

false_pos_rate4, true_pos_rate4, _ = roc_curve(y_test, y_test_score4)

roc_auc4 = auc(false_pos_rate4, true_pos_rate4)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)
ax2.plot(false_pos_rate3, true_pos_rate3, label='SVM $\gamma = 1$ ROC curve
(area = %0.2f)' % roc_auc3, color='b')
ax2.plot(false_pos_rate4, true_pos_rate4, label='SVM $\gamma = 50$ ROC curve
(area = %0.2f)' % roc_auc4, color='r')
ax2.set_title('Test Data')

for ax in fig.axes:
ax.plot([0, 1], [0, 1], 'k--')
ax.set_xlim([-0.05, 1.0])
ax.set_ylim([0.0, 1.05])
ax.set_xlabel('False Positive Rate')
ax.set_ylabel('True Positive Rate')
ax.legend(loc="lower right")
4 SVM with Multiple Classes

If the response is a factor containing more than two levels, then the svm()

function will perform multi-class classification using the one-versus-one approach. We explore
that setting here by generating a third class of observations:

np.random.seed(8)
XX = np.vstack([X, np.random.randn(50,2)])
yy = np.hstack([y, np.repeat(0,50)])
XX[yy ==0] = XX[yy == 0] +4

plt.scatter(XX[:,0], XX[:,1], s=70, c=yy, cmap=plt.cm.prism)

plt.xlabel('XX1')
plt.ylabel('XX2')

Fitting an SVM to multiclass data uses identical syntax to fitting a simple two-class model:

svm5 = SVC(C=1, kernel='rbf')

svm5.fit(XX, yy)
plot_svc(svm5, XX, yy)
Application to Handwritten Letter Data

We now examine Optical Recognition of Handwritten Digits Data Set, which contains
5,620 samples of handwritten digits 0..9. You can use these links to download the training data
and test data, and then we'll load them into python:

X_train = pd.read_csv('optdigits.tra', header=None)

y_train = X_train[64]
X_train = X_train.drop(X_train.columns[64], axis=1)

X_test = pd.read_csv('optdigits.tes', header=None)

y_test = X_test[64]
X_test = X_test.drop(X_test.columns[64], axis=1)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

print(X_train.shape)
print(X_test.shape)

This data set consists of preprocessed images of handwriting samples gathered from 43 different
people. Each image was converted into an 8x8 matrix (64 pixels), which was then flattened into a
vector of 64 numeric values. The final column contains the class label for each digit.

The training and test sets consist of 3,823 and 1,797 observations respectively. Let's see what one
of these digits looks like:

plt.imshow(X_train.values[1].reshape(8,8), cmap="gray")
plt.show()

That's a pretty messy digit. Let's peek at the true class:

y_train[0]

Phew, looks like our SVM has its work cut out for it! Let's start with a linear kernel to see how we
do:

svc = SVC(kernel='linear')
svc.fit(X_train, y_train)

# Print a nice confusion matrix

cm = confusion_matrix(y_train, svc.predict(X_train))
cm_df = pd.DataFrame(cm.T, index=svc.classes_, columns=svc.classes_)
print(cm_df)

We see that there are no training errors. In fact, this is not surprising, because the large number
of variables relative to the number of observations implies that it is easy to find hyperplanes that
fully separate the classes. We are most interested not in the support vector classifier’s performance
on the training observations, but rather its performance on the test observations:

cm = confusion_matrix(y_test, svc.predict(X_test))
print(pd.DataFrame(cm.T, index=svc.classes_, columns=svc.classes_))

We see that using cost = 10 yields just 71 test set errors on this data. Now try using the
GridSearchCV() function to select an optimal value for c.
Consider values in the range 0.01 to 100:

# Your code here

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)
Aim: Generate frequent item sets using Apriori Algorithm in python and
also generate association rules for any market basket data.

Solution:
• The Apriori algorithm is a well-known Machine Learning algorithm used for association
rule learning.
• Association rule learning is taking a dataset and finding relationships between items in
the data. For example, if you have a dataset of grocery store items, you could use
association rule learning to find items that are often purchased together.
• The Apriori algorithm is used on frequent item sets to generate association rules and is
designed to work on the databases containing transactions.
• The process of generating association rules is called association rule mining or association
rule learning. We can use these association rules to measure how strongly or weakly two
objects from the dataset are related.
• Frequent itemsets are those whose support value exceeds the user-specified minimum
support value.
The most common problems that this algorithm helps to solve are:
• Product recommendation
• Market basket recommendation

There are three major parts of the Apriori algorithm.

• Support
• Confidence
• Lift
Support
Support of item I is the ratio of the number of transactions in which item I appears to the total
number of transactions.

Confidence
Measures how often items in Y appear in transactions that contain X

Lift
Lift describes how much confident we are if B will be purchased too when the customer buys A:

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Example:

Let’s imagine we have a history of 3000 customers’ transactions in our database, and we have to
calculate the Support, Confidence, and Lift to figure out how likely the customers who buy
Biscuits will buy Chocolate.

Here are some numbers from our dataset:

• 3000 customers’ transactions

• 400 out of 3000 transactions contain Biscuit purchases
• 600 out of 3000 transactions contain Chocolate purchases
• 200 out of 3000 transactions described purchases when customers bought Biscuits and Chocolates
together

the support value for biscuits will be:

the confidence value shows the probability that customers buy Chocolate if they buy Biscuits

To calculate this value, we need to divide the number of transactions that contain Biscuits and
Chocolates by the total number of transactions having Biscuits:

the Lift value shows the potential increase in the ratio of the sale of Chocolates when you sell
Biscuits. The larger the value of the lift, the better:

5 Apriori Algorithm steps

1. Start with itemsets containing just a single item (Individual items)
2. Determine the support for itemsets
3. Keep the itemsets that meet the minimum support threshold and remove itemsets that do
not support minimum support
4. Using the itemsets kept from Step 1, generate all the possible itemset combinations.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Example:

Let’s take a look at these steps while using a sample dataset:

First, the algorithm will create a table containing each item set’s support count in the given
dataset – the Candidate set

Let’s assume that we’ve set the minimum support value to 3, meaning the algorithm will drop
all the items with a support value of less than three.

The algorithm will take out all the itemsets with a greater support count than the minimum
support (frequent itemset) in the next step:

Next, the algorithm will generate the second candidate set (C2) with the help of the frequent
itemset (L1) from the previous calculation. The candidate set 2 (C2) will be formed by creating
the pairs of itemsets of L1. After creating new subsets, the algorithm will again find the support
count from the main transaction table of datasets by calculating how often these pairs have
occurred together in the given dataset.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

After that, the algorithm will compare the C2’s support count values with the minimum support
count (3), and the itemset with less support count will be eliminated from table C2.

Program to Generate frequent item sets using Apriori Algorithm in

python and also generate association rules for any market basket data
Download the dataset Market_Basket_Optimization.csv
Step-1: Load the datatset and perform preprocessing using TransactionEncoder class
Code:

Sample Output:

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Step-2: Using Apriori algorithm, generate frequent itemsets with min_support=0.01 (1%)
Code:

Sample Output:

Step-3: Add a column ‘length’ and store the length of each frequent itemset
Code:

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Sample Output:

Step-4: Find the 3-itemsets(length=3) from frequent itemsets with min_support>=0.015

(15%)
Code:

Sample Output:

Step-5: Generate Association rules for the frequent item sets of step-4 with
confidence=50%
Code:

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

From the above output, the rules generated with support>=15% and confidence=50% are:

{ground beef,eggs} {mineral water}

{ground beef,milk} {mineral water}

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Instructions
Name your files Task1.py, Task2.py and so on. Compress them in a single file and
name it as LabXX YourRollNumber. Submit this file at end of Lab.

Before starting, import the following libraries from python:

import numpy as np
from keras.datasets import mnist
import matplotlib.pyplot as plt
from sklearn.preprocessing import MinMaxScaler
from sklearn.svm import SVC
from sklearn import metrics
from skimage.feature import hog
from sklearn.decomposition import PCA
from mnist import MNIST

Task 1: SVM on raw data

Raw Data
The data that we will be using in this lab is MNIST data. This is a collection of images of
handwritten digits, 0-9, along with their labels. The purpose of these tasks will be to identify the
digit in each image and predict its label.

Classifier
In supervised learning, the data is split into two parts; training data and testing data. We use the
training data along with its labels, to train our model. For testing, we only input the testing data
and not the labels. We use our trained model to predict the labels of this testing data and compare
it with the actual labels to see how accurate our model is.
The classifier that we will be using is Support Vector Machine (SVM). Without getting into
much detail, SVM tries to learn the boundary between different classes. You can imagine data as
a scatter plot, with data points of each class lying together as a cluster in a 2D plane. Now imagine
drawing a boundary such that the boundary separates/isolates each class from each other in the
plane. SVM tries to learn that boundary and use it to classify data.
Use the following line of code to load the training and testing data. You should have 60,000
training images, each with dimensions 28x28, and labels. The testing data should contain 10,000
images and labels. Check it after you have loaded the data.
(x_train, y_train), (x_test, y_test) = mnist.load_data()
1. Flatten each image into a 1D vector. Your training images should be of the shape
(60000,28,28). After flattening each image, the data should be of the shape (60000,784). Do
the same with the testing images. Also convert their type to

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)
‘float32’.
2. We will now scale the data so that each value is between -1 and 1. Use the
following code for scaling your data:
scaling = MinMaxScaler(feature_range=(-1, 1)).fit(x_train)
x_train = scaling.transform(x_train) x_test =
scaling.transform(x_test)

3. Use SVM from the sklearn library. Use a linear kernel and train the model using training
data. After training it, use the model to predict labels for the testing data.
Once we have the predicted labels, we can make a confusion matrix. It is a matrix with predicted
labels on one axis and actual labels on the other. The number in cell indicates the number of times
the corresponding actual label was classified as the corresponding predicted label. The diagonal of
course represents the correct predictions while the off diagonal terms are wrong predictions.
4. Using the in built function, ‘metrics’, find the confusion matrix of actual and predicted
labels of the testing data. You can plot it using the following code (cm is the confusion matrix):
plt.figure()
plt.imshow(cm, interpolation=’nearest’, cmap=’Pastel1’)
plt.title(’Confusion matrix’, size = 15) plt.colorbar()
tick_marks = np.arange(10)
plt.xticks(tick_marks, ["0", "1", "2", "3", "4", "5", "6", "7", "8", "9"]
plt.yticks(tick_marks, ["0", "1", "2", "3", "4", "5", "6", "7", "8", "9"]
plt.tight_layout() plt.ylabel(’Actual label’, size = 15) plt.xlabel(’Predicted label’,
size = 15) width, height = cm.shape
for x in range(width):
for y in range(height):
plt.annotate(str(cm[x][y]), xy=(y, x),
horizontalalignment=’center’,
verticalalignment=’center’)

plt.show()

Using the confusion matrix, we can calculate different merits of our model.
Two such merits are Accuracy and False Positive Rate (FPR) :
Number of correct predictions
Accuracy =
Total number of predictions
which can be calculated from the confusion matrix as:

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)
Sum of diagonal of the matrix
Accuracy =
Sum of all values in the matrix
FPR for each class corresponds to the probability of a false alarm. It is when the classifier
detects the data as a specific class but the data does not belong to that class. Each class will
have its own FPR

Total number of wrong predictions of class i

FPRi =

Total number of predictions of class

i
which can be calculated from the confusion matrix as:

Sum over the row of true label i except the diagonal

FPRi =
Sum over the row of true label i
5. Use the confusion matrix to calculate FPR for each class and overall accuracy
of the model.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)
Task 2: SVM with PCA
1. Load the MNIST data and reshape it and cast it to float32 as you did in Task
2. Use the following code to extract the principle components of data and then projects your
image along those components. The variable n components determines how many
components you wish to consider.
n_components=2 pca=PCA(n_components)

# Transform data pca.fit(x_train) x_train=pca.transform(x_train)

x_test=pca.transform(x_test)
P.S. Look at the shape of training and testing data now. The second dimension corresponds
to the number of components.
3. Scale the data and use SVM to predict labels as you did in Task 1.
4. Compute the confusion matrix along with the accuracy and FPR for each class.
Your accuracy might be less than the one you got while using raw data. This is because
perhaps the number of components you are using are not enough.
5. Perform this analysis again but for the following number of components and compute the
accuracy in each case:

• components = 5

• components = 11

• components = 44
P.S. You can compute the accuracy using the command metrics.accuracy score(y test, y
predict) where y predict are the predicted labels.
6. Comment on the time taken for classification using PCA as compared to using raw data.
Why do you think there is a difference? Why does increasing the number of components
leads to improved accuracy?

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Task 3: Apriori Algorithm

1. Implement apriori algorithm techniques as discussed in the example.

Conclusion:
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)
LAB 12: Genetic Algorithm
Objective:
To perform genetic algorithm and determine the outcome of the algorithm performance.

Background and Procedure:

Genetic Algorithm

The genetic algorithm is a search-based optimization technique. It is frequently used to find the
optimal or nearest optimal solution
In genetic we will use some biological terms such as population, chromosome, gene, selection,
crossover, mutation. Now, first of all, let‘s try to understand what these terms mean.

Population, Chromosome, Gene

At the beginning of this process, we need to initialize some possible solutions to this problem. The
population is a subset of all possible solutions to the given problem. In another way, we can say
that the population is a set of chromosomes. A chromosome is one of that solution to that current
problem. And each chromosome is a set of genes.
For simplicity, We can describe a chromosome as a string. So, we can say that a population is a
collection of some string(each character is a binary value, either 0 or 1 ). And each character of
the string is a gene.

For starting the process of the genetic algorithm, we first need to initialize the population. We can
initialize the population in two ways. The first one is random and the second one is heuristical. It
is always better to start the algorithm with some random population.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)
Fitness Function
After initializing the population, we need to calculate the fitness value of these chromosomes.
Now the question is what this fitness function is and how it calculates the fitness value.
As an example, let consider that we have an equation, f(x) = -x² + 5 .We need the
solution for which it has the maximum value and the constraint is 0≤x≤31.

Now, let consider that we have a random population of four chromosomes like below. The
length of our chromosome is 5 as 2⁵=32 and 0≤x≤31.

Our fitness function will calculate the functional value of each chromosome as stated in the
problem statement :
For the 1st chromosome, 01110 means 14 in integer. So, f(x) = -(14*14) + 5 = -191.
For the 2nd chromosome, 10101 means 21 in integer. So, f(x) = -(21*21) + 5 = -436.
For the 3rd chromosome, 00011 means 3 in integer. So, f(x) = -(3*3) + 5 = -4.
For the 4th chromosome, 10111 means 23 in integer. So, f(x) = -(23*23) + 5 = -524.

Parent Selection
Parent selection is done by using the fitness values of the chromosomes calculated by the fitness
function. Based on these fitness values we need to select a pair chromosomes with the highest
fitness value.
There are many ways for fitness calculation like Roulette wheel selection, rank selection. In
Roulette wheel selection, the chromosome with the highest fitness value has the maximum
possibility to be selected as a parent. But in this selection process, a lower can be selected. In rank
selection, chromosomes are ranked based on their fitness values from higher to lower. As an
example, According to those fitness values calculated above, we can rank those chromosomes
from higher to lower like 3rd>1st>2nd>4th. So, in the selection phase, 3rd and 1st chromosomes
will be selected based on the fitness valued calculated from the fitness function.

Crossover

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)
Crossover is used to vary the programming of the chromosomes from one generation to another
by creating children or offsprings. Parent chromosomes are used to create these
offsprings(generated chromosomes).
To create offsprings, there are some ways like a single-point crossover, two or multi-point
crossover.
For a single point crossover, first, we need to select a point and then exchange these portions
divided by this point between parent chromosomes to create offsprings. You can use the color
combination for easy understanding.

For a two-point crossover, we need to select two points and then exchange the bits.

Finally, these new offsprings are added to the population.

Mutation
Mutation brings diversity to the population. There are different kinds of mutations like Bit Flip
mutation, Swap mutation, Inversion mutation, etc. These are so so simple.
In Bit Flip mutation, Just select one or more bits and then flip them. If the selected bit is 0 then
turn it to 1 and if the selected bit is 1 then turn it to 0.

In Swap Bit mutation, select two bits and just swap them.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

In inverse mutation, just inverse the bits.

Implementation Of Genetic Algorithm In Python

Let‘s try to implement the genetic algorithm in python for function optimization.

Problem Statement

Let consider that we have an equation, f(x) = -x² + 5 . We need the solution for which it has
the maximum value and the constraint is 0≤x≤31. To select an initial population use the
probability 0.2.

Initial Population

Random Initialization is better than heuristic initialization. Therefore, here random

initialization is used for population initialization.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Fitness Function

The fitness function calculates the fitness value of chromosomes. The functionality of the
fitness function depends on the problem‘s requirements.

Selection

Fittest chromosomes are selected based on the fitness scores. Here rank selection process is
used.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

After selecting the fittest parents, a crossover is required to generate offsprings. Here, the
singlepoint crossover is used.

Mutation

After crossover is done, a mutation is done for maintaining the diversity from one generation
to another. Here, we will single point bit-flip mutation.

We need to iterate the whole process multiple times until we find our best solution.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

2. Implement genetic algorithm performing all steps as applied in the coding

example.

3. What is mutation and fitness in genetic algorithm?

________________________________________________________________________
___
________________________________________________________________________
___
________________________________________________________________________
___
______________________________________________________________________
_____ _______________________________________________________________
Conclusion:

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)
LAB 13: Search Algorithm
Objective:
To perform depth first search, breath first search and classical AI algorithm techniques.

Background and Procedure:

Breadh First Search

One of the foundation search algorithms called Blind Search.
The search algorithm is an algorithm to retrieve information stored within some data structure, or
calculated in the search space of a problem domain.

Breadth-first search is an algorithm for traversing or searching tree or graph data structures. It
starts at the root node and explores all nodes at the present depth before moving on to the nodes at
the next depth level.

In other words, it expands the shallowest unexpanded node which can be implemented by a First-
In-First-Out (FIFO) queue. Let‘s go through an example with the following GRAPH:

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

From the above step by step expansion diagram, we can see that the BFS algorithm prioritizes
edges closest to the starting vertex to perform a search.

Now, let‘s evaluate this algorithm:

Denote d as the depth of the least-cost solution and b as the maximum branching factor of the
search tree or graph. Assume a hypothetical state space where every node can be expanded into b
new nodes, solution of path-length d:

Time Complexity: How long does it take to find a solution?

1 + b + b² + b³ + …. + bᵈ = O(bᵈ)
Space Complexity: Maximum number of nodes in memory
Keeps every node in memory = O(bᵈ)
Completeness: Does it always find a solution if one exists? Yes
Optimality: Does it always find the best (least-cost) solution? Yes, when all steps cost equally.

The diagram is a schematic representation of the graph with vertices

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)
V={A, B, C, D, E, F, G, H, I, J, K, L, M}, E={{A, B}, {A, C}, {B, D}, {B, E}, {C, F}, {C,
G}, {D, H}, {D, I}, {E, J}, {E, K}, {F, L}, {F, M}}

BFS Implementation

Creating the function that takes in edges of the GRAPH which outputs the adjacency list for
undirected graph

Creating the function that takes in the adjacency list and starting vertex which output the sequence
of BFS search

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Depth First Search

Depth-first search is an algorithm for traversing or searching tree or graph data structures
[2]. Before explaining the DFS algorithm, let‘s introduce the graph data structure. A graph
G is a pair (V, E), where V is a finite set and E is a set of binary relations on V.

V is called the vertex set and its elements are vertices.

E is called the edge set and its elements are called edges. An edge is represented by a pair
of vertices.

The diagram is a schematic representation of the graph with vertices V={1, 2, 3, 4, 5, 6}, E={{1,
2},{1, 5},{2, 3},{2, 5},{3, 4},{4, 5},{4, 6}}

It starts at the root node and expands the deepest unexpanded node, backtracks only when no more
expansion. Let‘s go through an example with the following GRAPH:

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

From the above step by step expansion diagram, we can see that the DFS algorithm
prioritizes edges along the path to perform a search.

Now, let‘s evaluate this algorithm:

Denote m as the maximum depth of the state space and b as the maximum branching factor
of the search tree or graph

Completeness:
Infinite-depth spaces: No
Finite-depth spaces with loops: No
Finite-depth spaces with repeated-state checking: Yes
Finite-depth spaces without loops: Yes
Time Complexity: O(bᵐ)
Space Complexity: O(bm) Optimality:
No

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

The diagram is a schematic representation of the graph with vertices

V={A, B, C, D, E, F, G, H, I, J, K, L, M}, E={{A, B}, {A, C}, {B, D}, {B, E}, {C, F}, {C,
G}, {D, H}, {D, I}, {E, J}, {E, K}, {F, L}, {F, M}}

DFS Implementation

Creating the function that takes in edges of the GRAPH which outputs the adjacency list for
undirected graph.

Creating the function that takes in the adjacency list and starting vertex which output the
sequence of DFS search.

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan
Artificial Intelligence BS (AI)
Shaheed Zulfikar Ali Bhutto Institute of Science and Technology
(SZABIST)

Lab Exercises:

1. Implement search algorithm techniques BFS and DFS as discussed in the example.

Conclusion:

Shaheed Zulfikar Ali Bhutto Institute of Science and Technology (SZABIST)

Plot 67 Street 9, H-8/4 Karachi Campus: 99 3rd Ave, Block 5 Clifton Karachi City, Pakistan

AI Syllabus
No ratings yet
AI Syllabus
3 pages
1ET1080302 - Introduction To Artificial Intelligence and Machine Learning
No ratings yet
1ET1080302 - Introduction To Artificial Intelligence and Machine Learning
2 pages
AI LAB Manual R18 2020-21 10.02.2021
No ratings yet
AI LAB Manual R18 2020-21 10.02.2021
65 pages
AI-syllabus Class 11
No ratings yet
AI-syllabus Class 11
7 pages
Cse 1 Syll
No ratings yet
Cse 1 Syll
17 pages
Arti 1 Syll
No ratings yet
Arti 1 Syll
20 pages
Ai Cho
No ratings yet
Ai Cho
7 pages
315330-Ai & ML Algorithm
0% (1)
315330-Ai & ML Algorithm
7 pages
Csc-4101Artificial Intelligence: Course Information Sheet
No ratings yet
Csc-4101Artificial Intelligence: Course Information Sheet
3 pages
Artificial Intelligence BSC - Csit Syllabus IT College Nepal
No ratings yet
Artificial Intelligence BSC - Csit Syllabus IT College Nepal
3 pages
AI&ML
No ratings yet
AI&ML
3 pages
Ai 3
No ratings yet
Ai 3
54 pages
OCS351 - AI ML Fundamentals Syllabus
100% (1)
OCS351 - AI ML Fundamentals Syllabus
2 pages
19cs413 Artificial Intelligence
No ratings yet
19cs413 Artificial Intelligence
3 pages
Artificial Intelligence PDF
No ratings yet
Artificial Intelligence PDF
4 pages
AI MCAN2320 Coursepack
No ratings yet
AI MCAN2320 Coursepack
7 pages
Sec D EE 490 Artifical Intelligence
No ratings yet
Sec D EE 490 Artifical Intelligence
9 pages
2 III BTech Minor AI&ML Courses Syllabus
No ratings yet
2 III BTech Minor AI&ML Courses Syllabus
4 pages
L T P Credits Artificial Intelligence Lab - 2 1: Identifying Problems and Their AI Solutions
No ratings yet
L T P Credits Artificial Intelligence Lab - 2 1: Identifying Problems and Their AI Solutions
4 pages
COMP3411 Slides All Term
No ratings yet
COMP3411 Slides All Term
23 pages
AI Course Overview for Students
No ratings yet
AI Course Overview for Students
3 pages
18CS71-AI&ML Notes
100% (1)
18CS71-AI&ML Notes
159 pages
ACAD01
No ratings yet
ACAD01
2 pages
4-AI MCAN2320 Ajay Shiknder
No ratings yet
4-AI MCAN2320 Ajay Shiknder
9 pages
Syllabus
No ratings yet
Syllabus
3 pages
AI Lab Course for CS Students
100% (1)
AI Lab Course for CS Students
4 pages
18CSE305J AI Syllabus
No ratings yet
18CSE305J AI Syllabus
2 pages
Vikram Mooc Report
No ratings yet
Vikram Mooc Report
18 pages
Introduction To Artificial Intelligence
No ratings yet
Introduction To Artificial Intelligence
5 pages
AI Course for Computer Science Majors
No ratings yet
AI Course for Computer Science Majors
2 pages
Ai File
No ratings yet
Ai File
36 pages
Artificial and Computational Intelligence
No ratings yet
Artificial and Computational Intelligence
4 pages
CourseOutline For AI - CSEg3206
No ratings yet
CourseOutline For AI - CSEg3206
7 pages
CSA2001 - FUNDAMENTALS-IN-AI-AND-ML - LTP - 1.1 - 34 - CSA2001 - Fundamentals in AI and ML - LTP 4 - V1.1
No ratings yet
CSA2001 - FUNDAMENTALS-IN-AI-AND-ML - LTP - 1.1 - 34 - CSA2001 - Fundamentals in AI and ML - LTP 4 - V1.1
3 pages
JNTU CSE AI & ML Syllabus
No ratings yet
JNTU CSE AI & ML Syllabus
10 pages
Artificial Intelligence and Machine Learning
No ratings yet
Artificial Intelligence and Machine Learning
3 pages
Data Science
No ratings yet
Data Science
3 pages
Circular 20240429201652 Cho Ai and ML 22cs015 v2
No ratings yet
Circular 20240429201652 Cho Ai and ML 22cs015 v2
8 pages
Aiml
No ratings yet
Aiml
8 pages
Ai Outline
No ratings yet
Ai Outline
2 pages
1-Zeroth-Hour, Course Syllabus & Discussion-04-01-2024
No ratings yet
1-Zeroth-Hour, Course Syllabus & Discussion-04-01-2024
3 pages
AIML
No ratings yet
AIML
17 pages
AI Syllbus
No ratings yet
AI Syllbus
5 pages
23it21t1 - Artificial Intelligence Lab
No ratings yet
23it21t1 - Artificial Intelligence Lab
2 pages
21CS54 Module 1
100% (2)
21CS54 Module 1
35 pages
315330-Ai & ML Algorithm
No ratings yet
315330-Ai & ML Algorithm
7 pages
CSE-AI Third Year
No ratings yet
CSE-AI Third Year
3 pages
Sem 6
No ratings yet
Sem 6
18 pages
CSE3002 - ARTIFICIAL-INTELLIGENCE - ETH - 2.0 - 11 - CSE3002 - Artificial Intelligence - Revised - 2.0 (Done)
No ratings yet
CSE3002 - ARTIFICIAL-INTELLIGENCE - ETH - 2.0 - 11 - CSE3002 - Artificial Intelligence - Revised - 2.0 (Done)
3 pages
Lecture 1 - Intro
No ratings yet
Lecture 1 - Intro
31 pages
Ai Unit 3 Digital Notes
No ratings yet
Ai Unit 3 Digital Notes
99 pages
Instructor
No ratings yet
Instructor
3 pages
Intro 2 To AI
No ratings yet
Intro 2 To AI
2 pages
AI (AL-304) Lab Manual
No ratings yet
AI (AL-304) Lab Manual
37 pages
Sorghum Varieties for Ethiopian Farmers
No ratings yet
Sorghum Varieties for Ethiopian Farmers
2 pages
EBOOK - Netguru - Culture Book PDF
No ratings yet
EBOOK - Netguru - Culture Book PDF
50 pages
Pacific Ring of Fire Overview
No ratings yet
Pacific Ring of Fire Overview
49 pages
Technology Globalization and Ethics
No ratings yet
Technology Globalization and Ethics
30 pages
Study Material Accountancy Class 11TH 2024-25
No ratings yet
Study Material Accountancy Class 11TH 2024-25
134 pages
Blood Tests Report11
No ratings yet
Blood Tests Report11
8 pages
Top 10 Erp
No ratings yet
Top 10 Erp
86 pages
Physical Maps Latin America
No ratings yet
Physical Maps Latin America
4 pages
A35 Italia44 ENG
No ratings yet
A35 Italia44 ENG
8 pages
Library Management System Report IPT
No ratings yet
Library Management System Report IPT
6 pages
Conscious Oracle Card Booklet
No ratings yet
Conscious Oracle Card Booklet
44 pages
Public System Management Barun Kumar de Instant Download
No ratings yet
Public System Management Barun Kumar de Instant Download
82 pages
Tle10 Eim10 Q1 M5
No ratings yet
Tle10 Eim10 Q1 M5
12 pages
UU7PNM
No ratings yet
UU7PNM
2 pages
Reframing Organizations Intergrating The Four Frames
No ratings yet
Reframing Organizations Intergrating The Four Frames
10 pages
GLOBE - LATITUDE AND LONGITUDE (Key Notes)
No ratings yet
GLOBE - LATITUDE AND LONGITUDE (Key Notes)
2 pages
Keratograph 5m en PDF
No ratings yet
Keratograph 5m en PDF
16 pages
AscTec Falcon 8-User Manual V5 EU
No ratings yet
AscTec Falcon 8-User Manual V5 EU
158 pages
Semiconductor Material
No ratings yet
Semiconductor Material
55 pages
2022年港澳杯初赛 P1
No ratings yet
2022年港澳杯初赛 P1
4 pages
Financial Accounting and Reporting - QUIZ 5
100% (1)
Financial Accounting and Reporting - QUIZ 5
4 pages
China's Diverse Language Dialects
No ratings yet
China's Diverse Language Dialects
4 pages
Miguel Wiels
No ratings yet
Miguel Wiels
1 page
Highwoods Primary School Overview
No ratings yet
Highwoods Primary School Overview
18 pages
Prokon Course Outline 2 2021
No ratings yet
Prokon Course Outline 2 2021
3 pages
Technical Sales Cover Letter Guide
100% (2)
Technical Sales Cover Letter Guide
5 pages
Marketing Flexibility - Significance and Implications For Automobile Industry
No ratings yet
Marketing Flexibility - Significance and Implications For Automobile Industry
12 pages
Lesson Plan Surviving A Disaster
No ratings yet
Lesson Plan Surviving A Disaster
6 pages
Unit 4 SportsReporting
No ratings yet
Unit 4 SportsReporting
22 pages
Kyecu-Cocoa - 26.05.2025
No ratings yet
Kyecu-Cocoa - 26.05.2025
4 pages