Artificial Intelligence

Syllabus

5CSE

https://drive.google.com/file/d/1OA6ic4Pt2W5nBvD0hjqY_DGal_0rLH-D/view?usp=sharing

UNIT 1

Course outcomes and Syllabus

What is Intelligence ?

Capacity for logic, understanding, self-awareness, learning, emotional knowledge, reasoning, planning and problem solving. More generally it can be described as the ability to perceive information, and to retain it as knowledge to be applied towards behaviors within an environment.

Intelligence is a property of mind that encompasses many related mental abilities, such as the capabilities to

reason
plan
solve problems
think abstractly
comprehend ideas and
language and learn

What is Artificial Intelligence ?

Artificial intelligence refers to the simulation of human intelligence in machines. The goals of artificial intelligence include learning, reasoning, and perception.

Formal definition of AI

AI is a branch of computer science which is concerned with the study and creation of computer systems that exhibit some form of intelligence OR those characteristics which we associate with intelligence in human behavior.

AI is a broad area consisting of different fields, from machine vision, expert systems to the creation of machines that can "think". In order to classify machines as "thinking", it is necessary to define intelligence.

Categories and Foundation of AI

Categories of AI Systems

Systems that think like Humans

Black box. Thought process not clear

Area of Cognitive Science

Stimuli —> mental representation

Neural Network

Systems that act like Humans

Turing test to see if AI behaves like human. 1950.

Overall behaviour of system should be human like

Could be achieved by observation

Chatbots etc.

Systems that think rationally/logically

Logic or laws of thoughts

Logic formulas, prepositional logic are used for synthesizing outcomes

Not all intelligent behaviour are mediated by logical deliberation

Systems that act rationally/logically

agent

Rational behaviour is doing right thing

Even if method is illogical, observed behaviour must be rational .

Foundations of AI

Philosophy

Where does knowledge come from?

How does knowledge lead to action?

Mathematics

Formal logic methods

Boolean logic
Fuzzy logic

Uncertainity

Basis for modern approach to handle uncertainity can be handled by

Probability Theory
Modal and Temporal Logics

Economics

Decisions to maximize payoff

Decision to take if payoff far away

Optimizing certain values

Neuroscience

How brain works?

1824 studies on abnormal people
Recent studies accurate sensors to measure brain waves
Parallel computing, remapping, interconections, ANN, NNN

Control Theory

Machine modify behaviour in response to action

Water flow regulator
Steam engine generator

Theory of stable feedback systems

Transition from initial state to goal state with minimum energy
Control theory could only describe linear systems

Linguistics

How does language relate to thought

Speech depicts human intelligence

Children make sentences they never heard before

Language and thought go hand in hand

Computer Engineering

Building efficient computers

Application specific

Quantum computing

Psychology

How do humans think and act

History, Sub-areas and applications of AI

History

1943 - Artificial Neurons - Warren & Walter

1949 - Hebbian Learning - Donald Hebb

1950 - Turing Test - Alan Turing

1955 - Logic Theorist - Allen & Helbert

1956 - John McCarthy - AI made an academic field

High level computer languages COBOL FORTRAN LISP created

1966 - ELIZA (first chatbot) - Joseph

1972 - Intelligent Humanoid Robot WABOT-1 - made in Japan

1980 - Expert Systems - Emulate decision making ability of human

First National conference of AI at stanford

1997 - IBM Deep Blue beats world chess champion

2002 - Roomba - AI enters home

2006 - FAANG - AI enters business

2011 - IBM Watson wins quiz show - NLP

2012 - Google Now - predict according to user info

2014 - Eugene Goostman chatbot - came very close to pass turing test

Sub-areas of AI

Neural Networks

Inspired by human brain

Vision

Machine Learning

Speech Recognition

Robotics

Applications

Business - Financial Strategies

Engineering - Design, offer strategy

Manufacturing - Assembly, inspection

Mining

Medical

Education

Housing

Farming

Stucture of Agents and Problem Solving

Agent

An agent perceives its environment via sensores and acts upon that environment through its actuators

AI can be defined as a study of rational agents

Rational Agent

An agent could be anything which makes decisions like a person, firm or software. It carries out an action with the best outcome after considering past and current percepts(agent’s perceptual inputs at a given instance)

An AI System is composed of two main components :

Agent

Environment

Structure of Agents

Agent = Architecture + Agent Program

Architecture : the machinery that the agent executes on. It is a device with sensors and actuators, for example : a robotic car

Agent Program : Agent program is an implementation of an agent function. An agent function is a map from the percept sequence(history of all that an agent has perceived till date) to an action.

Examples

A software agent has Keystrokes, file contents, received network packages which act as sensors and displays on the screen, files, sent network packets acting as actuators.

A Human agent has eyes, ears, and other organs which act as sensors and hands, legs, mouth, and other body parts acting as actuators.

A Robotic agent has Cameras and infrared range finders which act as sensors and various motors acting as actuators.

Types of Environment

Fully Observable : Agent's sensors give it access to complete state of environment

Partially Observable : Either noisy/inaccurate sensors or parts of the state are missing from sensor data.

Defining Task Environment

PEAS

Ex : Self driving car

Performance Measure : Safe, Fast, Legal

Environment : Roads, other traffic

Actuators : steering wheel, brake, signal

Sensors : Cameras, LIDAR, GPS, odometer etc.

Types of Agents

Simple Reflex Agents

Act only on current percept. Ignore past percept.

Agent functions is based action-condition rule

A condition-action rule is a rule that maps a state i.e, condition to an action. If the condition is true, then the action is taken, else not.

Works only in fully observable environment

Cons

Limited Intelligence

No knowledge of non-perceptual parts of state

Usually too big to generate and score

If any update in environment, rules have to be changed

Model-based Agents

Works by finding a rule whose condition matches the current situation

Can work in partially-observable environment by use of model about the world

Agent keeps track of internal state adjusted by each percept and Depends on percept history

Two very important parts

Model : knowledge about "how things happen in the world"
Internal State: It is a representation of the current state based on percept history

These agents have the model, "which is knowledge of the world" and based on the model they perform actions.

Updating agent state requires information about

How the world evolves
How the agent actions effect the world

Goal-based Agents

Expand on the capabilities of a Model-based agent by having the goal information

The choose actions so that they can achieve the goal

Agents may have to consider a long sequence of possible actions before deciding whether the goal is achieved or not. Such considerations of different scenario are called searching and planning, which makes an agent proactive.

Utility-based Agents

Expand on the capabilities of a Goal-based agent by providing an extra component of Utility Measurement which provides measure of success for every state

They act based not only on goals, but also strive to obtain the best path of reaching the goal

Useful when there are multiple alternatives of reaching goal

Maps each state to a real number to check efficiency of action in achieving goal or 'success' rate

Learning Agent

Agent which can learn from it's past experiences

Starts with basic knowledge and then able to act and adapt automatically by learning

Four main components

Learning Element : Makes improvements by learning from environment

Critic : Learning Element takes feedback from critic to know how well it's doing compared to a standard performance

Performance Element : Responsible for selecting external action

Problem Generator : Responsible for suggesting actions that will lead to new/informative experiences

Formulating or Defining a Problem

To solve a problem defining it clearly is very important

A problem can be defined formally by five components

PITAG

Initial State

The state where agents start to perform the search to reach the goal state

Actions

Set of applicable actions to perform in state S

Transition Model

What each actions does. Transition each actions does.

Goal Test

This determines whether a given state is Goal Test or not.

Path Cost

Assigns numeric cost to each path

Problem Types

Single - State Problem

Observable

Deterministic

static

discrete

Multi - State Problem

Partially observable

Deterministic

static

discrete

Contingency Problem

Partially Observable

Non - Deterministic

State Space

No.of states the problem can go

Why is it needed ?

Analyzing problem
Representing the problem precisely

Together the initial state, actions, transition model implicitly define the state space of the problem. Set of all states reachable from initial state by any sequence of actions.

When graphically represented, state space forms directed graph in which nodes are states and edges are actions

A path is thus a sequence of state and sequence of actions taken to reach those states

Process of removing detail from a representation is called Abstraction

What is Heuristic ?

Technique designed to solve a problem quickly

Used to convert NP problem to P. [ NP = Non-polynomial ]

Gives good solution but does not guarentee optimal

Problems on State Space Representation

The 8 - Puzzle

We are given Initial State and Goal State

States : Individual Tiles

Actions : Move Up, Down, Left or Right

Goal Test := Compare with goal state

Path Cost : 1 per move

8 Puzzle has 9!/2 = 181440 states

There are two ways of solving this problem :

Uninformed Search ( Breadth First Search or DFS )

Just do a normal BFS where ALL possible states are searched and most optimal one is returned

Time Complexity : O ( BranchingFactor ^ DepthFactor ) for this problem its [ 3 ^ 20 ]

Informed / Heuristic Search ( Best First Search , A* Algo )

Find Heuristic Value ( R )

This value will consist of misplaced tiles when current state (initial state here) is compared to goal state [which is 3 in our case here]

Start Exploring all legal moves (up,down,left,right while in boudary)

Find Heuristic Value of all Legal Moves

Continue exploring with minimum heuristic value since we want ideally 0 misplaced tiles

Search Algorithms and Strategies

Search Strategies

Adversial Search

Evaluation Function

An evaluation function

also known as

heuristic evaluation function

static evaluation function

is a function used by game-playing computer programs to estimate the value or goodness of a position (usually at a leaf or terminal node) in a game tree.

A tree of such evaluations is usually part of a minimax or related search paradigm which returns a particular node and its evaluation as a result of alternately selecting the most favorable move for the side on move at each ply of the game tree.

The value is a quantized scalar,

The value is presumed to represent the relative probability of winning if the game tree were expanded from that node to the end of the game

Write the python code instead of illegible pseudo code

Mini - Max Algo Code [Python]

# A simple Python3 program to find
# maximum score that
# maximizing player can get
import math

def minimax (
							curDepth, 
							nodeIndex,
							maxTurn, 
							scores,
							targetDepth
						):

	# base case : targetDepth reached
	if (curDepth == targetDepth):
		return scores[nodeIndex]
	
	if (maxTurn):
		return max(
								minimax(
													curDepth + 1, 
													nodeIndex * 2,
													False, 
													scores, 
													targetDepth
												),
								minimax(
													curDepth + 1, 
													nodeIndex * 2 + 1,
													False, 
													scores, 
													targetDepth
												)
							)
	
	else:
		return min(minimax(curDepth + 1, nodeIndex * 2,
					True, scores, targetDepth),
				minimax(curDepth + 1, nodeIndex * 2 + 1,
					True, scores, targetDepth))
	
# Driver code
scores = [3, 5, 2, 9, 12, 5, 23, 23]

treeDepth = math.log(len(scores), 2)

print("The optimal value is : ", end = "")
print(minimax(0, 0, True, scores, treeDepth))

# This code is contributed
# by rootshadow

[OPTIMIZED] Mini - Max Algo with alpha-beta pruning [Pseudo Code]

The moment we encounter a Beta(Minimizer) which is less than Alpha (Maximizer), we automatically prune it because Apha won't chose anything less than or equal to it simply because it is a MAXIMIZER

function minimax(node, depth, isMaximizingPlayer, alpha, beta):

    if node is a leaf node :
        return value of the node
    
    if isMaximizingPlayer :
        bestVal = -INFINITY 
        for each child node :
            value = minimax(node, depth+1, false, alpha, beta)
            bestVal = max( bestVal, value) 
            alpha = max( alpha, bestVal)
            if beta <= alpha:
                break
        return bestVal

    else :
        bestVal = +INFINITY 
        for each child node :
            value = minimax(node, depth+1, true, alpha, beta)
            bestVal = min( bestVal, value) 
            beta = min( beta, bestVal)
            if beta <= alpha:
                break
        return bestVal

UNIT 2 skip ?

UNIT 3

Expert Systems and its Applications

Side Headings :

Introduction

Expert System is a computer program

Designed to provide Decision Making Ability (like a human expert)

Provides this ability by extracting knowledge from its knowledge base (using reasoning, inference rules, facts and heuristics (just like a human) on the User Query/input )

First system built in 1970

These systems are designed for specific domains such as medicine, law etc.

Performance of the system depends on quality/quantity of knowledge base

Suggestion of correct spelling in google is a well-known example of its use

Popular Examples

DENDRAL

Chemical Analysis Expert System

Used in organic chemistry to detect unknown organic molecules with the help of their mass spectra and knowledge base of chemistry.

MYCIN

Find bacteria causing infections like bacteraemia and meningitis

also used for the recommendation of antibiotics and the diagnosis of blood clotting diseases.

PXDES

determine the type and level of lung cancer.

Takes picture which identifies the type and degree of harm using knowledge base

CaDeT

Diagnostic support system that can detect cancer at early stages

Characteristics

1. High Performance

The expert system provides high performance for solving any type of complex problem of a specific domain with high efficiency and accuracy.

2. Understandable

It responds in a way that can be easily understandable by the user. It can take input in human language and provides the output in the same way.

3. Reliable

It is much reliable for generating an efficient and accurate output.

4. Highly Responsive

ES provides the result for any complex query within a very short period of time.

Components

User Interface

Interface helps a non-expert user to communicate with the expert system to find a solution.

Responsibilities

Accepting query in readable format

Sanitization of Query

Displays output to user

Inference Engine

Brain of the Expert System as it is the main processing unit of the system

⭐ It applies Inference rules to the knowledge base to extract an answer or conclusion

Types of Inference Engines

Deterministic Inference engine

Conclusions drawn are assumed to be true.

Based on facts and rules.

Probabilistic Inference engine

Conclusions contain uncertainity

Conclusions are based on probability

Modes of Deriving Solution

Forward Chaining

Starts with known facts and rules, applies inference rules to get conclusion and spits output.

Backward Chaining

Backward reasoning method

Starts from the goal and works backward to prove the known facts.

Knowledge Base

Type of storage that stores knowledge acquired from the different experts of the particular domain.

It can also be viewed as collection of Objects and their Attributes

Ex : Lion (object) is a Mammal (Attribute)

Components of Knowledge Base

Factual Knowledge

Based on facts and accepted by knowledge engineers

Heuristic Knowledge

Based on practice, the ability to guess, evaluation, and experiences.

Resources

https://www.javatpoint.com/expert-systems-in-artificial-intelligence

Phases in Building

Participants in Development

Domain Experts

The success of an ES much depends on the knowledge provided by human experts. These experts are those persons who are specialized in that specific domain.

Knowledge Engineers

Knowledge engineer is the person who gathers the knowledge from the domain experts and then encodes that knowledge to the system according to the formalism.

End - Users

This is a particular person or a group of people who may not be experts, and working on the expert system needs the solution or advice for his queries, which are complex.

Short - Explanation

There are five phases of processing of Expert system, such as-

Determine Problem Domain

Expert system must be bearing the problems which are solved by it.

Assign the domain expert for that ES.

Then, to build expert system that is cost- effective.

Design System

To build the high degree of integration this is able for another system and database.

Realize how the concepts can represent the domain knowledge best.

Build Prototype

To complete this phase, need knowledge expert who is able to do these tasks, such as-

Need domain knowledge from expert side.

And, then to represent this system into If-THEN-ELSE rules.

Testing the Prototype

End users allow testing the build prototypes of Expert system.

Development of Production Grade System

After testing, ensure that all interactions of expert system along with their elements of its environment, end users, all databases, and another system as well.

To keep all documents of the expert system assignment

Maintain

To update of ES continuously up to date.

Resources

http://digitalthinkerhelp.com/expert-system-in-artificial-intelligence-with-applications-examples-types-uses/

Long - Explanation

Identification [ Understand Problem ]

Precisely describe :

Knowedge Domain
Exact nature of the problem which system is intended to solve
State the precise goals that indicate exactly how system will contribute to solution

Iterative Procedure of improving Problem description in Engineer's head

Knowledge Engineer gains familiarity with that domain

Domain Expert describes several common problem states

Knowledge Engineer tries to extract fundamental concepts for those problem states

Domain Expert suggests changes ( Break; if description perfect. Fire ; if infinite loop )

Back to Square 1

Questions to ask

Is one Domain Expert enough ?

Should we consult more resources (books, research papers, scientists etc.) ?

Is a single engineer enough ?

Any deadlines ?

Conceptualization [ Conceptualize Problem ]

Purpose :

Analyse problem further to ensure specifics and generalities understood

Iterative procedure of conceptualizing problem in Engineer's head

Knowledge engineer draws out relation between objects and processes in problem domain

Divides problem into sub-problems

Create proper relationships between those sub-problems

Communicate with Domain Expert and make sure relations, objects and processes are as intended

Loop again if not ended

Note : If any weaknesses of previous stages are discovered, we go back to that stage add it and start again from that stage. Classic iterative procedure.

Formalization [ Problem is Connected to Proposed Solution]

Most interactive and time consuming stage

Not unusual for this stage to last for several years

Expert system is supplied by analyzing the relationships depicted in the last stage (conceptualization stage).

The knowledge engineer begins to select the techniques which are appropriate for developing this particular expert system.

Knowledge Engineer should be familier with

The various techniques of knowledge representation and intelligent search techniques used in expert systems.

The expert system tools which can greatly expedite the development process.

Other expert systems which may solve similar problems and thus may be adaptable to problem at hand.

Example : In rule based system

Engineer Develops set of rules to represent knowledge communicated by Expert

Many domain experts know "what" they do but not "why" they do it. This is a challenge for the engineer that he/she has to overcome

Domain Experts reviews rules

IF okay

Break ;

ELSE

Iterate again with suggestions

Implementation [ formalised concepts are programmed into the computer to build a prototype ]

Formalised concepts are programmed into the computer which has been chosen for system development using the predetermined techniques and tools to implement a ‘first-pass’ (prototype) of the expert system.

PROLOG was used as a programming language

The knowledge engineer may discover that the chosen techniques simply cannot be implemented.

It may not be possible, for example, to integrate the knowledge representation techniques selected for different sub-problems.

At that point, the concepts may have to be re-formalised, or it even may be necessary to create new development tools to implement the system efficiently. Basically go back and change stuff to make this stage work.

Once the prototype system has been refined sufficiently to allow it to be executed, the expert system is ready to be tested thoroughly to ensure that it expertise’s correctly.

Testing [ Identify weakness in Structure and Implementation to make appropriate corrections ]

Once the system has proven to be capable of correctly solving straight-forward problems, the domain expert suggests complex problems which typically would require a great deal of human expertise. These more demanding tests should uncover more serious flaws and provide ample opportunity to ‘fine tune’ the system even further.

Ultimately, an expert system is judged to be entirely successful only when it operates at the level of a human expert. The testing process is not complete until it indicates that the solutions suggested by the expert system are consistently as valid as those provided by a human domain expert.

Resources

https://www.engineeringenotes.com/artificial-intelligence-2/expert-systems/stages-to-develop-an-expert-system-artificial-intelligence/35592

Reading directly from PDF

Architecture

Knowledge Base

Warehouse of special heuristics or rules used directly by knowledge, facts (productions)

Has knowledge that is needed for understanding, formulating, & problem solving.

Working Memory

describes current running problem and record intermediate output.

Records Intermediate Hypothesis & Decisions in following pattern:

Plan,

Agenda

Solution

Inference Engine

Heart of expert system

Helps to manage entire structure of expert system

Uses different methodology for reasoning.

Knowledge Acquisition System

Knowledge acquisition is the process used to define the rules required for a knowledge-based system.

It is the process of extracting, organizing, and structuring the domain knowledge, specifying the rules to acquire the knowledge from various experts, and store that knowledge into the knowledge base.

Explanation System

Traces responsibility and justify the behavior of expert system by firing questions and answers

such as Why, How, What, Where, When, Who.

User Interface

It allows users to insert their queries with using own Natural Language Processing otherwise menus & graphics.

Knowledge Engeer

Main objective of this engineer is to design system for specific problem domain with using of expert system shell.

System Engineer

To design user interface and declarative format of knowledge base as well as to build inference engine

Users

They are non expert person who want to seek direct advice.

Resources

https://digitalthinkerhelp.com/components-and-architecture-of-expert-system-in-artificial-intelligence/

Applications

mnemonic : BFAME

Business

Marketing – It deal to large scale company to monitor discounts, quantity and other factors relating to sale.

Accounting – It allowing you to make financial decisions and credit authorization decisions.

Business – This allowing you to operate your business and trainees and evaluate their performance.

Finance

Expert system is used in the financial sector for solving these problems, such as –

For financial analysis of firms

For analyzing the causes of successful or unsuccessful business development

For market analysis

Agriculture

Crop Management – This expert system allows farmer for taking good decision about growing specific crop, such as rice, wheat, cucumber etc.

Production Planning – It helps to make profitable cropping plans.

Pest Management – This expert system deals to control pest/weed which are dangerous for cops.

Diagnostic Systems – It helps to manage different types of disease and infections to several crops and plants.

Conservation System – This expert system allows conservation of soil or water as well as it is used for management in agriculture.

Process Control Systems – In this expert system, to implement various sensors for monitoring different types of agricultural activities.

Marketing Advisory Systems – This expert system delivers good advice to farmers to get profits in the agricultural production.

Medical

MYCIN – It helps to diagnose infectious diseases in the blood and prescribing the appropriate antibiotics.

PUFF – To diagnose lung diseases

CADUCEUS – It is used for diagnosing of internal medicine.

Education

Computer Aided Instruction (CAI) – This expert system deals with expert heuristics for solving problems. It delivers for students for learning, and it uses games, puzzles, colorful graphs and sounds to keep the student attentive until the end of session.

Intelligent Tutoring System (ITS) – This expert system is used for learning students into pictorial form. It is totally based on the fuzzy logic, and it delivers best flexibility for representing information as well as responding to individual candidate as per need.

Intelligent Pascal Tutoring System (IPTS) – This expert system is able to keep a student database for identify individual performance and learning ability.

Advantages

These systems are highly reproducible.

They can be used for risky places where the human presence is not safe.

Error possibilities are less if the KB contains correct knowledge.

The performance of these systems remains steady as it is not affected by emotions, tension, or fatigue.

They provide a very high speed to respond to a particular query.

Limitations

The response of the expert system may get wrong if the knowledge base contains the wrong information.

Like a human being, it cannot produce a creative output for different scenarios.

Its maintenance and development costs are very high.

Knowledge acquisition for designing is much difficult.

For each domain, we require a specific ES, which is one of the big limitations.

It cannot learn from itself and hence requires manual updates.

[TODO] Expert System Vs Traditional System

Uncertainity

Types of Knowledge

Certain

Techniques such First-order logic and prepositional logic are used

This works due to knowledge being certain

Probability value is either 0 or 1, cannot be in between so no uncertainity

Uncertain

Here we are not sure about the predicates
A —> B means B is true iff A is true, but in this uncertain situation we are not sure if A or B is true or false
This begs the need of Probabilistic Reasoning

Causes of Uncertainity

Information taken from unreliable sources

Experimental Errors

Equipment Fault

Temprature Variation

Probabilistic Reasoning

Way of Knowledge Representation in which we apply concept of probability to indicate uncertainity in knowledge

Real world lots of scenarios depend on this

Need of Probabilistic Reasoning

There are unpredictable outcomes

Possibilities of predicates become too large to handle

Solve problems with Uncertain Knowledge

Probability Basics

Direct from PPT

Probability

Probability can be defined as a chance that an uncertain event will occur.

It is the numerical measure of the likelihood that an event will occur.

The value of probability always remains between 0 and 1 that represent ideal uncertainties.

Event

Each possible outcome of a variable is called an event

Sample Space

The collection of all possible events is called sample space

Random Variables

Random variables are used to represent the events and objects in the real world.

Prior Probability

The prior probability of an event is probability computed before observing new information.

Posterior Probability

The probability that is calculated after all evidence or information has taken into account. It is a combination of prior probability and new information

Conditional Probability

probability of occurring an event when another event has already happened

Bayes Rule

Determines the probability of an event with uncertain knowledge

Relates the conditional probability and marginal probabilities of two random events.

Bayesian inference is an application of Bayes' Theorem

Bayes' theorem allows updating the probability prediction of an event by observing new information of the real world.

If cancer corresponds to one's age then by using Bayes' theorem, we can determine the probability of cancer more accurately with the help of age.

Bayesian Network, Inference etc.

Videos to understand concepts

Listening to first 3 videos on 4x speed highly suggested to understand this concept

AI (IT) Lecture 13 Bayes rule

In this video the following topics covered:1. Bayes rule2. Examples of Bayes ruleSubscribe to my channel and press bell icon in order to get latest updates o...

https://www.youtube.com/watch?v=Qco5ilJeulc&list=PLUdotpZ2kw6MPQXqir1Fl7GaOPbQv94ah&index=11

AI (IT) Lecture 14 Bayesian Networks

In this video the following topic covered:Bayesian Network:1. Representation2. ConstructionSubscribe to my channel and press bell icon in order to get latest...

https://www.youtube.com/watch?v=_RnWJ-XSJ-c&list=PLUdotpZ2kw6MPQXqir1Fl7GaOPbQv94ah&index=13

AI (IT) Lecture 15 Inference in Bayesian Network

In this video the following topic covered:1. Inference in Bayesian Network2. Types of InferenceSubscribe to my channel and press bell icon in order to get la...

https://www.youtube.com/watch?v=K4ZTho2VlNE&list=PLUdotpZ2kw6MPQXqir1Fl7GaOPbQv94ah&index=9

Bayesian Network - Exact Inference Example (With Numbers, FULL Walk-Through)

Timestamps Relevant Equations - 0:12 Brief Aside - 1:52 Example Problem - 2:35 Solution - 3:41

https://www.youtube.com/watch?v=yS7hyFiwnPs

PPT covering topics

Bayesian Networks

Probabilistic Graphical model representing set of variables and their conditional dependencies

They are probabilitic because they are built from a probability distribution and use probability theory for prediction and anomaly detection

Bayesian Neworks can be used for building models form data.

The Bayesian Networks consist of Two Parts

Directed Acyclic Graph

Table of Conditional Probabilities

Give probabilities for value of random variables

Each row must sum to 1

All variables are boolean

Table for node with k-parents will have 2^k rows

Inference

Process of inferring unknown properties of a system given already calculated observations

Types of Inferences

Inference By Enumeration

By listing all variables

Inference By Variable Elimination

By variable removal

Fuzzy Logic

Must Watch Video Resources

Watch from 7th to 24th video in playlist. Each video approx 4-5 min.

Basics Of Fuzzy Logic ll Soft Computing Course Explained in Hindi

📚📚📚📚📚📚📚📚GOOD NEWS FOR COMPUTER ENGINEERSINTRODUCING 5 MINUTES ENGINEERING 🎓🎓🎓🎓🎓🎓🎓🎓SUBJECT :-Discrete Mathematics (DM) Theory Of Computation (...

https://www.youtube.com/watch?v=scIoh30WzIg&list=PLYwpaL_SFmcCPUl8mAnb4g1oExKd0n4Gw&index=7

Watch Gate smashers

Fuzzy Logic in Artificial Intelligence with Example | Artificial Intelligence

►Full Course of Artificial Intelligence:https://www.youtube.com/playlist?list=PLxCzCOWd7aiHGhOHV-nwb0HR5US5GFKFI►Subscribe to our new channel:https://www.you...

https://www.youtube.com/watch?v=vof2vhfqoBo

Various Operations in Fuzzy Logic with Example | Union, Intersection, Complement etc.

►Full Course of Artificial Intelligence:https://www.youtube.com/playlist?list=PLxCzCOWd7aiHGhOHV-nwb0HR5US5GFKFI►Subscribe to our new channel:https://www.you...

https://www.youtube.com/watch?v=o-2O4fmIu3E

Superset of Boolean logic

Represent with uncertainity

Represent with degree

Represent belongingness of a member of a crisp set to fuzzy set

Implementation

Can be implemented in Software and Hardware of various size

Can also be implemented in Micro controllers

Applications

It can control machines and consumer products

It may not give accurate reasoning, but acceptable reasoning

Helps dealing with uncertainity in engineering

Fuzzy Set

Collection of values which exist between 0 and 1

Operations

Union

When you find same value in both sets, include the one with max membership function

Intersection

When you find same value in both sets, include the one with min membership function

Complement Operation

( 1 - membership function ) for whole set

Fuzzy Logic Architecture

Fuzzification

Converts Crisp input to fuzzy input

Divides input signals into five states

Large Positive

Medium Positive

Small

Medium Negative

Large Negative

Rule Base

Contains all Rules and IF-ELSE conditions offered by experts to control decision making systems

Inference Engine

Degree of match between fuzzy input and the rules. Based on the % match

Determines which rules to implement

Defuzzification

Converts fuzzy output to Crisp output

Fuzzy Logic Algorithm

1) Initialization process: ▪ Define the linguistic variables. ▪ Construct the fuzzy logic membership functions that define the meaning or values of the input and output terms used in the rules. ▪ Construct the rule base (Break down the control problem into a series of IF X AND Y, THEN Z rules based on the fuzzy logic rules). 2) Convert crisp input data to fuzzy values using the membership functions (fuzzification). 3) Evaluate the rules in the rule base (inference). 4) Combine the results of each rule (inference). 5) Convert the output data to non-fuzzy values (defuzzification).

Examples and Membership function (theory)

Decision Making

AI (IT) Lecture 19 Utility Theory, Utility Functions

In this video the following topics covered:1. Basics on Utility Theory2. Axioms 3. Risk Aversion, Risk Loving, Risk Neutral

https://www.youtube.com/watch?v=S_YIiT3wlH8&list=PLUdotpZ2kw6MPQXqir1Fl7GaOPbQv94ah&index=5

Utility Theory

An agent's preferences over possible outcomes can be captured by a function that maps these outcomes to a real number; the higher the number the more that agent likes that outcome. This function is called a utility function

Once you have these numbers, then you can mix them in with probabilities and talk about expected utilities, optimal strategies, discounted future rewards, and many other fun things.

UNIT 4

Machine Learning and it's Paradigms

What is Learning ?

Change in system which enables it to perform same task more efficiently the next time

Constructing/Modifying representations of what is being experienced

Making useful changes in our minds

What is Machine Learning ?

First introduced by Arthur Sameul in 1959

It is a subset of Artificial Intelligence mainly concerned with development of algorithms

Automatically learn from data and past experiences, predict things without explicitly programmed to do so

Sample historical data (training data) is fed into these algorithms to produce a mathematical model

Brings Computer Science and Statistics together to create predictive models

The Three Paradigms in Machine Learning

Flow Chart with Brief Overview

Supervised Learning

Machine learning method in which we provide sample labelled data to the system in order to train it, on the basis of which it predicts the output

System creates a model using labelled data to understand the datasets

Goal is to map input data with output data

Example : Spam filtering

Classification

Process of categorizing data into classes (often referred as target,label or categories)

Can be performed on both structured and unstructured data

Example : Heart disease detection ( binary classification )

Algorithms

Decision Tree Algorithm

Support Vector Machines

Naive Bayes Classification

K Nearest Neighbour

Artificial Neural Networks

Regression

Algorithms attempting to map input variable (x) to numerical or continous output variables (y)

Regression prediction problems are usually quantities and sizes since output can be integer as well as float

Example : Predict house price from dataset of houses

Algorithms

Linear Regression

Logistic Regression

Regression Decision Tree

Artificial Neural Netorks

Unsupervised Learning

ML technique in which models themselves find hidden patterns and insights from given data

Models are trained using unlabelled data and are allowed to act on that data without any supervision

Cannot be directly applied to Classification/Regression since theres no output data to compare

Example : Cat and Dog

Clustering

Method of grouping objects into clusters such that object with most similarity remains in a group (cat,dog)

Algorithms

K - means Clustering

Hierarchial Clustering

Association

Used to find relationship between variables in a large database. Ex: People who buy X also buy Y

Inductive Learning

Knowledge extraction method for large amounts of data since expert system might not be feasible

Process where Learner discovers rules by observing examples to find patterns

System tries to make a general rule

Decision Trees

Supervised learning Technique

Can be used for Classification and regression but preferred for former

Internal Nodes : Features of Dataset

Branches : Decision Rules

Leaf Nodes : Represents outcome

Types of Nodes

Decision Node (Used to make any decision)
Leaf Node (Output)

Types of Decision Trees

Classification Trees ( Binary Yes / No )

Decision Variable is Categorical

Regression Trees ( Conintuous Data Types )

Oucome variable is continuous

Best Algorithm to construct a Decision Tree is the ID3 Algorithm

ID3 = Iterative Dichotomiser 3

It's a Classification Algo following greedy approach of building a decision tree by selecting a best attribute that yields Maximum Information Gain (IG) or Minimum Entropy (H)

Entropy is the measure of impurity, disorder or uncertainity

Entropy controls how Decision tree decides to split the data

Information gain measures how much info a feature gives us about the class

Information gain is the main key that is used by Decision Tree Algorithms to construct a Decision Tree.

Decision Trees algorithm always tries to maximize Information gain.

An attribute with highest Information gain will tested/split first

Problems and Formulae[TODO]

Complete Decision Tree using ID3 Problem in One Video

Solving this should be enough

1. Decision Tree | ID3 Algorithm | Solved Numerical Example | by Mahesh Huddar

1. Decision Tree - ID3 Algorithm Solved Numerical Example by Mahesh HuddarDecision Tree ID3 Algorithm Solved Example - 1: https://www.youtube.com/watch?v=gn8...

https://www.youtube.com/watch?v=coOTEc-0OGw

Sir PPT

Resources

https://medium.datadriveninvestor.com/learning-paradigms-in-machine-learning-146ebf8b5943

Artificial Neural Networks

Machine Learning approach that models Human Brain using artificial neurons

ANNs have fewer connections compared to human brain

Each Neuron receives a number of inputs

An activation function is applied to these inputs which result in activation level of neuron

Knowledge about the learning task is given in form of training data

Neuron Model - Information Processing Unit of the NN
Architecture - Links connecting Nerons, each link has an associated weight
Learning Algorithm - Training NN by modifying weights in order to model a learning task on training examples

Neuron and Neural Networks

Basic information processing unit of NN

Set of links with weights w1,w2,w3

Adder function to compute weighted sum of the inputs

Activation Function for limiting amplitude of neuron output

Bias is external parameter of neuron and can be modelled using extra input

Neuron, Activation Functions and BackPropagation Theory

Single Layer Feed-Forward Network

Theory from PPT

Multi-Layer Feed-Forward Network

Theory from PPT

Reinforcement Learning

Reinforcement learning is an area of ML concerned with how intelligent agents can take actions in an environment in order to maximize the reward

Training Data does have not answers the agent decides what to do to perform the given task

Learns from experience

Trained on Reward and Punishment Mechanism

PPT Theory

UNIT 5

Natural Language Processing (NLP)

Natural language processing (NLP) is a subfield of linguistics, computer science, and artificial intelligence

It is concerned with the interactions between computers and human language, in particular how to program computers to process and analyze large amounts of natural language data.

The goal is a computer capable of "understanding" the contents of documents, including the contextual nuances of the language within them.

The technology can then accurately extract information and insights contained in the documents as well as categorize and organize the documents themselves.

History of NLP

1950- NLP started when Alan Turing published an article called "Machine and Intelligence."

1950- Attempts to automate translation between Russian and English

1960- The work of Chomsky and others on formal language theory and generative syntax

1990- Probabilistic and data-driven models had become quite standard

2000- A Large amount of spoken and textual data become available

Phases of NLP

Five main Components of Natural Language processing in AI are:

Morphological and Lexical Analysis

Lexical analysis is a vocabulary that includes its words and expressions. It depicts analyzing, identifying and description of the structure of words. It includes dividing a text into paragraphs, words and the sentences

Individual words are analyzed into their components, and nonword tokens such as punctuations are separated from the words.

Syntactic Analysis

The words are commonly accepted as being the smallest units of syntax. The syntax refers to the principles and rules that govern the sentence structure of any individual languages.

Syntax focus about the proper ordering of words which can affect its meaning. This involves analysis of the words in a sentence by following the grammatical structure of the sentence. The words are transformed into the structure to show hows the word are related to each other.

Semantic Analysis

Semantic Analysis is a structure created by the syntactic analyzer which assigns meanings. This component transfers linear sequences of words into structures. It shows how the words are associated with each other.

Semantics focuses only on the literal meaning of words, phrases, and sentences. This only abstracts the dictionary meaning or the real meaning from the given context. The structures assigned by the syntactic analyzer always have assigned meaning

E.g.. "colorless green idea." This would be rejected by the Symantec analysis as colorless Here; green doesn't make any sense.

Discourse Integration

It means a sense of the context. The meaning of any single sentence which depends upon that sentences. It also considers the meaning of the following sentence.

For example, the word "that" in the sentence "He wanted that" depends upon the prior discourse context.

Pragmatic Analysis

Pragmatic Analysis deals with the overall communicative and social content and its effect on interpretation.

It means abstracting or deriving the meaningful use of language in situations. In this analysis, the main focus always on what was said in reinterpreted on what is meant.

Pragmatic analysis helps users to discover this intended effect by applying a set of rules that characterize cooperative dialogues.

E.g., "close the window?" should be interpreted as a request instead of an order.

Components of NLP

There are two components of NLP

Natural Language Understanding

Natural Language Understanding (NLU) helps the machine to understand and analyze human language by extracting the metadata from content such as concepts, entities, keywords, emotion, relations, and semantic roles.

NLU mainly used in Business applications to understand the customer's problem in both spoken and written language.

NLU involves the following tasks -

It is used to map the given input into useful representation.
It is used to analyze different aspects of the language.

Natural Language Generation

Natural Language Generation (NLG) acts as a translator that converts the computerized data into natural language representation.

It mainly involves Text planning, Sentence planning, and Text Realization.

An NLG System contains

Discourse Planner

What will be generated, which sentences

Surface Realizer

Realizes a sentence from its internal representation

Lexical Selection

Selecting the correct words describing the concepts

💡

NLU is more difficult tha NLG

Applications of NLP

Question Answering

Spam Filtering

Sentiment Learning

Machine Translation

Spelling Correction

Speech Recognition

Chatbot

Information Extraction

Natural Language Understanding (NLU)

Building an NLP Pipeline

Sentence Segmentation

Sentence Segment is the first step for building the NLP pipeline. It breaks the paragraph into separate sentences.

Example: Consider the following paragraph -

Independence Day is one of the important festivals for every Indian citizen. It is celebrated on the 15th of August each year ever since India got independence from the British rule. The day celebrates independence in the true sense.

Sentence Segment produces the following result:

"Independence Day is one of the important festivals for every Indian citizen."

"It is celebrated on the 15th of August each year ever since India got independence from the British rule."

"This day celebrates independence in the true sense."

Word Tokenization

JavaTpoint offers Corporate Training, Summer Training, Online Training, and Winter Training.

Word Tokenizer generates the following result:

"JavaTpoint", "offers", "Corporate", "Training", "Summer", "Training", "Online", "Training", "and", "Winter", "Training", "."

Stemming

Stemming is used to normalize words into its base form or root form. For example, celebrates, celebrated and celebrating, all these words are originated with a single root word "celebrate."

The big problem with stemming is that sometimes it produces the root word which may not have any meaning.

For Example, intelligence, intelligent, and intelligently, all these words are originated with a single root word "intelligen." In English, the word "intelligen" do not have any meaning.

Lemmatization

Lemmatization is quite similar to the Stemming. It is used to group different inflected forms of the word, called Lemma.

The main difference between Stemming and lemmatization is that it produces the root word, which has a meaning.

Identifying Stop Words

In English, there are a lot of words that appear very frequently like "is", "and", "the", and "a". NLP pipelines will flag these words as stop words.

Stop words might be filtered out before doing any statistical analysis.

Example: He is a good boy.

Dependancy Parsing

Dependency Parsing is used to find that how all the words in the sentence are related to each other.

POS Tags

POS stands for parts of speech, which includes Noun, verb, adverb, and Adjective. It indicates that how a word functions with its meaning as well as grammatically within the sentences. A word has one or more parts of speech based on the context in which it is used.

Example: "Google" something on the Internet.

In the above example, Google is used as a verb, although it is a proper noun.

Named Entity Recognition

Named Entity Recognition (NER) is the process of detecting the named entity such as person name, movie name, organization name, or location.

Example: Steve Jobs introduced iPhone at the Macworld Conference in San Francisco, California.

Chunking

Chunking is used to collect the individual piece of information and grouping them into bigger pieces of sentences.

What makes NLP Difficult ?

Ambiguity

Lexical Ambiguity

Syntactic Ambiguity

Referential Ambiguity

Grammars and Parsing

Theory from PPT

Some Extra Stuff

NLP APIs

IBM WATSON API

CHATBOT API

SPEECH TO TEXT API

GOOGLE CLOUD NATURAL LANGUAGE API

NLP Libraries

Scikit-learn

Natural language Toolkit (NLTK)

Pattern

TextBlob

Quepy

SpaCy

Gensim

Resources

Guru 99 Tutorial On NLP

Java T Point

Gate Smashers

Natural Language Processing in Artificial Intelligence in Hindi | NLP with Demo and Examples

►Full Course of Artificial Intelligence:https://www.youtube.com/playlist?list=PLxCzCOWd7aiHGhOHV-nwb0HR5US5GFKFI►Subscribe to our new channel:https://www.you...

https://www.youtube.com/watch?v=bPpwZxasJo0

Automatic Speech Recognition

Automatic Speech Recognition Software (ASR)

Automatic speech recognition (ASR) is the use of computer hardware and software-based techniques to identify and process human voice. It is used to identify the words a person has spoken or to authenticate the identity of the person speaking into the system.

Automatic speech recognition is also known as automatic voice recognition (AVR), voice-to-text or simply speech recognition

Easy to understand Infographic

Training of ASR

Human Tuning

This is a relatively simple means of performing ASR training.

It involves human programmers going through the conversation logs of a given ASR software interface and looking at the commonly used words that it had to hear but which it does not have in its pre-programmed vocabulary.

Those words are then added to the software so that it can expand its comprehension of speech.

Active Learning

Active learning is the much more sophisticated variant of ASR and is particularly being tried with NLP versions of speech recognition technology.

With active learning, the software itself is programmed to autonomously learn, retain and adopt new words, thus constantly expanding its vocabulary as it’s exposed to new ways of speaking and saying things.

This, at least in theory, allows the software to pick up on the more specific speech habits of particular users so that it can communicate better with them.

So for example, if a given human user keeps negating the autocorrect on a specific word, the NLP software eventually learns to recognize that particular person’s different use of that word as the “correct” version.

Famous ASR Systems

DRAGON

Dragon Dictate launched in 1990 for a staggering $9,000, touting a dictionary of 80,000 words and features like natural language processing (see the Infoworld article above).

These tools were time-consuming (the article claims otherwise, but Dragon became known for prompting users to ‘train’ the dictation software to their own voice). And it required that users speak in a stilted manner: Dragon could initially recognize only 30–40 words a minute; people typically talk around four times faster than that.

But it worked well enough for Dragon to grow into a business with hundreds of employees, and customers spanning healthcare, law, and more. By 1997 the company introduced Dragon NaturallySpeaking, which could capture words at a more fluid pace — and, at $150, a much lower price-tag⁸.

HARPY

the U.S. Department of Defense’s ARPA (the agency now known as DARPA) funded a five-year program called Speech Understanding Research.

This led to the creation of several new ASR systems, the most successful of which was Carnegie Mellon University’s Harpy, which could recognize just over 1000 words by 1976.

Resources

Automatic Speech Recognition (ASR) Software - An Introduction - Usability Geek

By Matthew Zajechowski In terms of technological development, we may still be at least a couple of decades away from having truly autonomous, intelligent artificial intelligence systems communicating with us in a genuinely "human-like" way. But, in many ways, we're progressing steadily towards this future scenario at a surprisingly fast pace thanks to the continuing development of what is known as automated speech recognition technology.

https://usabilitygeek.com/automatic-speech-recognition-asr-software-an-introduction/

Machine Vision

Machine Vision Hierarchy