Category: 06. Uncertain Knowledge

Bayesian Belief Networks or BBNs provide robust foundations for probabilistic models and inference in both the area of artificial intelligence and decision support. A Bayesian network is a state model as a probabilistic graphical model in which the dependencies among variables occur through a directed acyclic graph.

In addition, it is called a Bayes network, belief network, decision network or Bayesian model. Bayesian networks are marked by probability due to their essence, which is the foundation of the probability distribution and subsequent use of probability theories in prediction and anomaly detection.

Because real-world problems are primarily probabilistic and almost always involve relationships among events, we need a Bayesian network to model them. BBNs have gained significant use from their dependable treatment of uncertainty across the fields of healthcare, finance and environmental management, among many others. Its application continues in prediction, anomaly detection, diagnostics, automated insight, reasoning, time series prediction, and decision-making characterized by uncertainty.

Bayesian Network is modelled using both data and expertise knowledge and is divided into two categories:

• Directed Acyclic Graph
• Table of conditional probabilities

Directed Acyclic Graph

This is one representation of the network and between variables’ causal connections. DAG nodes are variables; edges are dependencies of the variables from one another. The arrows indicated on the graph indicate the direction of the arrow.

Table of Conditional Probabilities

Attached to every node of the DAG is a conditional table specifying the probability of each possible value of the node conditioning on its parents’ values in the DAG. Theses tables translate probabilistic relationships which exist between the variables in the network.

A Bayesian Belief Network Graph

The generalized version of the Bayesian network that holds and solves decision problems in the field of uncertain knowledge is referred to simply as an Influence diagram.

A Bayesian network graph consists of nodes and Arcs, where:

• Each node represents the random variables, and a variable can be continuous or discrete.
• Arc or arrows pointing in a particular direction mean causal relationships or conditional probabilities between random variables. These directed links, or arrows, link the pair of nodes of the graph.
• These links reflect that one node directly influences the other node, and the absence of a directed link means that nodes are independent of each other.
• In the above diagram, A, B, C, and D are random variables, which are represented by the nodes of the network graph.
• If we are taking Node B and we are dealing with an arrow coming from Node B onto Node A, then Node A becomes the parent Node to B.
• Node C is autonomous from node A.

Components of Bayesian Network

The Bayesian network has mainly two components:

• Causal Component
• Actual numbers

Causal Component

The causal part of a Bayesian network is the causal relationships between the variables of the system. It is formed by directed acyclic graphs (DAGs), which represent causal relationships between the variables in their direction. The nodes in the DAG are the system variables, and the edges denote causality between variables, i.e. causal relationships between variables. In most cases, the causal part of a Bayesian network is referred to as its “structure”.

The causal part of a Bayesian network is very important to understand to see how the variables in the system relate to each other. It presents the visual imagination of the causal links between the variables as a means to make predictions and to see how one variable will influence the rest.

Actual Numbers

The numerical part of a Bayesian network is comprised of conditional probability tables (CPTs) for all nodes of the DAG. These tables represent the probability of each variable provided the values taken by its parent variables. The numerical part in a Bayesian network is normally termed as the “parameters” of a network.

The modelling part of a Bayesian network supplies the numbers that are used to predict and work out probabilities. Every node in the network has its own CPT, which defines the probability of that node-dependent value of its parent nodes. These probabilities are used for computation in the determination of an overall likelihood of the system with some input or observations.

Each node in the Bayesian network has condition probability distribution P(Xi |Parent(Xi) ), which determines the effect of the parent on that node.

Bayesian network is based on Joint probability distribution and conditional probability. So, let’s first understand the joint probability distribution.

Joint Probability Distribution

If we have variables x1, x2, x3,….., xn, then the probabilities of a different combination of x1, x2, x3,.., xn, are known as Joint probability distribution.

P[x1, x2, x3,….., xn], it can be written in the following way in terms of the joint probability distribution.

= P[x1| x2, x3,….., xn]P[x2, x3,….., xn]

= P[x1| x2, x3,….., xn]P[x2|x3,….., xn]….P[xn-1|xn]P[xn].

In general, for each variable Xi, we can write the equation as:

P(Xi|Xi-1,………, X1) = P(Xi |Parents(Xi ))

Explanation of Bayesian Network

Let’s understand the Bayesian network through an example by creating a directed acyclic graph:

Example

Harry installed a new burglar alarm at his home to detect burglary. The alarm reliably responds to detecting a burglary but also responds to minor earthquakes. Harry has two neighbors David and Sophia, who have taken a responsibility to inform Harry at work when they hear the alarm. David always calls Harry when he hears the alarm, but sometimes he gets confused with the phone ringing and calls at that time, too. On the other hand, Sophia likes to listen to high music, so sometimes she misses to hear the alarm. Here, we would like to compute the probability of a Burglary Alarm.

Problem

Calculate the probability that the alarm has sounded, but there is neither a burglary nor an earthquake occurred, and David and Sophia both call Harry.

Solution

The Bayesian network for the above problem is given below. The network structure shows that burglary and earthquake are the parent nodes of the alarm and directly affect the probability of the alarm going off, but David and Sophia’s calls depend on alarm probability.

The network represents that our assumptions do not directly perceive the burglary and also do not notice the minor earthquake, and they also do not confer before calling.

The conditional distributions for each node are given as a conditional probabilities table or CPT.

Each row in the CPT must be summed to 1 because all the entries in the table represent an exhaustive set of cases for the variable.

In CPT, a boolean variable with k boolean parents contains 2K probabilities. Hence, if there are two parents, then CPT will contain four probability values.

List of all events occurring in this network:

• Burglary (B)
• Earthquake(E)
• Alarm(A)
• David Calls(D)
• Sophia calls(S)

We can write the events of the problem statement in the form of probability: P[D, S, A, B, E], can rewrite the above probability statement using joint probability distribution:

P[D, S, A, B, E]= P[D | S, A, B, E]. P[S, A, B, E]

=P[D | S, A, B, E]. P[S | A, B, E]. P[A, B, E]

= P [D| A]. P [ S| A, B, E]. P[ A, B, E]

= P[D | A]. P[ S | A]. P[A| B, E]. P[B, E]

= P[D | A ]. P[S | A]. P[A| B, E]. P[B |E]. P[E]

Let’s take the observed probability for the Burglary and earthquake component:

• P(B= True) = 0.002, which is the probability of burglary.
• P(B= False)= 0.998, which is the probability of no burglary.
• P(E= True)= 0.001, which is the probability of a minor earthquake
• P(E= False)= 0.999, Which is the probability that an earthquake does not occur.

We can provide the conditional probabilities as per the tables below:

Conditional Probability Table for Alarm A:

The Conditional probability of Alarm A depends on the Burglar and earthquake:

B	E	P(A= True)	P(A= False)
True	True	0.94	0.06
True	False	0.95	0.04
False	True	0.31	0.69
False	False	0.001	0.999

Conditional Probability Table for David Calls:

The Conditional probability of David that he will call depends on the probability of Alarm.

A	P(D= True)	P(D= False)
True	0.91	0.09
False	0.05	0.95

Conditional Probability Table for Sophia Calls:

The Conditional probability of Sophia that she calls depends on its Parent Node, “Alarm.”

A	P(S= True)	P(S= False)
True	0.75	0.25
False	0.02	0.98

From the formula of joint distribution, we can write the problem statement in the form of probability distribution:

P(S, D, A, ¬B, ¬E) = P (S|A) *P (D|A)*P (A|¬B ^ ¬E) *P (¬B) *P (¬E).

= 0.75* 0.91* 0.001* 0.998*0.999

= 0.00068045.

Hence, a Bayesian network can answer any query about the domain by using Joint distribution.

The Semantics of the Bayesian Network:

The semantics of the Bayesian network have two approaches to understanding, as shown below:

1. To understand the network as the representation of the Joint probability distribution.

It is of importance because we are able to model complex systems by using the utilization of the graph structure. In such a manner and the way of demonstrating the joint distribution in the form of the graph, we see what exactly the dependencies and independencies of the variables are, and it may be helpful in making further predictions and inferences about the system. What is more, it can develop a way to utilize by which we shall be in a position to identify the prospective reasons/ results of observed occurrences.

2. To understand the network as an encoding of a collection of conditional independence statements.

It is of very great essence to the style of producing effective inference procedures. The application of the conditional independence relationships that are provided in the network allows one to simplify the computational complexity of the inference processes greatly. This is because we are often able to factorize the joint distribution into the conditional ones, which are smaller and basil revised with the help of observed evidence.

Such a method is very useful in the situation addressing probabilistic reasoning – we have to try to deduce the probability distribution of those non-observed values that will take some particular values given some observed evidence.

Applications of Bayesian Networks in AI

Following are some of the applications of the Bayesian networks in the AI:

Prediction and Classification:

A set of inputs can be utilized by the Bayesian belief networks in order to carry out such kinds of operations as predicting the probability of events or classifying the data supplied to some class. This is relevant to areas such as the detection of fraud, the recognition of images, etc.

Decision Making:

Bayesian networks could also be used in decision-making despite the vague or incomplete pieces of information. For example, they can be applied in the determination of the best route that a truck full delivery vehicle should take in relation to the traffic and the delivery schedules.

Risk Analysis:

Bayesian belief networks may be employed in the examination of the risk that can be associated with definite actions or happenings. This is the case with such examples as financial planning, insurance and safety analysis.

Anomaly Detection:

Outliers or odd patterns are the cases’ instances where the Bayesian network can be applied in the process of monitoring the anomalies of the data. This is effective for a cyber security scenario where veracity that resists data traffic can brew up the scenario for the breach of security.

Natural Language Processing:

However, the Bayesian belief networks can probably be used whereby the probabilistic relationships between the words and phrases of the language of applicability exist. For example, the translation of the language and the sentiment analysis are developed.

Medical Diagnosis:

The Bayesian Belief Network is given in such a way that it would be able to search the relationships between the diseases and the symptoms, as well as the factors, which in turn can be dangerous for the users of them, closer to the application to the medicine. This fact is a common knowledge in the popular mode as BBN. In the above case, the BBNs help the doctors to insure them with the magnitude of prediction of the sickness and the best way of treating the sick utilizing two sources of information, which are the knowledge of the expert and the patient’s information.

For example, a BBN can make an approximate number of heart attack cases with the factor of chest pain, age and pressure, among others. The newly emanating symptoms are dynamically associated with having performed in order to revise the probabilities with a view of making the diagnoses as accurate as possible.

Machine Learning and Data Mining

The BBNs are used in machine learning and data searching, trying to discover an undiscovered pattern in the sets of data. They are used in order to make predictions, for instance, concerning such a type of results as fraud detection when the bank is checking the relations between the variables in the process of banking. In the context of data mining, BBNs are used to the end of building the relations between the features, which equals better predictive models.

They are priceless in the region where it is necessary to make decisions concerning complicated and ambiguous systems since they will have a chance to learn from the former information and the experts.

Advantages of Bayesian Belief Networks

Handling Uncertainty

Bayesian Belief Network (BBN) is rather competent in uncertainty handling since the probabilities are altered dynamically when new evidence comes before them. It, therefore, makes them good in such an environment where data is incomplete, noisy and dynamic.

Flexibility and Scalability

BBNs provide the possibility of flexibility because they show complex relations between the variants. They are likely to be applied to the real world of practice in the medical diagnosis process the financial forecasting. It is, however, not an easy thing for one to achieve, though by virtue of its modular architecture construction, it can be expanded to a bigger network with a little upgrade to itself.

Incorporating Expert Knowledge

One of the main strong points of BBNs is that such models can combine expert and data-oriented models. This is especially true in such fields as healthcare, whereby the role of a specialist will assure the accuracy and punctuality of the predictions. BBNs create an equilibrium between empirical data and knowledge on the domain in such a manner that decisions can be made in an uncertain state, which can be valid.

Challenges and Limitations

Computational Complexity

Many computations are required in the case of BBNs when it comes to the number of variables and the dependencies. When considering the scenario of such low numbers of nodes and connections in very large networks, the resource requirement is gargantuan in terms of resources.

Scaling Issues

The fact that the very large networks are also dynamic indeed results in the problem of extreme scalability. Suppose the question regarding increasing the size of the network is considered. In that case, the management of dependency and ensuring that the desired CPTs are maintained is an issue, and it makes it impossible for the model to function.

Defining Accurate Priors

High probabilities in advance are rather meaningful in the case of network reliability. It might, however, be quite challenging if a person is faced with a lack of historical data. When the incorrect priors that interfere with the validity of the model are obtained, then biased results may be obtained in such a case. Prior adjustment is a job, and only an expert can help to perform it; even the experts may fail to give accurate estimates for a complicated domain.

Conclusion

Bayesian Belief Networks (BBNs) are one of the integral components of artificial intelligence through which one can make various events of uncertain decision-making with the use of probabilities. An opportunity is given to the necessity that they are able to model complex dependency and update the probabilities in real-time, making it a necessity in such kinds of industries as healthcare, finance, and machine learning. BBNs will link the knowledge provided by the experts and the formed data with a view of making the predictions clearer.

While AI was increasing, both computational power and algorithms were going to improve the performance of BBNs, too. Such innovations will provide space and it will ensure that BBNs are one of the very fundamental aspects in the future AI and intelligent decision-making groups.

Bayes’ theorem is also known as Bayes’ rule, Bayes’ law, or Bayesian reasoning, which determines the probability of an event with uncertain knowledge.

In probability theory, it relates the conditional probability and marginal probabilities of two random events.

Bayes’ theorem was named after the British mathematician Thomas Bayes. The Bayesian inference is an application of Bayes’ theorem, which is fundamental to Bayesian statistics.

It is a way to calculate the value of P(B|A) with the knowledge of P(A|B).

Bayes’ theorem allows updating the probability prediction of an event by observing new information from the real world.

Core Concepts of Bayes’ Theorem

Bayes’ Theorem presupposes probabilities that can be changed based on a new piece of information. The blocks that make up Bayesian reasoning are prior probability, likelihood, posterior probability, and normalization constant, which are the most important entities of Bayesian reasoning.

Prior Probability

The initial guess of the probability of an event (which is conventionally referred to as “prior”) is an initial assumption concerning the likelihood of an event without the consideration of the new evidence. It is an advancement of our assumption or knowledge as regards the affairs that were in the state at the time, compared to the historical facts or intuition.

For example, a practitioner might have initial chances of telling them if a particular patient has the disease or not since the disease has/ has not affected individuals in the population.

Likelihood

Likelihood is the indicator of the assumed confirmation of some evidence regarding a certain hypothesis. It is the chance to see the observed data if you conclude that a hypothesis is right.

For instance, if a diagnostic test were able to state the disease in 95% of all the cases, then the probability of having positive results of the test if a person happens to have the disease would become 0.95.

Posterior Probability

Posterior probability is the re-evaluation of a particular hypothesis by new evidence. It is the prior probability multiplied by the likelihood that tells one the precision in believing in the given data.

Mathematically:

P(Hypothesis | Evidence) = P(Evidence | Hypothesis)⋅P(Hypothesis) / P(Evidence)

Here, P(Hypothesis | Evidence) is the posterior probability.

Normalisation Constant

Ensuring Probabilities Sum to One

The normalisation constant is going to make sure that all the likely hypotheses have their sum of probabilities equal to one. The sum of the probabilities of the previous occasions and the probability of each one of the hypotheses.

Mathematically:

P(Evidence) = ∑i P(Evidence | Hypothesisi) ⋅ P(Hypothesisi)

This is yet another title that makes the authenticity and the understandability of the posterior probabilistic answers possible.

Example

If cancer corresponds to one’s age, then by using Bayes’ theorem, we can determine the probability of cancer more accurately with the help of age.

Bayes’ theorem can be derived using the product rule and the conditional probability of event A with known event B:

From the product rule, we can write:

P(A ⋀ B)= P(A|B) P(B) or

Similarly, the probability of event B with known event A:

P(A ⋀ B)= P(B|A) P(A)

Equating the right-hand side of both equations, we will get:

The above equation (a) is called Bayes’ rule or Bayes’ theorem. This equation is the basis of most modern AI systems for probabilistic inference.

It shows the simple relationship between joint and conditional probabilities. Here,

P(A|B) is known as posterior, which we need to calculate, and it will be read as the Probability of hypothesis A when we have occurred evidence B.

P(B|A) is called the likelihood, in which we consider that the hypothesis is true, and then we calculate the probability of evidence.

P(A) is called the prior probability, the probability of the hypothesis before considering the evidence.

P(B) is called the marginal probability, the pure probability of an event.

In equation (a), in general, we can write P (B) = P(A)*P(B|Ai); hence, the Bayes’ rule can be written as:

Where A1, A2, A3,…….., An is a set of mutually exclusive and exhaustive events.

Role of Bayes’ Theorem in Artificial Intelligence

Decision-Making Under Uncertainty

• Dynamic Updating of Beliefs: Based on the way of Bayes’ theorem, AI systems can provide the first opinion (prior probability) and reconsider the provided opinion in terms of the condition of new appearing information (likelihood). In fact, in the case of autonomous vehicles, they can look back and determine the pre-probability of the likely obstacles based on the sensor data; here, the system can concentrate on braking, steering, or acceleration.
• Probabilistic Reasoning: On the contrary, in deterministic approaches, Bayes’ theorem is not one of a kind because it can describe the uncertainty that quantifies uncertainty. For example, in the medical diagnostic for symptoms where there is a provision of the probability of the disease, i.e., a pointer to the right test or treatment for doctors, then it increases the standard of trustworthiness in making decisions.
• Applications in Robotics: Robot works in a chaotic environment most of the time. In tasks like navigation, Bayesian reasoning will come in handy to support decision-making where the findings from the sensor may be noisy or are not inadequate, and the robots can take the most likely course to reach their target.

Data-Driven Learning

• Bayesian Inference for Model Training: Bayesian inference is an application of combinational values in prior knowledge as well as the data in the computation of posterior distributions that are a form of probabilistic reasoning about the model’s parameters. This is very useful, especially because of the lack of availability of abundant data, because it does not make models confident of what they present.
• Continuous Learning: In real-time systems, such as adaptive user interfaces or recommendation engines, Bayes’ Theorem facilitates ongoing learning. For instance, an AI that predicts user preferences can refine its model continuously as it gathers more interaction data.
• Feature Selection and Dimensionality Reduction: Bayesian techniques help identify the most relevant features for a given problem, reducing the complexity of models while preserving their accuracy. This capability is vital for applications like image recognition, where datasets have high-dimensional feature spaces.

Predictive Modelling and Risk Assessment

• Risk Assessment Models: The Bayesian calculation that is employed by AI systems is used to make predictions about the probability of adverse events. For instance, in the field of finance, the predictive models will provide a likelihood of default in credit or a crash in markets. This is a readiness that institutions need to have to be prepared for how they will contain threats that might arise at any time in the future.
• Personalized Predictions: The Bayesian procedures give rise to personalized determination of data, which relies on the information of the individual. For example, in the case of personalised health care, the possibility of an individual’s response to the treatment is derived from the Bayesian models and, therefore, more directive strategies of treatment.
• Scenario Analysis: The Bayes’ Theorem has been found to make scenario analysis very easy to use, as one only needs to input varying figures, and as the desired results will be computed automatically, there is no need for effort. For instance, supply chain management prepares for disruptions that may come from past trends and conditions required; thus, the businesses will be prepared in advance.

Applying Bayes’ rule:

Bayes’ rule allows us to compute the single term P(B|A) in terms of P(A|B), P(B), and P(A). This is very useful in cases where we have a good probability of these three terms and want to determine the fourth one. Suppose we want to perceive the effect of some unknown cause and want to compute that cause, then the Bayes’ rule becomes:

Example 1:

Que: What is the probability that a patient has meningitis with a stiff neck?

Given Data:

A doctor is aware that the disease meningitis causes a patient to have a stiff neck, and it occurs in 80% of cases. He is also aware of some more facts, which are given as follows:

• The Known probability that a patient has meningitis disease is 1/30,000.
• The Known probability that a patient has a stiff neck is 2%.

Let a be the proposition that the patient has a stiff neck, and let b be the proposition that the patient has meningitis. So we can calculate the following as:

P(a|b) = 0.8

P(b) = 1/30000

P(a)= .02

Hence, we can assume that 1 patient out of 750 patients has meningitis with a stiff neck.

Example 2:

Que: From a standard deck of playing cards, a single card is drawn. The probability that the card is a king is 4/52, then calculate the posterior probability P(King|Face), which means the drawn face card is a king card.

Solution:

P(king): probability that the card is King= 4/52= 1/13

P(face): probability that a card is a face card = 3/13

P(Face|King): probability of face card when we assume it is a king = 1

Putting all values in equation (i), we will get:

Application of Bayes’ theorem in Artificial Intelligence

Natural Language Processing (NLP)

Spam Detection Using Naive Bayes Classifier

It is the application of Bayes’ Theorem that helps the Naive Bayes classifier tell spam from non-spam emails.

• Mechanism: Certain words or phrases in an email are monitored by the classifier to determine whether it is spam.
• Example: Depending on whether the words “offer” and “free” often appear in spam or non-spam emails, the algorithm decides how likely an unread message is to be spam.

Sentiment Analysis

The point is to judge if the text is positive, negative or neutral.

• Mechanism: To determine sentiments, Bayes examines and counts the keywords found in the provided checked data.
• Example: Whenever someone describes a post as “excellent” or “poor,” the algorithm uses these words as terms to classify it.

Computer Vision

Image Recognition and Classification

Applying Bayesian methods helps to show the level of certainty in the results of image recognition.

• Mechanism: The features of an image are used to give it a label and a possibility based on its probability.
• Example: With Bayesian techniques, cars can identify traffic signs by looking at the examples received in the captured data.

Robotics

Localisation and Mapping (SLAM)

Robots using SLAM rely on Bayes’ Theorem to map and navigate the environment.

• Mechanism: The more data the sensors produce, the better Bayesian inference updates the position and map of the robot.
• Example: In a warehouse, it is up to SLAM to allow robots to carry goods, choosing a route that avoids any obstacles along the way.

Healthcare

Disease Prediction Models

If we look at patients’ symptoms, use the outcomes of various tests and recall previous estimates, Bayes’ Theorem helps predict diseases more accurately.

• Mechanism: Similar cases from the past are used to predict a successful treatment for the current patient.
• Example: In oncology, chemotherapy is recommended for patients using the Bayesian method.

Personalised Medicine

Bayesian models help by providing personalised treatment based on a patient’s genes and health records.

• Mechanism: Similar cases from the past are used to predict a successful treatment for the current patient.
• Example: In oncology, chemotherapy is recommended for patients using the Bayesian method.

Recommender Systems

Dynamic User Preference Predictions

Making recommendations, recommender systems rely on Bayes’ Theorem.

• Mechanism: They use someone’s past interaction data and further details to determine the probability of them liking the product.
• Example: It depends on Bayesian methods to observe users’ activities and select their preferred movies to suggest new programs.

Advantages and Challenges of Using Bayes’ Theorem in AI

Bayes’ Theory helps AI by providing a way to think about situations in which data is incomplete. At the same time, it is beneficial, but it does have some disadvantages. To use Bayes’ Theorem successfully in AI, one must know about its advantages and disadvantages.

Advantages

Robustness to Limited Data

Bayes’ Theorem can yield useful results even when there is scarcely any evidence. Unlike most forms of machine learning, Bayesian methods can manage with fewer observations.

Example

Often, when the sample is low, a Bayesian model analyses limited data and predicts the chances of disease from past observation rates and a few test results.

Interpretability of Probabilistic Models

This theorem produces results in the form of probabilities, which makes it less difficult for stakeholders to understand the predictions of the model. Being transparent helps a lot in activities like healthcare and finance.

• Key Feature: It displays clearly how new findings (data) contribute to updating pre-existing beliefs about the process of deciding.
• Real-World Use: Bayesian methods are used in fraud detection systems to help explain the reason why a transaction has been detected as suspicious.

Since AI is interpretable, people trust it and make good choices.

Challenges

Computational Complexity

Using Bayesian inference is complicated, especially when there are many parameters and a large amount of data. To do these calculations, statisticians often use MCMC, which requires a lot of time and effort.

• Impact: Bayesian techniques can be slow, so they are rarely used in real-time systems.
• Possible Solution: While they do not give exact results, variational inference and tools such as PyMC3 or Stan help cut down the time it takes Bayesian computation.

Dependence on Accurate Priors

How accurate Bayesian models are depends on the probabilities used as prior knowledge. If the prior used is not correct or fully appropriate, the information in the model can become misleading, and it may not be effective.

Challenge Example: Setting a suitable prior is not easy in new fields with very little data and may rely on personal opinions.

Mitigation Strategies:

• Do not use priors when prior knowledge is not available, as this gives more even results.
• Use hierarchical Bayesian models to calculate priors based on the available data.

Because of this, experts recommend validating the model and ensuring deep knowledge of the field.

Till now, we have learned knowledge representation using first-order logic and propositional logic with certainty, which means we were sure about the predicates. With this knowledge representation, we might write A→B, which means if A is true, then B is true, but consider a situation where we are not sure about whether A is true or not, then we cannot express this statement; this situation is called uncertainty.

So, to represent uncertain knowledge, where we are not sure about the predicates, we need uncertain reasoning or probabilistic reasoning.

Causes of Uncertainty

The following are some leading causes of uncertainty to occur in the real world.

• Information occurred from unreliable sources.
• Experimental Errors
• Equipment fault
• Temperature variation
• Climate change

Understanding Probabilistic Reasoning

Probabilistic reasoning is a way of knowledge representation where we apply the concept of probability to indicate the uncertainty in knowledge. In probabilistic reasoning, we combine probability theory with logic to handle uncertainty.

We use probability in probabilistic reasoning because it provides a way to handle the uncertainty that is the result of someone’s laziness and ignorance.

In the real world, there are lots of scenarios where the certainty of something is not confirmed, such as “It will rain today,” “the behavior of someone in some situations,” or “A match between two teams or two players.” These are probable sentences for which we can assume that it will happen, but we are not sure about it, so here we use probabilistic reasoning.

Need for probabilistic reasoning in AI:

• When there are unpredictable outcomes.
• When specifications or possibilities of predicates become too large to handle.
• When an unknown error occurs during an experiment.

In probabilistic reasoning, there are two ways to solve problems with uncertain knowledge:

• Bayes’ rule
• Bayesian Statistics

Note: We will learn the above two rules in later chapters.

As probabilistic reasoning uses probability and related terms, before understanding probabilistic reasoning, let’s know some common terms:

Probability: Probability can be defined as the chance that an uncertain event will occur. It is the numerical measure of the likelihood that an event will occur. The value of probability always remains between 0 and 1, which represents ideal uncertainties.

• 0 ≤ P(A) ≤ 1, where P(A) is the probability of an event A.
• P(A) = 0 indicates total uncertainty in event A.
• P(A) =1 indicates total certainty in event A.

We can find the probability of an uncertain event by using the following formula.

• P(¬A) = probability of an event not happening.
• P(¬A) + P(A) = 1.
• Event: Each possible outcome of a variable is called an event.
• Sample Space: The collection of all possible events is called the sample space.
• Random Variables: Random variables are used to represent the events and objects in the real world.
• Prior Probability: The prior probability of an event is the probability computed before observing new information.
• Posterior Probability: The probability that is calculated after all evidence or information has been considered. It is a combination of prior probability and new information.

Conditional probability:

Conditional probability is the probability of an event occurring when another event has already happened.

Let’s suppose we want to calculate event A when event B has already occurred, “the probability of A under the conditions of B” can be written as:

Where,

P(A⋀B)= Joint probability of A and B

P(B)= Marginal probability of B.

If the probability of A is given and we need to find the probability of B, then it will be given as:

It can be explained by using the below Venn diagram, where B is the occurrence of an event, so the sample space will be reduced to set B, and now we can only calculate event A when event B has already occurred by dividing the probability of P(A⋀B) by P( B ).

Example:

In a class, there are 70% of the students like English and 40% of the students like English and mathematics. What is the percentage of students who like English and also like mathematics?

Solution:

Let A be an event that a student likes Mathematics

B is an event where a student likes English.

Hence, 57% are students who like English and Mathematics.

Probabilistic Models in AI

In the essence of artificial intelligence, the probabilistic models can help the efficient administration of uncertainty and can assist in depicting complex relations between variables.

Bayesian Networks

Belief networks, Bayesian Networks, are a more common name and show probabilistic dependencies among variables in the form of a graphical structure. They are composed of:

• Nodes: Every node in the Bayesian network is equivalent to a random variable, which could be another variable or continuous.
• Edges: Edges going from one node to another represent that a variable at the starting node affects the conditional probability of the variable at the end node.
• Conditional Probability Tables (CPTs): Each node contains a CPT that indicates the degree of dependence of the node in relation to the variables presented by its parent nodes.

For illustration, in a medical diagnosis network, an individual variable, such as “Fever”, might depend on “Infection”, denoted by arrows between the nodes and a CPT, special probability values.

Markov Models

Markov Chains

It is a probabilistic model used in modelling systems that evolve via state changes. Key characteristics include:

• Memoryless Property: The following state is dependent on the present state and not the one before it.
• State Transition Matrix: Shows opportunities for changing from one state to another.

A weather model may attempt to display changes in weather through its “Sunny,” “Cloudy”, and “Rainy” states.

Hidden Markov Models (HMM)

HMMS is based on Markov Chains but adds hidden (latent) states:

• Observed States: Outputs generated by the system.
• Hidden States: Factors that are undetected to let us observe something.
• Emission Probabilities: The Possibility of observing particular states with hidden elements.

Dynamic Bayesian Networks (DBNs)

Dynamic Bayesian Networks generalise the Bayesian Networks setting to be able to follow evolving processes that extend over many time steps. They illustrate how variables act over time, including both static and dynamic connections.

• Temporal Dependencies: Demonstrate the way that variables change from one time step to another.
• Transition Models: Describe the chance of being in a different state in the long run.

Applications of Probabilistic Reasoning

Natural Language Processing (NLP)

• Language Modelling: N-grams and neural probabilistic language models come under modelling systems based on probability, which judge a sequence of words determined by probability and to which text generation and autocomplete features owe their development.
• Speech Recognition: The process of empowering spoken language alignment with HMMs and probabilistic algorithms increases the quality of transcriptions because of higher accuracy.
• Machine Translation: A variety of statistical machine translation systems take advantage of the employment of probabilistic algorithms, providing both good and poor-mode translations as they relate to their contextual meaning.
• Sentiment Analysis: Bayesian approaches calculate the probabilities of certain sentiments being presented in a text, improving opinion analysis and sentiment classification.

Robotics and Autonomous Systems

• Localisation and Mapping: Techniques, such as Monte Carlo Localisation and SLAM, allow robots to localise and map their environment for easy navigation.
• Path Planning: Robots that calculate the probability of a danger-free condition of specific routes can move safely.
• Decision-Making under Uncertainty: Robots are equipped with Bayesian networks and MDPs to handle uncertain data and respond appropriately, hence appropriate for situations with insufficient or noisy information.
• Human-Robot Interaction: Probabilistic models allow robots to recognise human intentions, and this increases their cooperation and communication.

Medical Diagnosis and Decision Support

• Disease Diagnosis: Based on processing symptoms and test results, Bayesian networks establish the probabilities of occurrence for specific diseases, facilitating good diagnosis calls by medical personnel.
• Predictive Analytics: Information processed using probabilistic models assists healthcare providers in predicting how a disease will develop and where preventive measures will be required.
• Treatment Recommendation Systems: Algorithms analyse the medical history of a patient, genetic details, and previous responses to treatments to personalise therapy recommendations.
• Clinical Decision Support: Machine-based systems utilise probabilistic analysis to recommend diagnostic checks and interpret their results.

Recommender Systems

• Collaborative Filtering: Probabilistic models analyse user interactions, identifying repeating patterns, and suggest items that match similar user behaviours.
• Content-Based Recommendations: Applying Bayesian techniques, with the help of characteristics and their historical interactions, the probability of a user liking an item by a user is known.
• Hybrid Approaches: More accurate recommendations can be achieved from a combination of synergised probabilistic, collaborative, and content-based methods.
• Dynamic Preferences: When users change their preferences, algorithms adjust their recommendations based on the application of probabilistic temporal models.

Fraud Detection

• Anomaly Detection: Bayesian and probabilistic methods estimate the anomalies for transactions and indicate signs of possible fraud.
• Risk Scoring: Fraud detection systems judge if a transaction is fraudulent by using previous data and situational information.
• Network Analysis: Probabilistic graph models reveal hidden connections and activities characteristic of fraud in financial or social networks.
• Real-Time Decision-Making: Instant algorithms judge while risking further racist behaviour or financial ruin.

Despite the effectiveness of probabilistic reasoning in the management of uncertainty in decision-making, it is prone to be hampered by practical issues undermining its successful implementation. Addressing these issues is a prerequisite for enlarging the application of probabilistic reasoning in artificial intelligence.

Challenges in Probabilistic Reasoning

Scalability Issues

The more complex the AI system is, the more problematic the task of probabilistic models to deal with data and calculations will be.

• Large-Scale Networks: The manipulation of such many variables and dependencies that Bayesian networks and their counterparts require compels a great deal of computational power. As an example, the complexities of weather or financial markets require handling enormous data sets to make a correct model design.
• High-Dimensional Data: The more variables that are added, the more one gets into a condition where probability distributions are exponentially increased, thus effectively depicting the “curse of dimensionality”.
• Real-Time Applications: In practical situations such as self-driving cars and the recommendation of websites, there is an urgent need for immediate and fast inference capabilities. Performance in finding a balance point between speed and accuracy continues to pose two great challenges to probabilistic reasoning models in such applications.
• Potential Solutions: To solve these problems, new algorithms like variational inference, parallel computation, and frameworks such as TensorFlow Probability are employed.

Computational Complexity

Probabilistic reasoning models have their share of beautiful computations that may soon necessitate large amounts of processing.

• Exact Inference: Techniques such as variable elimination and belief propagation have exponential complexities under conditions that restrict their applicability to large-scale systems.
• Sampling Methods: Such techniques (Monte Carlo and Gibbs Sampling) can be computationally expensive (and require a lot of computational capacity) if a high degree of precision is needed.
• Dynamic Systems: Integrating the time-varying dynamics into Bayesian networks, where dynamic models are used, places additional computational demands, requiring the iterative application of state transition updates.
• Potential Solutions: Using hybrid algorithms that combine both deterministic and probabilistic methods and using GPU and TPU technology, computational inefficiencies can be overcome.

Data Sparsity and Quality

The probability model’s accuracy largely depends on the availability of high-quality and large data. Poor or thin data may produce unreliable inferences and wrong predictions.

• Sparse Data: Routine acquisition of complete and reliable data samples for the successful testing of probabilities can be quite problematic. It is generally difficult to model complex events such as system outages or catastrophic weather events because they are poorly reflected in the data sets.
• Noisy Data: Unhandled or noisier datasets can easily lead to biased outcomes and compromise the validity of inferences. This problem is particularly critical in such areas as medical diagnostics, in which mistakes in data interpretation can cause severe health risks.
• Imbalanced Data: When this data is not balanced among the various categories, probabilistic methods may generate biased predictions.
• Potential Solutions: As a solution for data sparsity and maintaining the quality of the data, practitioners regularly implement techniques such as data augmentation, transfer learning, and reliable statistical estimation strategies. Subject matter experts’ insights can substantially enhance probabilistic models when the coverage of the dataset is limited.

Tools and Frameworks for Probabilistic Reasoning in Artificial Intelligence

Underlying Artificial Intelligence (AI), there lies probabilistic reasoning, and there are a lot of specialised techniques and frameworks that are available to promote its use. With these tools, the construction and implementation of probabilistic models with inbuilt inference, learning, and simulation features are simplified.

Pyro

Developed based on PyTorch, Pyro allows the developers of such models to instantly build and deploy probabilistic models that are scalable and flexible.

Key Features:

• Permits the Bayesian inference and stochastic processes.
• Simplifies the development of neural network-based probabilistic models by integrating with PyTorch.
• Provides support for both variational inference and Markov Chain Monte Carlo (MCMC) approaches.
• Enables the creation of customised probabilistic frameworks.

Use Cases:

• Complex hierarchical Bayesian models.
• Time-series forecasting using probabilistic approaches.
• Robust efficiencies in developing machine learning models that support scientific research and experimental techniques.

TensorFlow Probability (TFP)

TensorFlow Probability adds modules for probabilistic modelling and high-end statistical computation to the functionality of TensorFlow.

Key Features:

• Supports many distribution, densities, and transformation operations.
• Provides capabilities for Bayesian inference, Monte Carlo sampling, and optimisation techniques.
• Plugging into TensorFlow enables the generation of hybrid models based on the combination of deep learning with probabilistic methodologies.
• Automatic differentiation for gradient-based optimisation.

Use Cases:

• Creating combined deep learning and statistical models for use in applications like uncertainty quantification.
• Statistical modelling of financial and healthcare data analysis.
• Examining the possibility of optimising predictions via the use of Bayesian neural networks.

Pomegranate

Pomegranate is a probabilistic modelling library for Python, focused on simplicity and efficiency.

Key Features:

• The library provides implementations for many probabilistic models, such as Bayesian networks, Hidden Markov Models, and Gaussian Mixture Models.
• Provides speed increases by using Cython.
• The design is modular, and it makes one’s customisation easy, and experimenting with different approaches is easier.
• Allows Model parameter estimation even where data is missing.

Use Cases:

• An application of probabilistic models to sequential data in such areas as speech and transcription recognition and bioinformatics.
• Application of probabilistic algorithms for clustering and classification in unsupervised learning setups.
• Fast real-time probabilistic inference tailored to embedded systems and robotics.