-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathbayesian.html
238 lines (208 loc) · 11 KB
/
bayesian.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Bayesian Theorem</title>
<link rel="stylesheet" href="styles.css">
<!-- Latest compiled and minified CSS -->
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/[email protected]/dist/css/bootstrap.min.css"
integrity="sha384-BVYiiSIFeK1dGmJRAkycuHAHRg32OmUcww7on3RYdg4Va+PmSTsz/K68vbdEjh4u" crossorigin="anonymous">
<!-- Latest compiled and minified JavaScript -->
<script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/js/bootstrap.min.js"
integrity="sha384-Tc5IQib027qvyjSMfHjOMaLkfuWVxZxUPnCJA7l2mCWNIpG9mGCD8wGNIcPD7Txa"
crossorigin="anonymous"></script>
</head>
<body>
<nav class="navbar navbar-inverse">
<div class="container">
<div class="navbar-header">
<button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar"
aria-expanded="false" aria-controls="navbar">
<span class="sr-only">Toggle navigation</span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="#">Bootstrap theme</a>
</div>
<div id="navbar" class="navbar-collapse collapse">
<ul class="nav navbar-nav">
<li class="active"><a href="#">Home</a></li>
<li><a href="#">About</a></li>
<li><a href="#">Contact</a></li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true"
aria-expanded="false">Dropdown <span class="caret"></span></a>
<ul class="dropdown-menu">
<li><a href="#">Action</a></li>
<li><a href="#">Another action</a></li>
<li><a href="#">Something else here</a></li>
<li role="separator" class="divider"></li>
<li class="dropdown-header">Nav header</li>
<li><a href="#">Separated link</a></li>
<li><a href="#">One more separated link</a></li>
</ul>
</li>
</ul>
</div><!--/.nav-collapse -->
</div>
</nav>
<header>
<h1>Bayesian Theorem</h1>
</header>
<main>
<section id="fundamentals">
<h2>Fundamentals of Bayesian Theorem</h2>
<p>
Bayesian theorem, also known as Bayes' rule or Bayes' law, is a probabilistic theorem that describes the
relationship between the conditional probabilities of two events. It is named after Thomas Bayes, an
18th-century
English statistician and philosopher. The theorem is a fundamental concept in probability theory and
statistics,
providing a mathematical framework for updating our beliefs when given new evidence.
</p>
<p>
The theorem is written as P(A|B) = (P(B|A) * P(A)) / P(B), where P(A|B) is the probability of event A
happening,
given that event B has occurred, P(B|A) is the probability of event B happening, given that event A has
occurred, and P(A) and P(B) are the probabilities of events A and B happening independently.
<div>
<img src="bayesian.jpg" alt="Bayes' Theorem">
</div>
</p>
</section>
<section id="history">
<h2>History of Bayesian Theorem</h2>
<p>
Bayesian theorem is named after Reverend Thomas Bayes, who first formulated the theorem in the 18th
century. He
developed the theorem in the context of solving a specific problem in probability theory known as the
"inverse
probability problem." However, it was not until after Bayes' death that his work was published by
Richard Price,
another prominent mathematician of the time.
</p>
<p>
In the years that followed, Bayesian theorem gained wider acceptance and was further developed by
mathematicians
such as Pierre-Simon Laplace. Over time, the theorem became an essential tool in the field of statistics
and
probability, with numerous applications across various disciplines, including science, engineering,
economics, and
more recently, artificial intelligence and machine learning.
</p>
</section>
<section id="application">
<h2>How Bayesian Theorem Applies to Large Language Models</h2>
<p>
Bayesian theorem is an essential component of many machine learning algorithms, including large language
models
(LLMs). In the context of LLMs, Bayesian methods can be used to estimate the parameters of the model,
update
beliefs about the model's parameters as new data is observed, and make predictions based on the current
state of
the model.
</p>
<p>
One common application of Bayesian theorem in LLMs is through the use of Bayesian inference, where the
model's
parameters are updated iteratively as new data is processed. This allows the model to learn from the
data and
make more accurate predictions over time.
</p>
</section>
<section id="effectiveness">
<h2>Why Bayesian Theorem Works So Well in Large Language Models</h2>
<p>
Bayesian theorem works well in large language models because it provides a principled framework for
updating
</p>
</section>
<section id="application">
<h2>How Bayesian Theorem Applies to Large Language Models</h2>
<p>
Bayesian theorem is an essential component of many machine learning algorithms, including large language
models
(LLMs). In the context of LLMs, Bayesian methods can be used to estimate the parameters of the model,
update
beliefs about the model's parameters as new data is observed, and make predictions based on the current
state of
the model.
</p>
<p>
One common application of Bayesian theorem in LLMs is through the use of Bayesian inference, where the
model's
parameters
</p>
</section>
<section id="backpropagation">
<h2>How does back propogate and Bayesian Theorem Works Together</h2>
<p>
Bayesian theorem and backpropagation are two different ideas, but they can work together in some
situations.
Imagine that Bayesian theorem is like a detective who uses clues to guess who might be the criminal.
Backpropagation is like a teacher who helps a student learn by telling them which answers are right and
wrong.
When we have a big computer brain called a neural network, we sometimes want to help it learn using both
the detective's clues (Bayesian theorem) and the teacher's advice (backpropagation). In some cases, they
can work together to make the computer brain smarter and better at understanding things.
</p>
<!--- what you have assumed -->
<h1>
Reflections
</h1>
<p>I am assuming that P(A) is probability of taking the coin of A.</p>
<p> I am then assuming that P(B) is the proability of taking coin B</p>
<p> i have no idea what is the calculation for posterior distribution> seems like it is a
distribution, means a bunch of numbers or graph. it needs to be based of existing knowledge or smt
</p>
<!-- # what you have understood -->
<h1>
What I have understood
</h1>
<p>The theorem is written as P(A|B) = (P(B|A) * P(A)) / P(B), where P(A|B) is the probability of event A
happening,
given that event B has occurred, P(B|A) is the probability of event B happening, given that event A has
occurred, and P(A) and P(B) are the probabilities of events A and B happening independently.
</p>
<p>
i need to ignore the conditional probability first because i dont know what is that value.
</p>
<p>
coin a is 0.5 and coin b is 0.75. but given the coin tosses, it is telling us whether it is heads or tails
</p>
<p>
this means that i have P(T) is actually 5 tosses, 0.25 and P(H) is actually 0.75 probability which is 15 times.
total times is actually 20 times.
instead of assuming of which coin did i take.
i want to assume which probability of it is heads and which probability is tails
P(T)/P(H) = 0.25/0.75 = 0.3333333333333333
but what is the conditional probability of B given A?
P(B|A) = ?
P(B|A) = 0.25*0.75 = 0.1875
then what about p(A|B)?
= 0.1875*0.3333333333333333 = 0.0625
so the conditional probability of A given B is 0.0625
So what can i do with the conditional probability?
</p>
<h1> what went wrong </h1>
<p>
<ul>
<li>
0.5 assumption and 0.75 assumption is wrong.
</li>
<li>
i used the fraction of the probability to sub into the equation
</li>
<li>
instantly assuming that the probability is how much.
</li>
<li>
what helped is and using actual values to solve the equation.
</li>
<li>
another misconception is that the distribution is a single number. NO the distribution is a bunch of number. it distributes
</li>
</ul>
</p>