lec-1 probabilistic models.ppt

8/10/2019 lec-1 probabilistic models.ppt

1/29

Probabilistic models

Haixu Tang

School of Informatics


2/29

Probability

Experiment: a procedure involving

chance that leads to different results

Outcome:the result of a single trial of an

experiment;

Event:one or more outcomes of an

experiment;

Probability:the measure of how likely an

event is;


3/29

Example: a fair 6-sided dice

Outcome: The possible outcomes of

this experiment are 1, 2, 3, 4, 5 and

6;

Events: 1; 6; even

Probability: outcomes are equal ly

l ikelyto occur. P(A) = The Number Of Ways Event A Can Occur / The Total

Number Of Possible Outcomes

P(1)=P(6)=1/6; P(even)=3/6=1/2;


4/29

Probability distribution

Probability distribution: the assignment of a

probability P(x) to each outcome x.

A fair dice: outcomes are equally l ikelyto occur

the probability distribution over the all sixoutcomes P(x)=1/6, x=1,2,3,4,5 or 6.

A loaded dice: outcomes are unequal ly l ikelyto

occurthe probability distribution over the all

six outcomes P(x)=f(x), x=1,2,3,4,5 or 6, but

f(x)=1.


5/29

Example: DNA sequences

Event: Observing a DNA sequence S=s1s2sn:

si{A,C,G,T};

Random sequence model (or Independent and

identically-distributed, i.i.d.model): si occurs atrandom with the probability P(si), independent

of all other residues in the sequence;

P(S)=

This model will be used as a background

model (or called null hypothesis).

n

i

isP

1


6/29

Conditional probability

P(i|): the measure of how likely an event ihappens under the condition ; Example: two dices D1, D2

P(i|D1)probability for picking i using dicer D1

P(i|D2)probability for picking i using dicer D2


7/29

Joint probability

Two experiments X and Y

P(X,Y)joint probability (distribution) of experiments

X and Y

P(X,Y)=P(X|Y)P(Y)=P(Y|X)P(X) P(X|Y)=P(X), X and Y are independent

Example: experiment 1 (selecting a dice),

experiment 2 (rolling the selected dice)

P(y): y=D1or D2

P(i, D1)=P(i| D1)P(D1)

P(i| D1)=P(i| D2), independent events


9/29

Probability models

A system that produces different outcomes

with different probabilities.

It can simulate a class of objects (events),

assigning each an associated probability.

Simple objects (processes)probability

distributions


10/29

Example: continuous variable

The whole set of outcomes X (xX) canbe infinite.

Continuous variablex[x0,x

1]

P(x0xx1) ->0

P(x-dx/2 x x+dx/2) = f(x)dx; f(x)dx=1

f(x)probability density function (density, pdf)

P(xy)= y

x0f(x)dxcumulated density function (cdf)

x0

x1

x1

dx


11/29

Mean and variance

Mean

m=xP(x)

Variance

2= (k-m)2P(k)

: standard deviation


12/29

Typical probability distributions

Binomial distribution

Gaussian distribution

Multinomial distribution Dirichlet distribution

Extreme value distribution (EVD)


13/29

Binomial distribution

An experiment with binary outcomes: 0 or 1;

Probability distribution of a single experiment:

P(1)=p and P(0) = 1-p;

Probability distribution of N tries of the same

experiment

Bi(k 1s out of N tries) ~kNk

ppk

N

)1(


14/29

Gaussian distribution

N -> , Bi -> Gaussian distribution

Define the new variable u = (k-m)/ f(u)~ 2/exp 2

2

1 u


15/29

Multinomial distribution

An experiment with K independent outcomes

with probabilities i, i =1,,K, i =1.

Probability distribution of N tries of the same

experiment, getting nioccurrences of outcome i,

ni =N.

M(N|) ~

K

i

n

i

i

i

i

n

N

1!

!


16/29

Example: a fair dice

Probability: outcomes (1,2,,6) are equal ly

l ikelyto occur

Probability of rolling 1 dozen times (12) andgetting each outcome twice:

~3.410-3 1261

2

!126


17/29

Example: a loaded dice

Probability: outcomes (1,2,,6) are

unequal ly l ikelyto occur: P(6)=0.5,

P(1)=P(2)==P(5)=0.1

Probability of rolling 1 dozen times (12) and

getting each outcome twice:

~1.8710-4

102

2

!12

1.05.06


18/29

Dirichlet distribution

Outcomes: =(1, 2,,K)

Density: D(|a)~

(a1, a2,,aK) are constantsdifferent agives different probability distribution over

.

K=2Beta distribution

1

11

1K

i

i

K

i

ii

da


19/29

Example: dice factories

Dice factories produces all kinds of dices:

(1), (2),, (6)

A dice factory distinguish itself from the

others by parameters aa1,a2 ,a3 ,a4 ,a5 ,a6

The probability of producing a dice in thefactory ais determined by D(|a)


20/29

Extreme value distribution

Outcome: the largest number among N

samples from a density g(x)is larger than

x;

For a variety of densities g(x),

pdf:

cdf:


21/29

Probabilistic model

Selecting a model Probabilistic distribution

Machine learning methods Neural nets

Support Vector Machines (SVMs) Probabilistic graphical models

Markov models

Hidden Markov models

Bayesian models

Stochastic grammars Modeldata (sampling)

Datamodel (inference)


22/29

Sampling

Probabilistic model with parameter P(x| )for event x;

Sampling: generate a large set of eventsxiwithprobability P(xi|);

Random number generator ( function rand()picks a number randomly from the interval [0,1)with the uniform density;

Sampling from a probabilistic model

transforming P(xi| ) to a uniform distribution For a finite set X (xiX), find i s.t. P(x1)++P(xi-1)

lec-1 probabilistic models.ppt

Documents

Transcript of lec-1 probabilistic models.ppt