{
"cells": [
{
"cell_type": "markdown",
"id": "d3027ced",
"metadata": {},
"source": [
"# Bayesian Machine Learning\n",
"\n",
"\n",
"- **[1]** (#) (a) Explain shortly the relation between machine learning and Bayes rule. \n",
" (b) How are Maximum a Posteriori (MAP) and Maximum Likelihood (ML) estimation related to Bayes rule and machine learning?\n",
"\n",
"\n",
"- **[2]** (#) What are the four stages of the Bayesian design approach?\n",
"\n",
"\n",
"- **[3]** (##) The Bayes estimate is a summary of a posterior distribution by a delta distribution on its mean, i.e., \n",
"$$\n",
"\\hat \\theta_{bayes} = \\int \\theta \\, p\\left( \\theta |D \\right)\n",
"\\,\\mathrm{d}{\\theta}\n",
"$$\n",
"Proof that the Bayes estimate minimizes the expected mean-squared error, i.e., proof that\n",
"$$\n",
"\\hat \\theta_{bayes} = \\arg\\min_{\\hat \\theta} \\int_\\theta (\\hat \\theta -\\theta)^2 p \\left( \\theta |D \\right) \\,\\mathrm{d}{\\theta}\n",
"$$\n",
"\n",
"\n",
"- **[4]** (###) We make $N$ IID observations $D=\\{x_1 \\dots x_N\\}$ and assume the following model\n",
"$$\n",
"x_k = A + \\epsilon_k \n",
"$$\n",
" where $\\epsilon_k = \\mathcal{N}(\\epsilon_k | 0,\\sigma^2)$ with known $\\sigma^2=1$. We are interested in deriving an estimator for $A$. \n",
" (a) Make a reasonable assumption for a prior on $A$ and derive a Bayesian (posterior) estimate. \n",
" (b) (##) Derive the Maximum Likelihood estimate for $A$. \n",
" (c) Derive the MAP estimates for $A$. \n",
" (d) Now assume that we do not know the variance of the noise term? Describe the procedure for Bayesian estimation of both $A$ and $\\sigma^2$ (No need to fully work out to closed-form estimates). \n",
"\n",
" \n",
"- **[5]** (##) We consider the coin toss example from the notebook and use a conjugate prior for a Bernoulli likelihood function. \n",
" (a) Derive the Maximum Likelihood estimate. \n",
" (b) Derive the MAP estimate. \n",
" (c) Do these two estimates ever coincide (if so under what circumstances)? \n",
"\n",
""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "27db69a5",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Julia 1.6.3",
"language": "julia",
"name": "julia-1.6"
},
"language_info": {
"file_extension": ".jl",
"mimetype": "application/julia",
"name": "julia",
"version": "1.6.3"
}
},
"nbformat": 4,
"nbformat_minor": 5
}