Probability and inference
How much you trust your experiment?
As a new graduate student, you are picking up a project from a postdoc in the lab who is about to leave lab. This postdoc tells you that her experiments fail at a rate of 1 in 5. Your warmup job is to figure out why, and to see how you can improve that performance.
You start running the experiment with your own hands. After two consecutive success (ss), your experiment fails 3 times in a row
Do you need to start worrying about not performing the experiment correctly? (Calculate the posterior probability distribution of the probability of a failure f given the data you have, and draw conclusions.)
Just in case, you go back to the postdoc, who fixes some details in your protocol, and now you are confident you are running the experiment correctly. You continue, and in the next set, you obtain
What is your current estimate of f? What is the maximum probability value, and mean?
You want to be very certain about your conclusions before you go back to talk to the postdoc about the real failure rate of this experiment. So you decide to continue doing experiments until you have a confidence interval of . Here is a long series of consecutive outcomes for you to use (file homework.dat):
ssfsffffssfsfsssssssffsssssfsssffffsssssss fsssssfsssfsssffssfsfsfsfssssssssfsssssfss fssssffsssfsssffsssssssfssssffsfssssfsfffs ssfssssssfsssffssssssssssssssssssfffssssss ffssssssfsssfsfssfssssssffssssfs
At least how many tests do you need to use to get your estimate to a error, and what is the failure rate you are going to report back to the postdoc?