1. Should Anyone Panic? (40 points)

   Lucy goes to Hall Health about a sprained ankle, but while waiting for that, the nurse there chooses Lucy randomly to take part in a test for HPAI (high pathogenicity avian influenza), and this test involves a blood draw. Let's assume that in this season, one out of 1000 folks in Seattle are affected by HPAI. The HPAI test is 95% effective, meaning that there's only a 5% chance of a false positive. Let's assume the probability of a false negative is 0. Lucy's HPAI test result is positive. (a) What's the updated probability that she has HPAI?

   James attends a friend's marriage ceremony in Belize, and then he comes back to the U.S. Let's assume that 1 out of 80 people coming back from Belize come back with HPAI. James takes the same test that Lucy had, and his result is also positive. (b) What's the updated probably that he has HPAI?

   Should anyone panic? Should either of Lucy or James seek further assistance?

   Show your work, including whatever formulas you are using.

   Make a table showing the possible conditions, the possible outcomes, and their corresponding probabilities.

2. "The Mecha-Mouse at the Hostel for Travelling Droids" (60 points)

   The Hostel for Travelling Droids has four rooms: Dormitory (D), Lavatory (L), Pantry, and Mess Hall (M). There is a mechanical mouse ("Mecha-mouse") that inhabits the hostel, typically looking for a meal. The mouse has three actions: (X: exit current room; Y: alternative action; Z: remain as is). There is some danger than the "Compu-Cat" will ambush the mouse at any time, putting it in the Ambushed state, from which it can only go to the dead-end Kaput state. The activities in this hostel are governed by a Markov Decision Process with the following dynamics.

   s, a	Dormitory	Lavatory	Pantry	Mess Hall	Ambushed	Kaput
   Dormitory, X	0	0.4	0	0.6	0	0
   Dormitory, Y	0	0.6	0	0.4	0	0
   Dormitory, Z	0.75	0	0	0	0.25	0
   Lavatory, X	0.4	0	0.6	0	0	0
   Lavatory, Y	0.6	0	0.4	0	0	0
   Lavatory, Z	0	0.75	0	0	0.25	0
   Pantry, X	0	0.6	0	0.4	0	0
   Pantry, Y	0	0.4	0	0.6	0	0
   Pantry, Z	0	0	0.75	0	0.25	0
   Mess Hall, X	0.4	0	0.6	0	0	0
   Mess Hall, Y	0.6	0	0.4	0	0	0
   Mess Hall, Z	0	0	0	0.75	0.25	0
   Ambushed, *	0	0	0	0	0	1.0
   Kaput, *	0	0	0	0	0	1.0

   The reward here depends only on the current state s.

   s	R(s)
   Dormitory	0
   Lavatory	4
   Pantry	10
   Mess Hall	2
   Ambushed	-50
   Kaput	0

   1. Give the number of different policies that are possible for Mecha-mouse in the hostel.
   2. Manually apply the values iteration method to this problem for six iterations. Show the value at each state in each iteration. Assume that the discount factor is 0.5.
   3. Based on your analysis, give the optimal policy as an action for each state.

If necessary, updates and corrections will be posted here and mentioned in class or on GoPost.