Practice 1 Solutions

Note

Students may have different answers. Answers should be explained to justify any response. 

Use the following scenario for problems 1–5.  

A small restaurant group tests two menu designs to see if they impact the amount of money diners spend on their meals. They randomly selected 200 customers. By table, customers were either provided a new menu with photos and text or the standard menu with text only.

After the experiment, a simulation test with 10,000 trials was conducted. The standard deviation of the simulated differences was $1.66. An observed difference of $3.30 occurred in 234 simulations.

Group
n = 100
Average Check Amount
Treatment:
new menu
$31.95
Control:
standard menu
$28.65

Implementation details: New menus cost $7.50 each and are replaced every six months (compared to replacing the old menus every 2 years). An average of 6,000 customers are served monthly across all four locations.

  1. Calculate the z-score for the observed difference. Explain the statistical significance at the 5% level.

z=3.301.66=1.988

Because 1.988>1.96, the new menu shows statistically significant results for the average check amount at the 5% level.

Note

Q: What would you say to a person who says “1.988 is too close to 1.96 and not significant enough”?

A: Anything greater than 1.96 is significant, even by a few decimal places.

  1. Calculate the expected additional yearly revenue (earnings) if the new menu were to be implemented.

31.9528.65=3.303.306000=19,8001980012=237,600

One customer check increases $3.30. The restaurant should expect a revenue increase of $237,600 over one year (12 months) if an average of 6,000 customers are served monthly.

  1. Determine the annual cost for implementing 200 new menus at all locations. 

7.502=15154200=12,000

Each menu needs to be replaced every six months for an annual cost of $15 each. With 200 menus needed at each of the four locations, the total annual cost would be $12,000.

Note

Individual restaurants may need fewer/more menus. In inferential statistics, sometimes assumptions need to be made to provide estimates.

  1. Based on the revenue and costs, should the restaurant group adopt the new menu? Explain.

$237,600$12,000=$225,600

The restaurant group should adopt new menus. After the cost of the menus is subtracted from the revenue, the restaurant is expected to earn an additional $225,600 annually.

Note

“Revenue” is how much money the restaurant group brings in before deducting any expenses.

  1. What assumptions did you make to complete the cost analysis? What additional information would strengthen your recommendation?

Sample:
Assumptions: number of menus needed at each location, each check represents one customer, no other costs affect the decision about getting a new menu

A stronger recommendation would require the exact number of menus needed, as well as any other costs that would affect purchasing new menus.

Use the following scenario for problems 6–8.

A technology company claimed its brain games increased test scores for college students. The company randomly selected 400 students to participate. The games cost $3.75 per month, billed annually. A 10,000 trial simulation was performed assuming no difference between treatments.

Group
n = 200
Mean Test Score
Treatment: brain games 81.5% (B–)
Control: placebo games 79.8% (C+)
Standard deviation 5.86
  1. Calculate the z-score. Explain the statistical significance at the 5% level. 

81.579.8=1.7z=1.75.86=0.290

Because 0.290<1.96, the brain games do not produce significant results (at any level because the z-score is so close to zero).

  1. Press Release: “Brain games improve students’ test scores by a full letter grade! Sign up now to improve grades for a minimal cost.” Does the technology company have evidence to support this headline? Explain.

Sample: The press release is misleading because an increase of 1.7 percentage points is not a full letter grade. The results show improvement from a C+ to a B–, which is not a full letter grade (10% points).

  1. Suppose you purchased a month of the brain games and had no change in your test scores. Write a short response requesting a full refund. Remember to use statistics to justify your reasoning.

3.7512=45

Sample: To whom it may concern, I am requesting a full $45 refund. Your brain game products promised to improve my test scores by a full letter grade, which would be a 10% increase. Because my test scores have shown zero change since I started using this product, a refund is required.

Use the following scenario for problems 9–12.

A physical therapy office was testing a new rehabilitation technique on 90 patients recovering from shoulder surgery.

The new technique requires $15,000 in new equipment and a $2,000 certification course for physical therapists.

Group
n = 45
Recovery in Weeks Pain Score (0–10)
Control: standard technique 13.8 3.6
Treatment: new technique 11.2 2.8

The simulation results showed a simulated difference standard deviation of 1.12 weeks of recovery and 0.62 points for pain scores.

  1. Calculate the z-score for the observed differences of both recovery and pain score. Explain the statistical significance at the 5% level.

Recovery timePain score13.811.2=2.63.62.8=1.2z=2.61.12=2.321z=1.20.62=1.935

Because 2.321>1.96, the new technique shows statistically significant recovery times. While the pain scores are reduced with the new technique, the difference is not statistically significant at the 5% level.

  1. From the patient’s perspective, evaluate the practical significance of both recovery time and pain score.

Sample:  Practically speaking, both results are significant because patients are able to recover more quickly with the new treatment and have reduced pain at the same time.

  1. Suppose the office treats 200 shoulder patients annually, and three physical therapists would need to be certified if the new technique were implemented. Should the practice invest in the new technique? (Remember to consider statistics as well as practical outcomes.)

$15,000+3$2,000=$21,000

Sample: The initial cost of the new technique will be $21,000. If recovery time is reduced by 2.6 weeks for most patients, it is possible that additional patients could be seen at the office. If more patients are seen, this could cover the extra cost of switching to the new technique. Patients will like a shorter recovery time because it will reduce their medical costs. 

Note

Because of the variability of this outcome, here is another perspective-


Sample: The office should not invest in the new method because keeping the standard technique would increase patient time in the office, which would produce more guaranteed income. 

  1. Explain what might have resulted in the variables showing mixed results. What could this mean for the physical therapist when making decisions?

Sample: The results could be mixed because a pain score is the patient’s personal feeling about their pain, and each person has a different pain tolerance. On the other hand, the recovery time is tracking time. A physical therapist needs to consider that with the two variables, they can see that the treatment reduces time and pain. Both are critical to patient recovery.

Use the following scenario for problems 13–15.

A high school statistics class conducted an experiment to test whether using phones before bed affects sleep quality. The class randomly selected 150 schoolmates to participate. 

The class used a randomization test to simulate 10,000 trials. The standard deviation of the simulated differences was 0.41 hours.

Group
n = 75
Mean Hours of Sleep
Treatment: no phone one hour before bed 7.7
Control: normal phone use 6.8

Additional data: Health experts recommend teenagers sleep 8–10 hours each night.
Student participants who slept more reported feeling more alert and scored an average of four points higher on weekly tests.

  1. Calculate the z-score for the observed differences. Explain the statistical significance. 

7.76.8=0.9z=0.90.41=2.195

Because 2.195>1.96, not using your phone for one hour before bed is statistically significant at the 5% level.

  1. Is the treatment meaningful despite it falling short of the recommended 8–10 hours of sleep each night? Explain. 

Sample: The treatment is meaningful because it is both statistically and practically significant. Students reporting they “feel more alert” is important for learning, and it shows in their improved test scores. 

  1. Would you personally try putting your phone away one hour before bed? Explain your decision and consider the challenges of following this rule.

Sample: I would consider putting my phone away before bed. Based on the data, I would have a better night’s sleep. It could be challenging if I use a study app on my phone before bed or if I need to set a phone alarm. 

Customer Service

Monday–Thursday 8:30am–6pm ET