Laboratory 17: More on A/B Testing
In [100]:
# Preamble script block to identify host, user, and kernel
import sys
! hostname
! whoami
print(sys.executable)
print(sys.version)
print(sys.version_info)
Example 4: Times Roman or Times New Roman- That is the question!
The Daily Planet newspaper is considering swiching from the Times Roman Font to the Times New Roman as an attempt to to make it easier for their audience to read the newspaper and hence, increase the online purchases.¶
*hint: The Daily Planet is a fictional broadsheet newspaper appearing in American comic books published by DC Comics, commonly in association with Superman. Read more @ https://en.wikipedia.org/wiki/Daily_Planet
They have prepared two versions of their newspapers, one of each font. On the first day, they had a total of 1398 visitors. Randonmly, half of them (699 visitors) saw the original version (Time Roman) and the other half saw the alternative version (Times New Roman).
The first group purchased a total of 175 copies of the newspaper. This number was 200 copies for the second group. Based on the observations of the first day, first, calculate the purchase rate for each group and then, decide whether the Daily Planet should change their font in order to increase the online engagement time?
In [160]:
#Day 1
group1= 699
group2= 699
sold1= 175
sold2 = 200
rate1= sold1/group1
rate2 = sold2/group2
print(f"The ratio for the first group is {rate1:0.3f} copies sold per person")
print(f"The ratio for the second group is {rate2:0.3f} copies sold per person")
from scipy.stats import mannwhitneyu
import numpy as np
a_dist = np.zeros(group1)
a_dist[:sold1] = 1
b_dist = np.zeros(group2)
b_dist[:sold2] = 1
stat, p_value = mannwhitneyu(a_dist, b_dist, alternative="less")
print(f"Mann-Whitney U test for null hypothesis B <= A is {p_value:0.3f}")
After a week, they had a total of 10086 visitors. Randonmly, half of them (5043 visitors) saw the original version (Time Roman) and the other half saw the alternative version (Times New Roman).
The first group purchased a total of 1072 copies of the newspaper. This number was 1190 copies for the second group. Based on the observations of the first week, first, calculate the purchase rate for each group and then, decide whether the Daily Planet should change their font in order to increase the online engagement time?
In [161]:
#Week 1
group1= 5043
group2= 5043
sold1= 1072
sold2 = 1190
rate1= sold1/group1
rate2 = sold2/group2
print(f"The ratio for the first group is {rate1:0.3f} copies sold per person")
print(f"The ratio for the second group is {rate2:0.3f} copies sold per person")
from scipy.stats import mannwhitneyu
import numpy as np
a_dist = np.zeros(group1)
a_dist[:sold1] = 1
b_dist = np.zeros(group2)
b_dist[:sold2] = 1
stat, p_value = mannwhitneyu(a_dist, b_dist, alternative="less")
print(f"Mann-Whitney U test for null hypothesis B <= A is {p_value:0.3f}")
After a month, they had a total of 42000 visitors. Randonmly, half of them (21000 visitors) saw the original version (Time Roman) and the other half saw the alternative version (Times New Roman).
The first group purchased a total of 4300 copies of the newspaper. This number was 5700 copies for the second group. Based on the observations of the first month, first, calculate the purchase rate for each group and then, decide whether the Daily Planet should change their font in order to increase the online engagement time?
In [162]:
#Month 1
group1= 21000
group2= 21000
sold1= 4300
sold2 = 5700
rate1= sold1/group1
rate2 = sold2/group2
print(f"The ratio for the first group is {rate1:0.3f} copies sold per person")
print(f"The ratio for the second group is {rate2:0.3f} copies sold per person")
from scipy.stats import mannwhitneyu
import numpy as np
a_dist = np.zeros(group1)
a_dist[:sold1] = 1
b_dist = np.zeros(group2)
b_dist[:sold2] = 1
stat, p_value = mannwhitneyu(a_dist, b_dist, alternative="less")
print(f"Mann-Whitney U test for null hypothesis B <= A is {p_value:0.3f}")
More on Confidence Interval
Example5: From a normally distributed population, we randolmy took a sample of 100 adults with a mean height of 165 centimeters. Suppose the standard deviation of the population is 8:
What is the estimated true population mean for the 95% confidence interva?
How about 90% confidence interval?
How about 99% confidence interval?
In [163]:
# Step 1- Organize the data
n = 100 #Sample size
Xbar = 165 #Sample mean
std = 8 #Standard deviation (σ)
z_95 = 1.960 #The z value associated with 95% Confidence Interval
z_90 = 1.645 #The z value associated with 90% Confidence Interval
z_99 = 2.576 #The z value associated with 90% Confidence Interval
In [164]:
# Step2- Calculate the margin of error
import math
margin_95 = z_95*(std/math.sqrt(n))
print('The margin of error for 95% Confidence is equal to : ',margin_95)
margin_90 = z_90*(std/math.sqrt(n))
print('The margin of error for 90% Confidence is equal to : ',margin_90)
margin_99 = z_99*(std/math.sqrt(n))
print('The margin of error for 99% Confidence is equal to : ',margin_99)
In [165]:
# Step3- Find the estimated true population mean
low_95 = Xbar-margin_95
high_95 = Xbar+margin_95
print('the true population mean will be captured within the confidence interval of (',low_95,' , ',high_95,') and the confidence is 95%')
low_90 = Xbar-margin_90
high_90 = Xbar+margin_90
print('the true population mean will be captured within the confidence interval of (',low_90,' , ',high_90,') and the confidence is 90%')
low_99 = Xbar-margin_99
high_99 = Xbar+margin_99
print('the true population mean will be captured within the confidence interval of (',low_99,' , ',high_99,') and the confidence is 99%')
We took another sample of the same size and this time the mean height was 167 centimeters.
How does this new piece of information helps us in getting closer to a better estimation of the population mean?
In [166]:
# Step 1- Organize the data
n = 100 #Sample size
Xbar = 167 #Sample mean
std = 8 #Standard deviation (σ)
z_95 = 1.960 #The z value associated with 95% Confidence Interval
z_90 = 1.645 #The z value associated with 90% Confidence Interval
z_99 = 2.576 #The z value associated with 90% Confidence Interval
In [167]:
# Step2- Calculate the margin of error
import math
margin_95 = z_95*(std/math.sqrt(n))
print('The margin of error for 95% Confidence is equal to : ',margin_95)
margin_90 = z_90*(std/math.sqrt(n))
print('The margin of error for 90% Confidence is equal to : ',margin_90)
margin_99 = z_99*(std/math.sqrt(n))
print('The margin of error for 99% Confidence is equal to : ',margin_99)
In [168]:
# Step3- Find the estimated true population mean
low_95 = Xbar-margin_95
high_95 = Xbar+margin_95
print('the true population mean will be captured within the confidence interval of (',low_95,' , ',high_95,') and the confidence is 95%')
low_90 = Xbar-margin_90
high_90 = Xbar+margin_90
print('the true population mean will be captured within the confidence interval of (',low_90,' , ',high_90,') and the confidence is 90%')
low_99 = Xbar-margin_99
high_99 = Xbar+margin_99
print('the true population mean will be captured within the confidence interval of (',low_99,' , ',high_99,') and the confidence is 99%')
In [ ]:
