Topic Theory Manual Solving Simulation Coding
Introduction to T- - What is a t-test? - - Define the purpose - Simulate a sample - Write a Python function
Test Why is it used? - of a t-test.- Explain from a normal to check whether data
Assumptions of a t- why a t-test requires distribution and meets t-test assumptions.
test normally distributed check the validity of
data. t-test assumptions.
Types of T-Test - One-sample t-test - - Manually calculate - Simulate data for - Write Python functions
Two-sample t-test test statistics for all each type of t-test to implement all three
(independent) - three types of t-tests. and check if results types of t-tests.
Paired t-test align with
expectations.
Hypothesis Testing - Null and - Write down - Simulate data - Write a Python function
alternative hypotheses for a where you expect to that outputs a p-value and
hypotheses - P- practical example.- reject/fail to reject decision rule based on
value interpretation Compute test the null hypothesis. input data.
- Type I and Type II statistics and
errors interpret the p-value.
One-Sample T-Test - Formula and - Calculate test - Generate a single - Write a function to
interpretation - statistic and p-value sample and test if it perform a one-sample t-
When to use for a single dataset. differs from a test and display output.
known population
mean.
Two-Sample T-Test - Formula and - Calculate the test - Simulate two - Write a Python function
(Independent) interpretation - statistic for two samples and test if for a two-sample t-test
Equal vs unequal independent datasets they have equal with an option for
variance manually. means. equal/unequal variance.
Paired T-Test - Formula and - Manually calculate - Simulate - Write a Python function
interpretation - the test statistic for before/after to perform a paired t-test
When to use paired data. treatment data and and interpret the results.
test for a significant
effect.
Assumptions of T- - Normality - - Test whether - Simulate data to - Write a Python function
Test Independence - sample data meets t- test for violations of to check normality and
Homogeneity of test assumptions. assumptions. variance equality.
variance
Degrees of Freedom - Definition - How it - Calculate degrees of - Simulate how - Write a Python function
affects the t- freedom for a sample degrees of freedom to calculate degrees of
distribution manually. change with sample freedom for different
size. sample sizes.
P-Value and - What is a p-value? - Interpret a p-value - Simulate how the - Write a function to
Significance Level - How to choose a for different test p-value changes calculate and interpret a
significance level statistics. with sample size p-value.
and variability.
Confidence Intervals - How to calculate - - Manually compute - Simulate - Write a Python function
Relationship to t- confidence intervals confidence intervals to compute and visualize
test for a sample mean. and observe their confidence intervals.
variation.
T-Distribution vs - Differences and - Compute the shape - Simulate t- - Write a function to plot
Normal Distribution similarities - When of the t-distribution distribution vs and compare t-
to use each manually. normal distribution distribution vs normal
with different distribution.
degrees of freedom.
One-Tailed vs Two- - When to use each - Compute and - Simulate one- - Write a Python function
Tailed Tests type - Impact on p- compare one-tailed tailed vs two-tailed to perform one-tailed and
value vs two-tailed p- scenarios. two-tailed tests.
values.
Effect Size and - Cohen's d - - Manually calculate - Simulate datasets - Write a Python function
Power Statistical power Cohen's d and to estimate to calculate Cohen's d and
interpret results. statistical power. statistical power.
Alternative - When not to use a - Describe scenarios - Simulate data - Write Python code for a
Approaches t-test - Non- where a t-test is where assumptions Wilcoxon test or other
parametric inappropriate. are violated and non-parametric tests.
alternatives (e.g., explore
Wilcoxon test) alternatives.
Test Type When to Use
One-sample t-test Compare a sample mean to a known value (e.g., μ₀ = 7)
(scribbr.com)
Independent (two-sample) Compare means of two independent groups
t-test
Paired t-test Compare means of paired observations, like pre-post
measurements
Welch’s t-test Use when variances and/or sample sizes are unequal
One- vs Two-tailed Tests / Directionality
Two-tailed: Testing for any difference.
One-tailed: Testing a specific direction (e.g., Group A > Group B)
One-Sample t-test
🔍 Goal: Compare a sample mean to a known or target value.
📌 Use Case Example (Product Metrics – Amazon)
Scenario: Amazon’s product team claims that the average delivery time for a Prime order is within 2 days.
You collect a sample of 100 orders, and the average delivery time is 2.3 days.
💡 You want to test if this observed mean is significantly different from the target (2 days).
✅ Use:
# One-sample t-test checks if the sample average is different from 2.
# Null Hypothesis: μ = 2
💼 Why it's useful:
Helps validate service level agreements (SLAs).
Useful in benchmarking product KPIs against goals or promises.
✅ 2. Two-Sample (Independent) t-test
🔍 Goal: Compare means of two independent groups.
📌 Use Case Example (A/B Testing – Netflix UI)
Scenario: Netflix is testing two different UI layouts:
Group A (old UI)
Group B (new UI)
You want to test if the average watch time per user is higher with the new UI.
Why it's useful:
Commonly used in A/B testing, marketing campaigns, pricing models.
Helps data-driven decisions about feature rollouts or UI changes.
Paired t-test (Dependent Samples)
🔍 Goal: Compare means from the same group at two different times or under two conditions.
📌 Use Case Example (Before/After Feature Rollout – Spotify Personalization)
Scenario: Spotify introduces a new recommendation algorithm.
You compare user listening hours before and after the algorithm update for the same users.
Why it's useful:
Helps measure impact of features or updates on the same user base.
Controls for user-level variability.
Difference between 2-sample and paired t test
https://chatgpt.com/c/684eab1c-0ecc-8006-998a-5dfac04562c1
ASSUMPTION VIOLATION
Randomness: The data needs to be randomly sampled from the population.
This assumption can be checked by examining the design of the experiment and the sampling strategy.
If the randomness assumption is violated, we need to consider re-design the experiment and re-sample
the data.
Normality: The t-test assumes that both populations are normally distributed.
The normality of the data can be checked using visualization such as histogram or QQ plot. It can also be
checked by statistical tests such as the Shapiro-Wilk test, the D’Agostino’s K-Squared test, or the
Kolmogorov-Smirnov test.
When the normality assumption is violated, try data transformation such as logarithm or square root, and
check the normality of the transformed dataset.
If the dataset is still not normally distributed after transformation, we can use the Wilcoxon test, a non-
parametric test that does not assume a normal data distribution.
Equal Variance: The population variances are the same.
Equal variance can be checked using visualization such as boxplot or histogram. It can also be checked
by statistical tests such as Levene’s test or F test.
The Welch test should be used when the equal variance assumption is violated and the normality
assumption is satisfied.
When both the equal variance and the normality assumptions are violated, the non-parametric Wilcoxon
test should be used.
CODING
https://www.geeksforgeeks.org/t-test/
T-DISTRIBUTION
make me under the technical or conceptual understanding 'why though different t-test has different
formulas but all follow t-distribution'?
https://chatgpt.com/c/684eb2a6-e30c-8006-9d52-3e46216ab9c1
Z-TEST
CODING
https://www.geeksforgeeks.org/z-test/
Z-distribution
https://chatgpt.com/c/684eb63b-1a20-8006-a1ee-0fd33e27bc9c
chi-square
https://chatgpt.com/c/684eb7e9-4e90-8006-869c-76489a22b668
https://chatgpt.com/c/684eb7b6-8ce0-8006-a838-fd71c87ec132
ANOVA
https://www.datacamp.com/tutorial/anova-test
https://www.geeksforgeeks.org/maths/anova-formula/
https://chatgpt.com/c/684eb8c8-42ac-8006-a80b-2f0089558d74
