FROM MY PT SCHOOL NOTES · EVIDENCE-BASED PHYSICAL THERAPY · CHAPTER 7
Does a P-Value Tell You a Treatment Works?
A hypothesis test can help interpret a study, but the p-value is not the probability that treatment works.
Understand the type of comparison
Inferential statistics evaluate how sample differences or associations inform claims about a broader population under specified assumptions. The course distinguishes independent-group from paired/repeated observations and tests of relationships from tests of differences.
The lecture surveys parametric approaches (such as t-tests, ANOVA and Pearson correlation/regression) and nonparametric or rank/frequency approaches (such as Mann–Whitney, Wilcoxon, Kruskal–Wallis and chi-square). Proper choice depends on measurement scale, design, dependence, assumptions and question.
Interpret uncertainty
Correlation summarizes association rather than establishing causation. Regression can describe or predict outcomes from one or more predictors. The course introduces r, r-squared, confidence intervals, standard errors, reliability coefficients and minimal detectable change in the measurement context.
Decide whether the result matters
Alpha is the prespecified threshold for Type I error (rejecting a true null hypothesis). Type II error is failure to reject a false null hypothesis, and power is 1 minus beta. Multiple comparisons can increase false-positive risk; the original lecture mentions Bonferroni adjustment.
Check your reasoning: Does a nonsignificant result prove two treatments are equivalent?
No. Failing to reject a null hypothesis is not proof of equality; effect magnitude, confidence intervals, design and power still matter.
Updated-standard note
A p-value is not the probability that the result occurred ‘due to chance,’ and a nonsignificant p-value is not proof of equal outcomes. Estimate magnitude, uncertainty and clinical relevance alongside any hypothesis test. Check the exact assumptions and definitions used in your course.
Put it into practice
Scenario: A trial reports p = 0.08 and a confidence interval that includes both a potentially useful effect and no effect.
What should you conclude?
Reasoning: This does not establish equivalence. The result may be uncertain because of sample size, variability or a small true effect. Interpret magnitude, interval, design and clinical consequences rather than declaring ‘no difference.’
Common trap: Calling a nonsignificant result proof that two interventions work equally well. Remember: Statistical thresholds do not replace estimation and clinical interpretation.
Current reference (separate from my class notes): American Statistical Association statement on p-values. A p-value does not by itself express effect size, clinical importance or the probability that a hypothesis is true.
Course basis: my Evidence-Based PT school notes, source lecture Chapter 10. New questions and headings are teaching aids, not official exam items. For full definitions and examples, use the class notes and assigned readings.
