You are currently on:

Construction of confidence intervals in QILT reporting

Confidence intervals (90%) are displayed to indicate accuracy. The confidence intervals presented in the QILT National reports are calculated using the method described by Agresti and Coull (1998).

Confidence intervals provide a range of values that reflect the uncertainty involved in estimating a population parameter from sample data. In our reports, we use 90 per cent confidence intervals, which means that if we were to draw many samples from the population and calculate the interval each time, the true population value would fall within the interval 90 per cent of the time.

The confidence intervals presented in the QILT National reports are calculated using the method described by Agresti and Coull (1998)1. This method is an improvement over the traditional Wald method, offering more reliable results across a wider range of sample sizes.

The Wald method

The Wald confidence interval is calculated using the formula:

Where:

  • p is the proportion of positive responses (n1 / n) 
  • n1 is the number of positive responses
  • n is the total number of valid responses
  • z is the z-score for the desired confidence level (1.645 for a 90%). 

The Agresti-Coull adjustment

The Agresti-Coull method adjusts both the sample size and the proportion:

  • Adjusted sample size: 
  • Adjusted proportion:  

 The confidence interval is then calculated as:

 

Finite population correction

When the sample size is large relative to the population, a finite population correction (FPC) is applied to narrow the interval. This applies to the Graduate Outcomes Survey (GOS), Graduate Outcomes Survey – Longitudinal (GOS‑L), and Student Experience Survey (SES), but not to the Employer Satisfaction Survey (ESS).

This is done by multiplying the term to the right of the ± symbol by a finite population correction factor, given as where N is the population size.

Important notes

The confidence intervals are calculated around the adjusted proportion (p ̃), but the raw proportions (p) are reported in the results. 

Like other approximation methods, this approach may produce unreliable intervals when proportions are very close to 0 per cent or 100 per cent. In such cases, the confidence intervals are not shown and are flagged in the reports.


[1] Agresti, A., & Coull, B. A. (1998). Approximate Is Better than “Exact” for Interval Estimation of Binomial Proportions. The American Statistician, 52(2), 119–126. https://doi.org/10.2307/2685469.