6.2 Sampling, observational studies, and experiments

The way data are collected determines which conclusions are defensible. A large biased sample can be less useful than a smaller well-designed sample.

6.2.1 Bias and sampling methods

Sampling bias. A systematic tendency for a sampling method to overrepresent or underrepresent parts of the population.

Convenience samples select cases that are easy to reach. Non-response bias occurs when respondents differ systematically from nonrespondents. Simple random sampling gives each case an equal selection opportunity. Stratified sampling draws from each relatively similar subgroup. Cluster sampling selects whole mixed groups, while multistage sampling combines selection stages.

Worked Example: Recognizing convenience bias

Testing only taps nearest the treatment plant may not represent the full distribution network. Distance, pipe age, and residence time may differ elsewhere, creating convenience sampling bias.

Try It. Why might a voluntary online water-taste survey be biased?
Check Your Work People with unusually strong opinions may be more likely to respond.

Worked Example: Choosing a stratified sample

To compare pipe conditions across three material types, divide connections into material strata and randomly sample within each. This ensures that every material is represented.

Try It. Suggest strata for sampling water quality across a city with four pressure zones.
Check Your Work Use the four pressure zones as strata and randomly select locations within each.

6.2.2 Observational studies and experiments

Observational study. A study that records naturally occurring variables without assigning treatments.

Observational studies can establish association but not causation because confounding variables may explain the pattern. Experiments assign treatments and can support causal conclusions when well designed. Core principles are comparison, random assignment, replication, and blocking. Blinding reduces behavioural and measurement effects.

Worked Example: Distinguishing study types

Comparing historical filter records for two coagulants is observational if operators chose the coagulant. Randomly assigning comparable filter runs to the two coagulants would create an experiment.

Try It. Is recording rainfall and overflow events observational or experimental?
Check Your Work Observational. Rainfall is not assigned by the researchers.

Worked Example: Blocking a treatment experiment

If raw-water temperature may affect removal, group test runs into low- and high-temperature blocks, then randomly assign coagulant doses within each block.

Try It. Suggest a blocking variable for testing pump efficiency across old and new pumps.
Check Your Work Pump age or model could define blocks before treatments are randomized.
Applied Problem: Evaluating a filter-media study. A facility installs new media in its newest filter and compares its turbidity with an older filter using standard media. Can the result establish that media caused any difference? Identify a confounder and propose a stronger design.
One Possible Answer No. Filter age or condition is confounded with media type. A stronger design would use several comparable filters, randomly assign media where feasible, replicate runs, and block by relevant conditions such as raw-water turbidity.

6.2.3 Practice Problems

  1. Define convenience sampling.
    Check Your Work Selecting cases mainly because they are easy to reach.
  2. What is simple random sampling?
    Check Your Work A random method giving each population case an equal selection opportunity.
  3. Distinguish strata from clusters.
    Check Your Work Strata are internally similar and sampled within; clusters should each resemble the population and selected clusters are sampled.
  4. Can an observational association prove causation?
    Check Your Work No.
  5. Name the four experiment principles.
    Check Your Work Comparison, randomization, replication, and blocking.
  6. Sampling only daytime flows creates what concern?
    Check Your Work Time-of-day sampling bias.
  7. A utility randomly samples within each pressure zone. Name the method.
    Check Your Work Stratified sampling.
  8. Operators randomly assign two coagulant doses within high- and low-turbidity blocks. Identify randomization and blocking.
    Check Your Work Dose is randomly assigned; raw-water turbidity defines the blocks.