1.7 Chapter Review

1.7.1 Chapter summary

An applied data science project is a bounded, iterative investigation designed to improve understanding or support a real decision. Strong projects connect the client’s concern to a precise analytical question, define success in observable terms, and treat planning, data work, analysis, communication, and implementation as interdependent responsibilities.

1.7.2 Common mistakes

  • accepting the initial request without clarifying what decision it supports;
  • defining success only as submitting the final files;
  • selecting a preferred method before understanding the data;
  • treating every requested analysis as equally useful;
  • confusing a conditional projection with a guaranteed forecast; and
  • delaying integration until each team member believes their individual work is finished.

1.7.3 Exercises

Exercise 1. Write a project brief for NVRW that identifies the decision, audience, population, time period, deliverables, exclusions, and success criteria.

One Possible Answer

Decision: Determine where NVRW should focus retention and capacity-planning efforts. Audience: NVRW leadership and facility managers. Population and period: Members, visits, and program activity from 2022 through 2025. Deliverables: Data dictionary, EDA report, retention analysis, utilization projection, recommendations, final report, and presentation. Exclusions: Claims about causal effects or motives not recorded in the data. Success criteria: Reproducible results, validated definitions and denominators, documented limitations, and recommendations that can be traced to evidence.

Exercise 2. Trace one possible error from data definition through analysis, recommendation, implementation, and evaluation. Explain where a review gate could prevent the error from travelling further.

One Possible Answer

If censored membership periods are incorrectly coded as non-renewals, the analysis understates retention. A model then identifies false risk patterns, the team recommends targeting the wrong members, and the pilot appears ineffective. A data-dictionary review should identify the censoring rule before modelling. A second review gate should compare model eligibility counts with independently calculated retention totals.

Exercise 3. Classify each proposed project output as evidence, interpretation, action, or evaluation. Revise any output that does not contribute to the decision.

Check Your Work

Evidence includes validated tables, model results, forecasts, and documented source findings. Interpretation explains what those results mean and how strong the evidence is. Action specifies what should be done, by whom, and under what conditions. Evaluation defines how implementation will be assessed. An output that fits none of these categories should be removed, moved to an appendix, or revised so its decision purpose is clear.

Exercise 4. Choose an organizational concern from health, retail, government, education, or nonprofit work. Write two analytical questions that could support the same concern, then explain how the required data and methods would differ.

One Possible Answer

Concern: A food bank is struggling to match inventory with demand. Question 1: How does weekly demand vary by location, household size, and season? This requires transaction-level service records and descriptive or time-series analysis. Question 2: Which products are most likely to experience a shortage during the next four weeks? This requires inventory, deliveries, product demand, and lead-time data, followed by item-level forecasting or scenario analysis.

Exercise 5. Create a one-page evidence map that connects a decision to required evidence, analytical outputs, limitations, and a possible recommendation.

One Possible Answer

For an NVRW retention decision, the map could connect member eligibility, renewal outcomes, early visits, facility, membership type, and acquisition channel to a validated retention table and risk model. The main limitations would include observational data, unavailable reasons for cancellation, and incomplete follow-up for recent members. A supported recommendation would be a measured early-engagement pilot for an eligible high-risk group, not a claim that low visit frequency causes cancellation.

1.7.4 Questions for discussion

  1. Why might a technically accurate model still fail as a professional deliverable?
  2. What evidence would show that a project has become too broad?
  3. When should a team revise its original analytical question?
  4. Which success criteria would matter most for a public-facing dashboard? Which would matter most for an internal forecasting model?