7.9 Chapter Review
7.9.1 Chapter summary
A strong data dictionary documents the dataset’s grain, keys, coverage, provenance, and limitations as well as each variable’s meaning, type, units, allowed values, missing-value rules, source, and transformations. It should be created early, tested against the data, and updated when definitions or derived variables change.
7.9.2 Common mistakes
- copying the column name into the definition;
- documenting technical type without business meaning;
- omitting the unit of analysis;
- listing observed values as though they were necessarily valid;
- failing to document special missing codes;
- recording missing counts without a dataset version;
- overwriting source fields;
- omitting derived variables; and
- allowing report labels to drift away from dictionary definitions.
7.9.4 Exercises
Exercise 1. An analyst defines active_member as “active member” and codes recent members without a full twelve-month observation period as FALSE. Identify two problems and rewrite the entry.
One Possible Answer
The definition repeats the name without specifying the rule, and members who cannot yet be observed for twelve months are incorrectly treated as known non-retained cases. A stronger variable is retained_12m: “Member has an active membership on the date twelve months after the initial membership start.” Allowed values are TRUE, FALSE, and missing when twelve months of follow-up are unavailable. The entry should name the source dates and extract date used.
Exercise 2. Open data/nvrw/data_dictionary.csv. Select five variables from at least three datasets and evaluate whether each definition identifies the unit, timing, allowed values, and interpretation clearly enough for a new analyst.
One Possible Answer
The existing dictionary is a useful starting point but several entries need more detail for independent use. For example, renewed_next_period should define the eligibility and censoring date; contribution_margin should name the revenue and cost fields and period; duration_minutes should document the raw quality rule; campaign_spend should state the campaign period and allocation; and population should distinguish estimates from projections and identify geography and reference year.
Exercise 3. Compare data/nvrw/raw/members.csv with data/nvrw/clean/members.csv. Document the standardization rule for membership_type, service_area, and postal_fsa without describing the cleaned result as the original source value.
Check Your Work
The rule should state that raw strings are mapped to the approved category labels for membership type and service area, while postal FSA values are trimmed and standardized to uppercase three-character codes. The raw field must remain identifiable as the source, and the dictionary or cleaning log should record the mapping, unmatched values, and the clean variable’s provenance. If the supplied raw and clean values already match, that is a validation result, not evidence that no rule exists.
Exercise 4. Design dictionary entries for annual visits per active member and program fill rate. Include the formula, denominator, exclusions, time window, and source fields.
One Possible Answer
Annual visits per active member: Count of valid, deduplicated visits during the calendar year divided by members with an active membership in the defined year; exclude invalid visit identifiers and document how partial-year memberships are treated. Program fill rate: Eligible registrations divided by the program capacity measured at the stated reference point; define the treatment of cancelled registrations, wait-list records, missing capacity, and zero capacity. Both entries should list the source tables, join keys, and calculation date.