Skip to content

/data-dictionary

Generates a codebook from a Stata .dta file in summary, full or analysis-ready mode: variable list, summary statistics, value labels, string variables and high-missingness flags.

Category: data-handling
Field: economics
License: CC-BY-4.0 (repo LICENSE, "Creative Commons Attribution 4.0 International, Copyright (c) 2026 Emily Beam")
Updated: 2026-04-20
Stages: data-analysis

Data Dictionary Generator

v1.0 — Auto-generate comprehensive codebooks from .dta files

Read Stata .dta files and produce a structured markdown data dictionary with variable names, types, labels, value labels, summary statistics, and missingness. Outputs a ready-to-use codebook document.

Argument: $ARGUMENTS - Path to a .dta file or directory containing .dta files

Modes (append to argument): - summary (default) — One-page overview: variable list with types, labels, missingness - full — Comprehensive codebook: summary + value labels + summary stats + distributions for key variables - analysis — Analysis-ready: full + notes on which variables are outcomes vs controls, indices vs components

Flags: - vars:consumption,assets — Only document variables matching these patterns - output:path/to/output.md — Custom output path (default: same directory as input, named codebook_[filename].md) - format:md (default) | format:csv — Output format

Example: /data-dictionary data/for_analysis/endline_analysis.dta full Example: /data-dictionary . vars:consumption,exp_ Example: /data-dictionary data/for_analysis/ summary

Instructions

Step 0: Locate and Load Data

  1. Resolve $ARGUMENTS to find .dta file(s):
  2. If a file path: read that file directly
  3. If a directory: glob for *.dta files in it
  4. If a bare name: search the current working directory and subdirectories (data/, data/for_analysis/, output/)
  5. For directories with many files (>10), generate an index page + individual codebooks
  6. Parse mode and flags from $ARGUMENTS. Default to summary.

Step 1: Extract Metadata

Write a temporary Python script (use uv run python or python3). Never use python -c.

Python
import pandas as pd
import pyreadstat

df, meta = pyreadstat.read_dta("path/to/file.dta",
                                apply_value_formats=False)

## Available metadata:
## meta.column_names          — variable names (list)
## meta.column_labels          — variable labels (dict: name -> label)
## meta.variable_value_labels  — value label mappings (dict: name -> {code: label})
## meta.original_variable_types — Stata storage types (dict: name -> type string)
## meta.number_columns         — count of numeric columns
## meta.number_rows            — row count
## meta.file_label              — dataset label
## meta.notes                   — dataset notes

For each variable, extract: - Name: column name - Label: from meta.column_labels (may be empty) - Type: from meta.original_variable_types (e.g., float, double, byte, int, long, str#) - Value labels: from meta.variable_value_labels (categorical mappings) - N non-missing: count of non-null values - N missing: count of null values - Missing %: percentage missing - Unique values: nunique

Step 2: Summary Statistics (full and analysis modes)

For numeric variables: - Mean, SD, Min, P25, Median, P75, Max - Flag: negative values in typically-positive variables

For string variables: - Number of unique values - Top 5 most frequent values with counts - Max string length

For binary (0/1) variables: - Proportion = 1 (with count)

For categorical variables (those with value labels): - Frequency table: code, label, count, percentage

Step 3: Variable Classification (analysis mode only)

Attempt to classify variables by role based on naming patterns:

Pattern Classification
hhid, id, *_id Identifier
treat*, arm*, t_* Treatment
strat*, block*, pair* Stratification
i_*, idx_*, index_* Index/Composite
z_* Z-score
exp_*, cons_*, income_* Outcome (economic)
is_*, has_*, any_* Binary indicator
mi_*, miss_* Missing flag
_wins, _w, _tr Winsorized/trimmed
bl_*, base_* Baseline
wt_*, weight* Sampling weight

Present classifications as suggestions, not assertions. Include an "Unclassified" category.

Step 4: Cross-Variable Relationships (analysis mode only)

  • Index decomposition: For index variables (i_, idx_), identify likely component variables and report correlations
  • Treatment balance: If treatment variable detected, report mean of key variables by treatment arm (first 10 numeric variables)
  • Missing patterns: Identify clusters of variables with correlated missingness

Step 5: Generate Output

Output location: - Default: same directory as input file, named codebook_[filename]_[YYYY-MM-DD].md - Override with output: flag - For directory inputs: codebook_index_[YYYY-MM-DD].md + individual codebooks

Tell the user the full absolute path to the output file.

Output Format — Summary Mode

Markdown
## Data Dictionary: [filename]

**Generated:** [YYYY-MM-DD]
**Source:** [full path]
**Dataset label:** [meta.file_label if available]
**Observations:** [N rows]
**Variables:** [N columns]

---

### Variable List

| # | Variable | Label | Type | Non-missing | Missing % | Unique |
|---|----------|-------|------|-------------|-----------|--------|
| 1 | hhid | Household ID | long | 5,000 | 0.0% | 5,000 |
| 2 | treat | Treatment arm | byte | 5,000 | 0.0% | 3 |
| ... | ... | ... | ... | ... | ... | ... |

---

### Variables Without Labels

[List any variables that have no label — these may need documentation]

---

### Notes

- [N] variables are entirely missing (0 non-missing values)
- [N] variables have >50% missing values
- [N] categorical variables have value labels defined

Output Format — Full Mode

Adds to summary:

Markdown
---

### Summary Statistics — Numeric Variables

| Variable | N | Mean | SD | Min | P25 | Median | P75 | Max |
|----------|---|------|-----|-----|-----|--------|-----|-----|
| ... | ... | ... | ... | ... | ... | ... | ... | ... |

---

### Summary Statistics — Binary Variables

| Variable | Label | N | Prop = 1 | Count = 1 |
|----------|-------|---|----------|-----------|
| ... | ... | ... | ... | ... |

---

### Value Labels — Categorical Variables

#### [variable_name]: [label]

| Code | Label | Count | Percent |
|------|-------|-------|---------|
| 0 | Control | 2,500 | 50.0% |
| 1 | Treatment | 2,500 | 50.0% |

[Repeat for each categorical variable with value labels]

---

### String Variables

| Variable | Label | N | Unique | Max Length | Top Values |
|----------|-------|---|--------|------------|------------|
| ... | ... | ... | ... | ... | [val1 (N), val2 (N), ...] |

---

### High Missingness Variables (>20%)

| Variable | Label | Missing % | Non-missing N |
|----------|-------|-----------|---------------|
| ... | ... | ... | ... |

Output Format — Analysis Mode

Adds to full:

Markdown
---

### Variable Classification (suggested)

#### Identifiers
| Variable | Label |
|----------|-------|
| hhid | Household ID |

#### Treatment & Stratification
| Variable | Label | Values |
|----------|-------|--------|
| treat | Treatment arm | 0: Control, 1: Treatment |

#### Outcome Indices
| Variable | Label | Mean | SD | Likely Components |
|----------|-------|------|-----|-------------------|
| i_consumption | Consumption index | 0.00 | 1.00 | exp_food, exp_nonfood, ... |

#### Outcome Variables
[table]

#### Control Variables / Baseline
[table]

#### Missing Flags
[table]

#### Unclassified
[table]

---

### Treatment Balance (first 10 numeric variables)

| Variable | Control Mean | Treatment Mean | Diff | p-value |
|----------|-------------|----------------|------|---------|
| ... | ... | ... | ... | ... |

Principles

  • Comprehensive but scannable. The dictionary should work as both a reference document (ctrl+F for a variable) and a quick overview (scan the summary table).
  • Metadata-first. Always use Stata's own metadata (labels, value labels, types) rather than inferring. Only infer when metadata is missing.
  • Flag gaps. Unlabeled variables, undocumented value labels, and high missingness are all worth flagging — they're the most likely sources of confusion.
  • Analysis mode is suggestive, not prescriptive. Variable classification is based on naming patterns and may be wrong. Present as suggestions.
  • Reproducible. The output includes the source path and generation date so it's clear what version of the data was documented.