`/education-data-source-scorecard`¶

Source guide to College Scorecard earnings, repayment and debt outcomes built from aid and tax records, stressing that it tracks only Title IV aid recipients and comparing it with IPEDS.

Pack: DAAF — Data Analyst Augmentation Framework (Brian Heseung Kim)

Category: data-handling

Field: social-sciences

License:

LGPL-3.0-or-later (as declared in the README, badge and CITATION.cff; the repo ships the GPL-3.0 text as LICENSE plus COPYING.LESSER)

Updated: 2026-08-09

Stages: data-acquisition

↗ view SKILL.md on source · GitHub stars

Scorecard Data Source Reference¶

College Scorecard — the primary institutional-level source for post-enrollment labor market outcomes, linking NSLDS financial aid records to IRS/Treasury earnings data. Use when comparing institutions on post-graduation earnings, loan repayment, or student debt, or when actual tax-record-based earnings are required rather than survey estimates. The Portal mirrors six institution-level families and does not mirror program/field-of-study data; use the official Scorecard Data page or documented API for program-level work. Critical limitation: tracks only Title IV federal aid recipients, not all students.

Federal data on post-college outcomes including earnings, debt, and repayment for students who received Title IV financial aid. Links education records to IRS tax data for actual earnings, making it the primary source for post-college labor market outcomes.

CRITICAL: Value Encoding and Missing Data

The Education Data Portal uses integer encodings for all categorical variables and lowercase, restructured variable names that differ from the original Scorecard column names. Suppression encoding differs by dataset:

Earnings/counts: -3 integer code is the primary suppression indicator

Yes/No flags (institutional characteristics): null for missing, 0/1 for valid

Rates (repayment, default): null for missing

The original Scorecard string "PrivacySuppressed" does NOT appear in Portal data

Context pred_degree_awarded_ipeds HBCU / tribal flags religious_affiliation

Portal (integer) 0-4 0 / 1 Integer codes 22-200

Original Scorecard String labels String labels String labels

See ./references/variable-definitions.md for complete encoding tables.

What is College Scorecard?¶

Publisher: U.S. Department of Education
Primary value: Post-college labor market outcomes (earnings) and debt/repayment metrics
Data sources: NSLDS (loans/aid), IRS/Treasury (earnings), IPEDS (institutional characteristics)
Coverage: Title IV federal aid recipients only — not all students
Unique feature: Links education to IRS tax records for actual earnings data
Access: Education Data Portal mirrors (parquet/CSV); see datasets-reference.md for paths, mirrors.yaml for mirror config, fetch-patterns.md for fetch code
Primary identifier: unitid (IPEDS institution ID)

Reference File Structure¶

File	Purpose	When to Read
`earnings-data.md`	Post-college earnings methodology, cohorts, time horizons	Analyzing earnings outcomes
`debt-repayment.md`	Student debt, repayment rates, default rates	Analyzing debt or loan outcomes
`completion-rates.md`	Completion metrics vs IPEDS	Comparing graduation rates
`population-coverage.md`	Title IV limitation details, who is included/excluded	Understanding data representativeness
`variable-definitions.md`	Key variables, naming conventions, special values	Building queries or interpreting results
`data-quality.md`	Suppression rules, selection bias, known limitations	Assessing data reliability
`field-of-study.md`	Program-level earnings and debt data	Analyzing outcomes by major/CIP code

Decision Trees¶

What outcome am I researching?¶

Text Only

Outcome type?
├─ Post-college earnings
│   ├─ Institution-level → Portal earnings family; ./references/earnings-data.md
│   └─ Program/field of study → NOT in Portal; use https://collegescorecard.ed.gov/data/
│       └─ Choose "Most Recent Data by Field of Study" or the documented API; see ./references/field-of-study.md
├─ Student debt levels
│   ├─ Institution-level borrowing → Portal repayment families; ./references/debt-repayment.md
│   └─ Debt by program/field → NOT in Portal; use the official Data page or documented API
├─ Loan repayment/default
│   └─ Repayment rates → ./references/debt-repayment.md
├─ Completion rates
│   └─ Scorecard completion → ./references/completion-rates.md
└─ Understanding limitations
    ├─ Who is included → ./references/population-coverage.md
    └─ Data quality issues → ./references/data-quality.md

How do I interpret this data?¶

Text Only

Interpretation question?
├─ Why are earnings suppressed?
│   └─ Privacy thresholds → ./references/data-quality.md
├─ What does "6-year earnings" mean?
│   └─ Cohort timing → ./references/earnings-data.md
├─ Why don't Scorecard rates match IPEDS?
│   └─ Different cohorts → ./references/completion-rates.md
├─ What loans are included in debt?
│   └─ Federal only → ./references/debt-repayment.md
└─ How representative is this data?
    └─ Title IV coverage → ./references/population-coverage.md

Building a query?¶

Text Only

Query construction?
├─ Portal institution-level family → Use exact path in "All Scorecard Datasets" below
├─ Variable names and codes → ./references/variable-definitions.md
├─ Suppression flags to handle → ./references/data-quality.md
├─ Understanding cohort years → ./references/earnings-data.md
└─ Program/field-level query → Not mirrored by Portal
    └─ Use https://collegescorecard.ed.gov/data/ or documented API; see ./references/field-of-study.md

Quick Reference: Scorecard Variables¶

Portal Data Structure (CRITICAL)¶

The Portal uses LONG format with time horizon as a column, NOT the WIDE format from original Scorecard bulk download files. Portal column names are all lowercase and differ significantly from original Scorecard names.

Original Scorecard (WIDE)	Portal Column (LONG)	How to Get
`MD_EARN_WNE_P6`	`earnings_med`	Filter: `years_after_entry == 6`
`MD_EARN_WNE_P10`	`earnings_med`	Filter: `years_after_entry == 10`
`COUNT_WNE_P6`	`count_working`	Filter: `years_after_entry == 6`
`MN_EARN_WNE_P6`	`earnings_mean`	Filter: `years_after_entry == 6`
`CONTROL`, `INSTNM`	NOT IN EARNINGS	Join to IPEDS directory or `inst_characteristics` dataset
`CDR3`	`default_rate`	In `repayment_fsa` dataset; filter: `years_since_entering_repay`
`RPY_3YR_RT`	`repay_rate`	In `repayment_nslds` dataset; filter: `years_since_entering_repay == 3`

Earnings Dataset Columns (Actual Portal Names)¶

Source dataset: scorecard/colleges_scorecard_earnings (203,066 rows x 33 columns)

Portal Column	Type	Description	Original Scorecard
`unitid`	Int64	IPEDS institution ID	`UNITID`
`opeid`	String	OPE ID (8-digit, zero-padded)	`OPEID`
`year`	Int64	Data year (2003-2014, 2018)	File year
`years_after_entry`	Int64	Years since first enrollment (6-10)	Encoded in variable name
`cohort_year`	Int64	Entry cohort year	Encoded in variable name
`earnings_med`	Int64	Median earnings (W-2)	`MD_EARN_WNE_P*`
`earnings_mean`	Int64	Mean earnings	`MN_EARN_WNE_P*`
`earnings_sd`	Int64	Standard deviation of earnings	`SD_EARN_WNE_P*`
`earnings_pct10`	Int64	10th percentile earnings	`PCT10_EARN_WNE_P*`
`earnings_pct25`	Int64	25th percentile earnings	`PCT25_EARN_WNE_P*`
`earnings_pct75`	Int64	75th percentile earnings	`PCT75_EARN_WNE_P*`
`earnings_pct90`	Int64	90th percentile earnings	`PCT90_EARN_WNE_P*`
`count_working`	Int64	Count working and not enrolled	`COUNT_WNE_P*`
`count_not_working`	Int64	Count not working and not enrolled	`COUNT_NWNE_P*`
`earnings_greater_than_25k_pct`	Float64	Share earning > $25K	`GT_25K_P*`
`earnings_lowinc_mean`	Int64	Mean earnings, low-income	`MN_EARN_WNE_INC1_P*`
`earnings_midinc_mean`	Int64	Mean earnings, mid-income	`MN_EARN_WNE_INC2_P*`
`earnings_highinc_mean`	Int64	Mean earnings, high-income	`MN_EARN_WNE_INC3_P*`
`earnings_dep_mean`	Int64	Mean earnings, dependent students	—
`earnings_dep_lowinc_mean`	Int64	Mean earnings, dependent low-income	—
`earnings_ind_mean`	Int64	Mean earnings, independent students	—
`earnings_female_mean`	Int64	Mean earnings, female	—
`earnings_male_mean`	Int64	Mean earnings, male	—
`count_working_*`	Int64	Count working by subgroup	—

Key Identifiers¶

ID	Format	Level	Example	Notes
`unitid`	6-digit integer	Institution	`110635`	Same as IPEDS unitid; primary join key
`opeid`	8-digit string	OPE (Title IV)	`"00100200"`	Zero-padded; present in all datasets
`opeid6`	Integer	6-digit OPE	`1002`	Numeric, no zero-padding

Data Timing¶

Metric	Dimension Column	Values	Typical Lag
Earnings	`years_after_entry`	6, 7, 8, 9, 10	Data from 7+ years ago
Default	`years_since_entering_repay`	2, 3	Varies
Repayment	`years_since_entering_repay`	1, 3, 5, 7	Varies

"After entry" means after first enrollment, not after graduation.

Categorical Value Encodings (Institutional Characteristics Dataset)¶

Variable	Values
`pred_degree_awarded_ipeds`	0=Not classified, 1=Certificate, 2=Associate's, 3=Bachelor's, 4=Graduate
Yes/No flags (HBCU, tribal, etc.)	0=No, 1=Yes, null=Missing
`religious_affiliation`	76 integer codes 22-200 (see variable-definitions.md for complete mapping), null=None/Missing

Missing Data Codes¶

Code	Meaning	Which Datasets
`-3`	Suppressed for privacy	Earnings dataset (earnings and count columns) — primary suppression indicator
`null`	Missing/not applicable	Institutional characteristics (yes/no flags), repayment/default (rates)
Positive numeric	Actual value	Earnings, debt, counts, rates

Python

import polars as pl

## Filter for valid earnings (handle -3 suppression code)
valid = df.filter(
    (pl.col("earnings_med").is_not_null()) &
    (pl.col("earnings_med") != -3)
)

## Filter for 6-year earnings specifically
six_yr_valid = valid.filter(pl.col("years_after_entry") == 6)

S

## Filter for valid earnings (handle -3 suppression code)
valid <- df |> filter(!is.na(earnings_med), earnings_med != -3)

## Filter for 6-year earnings specifically
six_yr_valid <- valid |> filter(years_after_entry == 6)

Data Access¶

Datasets for Scorecard are available via the mirror system. See datasets-reference.md for canonical paths, mirrors.yaml for mirror configuration, and fetch-patterns.md for fetch code patterns.

Codebooks are .xls files co-located with data in all mirrors. Use get_codebook_url() from fetch-patterns.md to construct download URLs.

Truth Hierarchy: When interpreting variable values, apply this priority: 1. Actual data file (what you observe in the parquet/CSV) — this IS the truth 2. Live codebook (.xls in mirror) — authoritative documentation, may lag 3. This skill documentation — convenient summary, may drift from codebook

If this documentation contradicts the codebook, trust the codebook. If the codebook contradicts observed data, trust the data and investigate.

All Portal Scorecard Datasets (6 Institution-Level Families)¶

The Portal manifest exposes the six families below. It does not expose a field-of-study/program-level family. For current program data, go to https://collegescorecard.ed.gov/data/ and choose Most Recent Data by Field of Study, or use the documented Scorecard API. Do not make a date-stamped ZIP URL the only access route.

Dataset	Path	Codebook	Type	Years
Earnings	`scorecard/colleges_scorecard_earnings`	`scorecard/codebook_colleges_scorecard_earnings`	Single	varies
Default	`scorecard/colleges_scorecard_repayment_fsa`	`scorecard/codebook_colleges_scorecard_default`	Single	1996-2020
Institutional Characteristics	`scorecard/colleges_scorecard_inst_characteristics`	`scorecard/codebook_colleges_scorecard_institutional-characteristics`	Single	1996-2020
Repayment	`scorecard/colleges_scorecard_repayment_nslds`	`scorecard/codebook_colleges_scorecard_repayment`	Single	2007-2016
Student Characteristics (Aid)	`scorecard/colleges_scorecard_student_body_nslds`	`scorecard/codebook_colleges_scorecard_student-characteristics_aid-applicants`	Single	1997-2016
Student Characteristics (Home)	`scorecard/colleges_scorecard_student_body_treasury`	`scorecard/codebook_colleges_scorecard_student-characteristics_home-neighborhood`	Single	1997-2016

Scorecard naming note: Data file paths differ significantly from codebook paths. Notable mismatches: data repayment_fsa vs codebook default; data inst_characteristics vs codebook institutional-characteristics; data repayment_nslds vs codebook repayment; data student_body_nslds vs codebook student-characteristics_aid-applicants; data student_body_treasury vs codebook student-characteristics_home-neighborhood. Always use the exact paths shown above.

Fetching Data¶

Python

import polars as pl

## Load `education-data-query` before authoring the Stage 5 script, then copy its
## documented `fetch_from_mirrors()` helper contract inline from
## `education-data-query/references/fetch-patterns.md`. There is no local
## `fetch_utils` module to import. The inline helper handles mirror priority,
## failover, and local filtering under the shared query contract.

## Fetch earnings data
earnings = fetch_from_mirrors("scorecard/colleges_scorecard_earnings")

## Filter by time horizon (LONG format — filter, don't use wide column names)
six_yr = earnings.filter(pl.col("years_after_entry") == 6)

## Filter for valid earnings (exclude -3 suppression code)
valid = six_yr.filter(
    (pl.col("earnings_med").is_not_null()) &
    (pl.col("earnings_med") != -3)
)

## Institution names/control are NOT in the earnings dataset.
## Join to inst_characteristics or IPEDS directory:
inst = fetch_from_mirrors("scorecard/colleges_scorecard_inst_characteristics",
                          years=[2020])
valid = valid.join(
    inst.select("unitid", "inst_name", "pred_degree_awarded_ipeds"),
    on="unitid", how="left"
)

S

## Fetch earnings data.
## fetch_from_mirrors() is a Python helper; in R, build the URL from the mirror
## root in mirrors.yaml and read directly.
## Mirror failover: see `education-data-query/references/fetch-patterns.md` (R pattern).
config <- yaml::read_yaml("mirrors.yaml")
mirror <- config$mirrors[[1]]
url <- paste0(mirror$root_url, "/", "scorecard/colleges_scorecard_earnings", ".", mirror$format)
## NOTE: illustrative only — mirror parquet files are Polars-written and may
## declare string_view columns, so a plain read can fail under R arrow
## ("cannot handle Array of type <utf8_view>"). Real fetch scripts must use the
## view-safe parquet read from `education-data-query/references/fetch-patterns.md`.
earnings <- arrow::read_parquet(url)

## Filter by time horizon (LONG format — filter, don't use wide column names)
six_yr <- earnings |> filter(years_after_entry == 6)

## Filter for valid earnings (exclude -3 suppression code)
valid <- six_yr |> filter(!is.na(earnings_med), earnings_med != -3)

## Institution names/control are NOT in the earnings dataset.
## Join to inst_characteristics or IPEDS directory (filter years locally):
url <- paste0(mirror$root_url, "/", "scorecard/colleges_scorecard_inst_characteristics", ".", mirror$format)
inst <- arrow::read_parquet(url) |> filter(year == 2020)
valid <- valid |> left_join(
  inst |> select(unitid, inst_name, pred_degree_awarded_ipeds),
  by = "unitid"
)

Common Pitfalls¶

Pitfall	Issue	Solution
"All graduates" claims	Scorecard covers Title IV recipients only, not all students	Note Title IV limitation prominently in any analysis
Wage comparison	Comparing to BLS wages or Census income uses different populations	Use for relative comparisons, not absolute claims; document population differences
Ignoring suppression	Many programs have no data due to privacy thresholds	Check suppression rates before analyzing; document coverage
Time lag ignored	Earnings reflect old cohorts (6-year = data from 7+ years ago)	Document data vintage and cohort years explicitly
Total borrowing assumption	Scorecard debt includes only federal loans, not private	State "federal loans only" when reporting debt figures
String codes from docs	Original Scorecard uses string labels; Portal uses integers	Verify actual data types in Portal parquet files; use integer codes
Wide-format variable names	Using `MD_EARN_WNE_P10` column name on Portal data	Portal uses LONG format — filter `years_after_entry` instead
Assuming null = suppressed	Earnings dataset uses `-3` for suppression, not null	Filter both: `is_not_null() & != -3`
Using uppercase names	Original Scorecard uses `MD_EARN_WNE_P6`; Portal uses `earnings_med`	Always use lowercase Portal names from actual data

Critical Limitation: Title IV Recipients Only¶

The single most important caveat for all Scorecard analysis:

Scorecard tracks ONLY students who received federal financial aid (Title IV): - Pell Grants - Federal student loans (Direct, Perkins, PLUS) - Federal work-study

Excluded Group	Impact
Full-pay students	Often higher-income; different outcomes
Students with only state/institutional aid	Missing from data
International students	Not eligible for federal aid
Some graduate students	If they received no federal aid

Coverage varies dramatically by institution type:

Institution Type	Typical Title IV Coverage
For-profit colleges	80-90%+
Community colleges	60-80%
Public flagships	50-70%
Selective private colleges	30-50%

Data systematically overrepresents lower-income students who are more likely to need federal aid.

What Scorecard Data Does NOT Include¶

Excluded	Why It Matters
Non-Title IV students	Often higher-income; different outcomes
Self-employment income	1099 income excluded from earnings
Students still in school	Not working = not in earnings data
Private student loans	Only federal loans tracked
Students who left the country	Lost to follow-up

Comparison: Scorecard vs IPEDS¶

Aspect	College Scorecard	IPEDS
Who's tracked	Title IV aid recipients	First-time, full-time students
Includes part-time	Yes	No (for grad rates)
Includes transfers-in	Yes	No (tracked at origin)
Outcome focus	Earnings, debt, repayment	Completion, retention
Data source	NSLDS + IRS	Institution-reported

Source	Relationship	When to Use
`education-data-source-ipeds`	Institutional characteristics, enrollment, finance	Join on `unitid` for institution names, control type, enrollment context
`education-data-source-pseo`	Alternative post-college earnings (Census LEHD)	When broader population coverage needed (not limited to Title IV)
`education-data-source-fsa`	Federal student aid details	Deeper analysis of aid types and disbursements
`education-data-explorer`	Parent discovery skill	Routing questions to mirror dataset files
`education-data-query`	Data fetching	Downloading parquet/CSV files

Topic Index¶

Topic	Reference File
Earnings methodology	`./references/earnings-data.md`
Cohort definitions	`./references/earnings-data.md`
IRS data matching	`./references/earnings-data.md`
Earnings suppression	`./references/data-quality.md`
Debt metrics	`./references/debt-repayment.md`
Repayment rates	`./references/debt-repayment.md`
Default rates	`./references/debt-repayment.md`
NSLDS data	`./references/debt-repayment.md`
Completion methodology	`./references/completion-rates.md`
IPEDS comparison	`./references/completion-rates.md`
Title IV coverage	`./references/population-coverage.md`
Who is excluded	`./references/population-coverage.md`
Selection bias	`./references/population-coverage.md`
Variable names	`./references/variable-definitions.md`
Special values	`./references/variable-definitions.md`
Privacy suppression	`./references/data-quality.md`
Data limitations	`./references/data-quality.md`
Program-level data	`./references/field-of-study.md`
CIP codes	`./references/field-of-study.md`

Context	`pred_degree_awarded_ipeds`	HBCU / tribal flags	`religious_affiliation`
Portal (integer)	`0`-`4`	`0` / `1`	Integer codes 22-200
Original Scorecard	String labels	String labels	String labels

/education-data-source-scorecard¶

Scorecard Data Source Reference¶

What is College Scorecard?¶

Reference File Structure¶

Decision Trees¶

What outcome am I researching?¶

How do I interpret this data?¶

Building a query?¶

Quick Reference: Scorecard Variables¶

Portal Data Structure (CRITICAL)¶

Earnings Dataset Columns (Actual Portal Names)¶

Key Identifiers¶

Data Timing¶

Categorical Value Encodings (Institutional Characteristics Dataset)¶

Missing Data Codes¶

Data Access¶

All Portal Scorecard Datasets (6 Institution-Level Families)¶

Fetching Data¶

Common Pitfalls¶

Critical Limitation: Title IV Recipients Only¶

What Scorecard Data Does NOT Include¶

Comparison: Scorecard vs IPEDS¶

Related Data Sources¶

Topic Index¶

`/education-data-source-scorecard`¶