CPS sample size cut may save some money but would cost policymakers
Degrading data quality makes it harder to know when and how to act
The Current Population Survey (CPS) is the sole timely source of several valuable labor market indicators, including the unemployment rate, the labor force participation rate, and measures of within-person wage growth. The Bureau of Labor Statistics (BLS) will reportedly reduce the sample size of the CPS by 5,000 households (about 8%) as a cost-saving measure. Combined with continuing declines in response rates, this reduction would likely put the number of responding households just below its pandemic-induced nadir. A smaller sample size makes it more difficult for policymakers at all levels of government, including the Federal Reserve, to respond to changes in the economy. Detecting changing labor market outcomes among many population subgroups of interest would be more difficult given the already-small sample sizes. Less populous states and rural areas within larger states also have relatively few residents sampled each month.
Several key indicators are unique to the CPS
While some labor market indicators can be produced from multiple data sources, several important measures can only be produced in a timely fashion using CPS data. Timeliness is especially important when labor market conditions are changing rapidly or highly uncertain, such as during the Covid-19 pandemic (production lags and lower frequency make even comprehensive data like that provided by the Quarterly Census of Employment and Wages less useful in this context). As the labor market recovered from its initial contraction, the CPS was the only reliable data source that could capture both economic and health-related shifts in workers’ behavior, and getting updates every month was invaluable. That reliability was and remains especially valuable given the challenges other labor market indicators like unemployment insurance claims (fluctuations driven by administrative rather than economic causes) and data from the Job Openings and Labor Turnover Survey (large decline in the response rate) face with providing dependable estimates.
Even when updates are needed less urgently, the CPS is uniquely positioned to produce measures like labor force participation and unemployment rates. These measures must be constructed using surveys of workers, since they hinge in part on the job search efforts of people who are not working, which cannot be captured by surveys of employers or administrative data. Measuring duration of or reason for unemployment, which can provide key insights into the business cycle, requires identifying unemployed people and therefore can also only be produced through the CPS.
The panel structure of the CPS (respondents are in sample for four months, out of sample for eight months, and then back in sample for four more months, the same calendar months as their initial interviews) allows it to measure how wages change within a fixed group of workers over time, and the demographic information collected enables analysis of how wage growth differs across groups of workers. These capabilities are critical to understanding wage dynamics (especially at a time when inflation is above target) because wage indicators based on other data sources often lack demographic information and are based on repeated cross-sections of workers or jobs. When collected through other sources, information on worker demographics is extremely limited (e.g. the only demographic characteristic covered by the establishment survey is sex) or dramatically less timely (e.g. the American Community Survey collects similarly detailed demographic information but is released only once per year several months after data collection is complete).
The CPS provides information on who wants a job or more hours but can’t get them, how people look for jobs, who was not at work last week and why (which has been very valuable since the pandemic began), and who has multiple jobs. These questions can only be answered by asking workers, and the share of people providing a particular response is often low, making them especially vulnerable to sample size reductions.
The proposed cuts would likely drop the number of responding households below the lowest levels seen during the pandemic
The CPS currently attempts to interview members of about 60,000 households each month. Historically, response rates were high. From 2006 through 2015, for example, just over 53,000 households completed interviews in a typical month, corresponding to a response rate of about 90 percent. However, since roughly late 2016, response rates have trended down markedly. By 2019, only about 82 percent of surveyed households agreed to be interviewed. In 2020, the pandemic disrupted in-person interviews, and the response rate fell below 65 percent that June (just under 39,000 households) before largely (but incompletely) recovering later in the year, as shown in the following figure.
The downward trend in response rates has continued since 2020. Over the last twelve months for which data are available, the response rate has been about 70 percent, with nearly 42,000 households responding on average. If that response rate were maintained alongside the proposed sample size reduction, the number of responding households would fall by about 3,500 to just over 38,000. This would not only be below the level seen during the most logistically challenged pandemic months but also 15,000 households (28 percent) below the level seen in the typical month during the decade before the response rate decline began in earnest.
While efforts can and have been made to increase response rates, these efforts require resources. BLS saw its budget fall by nine percent in real terms between FY 2008 and FY 2023. Over that same period, the labor force it was expected to measure grew by eight percent.
Many groups are already represented by small numbers of respondents
Even as they decline, these national sample sizes may obscure the extent to which interesting and important population subgroups are represented by small numbers of respondents. Policymakers are often interested in understanding labor market outcomes within and across groups defined by demographic characteristics such as age, sex, race/ethnicity, and education. Cell sizes defined by these characteristics can be quite small. For prime-age people, 36 out of 56 sex-race/ethnicity-education cells have fewer than 500 observations on average over the last 12 months, including groups with full populations approaching 1.5 million; some cells have only a handful of monthly observations nationwide, as the figure below illustrates.
Estimates related to wages are based on substantially smaller sample sizes than estimates related to other outcomes. Information on wages and hours is only collected from about one quarter of the sample each month (respondents in their fourth and eighth months in sample, the so-called “outgoing rotation groups” that will not be surveyed the following month).1 Sample sizes for estimates of wage growth are slightly smaller still because they can only be produced for people who continue completing their interviews through the final month of the panel.
The CPS also collects information on other characteristics that are of great interest in certain contexts, such as disability, veteran status, and foreign-born status. The prevalence of these characteristics in the full population ranges from about six percent for veteran status to nearly 19 percent for foreign-born status in the May 2024 data. Given that these characteristics are often of interest within groups defined by the demographics discussed above, sample sizes can get quite small. Analyses of the characteristics of people who have certain labor market outcomes, such as being unemployed, can face the same issue. In May, the unemployment rate was 4.0 percent, and only about 1,700 people in the underlying data were unemployed. Analyzing the demographics of those people, or the duration of or reasons for their unemployment, can result in some very small cells.
Sample sizes get dramatically smaller at the state level
Taking this same degree of demographic detail to the state level leaves most cells statistically unusable, even before the proposed sample size reduction. Half have fewer than 11 observations in an average month over the last year, and 75 percent have fewer than 29. None has more than about 350.
Accepting less demographic detail makes the data more usable but still leaves many cells with few supporting observations. For state-level cells defined using age, sex, and race/ethnicity but not education, the median cell contains about 19 observations in a typical month, and the cell at the 75th percentile contains about 84. Further demographic aggregation yields larger sample sizes but comes at the cost of coarser analysis, and cutting the overall sample size would add tension to this tradeoff.
Setting demographics aside and focusing on geographic groups within states reveals the limited sample sizes outside of cities. In only 12 states is the average number of observations in all nonmetropolitan areas combined over the last 12 months greater than 500. Those nonmetropolitan areas may be quite heterogeneous in some cases, but the data generally cannot speak to that.
Sample size savings come with a cost
Degrading the CPS’s ability to produce precise estimates of labor market outcomes and workers’ characteristics could make it harder for policymakers, including and especially those within the Federal Reserve system, to identify and respond to turning points in the trajectory of the labor market. Even modest decreases in the precision of headline indicators can contribute to increased uncertainty about the state of the economy and slow the policy response to changing conditions. Degrading the picture the data provide of how workers in smaller communities are doing makes it harder at the margin to achieve and maintain inclusively full employment. Good policy requires good data. Every effort should be made to maintain (and ideally increase) the CPS sample size to ensure policymakers have access to high-quality labor market information.
Information on union status and overtime/tip/commission compensation is also only collected from one-quarter of the sample. The limited sample size is especially noteworthy for union status given the lack of alternative sources for this information.