Skip to main content
Searching the BLS website

Pandemic, racial inequities underscore need for better labor market data

by Erica Groshen

The U.S. needs and could have much better official labor market statistics. To get there, we must tap our most underutilized national data asset: Unemployment Insurance (UI) administrative records. As a former commissioner of the U.S. Bureau of Labor Statistics (BLS), I was and am proud of the bureau’s gold-standard economic indicators, such as the unemployment rate and payroll job growth. Yet, all statistics have limitations. The recent experiences of the COVID-19 pandemic and the national reckoning on racial inequities call for upgrading our current suite of statistics. Making changes will be complicated, but it is also doable, overdue, and will help fuel a brighter economic future, locally and nationally.

Labor Statistics Reform Use Cases

With just one fix – enhancing UI system wage and claims records (by including data on hours, job title, work site and demographics) and adding them to our national data infrastructure -- BLS could provide more timely and specific labor market information to policymakers, educators, state labor market information offices, businesses, investors and job-seekers.

With the fix proposed here, BLS could use matching (with survey data) and modelling to unpack local labor trends quickly. Consider three examples of what BLS could do:

  • Identify in-demand and declining occupations and industries, by state and metro area;
  • Reduce revisions to monthly payroll statistics; and
  • Publish weekly layoff indicators, with greater detail (including demographics) and not confounded by administrative peculiarities of the UI system.

We know these are doable because matching and modelling are being applied successfully elsewhere. In a close example, the Census Bureau’s Longitudinal Employer Household Dynamics (LEHD) program already publishes valued statistics using UI wage records (without enhancements) merged with other information.[1]

With upgraded BLS statistics, policymakers can target responses (whether monetary, fiscal, or regulatory) more accurately and quickly. Effective targeting (by geography, industry, demographics, income, etc.) conserves taxpayer dollars and concentrates impact when and where it is most needed. This is essential for crafting responses to disruptions (such as recessions or natural disasters) and longer-run issues (such as perceived skill shortages). Workforce development specialists would work from better forecasts. Researchers could better study causes and consequences of labor market inequities. Investors would know more about local labor market assets. The Department of Labor’s Employment and Training Administration (ETA) which has always relied on BLS data to satisfy many of its wage determination obligations for its foreign labor certification operations, would be able to provide wage information for certain immigration programs that are more occupationally-specific and thus likely more accurate. With access to clean microdata, ETA and states would be better able to investigate UI claims fraud. Policy and program evaluations would be faster, less expensive, and more accurate, improving understanding of the impact of job training, environmental, and economic development programs, to name a few.

As part of its adjustment to the new data source, BLS can likely eventually reduce or replace two large employer data collections. The two programs are the Occupation Employment Survey—the basis of Occupational Projections and the Occupational Outlook Handbook (BLS’s most popular product), and the National Compensation Survey—the basis of the Employment Cost Index and Employer Costs for Employee Compensation.

More Timely and Detailed Labor Statistics – the Value Proposition

With the fix proposed here, we could address at least two critical data deficiencies. First, many decisions would be easier if data were more timely and frequent. In early 2020, the COVID-19 pandemic caused the steepest recorded economic shock in U.S. history. Each month, BLS provided the most reliable and timely measures of labor market activity in its Employment Situation report. That release reflects conditions at mid-month (with two-week lag), gathered via a household survey and an employer survey. That pace seemed very slow and infrequent during the pandemic, particularly from March to June 2020. Indeed, the breathless attention paid to weekly UI Initial Claims releases during most of 2020 testifies to a need for higher frequency and timelier statistics, particularly during crises and business cycle turning points. Unfortunately, the claims releases (produced by ETA as part of running the UI system) are administrative totals. They are not constructed as official indicators with known statistical properties, so they can be misleading when used to assess economic conditions.[2] For that, we need the expertise of a statistical agency.

Second, our statistics need more detail. COVID-19’s impacts varied strongly within and between communities, industries, and demographic groups, etc. To address immediate distress and long-run inequities, we must measure them with enough granularity that policymakers can investigate and target them appropriately. The same is true for private sector decision-makers to make the best choices for their operations and investments. Relying on surveys to provide such nuance is particularly challenging because only the very largest surveys have sufficient representation of people in small population groups. Even the large monthly Current Population Survey of 60,000 families produces very volatile employment estimates for African-American, Hispanic, and Asian workers. One solution, oversampling certain populations, is piecemeal and expensive when applied to many surveys.[3] Furthermore, response rates for household surveys have been declining steadily in recent decades, leading to higher costs and lower data quality. By contrast, the UI data cover the people holding almost every U.S. job. Together, the wage records cover about 96 percent of US jobs and may cover more in the future.[4]


Getting to this new day entails improving and harnessing the claims and quarterly wage records that each state UI program already collects. To summarize, we must take four steps.

  1. Provide states with adequate resources to curate the records well, under BLS guidance. Curation refers to quality control steps to ensure that records are complete, accurate, and properly combined. BLS and the states already cooperate to compile UI employer records into the Quarterly Census of Employment and Wages (QCEW).[5] A similar arrangement is feasible for wage and claims records. States and BLS must also have adequate resources for analysis of the data.
  2. Enhance the records with additional information. Most states’ wage records still contain little more than identifiers (for the worker and employer) with earnings for each month in the quarter. Some States have already enhanced their records to help reveal local skill needs.[6] To this end, the BLS Labor Market Information Oversight Council has sponsored a set of studies of enhanced wage records.[7] The Workforce Information Advisory Council has also recommended enhancements to the Secretary of Labor.[8] To make the records really useful, they should include the following enhancements:
    1. Hours worked, so that we can calculate hourly earnings and track hours changes;
    2. Job title, which can be converted to a Standard Occupational Classification via text analysis;
    3. Work location, because some employers’ accounts cover multiple work sites; and
    4. Demographics, such as race, ethnicity, sex, education, and age to promote inclusion.
  3. Store the data in a secure warehouse that allows qualified access to statistical agencies, state labor market information shops, researchers, and program evaluators via standard memoranda of understanding. State-of-the-art privacy controls and security will also be essential for success. BLS has already developed a model memorandum of understanding with seven states in a pilot project. Despite many states’ interest in sharing these data with each other, there has never been an overall sharing agreement, although some states have joined together for limited data exchanges.[9]
  4. Encourage employers to adopt consistent wage and employment record-keeping practices, such as those under development by the T3 Innovation Network.[10] Widespread adoption of common data schema will dramatically lower the reporting burden on employers, raise the quality of the data reported, and facilitate employers’ comparisons of their internal data with published statistics.


Taking this on will require a cooperative network of diverse stakeholders who recognize its importance and prioritize its success. To begin with, the Biden administration and Hill appropriators will need to fund the states, ETA, and BLS adequately to cover these new activities. A rough ballpark estimate of the ongoing budget to cover BLS and state costs is about $100 million per year.[11] The benefits of improved statistics accrue very broadly and often indirectly, so it can be difficult to galvanize support for efforts like this proposal. Beneficiaries will include workers who are not laid off because companies and the Federal Reserve can forecast better, job-seekers who will know more about opportunities, communities that attract more investment, data analytic companies who can better benchmark their statistics, and advocates who can make their case with more precision. Over 40 forward-thinking organizations (such as professional and industry associations, labor unions, tech companies, thinktanks and more), and more than 150 people, including notable former public officials, recently signed a letter to appropriators in support of funding for this proposal.[12]

BLS will need to supervise the states and work with ETA in preparing the data. It will also need to use the new information to create new products and upgrade existing programs. Funding and training federal and State analytical staff will also be important.

The Department of Labor leadership and ETA will need to work with BLS and the states to help meet higher data quality standards. This will include ongoing consultation with BLS about institutional glitches and changes to reporting requirements. ETA will also need to help BLS as it designs and produces new economic indicators from the claims data. Fortunately, more complete and reliable data should benefit UI operations also.

The states will need to sign memoranda of understanding with BLS, revise data collection protocols, curate the data, and incorporate the new information into their workforce information analytics. Represented through the BLS Labor Market Oversight Council and Workforce Information Advisory Council, they should also help BLS shape the new products they need.[13]


Yes, this is complicated. However, it is also doable, overdue, and can ride some helpful tides. Public awareness of our data deficiencies is high. The initiative addresses many priorities of Congress and the current administration, including measuring and addressing racial inequities, reliance on evidence for policymaking, capitalizing on federal administrative data, UI system reform and modernization, building an effective workforce development system, and enhancing our national infrastructure. Fortunately, we have in-hand some essential proofs-of-concept, including a BLS pilot for the memoranda of understanding with the states, the QCEW model for curation and supervision, and examples of uses of wage records from the LEHD program. The T3 Innovation Network has engaged the private sector to design data schema that allow record enhancement with lower burden on employers. Last, but not least, advances in IT hardware and software have lowered costs and vastly expanded potential uses for these data.

Let’s strike while the iron is hot. Our decision-makers need more timely and granular labor market data. Enhanced, securely shared UI wage and claims records based on common data schema are the clear means to obtain it. If we add this asset to our national data infrastructure, we can improve policy and private outcomes, job search and recruitment, training for in-demand jobs, diversity, economic development, and more. We will also enhance our competitiveness by providing domestic and international investors with better information about where to locate their operations. The result will be a brighter economic future, locally and nationally.

[1] The Census Bureau has gained conditional access to UI wage records via individual agreements with all 50 states for the LEHD program and providing valued aggregated statistics, such as job flows. This arrangement does not have a guarantee of continuity or include funds for states to improve the underlying records or produce them expeditiously. Furthermore, commingling with IRS data and state-specific restrictions prohibit many uses and forms of sharing.

[2] See Government Accountability Office report GAO-21-191, “COVID-19: Urgent Actions Needed to Better Ensure an Effective Federal Response,”, pages 212-225 Also,

[3] See the American Economic Association’s Committee on Economic Statistics recent summary of a workshop to discuss measuring the economic effects of systemic racism and discrimination

[4] Employers submit wage records to the State for all covered workers who worked or received pay during the quarter. Covered employees include most corporate officials, all executives, all supervisory personnel, all professionals, all clerical workers, many farmworkers, all wage earners, all piece workers, and all part-time workers. Workers on paid sick leave, paid holiday, paid vacation, and the like are also covered. Workers on the payroll of more than one firm receive a record from each that is subject to UI. Among the workers excluded are elected officials, proprietors, the unincorporated self-employed, unpaid family members, certain farm, domestic, and railroad workers, and those who earned no wages because of work stoppages, temporary layoffs, illness, or unpaid vacations. Note that the recent CARES Act extension of UI eligibility to many non-traditional jobs means that some currently excluded positions could be included in the future. For all the current exclusions, see

[5] For information about the QCEW, see

[6] States have a keen interest in understanding the skills needs of local employers to assess their needs for Career and Technical Education and other training programs. Many States now match their wage records to educational administrative data-based courses and training received at community colleges and 4-year colleges and universities. For example, see projects conducted by the Administrative Data Research Facility sponsored by the Coleridge Initiative. See There are also numerous federal mandates to measure in-demand occupations and industries.

[9] Currently, states provide wage records under mandatory requirements to the Department of Health and Human Services for the National Directory of New Hires program and voluntarily to Census for the LEHD program. Neither program facilitates sharing of consistent records among the States. The various mutual State sharing agreements allow the States to exchange data to examine issues such as training and job mobility between States and labor sheds or job clusters that spread across State lines.

[10] The U.S. Chamber of Commerce Foundation (Chamber Foundation) and the T3 Innovation Network (T3 Network) recently partnered with the HR Open Standards Consortium (HR Open) to develop public-private open data standards for employment and earnings records. Employers use these records in business planning and managing their human resources and report them to federal and state governments, including State UI systems. The HR Open standards establish common definitions, clarify data relationships, and provide guidance for employers and their human resources (HR) technology service providers on maintaining and reporting the records, and for government agencies on establishing reporting requirements and data collection systems. These standards are designed to reduce data reporting costs for employers and government and improve labor market information. The Chamber Foundation and T3 Network also partnered with the National Association of State Workforce Agencies (NASWA) to assist in engaging states and other stakeholders in developing and reviewing the standards and exploring their applications for enhancing State UI wage records and improving labor market information. For more information, see

[11] For a recent summary of the budgetary situation of the BLS with a funding recommendation that includes this proposal, see the Council of Professional Associations on Federal Statistics document “Bureau of Labor Statistics Priorities for the 117th Congress and 2021-2025” Despite rising costs and the importance of its statistics to decisions made throughout the economy, BLS finding has fallen 13 percent in real terms since 2009.

[13] For information about these advisory groups, see and

Erica Groshen

  • Extension Faculty, ILR School

Weekly Inbox Updates