Effect on background checks of newly-enacted comprehensive background check policies in Oregon and Washington: a synthetic control approach
Injury Epidemiology volume 6, Article number: 45 (2019)
Comprehensive background check (CBC) laws extend background check requirements to private party firearm transfers to prevent firearm acquisitions by prohibited persons. The aim of our study was to estimate the association between CBC policies and changes in background check rates for firearm acquisition in two states (Oregon and Washington) that have newly-enacted CBC policies.
We used data on handgun background checks from January 1999 to December 2018 from the National Instant Criminal Background Check System. Observed trends in exposed states were contrasted with counterfactual trends estimated with the synthetic control group method.
CBC policies were associated with increases in background checks in Oregon (by 18.0%; p = 0.074), but not in Washington (4%; p = 0.321). A gradual increase in private party checks was seen following enactment in Washington; however, firearm transactions coded as “private” represent less than 5% of total background checks in that state.
Comprehensive background check policies appear to be effective in increasing pre-firearm-sale background checks in Oregon but not in Washington. Differences appear to be related to variations in the proportion of firearm sales that are private party transfers and to gradual adaptation to the new law by private gun sellers.
Public mass shootings and rising rates of firearm homicide and suicide have given new urgency to efforts to prevent firearm violence in the United States. Comprehensive background check (CBC) policies are one such effort, and they enjoy broad public support (Barry et al. 2018). CBC laws extend background check requirements to private party firearm transfers to prevent firearm acquisitions by prohibited persons. In states without CBC policies, approximately 57% of private party transfers precede without background checks; this decreases to 26% in states where CBC policies are in effect (Miller et al. 2017). Nearly all prohibited persons (95%) who acquire firearms for criminal purposes rely on sellers not required to conduct checks (Vittes et al. 2013).
Studies of CBC laws adopted in the 1990s that did not require a permit or license to purchase firearms have not produced evidence that the laws reduce homicides or suicides (Kagawa et al. 2018; Castillo-Carniglia et al. 2019). Recently-adopted CBC laws did not even reliably increase the overall number of background checks (Castillo-Carniglia et al. 2018). Such an increase would be an expected intermediate outcome if CBC policies are to have their intended effect on firearm violence. In this report, we assess the effect of the enactment of CBC laws on handgun background check rates in two US states, extending our preliminary findings for Washington, which considered 2 post-intervention years through December 2016 (Castillo-Carniglia et al. 2018), and adding data for Oregon.
We used monthly counts from the Federal Bureau of Investigation’s National Instant Criminal Background Check System (NICS) from January 1999 through December 2018. Washington implemented its CBC law in December 2014 and Oregon in August 2015. We used the synthetic control group method (Abadie et al. 2010) to assess whether the number of monthly handgun background checks per 100,000 people in each state changed following policy enactment in ways different from those forecasted by state-specific counterfactuals. With this method, synthetic controls for each of the two new CBC states were created from a weighted average of a pool of potential control states—the 28 states that did not implement CBC policies during the study period—that minimizes the root mean squared prediction error (RMSPE) for CBC state outcome measures for the pre-CBC law period. The states with potential of being part of the synthetic controls were: Alabama, Alaska, Arizona, Arkansas, Florida, Georgia, Idaho, Kansas, Kentucky, Louisiana, Maine, Minnesota, Mississippi, Montana, Nevada, New Hampshire, New Mexico, North Dakota, Ohio, Oklahoma, South Carolina, South Dakota, Texas, Utah, Virginia, West Virginia, Wisconsin, Wyoming.
Weights for control states sum to 1 and are estimated based on state-specific values for variables shown to predict background check rates in the two study states during the time before their CBC laws took effect. We again used the following variables (Castillo-Carniglia et al. 2018): percentages of the population between ages 18–24 years, male, Latino, black, having a high school diploma or higher, under the poverty line, or living in urban areas, and median household income, Gini coefficient measure of income inequality, violent crime rate, prevalence of firearm ownership (estimated as the proportion of firearm suicide among all suicides) and values of the outcome at 3 pre-intervention time points: January 1999, 2007, and 2012.
The synthetic control group method does not produce a traditional measure of uncertainty or a test of statistical significance. Inference is based on permutation tests (also called placebo interventions), in which the same analysis is replicated, but the intervention assignment is “permuted” across control states. There is no theoretical reason to expect an effect of a non-existent intervention, but random variation in the post-intervention period can exist. The original results are therefore compared with those from the permutation tests. P-values are estimated as the quotient of the number of control states with standardized differences (estimated as post-intervention RMSPE/pre-intervention RMSPE) larger than the treated states and the total number of states in the donor pool.
States with and without CBC policies exhibited an upward trend in monthly handgun background check rates from early 2000s until 2016–2017 and a small decrease more recently (Additional file 1: Figure S1). Rates were lower in states with CBC policies.
Table 1 displays weights for control states and RMSPEs from synthetic control group comparisons. For Oregon and Washington, eight and five control states, respectively, had nonzero weights. The largest weights for Oregon were assigned to Arizona, New Hampshire, North Dakota, and Montana; for Washington, the largest weights were assigned to Florida, Minnesota, and South Dakota.
There was a good match between the pre-intervention trend in background checks in each of the new CBC states and its synthetic control (Fig. 1). Further detail is in Table 2 and Fig. 2. The observed average background check rates in the post-intervention period, and the counterfactual rates for the two case states are presented in the first and second rows of the table. The estimated absolute effect of the CBC laws is the difference between the observed and the counterfactual rates, presented in the third row. The last row shows the number of control states with a difference associated with a hypothetical CBC policy greater than the observed difference in the case state. CBC law enactment was associated with a background check rate increase of 52.8 per 100,000 population in Oregon (an 18.0% increase); only 2 out of 27 states in the comparison pool experienced an increase greater than Oregon’s. No change was observed for Washington (4.9%; P = 0.3214).
Washington did experience an increase in checks arising from private-party sales following enactment of its CBC law (Additional file 1: Figure S2). However, checks for private-party transfers accounted for only 3.4% of background checks during the study period; 1.3% in the first year following implementation and 4.8% in the last year available.
Inconsistent association between CBC laws and the overall rate of background checks in these two neighboring states may reflect differences in the proportion of firearm transfers that occur between private parties, in compliance (i.e., adaptation to the new law by private gun sellers), and implementation (e.g., mechanism to detect prohibited persons through integrated data systems, police enforcement) (Vittes et al. 2013; Crifasi et al. 2018). For example, while Oregon showed a consistent increase in handgun background checks following CBC implementation, Washington only experienced a modest nonsignificant increase in background checks by mid-2017, almost two years after implementation of the law. Further comparisons between these two states on factors potentially associated with the effects of CBC policies on background checks such as measures of enforcement (e.g., persons charged with CBC law violations) would be very helpful.
The new laws may have had effects that we were unable to measure or detect. For example, Washington’s increase in private-party checks after its CBC policy took effect suggests a gradual adaptation to the new law by private gun sellers, which warrants the use of longer post-intervention data when evaluating CBC policies. In the case of Washington, however, these checks accounted for less than 5% of total handgun background checks, which is still a very small proportion compared with what has been estimated at the national level (Miller et al. 2017). No pre-enactment data were available, and the increase cannot be attributed to the policy with any certainty.
In contrast to the CBC laws studied here, permit-to-purchase (PTP) laws require handgun purchasers to apply for purchase permits directly to law enforcement agencies who perform background checks and ensure that applicants meet all eligibility requirements (e.g., safety training). PTP laws have been associated with reduced firearm homicides and suicides (Rudolph et al. 2015; Hasegawa et al. 2019). Further research will be needed to determine whether newly-enacted CBC laws in our study states and Vermont (enactment in April 2018) (Vermont General Assembly 2018) are associated with similar benefits.
CBC laws have inconsistent effects on background checks, potentially by several mechanisms acting jointly. All these mechanisms, along with incomplete reporting of prohibiting events, may be responsible for the observed modest or absent effect of CBC laws on rates of firearm violence (Wintemute 2019). Identifying and implementing measures to improve their effectiveness is an urgent public health priority.
Availability of data and materials
Software code used in Additional file 2. The datasets analyzed during the current study are available in the Federal Bureau of Investigation’s National Instant Criminal Background Check System at https://www.fbi.gov/file-repository/nics_firearm_checks_-_month_year_by_state_type.pdf/view
Abadie A, Diamond A, Hainmueller J. Synthetic control methods for comparative case studies: estimating the effect of California’s tobacco control program. J Am Stat Assoc. 2010;105(490):493–505.
Barry CL, Webster DW, Stone E, Crifasi CK, Vernick JS, McGinty EE. Public support for gun violence prevention policies among gun owners and non-gun owners in 2017. Am J Public Health. 2018;108(7):878–81.
Castillo-Carniglia A, Kagawa RMC, Cerdá M, Crifasi CK, Vernick JS, Webster DW, et al. California's comprehensive background check and misdemeanor violence prohibition policies and firearm mortality. Ann Epidemiol. 2019;30:50–6.
Castillo-Carniglia A, Kagawa RMC, Webster DW, Vernick JS, Cerda M, Wintemute GJ. Comprehensive background check policy and firearm background checks in three US states. Inj Prev. 2018;24(6):431–6.
Crifasi CK, Merrill-Francis M, Webster DW, Wintemute GJ, Vernick JS. Changes in the legal environment and enforcement of firearm transfer laws in Pennsylvania and Maryland. Inj Prev. 2018. https://doi.org/10.1136/injuryprev-2017-042582 [Published ahead of print].
Hasegawa RB, Webster DW, Small DS. Evaluating Missouri's handgun purchaser law: a bracketing method for addressing concerns about history interacting with group. Epidemiology. 2019;30(3):371–9.
Kagawa RMC, Castillo-Carniglia A, Vernick JS, Webster D, Crifasi C, Rudolph KE, et al. Repeal of comprehensive background check policies and firearm homicide and suicide. Epidemiology. 2018;29(4):494–502.
Miller M, Hepburn L, Azrael D. Firearm acquisition without background checks: results of a national survey. Ann Intern Med. 2017;166(4):233–9.
Rudolph KE, Stuart EA, Vernick JS, Webster DW. Association between connecticut's permit-to-purchase handgun law and homicides. Am J Public Health. 2015;105(8):e49–54.
Vermont General Assembly. An act relating to the disposition of unlawful and abandoned firearms, S.55 Act 94 2018. Available from: https://legislature.vermont.gov/bill/status/2018/S.55.
Vittes KA, Vernick JS, Webster DW. Legal status and source of offenders’ firearms in states with the least stringent criteria for gun ownership. Inj Prev. 2013;19(1):26–31.
Wintemute GJ. Background checks for firearm purchases: problem areas and recommendations to improve effectiveness. Health Aff (Millwood). 2019;38(10):1702–10.
We gratefully acknowledge funding from the Joyce Foundation (grant 15–36377), The California Wellness Foundation (grant 2014–255), and the Heising Simons Foundation (grant 2016–219).
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Figure S1. Trend in the rate (per 100,000 people) of monthly handgun background checks in the United State by CBC status, 1999 – 2018. Figure S2. Trend in the rate (per 100,000 people) of monthly private party handgun background checks in Washington, July 2013 to December 2018.
Stata code for synthetic control group method and figures.
About this article
Cite this article
Castillo-Carniglia, A., Webster, D.W. & Wintemute, G.J. Effect on background checks of newly-enacted comprehensive background check policies in Oregon and Washington: a synthetic control approach. Inj. Epidemiol. 6, 45 (2019). https://doi.org/10.1186/s40621-019-0225-8