5.3 Difference-in-differences

Event studies and placebo dates: reading pre-trends and ranking the policy date

An event study computes a separate estimate for each period before and after a policy starts. The estimates for the periods before the start show whether the treated group and the comparison group already moved apart before the policy. A placebo check repeats the estimate at dates when nothing happened and asks how unusual the estimate at the policy date is.

In brief

In Part A, some stores of a chain start a policy in period 7, and the others do not. Part B follows the daily orders of an online store that starts a policy on a Monday.

An event time is the distance in periods from the start of the policy. Each event time gets a difference-in-differences (DiD) estimate, measured against event time −1. Event times −6 to −2 give the leads, and event times 0 to 5 give the lags.
The pre-trend test asks whether all five leads are zero at the same time. A small p-value means that leads this far from zero would be unlikely if the trends before the policy were the same.
The actual estimate compares the w days from the policy date on with the w days before it, where w is the window length.
The naive t-test compares these two windows as if all days were independent.
The placebo p-value is the rank of the policy date among K candidate dates, divided by K. The other candidate dates are placebo dates: dates when nothing happened.

Move the sliders and watch these numbers. The explanation below the simulation gives the reasons.

Try it

The page has two simulations. Part A shows an event study with many stores, and Part B shows placebo dates in one daily series. The data are simulated, so the page knows the true effect.

Part A. An event study with leads and lags

The simulation has 40 treated stores and 40 comparison stores in periods 1 to 12. The treated stores start the policy in period 7 (event time 0). Each store has its own level and its own noise. Both groups share the same period-to-period changes, unless you move the extra-growth slider.

Dots show the estimate for each event time. The vertical line through each dot is its 95% confidence interval. The hollow dot at event time −1 is the reference period. Hover over a dot to see its value.

Average estimate, event times 0 to 5
True average effect, event times 0 to 5
Average estimate minus true average effect
Pre-trend test p-value (all five leads)

Estimates for each event time
Event time	Estimate	95% confidence interval	True effect

Part B. Placebo dates in daily data

The simulation has 60 weeks of daily orders. The first day is a Monday. The policy starts on the first day of week 53, also a Monday. Every candidate date uses windows of the same length. The window starts at 14 days, so each window holds every weekday twice.

Daily orders around the policy date

The vertical line marks the policy date. The two shaded areas are the before window and the after window of the policy date. Short gray marks at the bottom show the placebo dates in the last 16 weeks before the policy. Earlier placebo dates are not shown.

Estimates at all candidate dates

Each dot is the estimate at one candidate date, sorted from lowest to highest. The shaded band runs from minus to plus the size of the actual estimate. Dots on the edge of the band or outside it are at least as far from zero as the actual estimate. Hover over a dot to see its value.

Actual estimate (after minus before)
Rank of the policy date among K candidate dates (1 = farthest from zero)
Placebo p-value (rank/K)
Smallest possible placebo p-value (1/K)
Naive t-test p-value
Spread of the placebo estimates (standard deviation)

What to notice

Part A: keep the extra-growth slider at 0 and draw several samples. Most intervals of the leads contain zero, and the pre-trend test p-value is above 0.05 in most samples. Now set the noise to 0.5 and the extra growth to 0.2. The leads climb toward zero at event time −1, and most estimates after the start sit above the dashed true effect. The average estimate now exceeds the true average effect by about 0.7. The p-value falls below 0.001.
Part A: set the noise to 3 and the extra growth to 0.3, then draw several samples. In most samples the pre-trend test p-value is above 0.05, so the test does not detect the pre-trend. In most of these samples, the average estimate still exceeds the true average effect by more than 0.7.
Part B: set the effect to 0 and the day-to-day correlation to 0.9, then draw several samples. The naive t-test p-value is below 0.05 in about half of the samples, although nothing happened. The placebo p-value is rarely below 0.05.
Part B: press Reset. Then set the window to 10 days and the weekly pattern to 4. The actual estimate drops, in most samples below zero, although the true effect is still 1. The reason: the after window now holds more of the weekdays with few orders.
Part B, with the window still at 10 days: the same-weekday placebo estimates drop too. They center near −1.5, in most samples farther from zero than the actual estimate. So the placebo p-value is large, in most samples above 0.5. Now turn the same-weekday option off. The placebo estimates spread out, and their standard deviation grows by about 0.3 to 0.5. With a 14-day window, the same switch changes the spread by less than 0.1 in most samples.
Part B: look at K, the second number in the rank readout. With the same-weekday option off, K is more than 300, and the smallest possible placebo p-value is about 0.003. Turn the option on. K falls to about 50, so the placebo p-value cannot fall below about 0.02, even with a large effect.

The idea

Leads and lags

A chain of stores starts a new policy in some of its stores in period 7. The other stores do not start the policy. We call the first set of stores the treated group and the second set the comparison group. We observe an outcome, for example weekly sales, in periods 1 to 12.

Difference-in-differences (DiD) compares the change in the treated group with the change in the comparison group over the same time. An event study computes one DiD estimate for each period. We label each period t by its event time \(e = t - 7\), the distance in periods from the start of the policy. Event time 0 is the first period with the policy. Event time −1 is the last period before the policy.

Each estimate compares event time e with event time −1:

\[\begin{aligned} & \text{estimate for event time } e \\ = {} & (\text{treated at } e - \text{treated at } {-1}) \\ & - (\text{comparison at } e - \text{comparison at } {-1}) \end{aligned}\]

Event time −1 is the reference period. Its estimate is 0 by construction, so it has no confidence interval. All other estimates are measured against it. So chance noise in the reference period moves all estimates up or down together.

The standard errors in symbols

Write \(y_{it}\) for the outcome of store \(i\) in period \(t\). Let \(d_{ie} = y_{ie} - y_{i,-1}\) be the change of store \(i\) from event time −1 to \(e\). The standard error is the square root of \(\operatorname{var}_T(d_e)/n_T + \operatorname{var}_C(d_e)/n_C\). Here \(T\) and \(C\) mark the treated and comparison groups. The 95% confidence interval is the estimate plus or minus 1.96 standard errors.

The estimates for event times −6 to −2 are the leads. The estimates for event times 0 to 5 are the lags. The lags show how the effect changes over time.

The same estimates from a regression

The same numbers come from a regression. It has a separate intercept for each store (unit fixed effects) and a separate intercept for each period (period fixed effects). It also has one indicator for each event time, which is 1 for treated stores in that period.

The regression must leave out one of these indicators, or it has no unique solution. This page leaves out the indicator for event time −1, the last period before the policy, as Huntington-Klein (2025, ch. 18) does.

In symbols, write \(y_{it}\) for the outcome of store \(i\) in period \(t\), and \(D_i = 1\) for a treated store. The event-study regression is

\[\begin{aligned} y_{it} = {} & \alpha_i + \gamma_t \\ & + \sum_{e \neq -1} \beta_e D_i \mathbf{1}[t - 7 = e] + \varepsilon_{it} \end{aligned}\]

With one start date and a comparison group that never starts the policy, the estimate of \(\beta_e\) equals the DiD formula above.

Some books number the periods in another way. For example, Huntington-Klein (2025, ch. 18) calls the last period before the policy period 0.

What the leads can and cannot show

In this simulation, the policy has no effect before it starts (no anticipation). So if the two groups had the same trend before the policy, the leads are near zero, apart from noise. Leads that move away from zero then show a pre-trend: the trends of the two groups already differed before the policy. In real data, no anticipation is an assumption. A pre-trend test asks whether all five leads are zero at the same time. A small p-value means that leads this far from zero would be unlikely if the trends before the policy were the same.

The pre-trend test in symbols

The pre-trend test statistic is \(W = b' V^{-1} b\). Here \(b\) holds the five leads. Their covariance matrix is \(V = \operatorname{cov}_T(d)/n_T + \operatorname{cov}_C(d)/n_C\). It is computed from the changes \(d\) at event times −6 to −2.

The p-value comes from a chi-square distribution with 5 degrees of freedom. This distribution is an approximation for large samples. With 40 stores per group and no pre-trend, about 7 percent of the samples in this simulation give a p-value below 0.05, not 5 percent. Suppose that the treated group grows by \(\delta\) more per period without the policy. Then \(\beta_e\) estimates \(\tau_e + \delta(e + 1)\), where \(\tau_e\) is the true effect.

Passing the pre-trend test does not show that parallel trends holds. The parallel trends assumption says that without the policy, the treated group would have changed by the same amount as the comparison group. It concerns the periods after the start, and nobody observes the treated group without the policy in those periods. So no test of the data can check the assumption. The leads show only whether the trends differed before the policy (Huntington-Klein 2025, ch. 18).

A pre-trend test can also miss a pre-trend that exists. The power of a test is the chance that it detects a difference that exists. Pre-trend tests can have low power (Roth 2022). A small pre-trend in noisy data often passes the test, and the estimates after the start are still biased.

Bias after a passed pre-trend test

Roth (2022) also shows a second problem. Suppose that an analysis goes ahead only after it passes a pre-trend test. Then the bias of the estimates can grow, and the confidence intervals can miss the true effect more often.

We ran the simulation in Part A 4,000 times with noise 3 and extra growth 0.3. About 70 percent of the samples passed the test (p-value above 0.05). In the samples that passed, the average estimate after the start missed the true average effect by about 1.2. In the samples that failed, it missed by about 0.7.

An estimate is statistically significant at the 5 percent level if its test gives a p-value below 0.05. For a single lead or lag, this is the case when its 95% confidence interval does not contain zero. With many leads, some can look significant by chance alone, even when nothing is wrong (Huntington-Klein 2025, ch. 18).

Placebo dates in daily data

Now take one daily series, for example the daily orders of an online store. The store starts a policy on a Monday. We call this day the policy date.

A simple estimate compares the average of the w days from the policy date on with the average of the w days before it. Here w is the window length in days. We call these two sets of days the after window and the before window. We call the estimate at the policy date the actual estimate.

Daily noise can be correlated from one day to the next: a busy day tends to be followed by another busy day. A naive t-test compares the two windows as if all days were independent. Huntington-Klein (2025, ch. 17) describes what happens in data with this kind of correlation. Pick a date at random, when nothing happened. A test that treats the days as independent then finds a statistically significant effect about half the time. A test at the 5 percent level should find a significant effect at such a date only 5 percent of the time.

The daily series, the estimate, and the naive t-test in symbols

Write \(y_d\) for the orders on day \(d\) and \(D_0\) for the policy date. The simulation draws

\[\begin{aligned} y_d = {} & 100 + s \cdot \operatorname{pattern}(\text{weekday of } d) \\ & + \tau \cdot \mathbf{1}[d \ge D_0] + u_d, \\ u_d = {} & \rho \, u_{d-1} + \varepsilon_d \end{aligned}\]

The pattern from Monday to Sunday is −0.8, −0.6, −0.4, −0.2, 0.2, 1.0, 0.8, and \(s\) is the size of the weekly pattern. The shocks \(\varepsilon_d\) are independent normal draws with standard deviation \(\sigma\). The estimate at date \(c\) is

\[\begin{aligned} \operatorname{est}(c) = {} & \operatorname{mean}(y_c, \ldots, y_{c+w-1}) \\ & - \operatorname{mean}(y_{c-w}, \ldots, y_{c-1}) \end{aligned}\]

The naive t-test is a pooled two-sample t-test of the two windows with \(2w - 2\) degrees of freedom.

A placebo date is a date when nothing happened. Abadie, Diamond, and Hainmueller (2015) call checks at such dates in-time placebos: they apply the method to dates when the intervention did not occur. On this page, each placebo date uses only data from before the policy: both of its windows end before the policy date. Huntington-Klein (2025, ch. 18) describes this check. He also suggests trying many placebo dates and seeing where the actual estimate falls among the placebo estimates. He calls this approach randomization inference (see the key terms).

The logic is this. Suppose that the policy changed nothing and that nothing else happened on the policy date. Then the policy date is one more date like the placebo dates, and its estimate is one more draw from the same distribution as theirs. The ranking uses no standard error, so it needs no formula that treats the days as independent. It needs another condition instead: without the policy, the policy date must be like the placebo dates.

The page computes the estimate at every candidate date: the policy date and each placebo date. K is the number of candidate dates. The page ranks the candidate dates by the distance of their estimate from zero, and rank 1 is the largest distance. The placebo p-value is the rank of the policy date divided by K:

\[\text{placebo p-value} = \frac{\begin{gathered} \text{number of candidate dates} \\ \text{with an estimate} \\ \text{at least as far from zero} \\ \text{as the actual estimate} \end{gathered}}{K}\]

The policy date counts in both parts of the fraction. So the smallest possible p-value is \(1/K\).

One-sided tests, the floor of \(1/K\), and when the p-value is exact

In symbols, a placebo date \(c\) needs \(c - w \ge 0\) and \(c + w - 1 < D_0\). The placebo p-value is the number of candidate dates \(c\) with \(|\operatorname{est}(c)| \ge |\operatorname{est}(D_0)|\), divided by \(K\).

Ranking by the distance from zero looks for a change in either direction, so this is a two-sided test. If only an increase counts as evidence, a one-sided version ranks the signed estimates instead, from the largest increase down.

Abadie, Diamond, and Hainmueller (2015) define a p-value of this kind for placebo checks across comparison units. Their p-value is the fraction of placebo effects that are greater than or equal to the effect for the treated unit. This page applies the same rule to placebo dates.

The floor of \(1/K\) matters in real studies. In their study of German reunification, West Germany ranked first among 17 countries in a placebo test across countries. So its p-value was 1/17, about 0.06, and no result in that study could fall below 0.05 (Abadie, Diamond, and Hainmueller 2015).

Suppose that the policy date was chosen as if at random from the candidate dates. Then the placebo p-value is exact. When the policy has no effect, the chance of a p-value at or below 0.10 is at most 0.10. The same holds for every other level. If the policy date was not chosen in this way, the p-value describes how unusual the actual estimate is among dates when nothing happened. Abadie, Diamond, and Hainmueller (2015) give the same two readings for their p-value across units.

Why the placebo dates are on the same weekday

Daily orders often follow a weekly pattern. In this simulation, orders are highest on weekends. A window of 7, 14, 21, or 28 days holds each weekday the same number of times. So the weekly pattern cancels out of the estimate. A window of another length does not.

For example, a 10-day after window that starts on a Monday holds two Mondays, two Tuesdays, and two Wednesdays. The 10 days before that Monday hold two Fridays, two Saturdays, and two Sundays. So the estimate also contains a difference between weekdays.

A placebo date on another weekday covers a different mix of weekdays, so its estimate contains a different weekday difference. A placebo date on the same weekday as the policy date has the same mix as the policy date. So when the policy has no effect, the actual estimate is like one more placebo estimate. For this reason, the page uses only placebo dates on the same weekday by default. This is a design choice. One applied example is Cattaneo, Díaz, and Titiunik (2026), who use placebo adoption dates on the same weekday in daily crime data.

Same-weekday placebo dates do not remove the weekday difference. It stays in the actual estimate and in every placebo estimate. In this simulation, the weekday difference is negative for every window length that is not a multiple of 7.

A positive effect of the policy moves the actual estimate up, so the actual estimate can end up closer to zero than the placebo estimates. Because the placebo p-value ranks the estimates by their distance from zero, it can then be large although the policy has an effect. A window of 7, 14, 21, or 28 days has no weekday difference. For this reason, the page starts with a 14-day window.

References

Abadie, A., Diamond, A., & Hainmueller, J. (2015). Comparative politics and the synthetic control method. American Journal of Political Science, 59(2), 495-510. https://doi.org/10.1111/ajps.12116
Cattaneo, M. D., Díaz, C., & Titiunik, R. (2026). Randomization inference for before-and-after studies with multiple units: An application to a criminal procedure reform in Uruguay. Journal of the Royal Statistical Society Series A: Statistics in Society, 189(4), 2127-2146. https://doi.org/10.1093/jrsssa/qnaf192
Huntington-Klein, N. (2025). The effect: An introduction to research design and causality (2nd ed.). Chapman & Hall. Chapter 17, Event Studies, and chapter 18, Difference-in-Differences. Free online edition: chapter 17 and chapter 18.
Roth, J. (2022). Pretest with caution: Event-study estimates after testing for parallel trends. American Economic Review: Insights, 4(3), 305-322. https://doi.org/10.1257/aeri.20210236