# 25 Matching

In Chapter 4, we discussed three basic causal diagrams and suggested that the concept of nonparametric conditioning on \(Z\) is more demanding than simply including \(Z\) as another regressor in a linear regression model. This chapter develops this idea more fully and examines practical approaches to properly conditioning on confounding variables.

The code in this chapter uses the following packages. For instructions on how to set up your computer to use the code found in this book, see Section 1.2. Quarto templates for the exercises below are available at https://github.com/iangow/far_templates.

## 25.1 Background on auditor choice

For concreteness, we will explore the basic ideas of this chapter using the setting of auditor choice and its effects. An extensive literature has examined the question of whether “Big N” auditors produce higher audit quality. This question is examined in two of the papers we will study in this chapter (DeFond et al., 2017; Lawrence et al., 2011).

For data on auditors, the table `comp.funda`

has a column labelled `au`

, which lines up with `aucd`

in `comp.r_auditors`

.

While Lawrence et al. (2011) refers to the “Big Four”, they are actually referring to the current group that DeFond et al. (2017) refer to as the “Big N”. The latter term alludes to the prior incarnations of grouping of the top audit firms, including the “Big Five” (prior to the demise of Arthur Andersen” and the “Big Eight” (prior to mergers in the late 1980s) (see Gow and Kells, 2018 for more on this history).

`r_auditors |> collect(n = 12)`

aucd | audesc |
---|---|

0 | Unaudited |

1 | Arthur Andersen |

2 | Arthur Young |

3 | Coopers & Lybrand |

4 | Ernst & Young |

5 | Deloitte & Touche |

6 | KPMG |

7 | PricewaterhouseCoopers |

8 | Touche Ross |

9 | Other |

10 | Altschuler, Melvoin, and Glasser |

11 | BDO |

Looking closer at `r_auditors`

, we see that Arthur Andersen has an `aucd`

of 1. Note that Arthur Young is now part of Ernst & Young, Coopers & Lybrand is now part of PricewaterhouseCoopers, and Touche Ross is now part of Deloitte & Touche. So a reasonable approach to “Big N” would appear to be `big4 = aucd %in% 1:8L`

.^{1} It’s commonly understood that most large firms choose a Big Four auditor. But rather than simply accepting that, we can look at the data.

Here we focus on firms with meaningful financial statements (`sale > 0, at > 0`

) and fiscal 2019, the latest fiscal year with complete data at the time of first writing this chapter.

```
size_big4 <-
funda |>
filter(indfmt == "INDL", datafmt == "STD",
consol == "C", popsrc == "D") |>
filter(sale > 0, at > 0, fyear == 2019, !is.na(au)) |>
mutate(au = as.integer(au)) |>
select(gvkey, datadate, fyear, au, prcc_f, csho) |>
mutate(big4 = au %in% 1:8L,
mkt_cap = prcc_f * csho * 1e6) |>
arrange(gvkey, datadate) |>
collect()
```

We calculate market share by “bins” where each bin represents an order of magnitude of market capitalization. For example, `(9, 10]`

includes all firms with market capitalization over $1 billion (\(10^9\)) and less than or equal to $10 billion (\(10^{10}\)).

```
size_big4 |>
filter(mkt_cap > 1e6) |>
mutate(log_mkt_cap = log10(mkt_cap)) |>
mutate(bin = cut(log_mkt_cap, breaks = seq(6, 13, 1))) |>
group_by(bin) |>
summarize(big4_perc = 100 * mean(big4),
big4_num = sum(big4),
non_big4_num = sum(!big4))
```

bin | big4_perc | big4_num | non_big4_num |
---|---|---|---|

(6,7] | 16.364 | 63 | 322 |

(7,8] | 27.555 | 337 | 886 |

(8,9] | 58.034 | 1051 | 760 |

(9,10] | 89.671 | 1580 | 182 |

(10,11] | 96.761 | 687 | 23 |

(11,12] | 97.849 | 91 | 2 |

(12,13] | 100.000 | 1 | 0 |

Interestingly, one of the two cases of companies with an apparent market capitalization over $100 billion and a non–Big Four auditor is Vanjia, which apparently misstated the number of its shares outstanding as 8,550 million rather than 30 million.^{2}

```
size_big4 |>
filter(!big4, mkt_cap > 1e11)
```

gvkey | datadate | fyear | au | prcc_f | csho | big4 | mkt_cap |
---|---|---|---|---|---|---|---|

014447 | 2019-12-31 | 2019 | 9 | 92.988 | 2518.262 | FALSE | 234168146856 |

026478 | 2019-12-31 | 2019 | 9 | 14.380 | 8550.000 | FALSE | 122949000000 |

We can also present the data as a histogram. From either the table or the histogram, it is clear that the Big Four have overwhelming market share among the largest firms and have most of the market even among firms with market capitalization in the $100 million-to-$1 billion range.

## 25.2 Simulation analysis

We will use simulation analysis to examine more closely how propensity-score matching works and what it can and cannot do. We take the actual empirical distribution of market capitalization from the data depicted in the histogram above and give each firm a probability of using a Big Four auditor equal to the observed conditional probability for the histogram bin it is in.

We then draw 5000 firms from this distribution and assign each firm an auditor based on the conditional probabilities calculated above. A critical assumption here is that *conditional on market capitalization* whether a Big Four auditor is chosen is completely random. Finally, we calculate a measure of audit quality that is a non-linear function of market capitalization and a random noise component.

```
n_firms <- 5000
set.seed(2021)
sim_auditor <-
prob_big4 |>
sample_n(size = n_firms) |>
mutate(rand = runif(nrow(pick(everything()))),
big4 = rand < p_big4,
epsilon = rnorm(nrow(pick(everything()))),
id = 1:nrow(pick(everything()))) |>
mutate(audit_quality = big4 * 0 + mkt_cap ^ (1/3) * 0.003 + epsilon) |>
select(id, big4, mkt_cap, log_mkt_cap, audit_quality)
```

Given this is a simulation, we *know* that the correct causal diagram is the following:

We draw the arrow connecting \(X\) and \(Y\) with a dashed line because the true coefficient on `big4`

is zero, so *in this simulation* there is no actual causal relation between `big4`

and `audit_quality`

.

The good news from this causal diagram is that correctly conditioning on \(Z\) should give us an unbiased estimator of the causal effect of \(X\) on \(Y\). So let’s regress `audit_quality`

on `big4`

and `mkt_cap`

.

```
fm <- lm(audit_quality ~ big4 + I(mkt_cap/1e9),
data = sim_auditor)
modelsummary(fm,
estimate = "{estimate}{stars}",
gof_map = c("nobs", "r.squared"),
stars = c('*' = .1, '**' = 0.05, '***' = .01))
```

(1) | |
---|---|

(Intercept) | 1.561*** |

(0.050) | |

big4TRUE | 2.310*** |

(0.064) | |

I(mkt_cap/1e+09) | 0.057*** |

(0.001) | |

Num.Obs. | 5000 |

R2 | 0.555 |

So we have a clearly positive coefficient on `big4`

even “controlling for” `mkt_cap`

, and we might be tempted to conclude that we have identified a causal effect. Of course, we know that this conclusion is invalid. The issue is that we are effectively assuming a linear relation between `mkt_cap`

and audit quality, whereas the true relation is non-linear in `mkt_cap`

. As a result, “controlling for” `mkt_cap`

by including `mkt_cap`

does not adequately control for the relation between `mkt_cap`

and audit quality. Because there is a non-linear relation between `big4`

and `mkt_cap`

, the regression specification can use information in `big4`

to explain variation in `audit_quality`

in addition to that variation explained by a linear function of `mkt_cap`

. This additional explanatory power shows up as a statistically significant coefficient on `big4`

in the regression above.

Would some kind of matching analysis help?

Given its prominence in accounting research, we will focus on propensity-score matching here. The first step is the estimation of propensity scores. The most common approach is to use logistic regression, which we can do using the `glm`

function (which fits “generalized linear models”) with `family = binomial(link = "logit"))`

.

We can estimate propensity scores as the fitted values from the estimated logit model (we specify `type = "response"`

to get fitted values on the original \([0, 1]\) scale of the dependent variable).

But even with the propensity score, we need an algorithm to match treatment observations to their nearest counterparts among the control observations, where “nearest” means closest in terms of propensity scores. To this end, we can use the `matchit()`

function from the `MatchIt`

package. This function returns a `matchit`

object and we can apply the `match.data()`

function to this to retrieve the resulting matches and store these in `sim_matches`

. Following Lawrence et al. (2011), we use a caliper of 3%.

```
sim_matches <-
matchit(!big4 ~ mkt_cap, data = sim_match_pscores) |> # caliper = 0.03) |>
match.data()
```

Inspecting `sim_matches`

, we see that it contains the variable `distance`

that exactly matches `pscore`

that we estimated above.

`sim_matches |> count(pscore == distance)`

```
# A tibble: 1 × 2
`pscore == distance` n
<lgl> <int>
1 TRUE 3728
```

In addition to the original contents of `sim_auditor`

for the matched observations, `sim_matches`

contains data on the matched pairs in the column `subclass`

. Picking observations for three arbitrary values of `subclass`

, we can see that each value is associated with a pair of observations, one with `big4 == FALSE`

and one with `big4 == TRUE`

and very similar values of `mkt_cap`

and hence very similar `pscore`

values.

```
sim_matches |>
filter(subclass %in% c(220, 481, 333)) |>
select(id, big4, mkt_cap, pscore, subclass) |>
arrange(subclass, big4)
```

```
# A tibble: 6 × 5
id big4 mkt_cap pscore subclass
<int> <lgl> <dbl> <dbl> <fct>
1 1523 FALSE 238947920 0.509 220
2 4404 TRUE 1220480140 0.420 220
3 1783 FALSE 303631240 0.504 333
4 4510 TRUE 1386042900 0.406 333
5 2169 FALSE 17701600 0.530 481
6 2740 TRUE 157943150 0.517 481
```

To examine the success of the match, we can compare two pairs of histograms. The first pair Figure 25.3 shows the propensity scores for all observations. Here we see that there are many treatment observations with p-scores above say 0.70, but very few control observations with comparable p-scores to match with these. In this relatively simple setting, this is really just another way of saying there are many large Big Four clients and few similarly sized non–Big Four clients to match with them. This suggests that many observations will be unmatched.

The first pair of histograms Figure 25.4 shows the propensity scores for the *matched* observations. We can see that the distributions of p-scores for treatment and control observations look very similar, suggesting a fairly good match.

```
sim_match_pscores |>
ggplot(aes(x = pscore, fill = big4)) +
geom_histogram(binwidth = 0.025) +
facet_grid(big4 ~ .)
```

```
sim_matches |>
ggplot(aes(x = pscore, fill = big4)) +
geom_histogram(binwidth = 0.025) +
facet_grid(big4 ~ .)
```

If we are happy with the match, we can finish our analysis by estimating regressions. Like Lawrence et al. (2011), we estimate regressions with and without the variables from the p-score regression (Shipman et al., 2016 refer to these approaches as “MR” and “t-test,” respectively).

```
modelsummary(fms,
estimate = "{estimate}{stars}",
gof_map = c("nobs", "r.squared"),
stars = c('*' = .1, '**' = 0.05, '***' = .01))
```

(1) | (2) | |
---|---|---|

(Intercept) | 1.601*** | 1.505*** |

(0.037) | (0.033) | |

big4TRUE | 0.971*** | 0.908*** |

(0.053) | (0.047) | |

I(mkt_cap/1e+09) | 0.135*** | |

(0.004) | ||

Num.Obs. | 3728 | 3728 |

R2 | 0.083 | 0.275 |

Interestingly, we see that `big4`

still has a statistically significant relation with audit quality. This is likely due to a residual effect of `mkt_cap`

being picked up by `big4`

due to the imperfection of the propensity-score match.

### 25.2.1 Exercises

- What happens in the above analyses if the true effect of
`big4`

on audit quality is`10`

instead of`0`

? - What happens in the above analyses if the caliper is reduced to
`0.01`

from`0.03`

(but the true effect of`big4`

on audit quality is still`0`

)? - What happens in the above analyses if the caliper is reduced to
`0.01`

from`0.03`

and the true effect of`big4`

on audit quality is`10`

? - Given the above, what are your conclusions about the value of propensity-score matching and OLS in causal inference?
- What regression results do you get if you repeat the analysis shown in Table 25.2, but replace
`I(mkt_cap/1e+09)`

with`I(mkt_cap ^(1/3) * 0.003)`

? Does this suggest a solution to the issues we see above? If so, what challenges does such a solution face? - Was endogeneity an issue in the simulation above? If not, why not? If so, in what way, and what could you do to address it?

## 25.3 Replication of Lawrence et al. (2011)

To facilitate discussion, we conduct an approximate replication of Lawrence et al. (2011) and begin by constructing a subset of Compustat with calculations used in that paper.

```
comp <-
funda |>
filter(indfmt == "INDL", datafmt == "STD",
consol == "C", popsrc == "D") |>
filter(!is.na(sich), !between(sich, 6000, 6999),
sale > 0, at > 0, (dltt >= 0 | is.na(dltt)) & (dlc >= 0 | is.na(dlc)),
prcc_f * csho > 0, ceq > 0) |>
select(gvkey, datadate, fyear, au, prcc_f, csho, at, ib,
dltt, dlc, rect, ppent, sale, act, lct, sich,
oancf) |>
mutate(sic2 = floor(sich/100)) |>
filter(between(fyear, 1988, 2006)) |>
collect() |>
group_by(gvkey) |>
arrange(fyear) |>
mutate(lag_fyear = lag(fyear),
avg_at = (lag(at) + at)/2,
log_at = log(at),
aturn = if_else(lag(at) > 0, sale/lag(at), NA_real_),
inv_avg_at = 1/avg_at,
d_sale = sale - lag(sale),
d_rect = rect - lag(rect),
d_sale_at = (d_sale - d_rect)/avg_at,
ppe_at = ppent/avg_at,
curr = if_else(lct > 0, act/lct, NA_real_),
lag_curr = lag(curr),
lev = if_else(avg_at > 0,
(dltt + coalesce(dlc, 0))/avg_at, NA_real_),
lag_lev = lag(lev),
roa = if_else(avg_at > 0, ib/avg_at, NA_real_),
lag_roa = lag(roa),
accruals = ib - oancf,
ta = accruals/avg_at,
mkt_cap = prcc_f * csho,
log_mkt = if_else(mkt_cap > 0, log(mkt_cap), NA_real_)) |>
ungroup() |>
filter(lag_fyear == fyear - 1L)
```

Lawrence et al. (2011) estimate discretionary accrual models by industry and year, subject to a requirement of at least 10 data points for each model. So we compile a list of industry-years that meet this criterion.

We next construct a function to estimate discretionary accruals along the lines described in footnote 11 of Lawrence et al. (2011).

```
get_klw_data <- function(sic2, fyear) {
reg_data <-
comp |>
filter(sic2 == !!sic2, fyear == !!fyear)
fm <- tryCatch(lm(ta ~ inv_avg_at + d_sale_at + ppe_at,
data = reg_data, na.action = na.exclude),
error = function(e) NULL)
if (is.null(fm)) return(NULL)
results <-
reg_data |>
select(gvkey, fyear, roa) |>
bind_cols(da = resid(fm)) |>
filter(!is.na(da))
ada_data <-
results |>
inner_join(results, by = "fyear",
suffix = c("", "_match"),
relationship = "many-to-many") |>
filter(gvkey != gvkey_match) |>
mutate(roa_diff = abs(roa - roa_match)) |>
group_by(gvkey, fyear) |>
filter(roa_diff == min(roa_diff, na.rm = TRUE)) |>
mutate(ada = abs(da - da_match)) |>
select(gvkey, fyear, da, ada) |>
ungroup()
ada_data
}
```

We can then `Map`

this function onto our industry-years and store the results in `klw_results`

.^{3}

```
full_sample <-
comp |>
inner_join(klw_results, by = c("gvkey", "fyear")) |>
filter(avg_at > 0) |>
mutate(big4 = au %in% 1:8L,
mkt_cap = prcc_f * csho,
log_mkt = if_else(mkt_cap > 0, log(mkt_cap), NA_real_))
fms <- list()
fms[[1]] <- feols(ada ~ big4 + log_mkt + lag_roa +
lag_lev + lag_curr | sic2 + fyear,
data = full_sample)
```

`NOTE: 13,505 observations removed because of NA values (RHS: 13,505).`

Lawrence et al. (2011) use the widely used approach of winsorizing variables used in their analysis. We use the `winsorize`

function from the `farr`

package (as discussed in Chapter 24) to do this.

```
win_vars <- c("ada", "log_mkt", "lag_roa", "lag_lev", "lag_curr",
"roa", "lev", "curr", "mkt_cap")
full_sample_win <-
full_sample |>
mutate(across(all_of(win_vars),
\(x) winsorize(x, prob = 0.01)))
fms[[2]] <-
feols(ada ~ big4 + log_mkt + lag_roa + lag_lev + lag_curr |
sic2 + fyear,
~ gvkey + fyear,
data = full_sample_win)
```

`NOTE: 13,505 observations removed because of NA values (RHS: 13,505).`

```
full_sample_win_no_miss <-
full_sample_win |>
filter(!is.na(log_at) &
!is.na(log_mkt) &
!is.na(aturn) &
!is.na(curr) &
!is.na(lev) &
!is.na(roa))
lmz_match_win <- matchit(!big4 ~ log_at + log_mkt + aturn +
curr + lev + roa,
data = full_sample_win_no_miss,
caliper = 0.0300, std.caliper = FALSE,
m.order = "largest", discard = "both")
```

We can estimate propensity scores as the fitted values from the estimated logit model (we specify `type = "response"`

to get fitted values on the original \([0, 1]\) scale of the dependent variable).

Following Lawrence et al. (2011), we discard matches where the difference in p-scores is greater than 3%.

```
lmz_matched <- match.data(lmz_match)
fms[[3]] <-
feols(ada ~ big4 + log_mkt + lag_roa + lag_lev + lag_curr |
sic2 + fyear, ~ gvkey + fyear,
data = lmz_matched)
```

`NOTE: 4,702 observations removed because of NA values (RHS: 4,702).`

```
lmz_matched_win <- match.data(lmz_match_win)
fms[[4]] <-
feols(ada ~ big4 + log_mkt + lag_roa + lag_lev + lag_curr |
sic2 + fyear,
~ gvkey + fyear,
data = lmz_matched_win)
```

`NOTE: 4,665 observations removed because of NA values (RHS: 4,665).`

```
modelsummary(fms,
estimate = "{estimate}{stars}",
gof_map = c("nobs", "r.squared", "adj.r.squared"),
stars = c('*' = .1, '**' = 0.05, '***' = .01))
```

(1) | (2) | (3) | (4) | |
---|---|---|---|---|

big4TRUE | −0.016*** | −0.014*** | −0.007*** | −0.006*** |

(0.002) | (0.001) | (0.002) | (0.002) | |

log_mkt | −0.006*** | −0.005*** | −0.007*** | −0.005*** |

(0.001) | (0.000) | (0.001) | (0.001) | |

lag_roa | −0.108*** | −0.124*** | −0.107*** | −0.136*** |

(0.006) | (0.005) | (0.017) | (0.005) | |

lag_lev | −0.045*** | −0.045*** | −0.054*** | −0.050*** |

(0.003) | (0.002) | (0.005) | (0.005) | |

lag_curr | 0.000*** | −0.001*** | 0.000 | −0.001** |

(0.000) | (0.000) | (0.000) | (0.000) | |

Num.Obs. | 73011 | 73011 | 24210 | 24237 |

R2 | 0.127 | 0.150 | 0.116 | 0.134 |

R2 Adj. | 0.127 | 0.149 | 0.113 | 0.132 |

## 25.4 DeFond et al. (2017)

DeFond et al. (2017) re-examine the question of Lawrence et al. (2011), but with some differences in approach.

The first set of differences relates to design choices in the application of propensity-score matching. While Lawrence et al. (2011) focus on a single propensity-score model with five primary covariates, DeFond et al. (2017) augment the propensity-score model with twenty additional variables, namely the squares and cubes of these covariates and the ten interactions between the five covariates. DeFond et al. (2017) then select 1,500 random subsets from the twenty variables to develop propensity-score models. For each estimated propensity-score model, the caliper is set at a random value (less than 30%) that yields a “balanced matched sample”.

A second design-choice variation that DeFond et al. (2017) consider is matching with replacement. While Lawrence et al. (2011) focus on matching one treatment observation with (at most) one control observation without replacement (i.e., once a control observation is matched to a treatment observation, it is not available to match with another treatment observation), DeFond et al. (2017) consider matches *with* replacement and consider matches where each treatment observation is matched to one, two, or three control firms.

The second set of differences between DeFond et al. (2017) and Lawrence et al. (2011) relate to the measures of audit quality considered. Lawrence et al. (2011) consider cost of equity capital and analyst forecast accuracy, while DeFond et al. (2017) omit these two measures on the basis that they have “poor construct validity, and as a result are rarely used in the prior literature” (DeFond et al., 2017, p. 3637). Instead, DeFond et al. (2017) consider three other measures of audit quality: income-increasing discretionary accruals, restatements, and going concern opinions.

## 25.5 Discussion questions

Given the evidence presented in the simulation, how do you interpret the regression results presented above? How do DeFond et al. (2017) appear to interpret the results?

Referring back to the basic causal relations described in Section 4.2, what is the causal diagram implied by equation (1) of and Table 2 of Lawrence et al. (2011)? What variables are in

*PROXY_CONTROLS*for Table 2? Why is*LOG_ASSETS*not found in Table 2?Do Lawrence et al. (2011) report results from estimating equation (1)? What happens to the “difference in means” between the Full Sample and the Propensity-Score Matched Sample in Table 1? Does this make sense?

Why do you think the difference in means is still statistically significant for two variables in the Propensity-Score Matched Sample column of Table 1 in Lawrence et al. (2011)? Do you think this is a concern?

Lawrence et al. (2011) evaluate three outcomes as measures of audit quality: discretionary accruals, cost of equity, and analyst forecast accuracy? Evaluate each of these measures. Which do you think is best? What are the strengths and weaknesses of this best measure?

Can you think of other measures of audit quality that might make sense?

Apply the check list of Shipman et al. (2016, pp. 217–218) and the questions in Panels B and C of Table 1 of Shipman et al. (2016) to Lawrence et al. (2011). How does Lawrence et al. (2011) stack up against these?

This appears to be the approach used in DeFond et al. (2017), but whether this lines up with the approach used in Lawrence et al. (2011) is hard to say.↩︎

The higher number can be seen here, but this appears to have been quietly corrected here. The other company is LVMH, the French luxury goods firm, which appears to have been audited in part by Ernst & Young.↩︎