Package {evalHTE}

Title: Evaluating Heterogeneous Treatment Effects
Version: 0.2.0
Description: Provides various statistical methods for evaluating heterogeneous treatment effects (HTE) in randomized experiments. The package includes tools to estimate uniform confidence bands for estimation of the group average treatment effect sorted by generic machine learning algorithms (GATES). It also provides the tools to identify a subgroup of individuals who are likely to benefit from a treatment the most "exceptional responders" or those who are harmed by it. Detailed reference in Imai and Li (2023) <doi:10.48550/arXiv.2310.07973>.
License: MIT + file LICENSE
Depends: R (≥ 3.5.0), dplyr (≥ 1.0.10)
Imports: cli, evalITR (≥ 1.1.0), ggplot2, ggthemes, rlang, zoo, furrr, ggdist, scales, tidyr, stats, purrr, caret
Encoding: UTF-8
RoxygenNote: 7.3.3
Suggests: testthat (≥ 3.0.0), knitr, rmarkdown, future, grf, magrittr, tibble
VignetteBuilder: knitr
NeedsCompilation: no
Packaged: 2026-09-06 19:22:11 UTC; shazn
Author: Michael Lingzhi Li [aut, cre], Kosuke Imai [aut], Jialu Li [ctb]
Maintainer: Michael Lingzhi Li <mili@hbs.edu>
Repository: CRAN
Date/Publication: 2026-09-07 08:30:02 UTC

Estimation of the Grouped Average Treatment Effects (GATEs) in Randomized Experiments

Description

Estimates grouped average treatment effects from a continuous score.

Usage

GATE(D, tau, Y, ngates = 5, centered = FALSE)

Arguments

D

Binary treatment indicator (0 or 1).

tau

Continuous score vector.

Y

Outcome vector.

ngates

Number of groups (at least 2).

centered

Whether to center outcomes before estimation.

Details

Uses GATE for inference.

Value

A list with group estimates gate and standard errors sd, ordered by increasing score.

Author(s)

Michael Lingzhi Li, Technology and Operations Management, Harvard Business School mili@hbs.edu, https://www.michaellz.com/;

References

Imai and Li (2022). “Statistical Inference for Heterogeneous Treatment Effects Discovered by Generic Machine Learning in Randomized Experiments”,

Examples

D = c(1,0,1,0,1,0,1,0)
tau = c(0,0.1,0.2,0.3,0.4,0.5,0.6,0.7)
Y = c(4,5,0,2,4,1,-4,3)
gatelist <- GATE(D,tau,Y,ngates=2)
gatelist$gate
gatelist$sd

Estimation of the Grouped Average Treatment Effects (GATEs) in Randomized Experiments Under Cross Validation

Description

Estimates grouped average treatment effects from cross-validated scores.

Usage

GATEcv(D, tau, Y, ind, ngates = 5, centered = FALSE)

Arguments

D

Binary treatment indicator (0 or 1).

tau

A matrix of scores with one column per fold. Column i contains predictions for all observations from a model trained without fold i.

Y

Outcome vector.

ind

Integer validation-fold labels starting at 1.

ngates

Number of groups (at least 2).

centered

Whether to center outcomes before estimation.

Details

Uses GATEcv for inference.

Value

A list with group estimates gate and standard errors sd, ordered by increasing score.

Author(s)

Michael Lingzhi Li, Technology and Operations Management, Harvard Business School mili@hbs.edu, https://www.michaellz.com/;

References

Imai and Li (2022). “Statistical Inference for Heterogeneous Treatment Effects Discovered by Generic Machine Learning in Randomized Experiments”,

Examples

D = c(1,0,1,0,1,0,1,0)
tau = matrix(c(0,0.1,0.2,0.3,0.4,0.5,0.6,0.7,-0.5,-0.3,-0.1,0.1,0.3,0.5,0.7,0.9),nrow = 8, ncol = 2)
Y = c(4,5,0,2,4,1,-4,3)
ind = c(rep(1,4),rep(2,4))
gatelist <- GATEcv(D, tau, Y, ind, ngates = 2)
gatelist$gate
gatelist$sd

This function use individualized treatment rule to identify exceptional responders. The details of the methods for this design are given in Imai and Li (2023).

Description

This function use individualized treatment rule to identify exceptional responders. The details of the methods for this design are given in Imai and Li (2023).

Usage

URATE(D, tau, Y)

Arguments

D

A vector of the unit-level binary treatment receipt variable for each sample.

tau

A vector of the unit-level continuous score. Conditional Average Treatment Effect is one possible measure.

Y

A vector of the outcome variable of interest for each sample.

Value

A list that contains the following items:

rate

The estimated vector of URATE of length Y.

sd

The estimated vector of standard deviation of URATE.

Author(s)

Michael Lingzhi Li, Technology and Operations Management, Harvard Business School mili@hbs.edu, https://www.michaellz.com/;

References

Imai and Li (2022). “Statistical Inference for Heterogeneous Treatment Effects Discovered by Generic Machine Learning in Randomized Experiments”,

Examples

D <- c(1, 0, 1, 0, 1, 0, 1, 0)
tau <- c(0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7)
Y <- c(4, 5, 0, 2, 4, 1, -4, 3)
ratelist <- URATE(D, tau, Y)
ratelist$rate
ratelist$sd

Compute Quantities of Interest (GATE, GATEcv, URATE)

Description

Compute Quantities of Interest (GATE, GATEcv, URATE)

Usage

compute_qoi(fit_obj, algorithms)

Arguments

fit_obj

An output object of estimate_itr function from the evalITR package.

algorithms

Machine learning algorithms

Compute Quantities of Interest (GATE, GATEcv, URATE) with user defined functions

Description

Compute Quantities of Interest (GATE, GATEcv, URATE) with user defined functions

Usage

compute_qoi_user(user_hte, Tcv, Ycv, data, ngates, ...)

Arguments

user_hte

A user-defined function to estimate heterogeneous treatment effects (HTE). The function should take the data as input and return an unit-level continuous score for treatment assignment. We assume those that have score less than 0 should not have treatment. The default is NULL, which means the heterogeneous treatment effects will be estimated from by the package.

Tcv

A vector of the unit-level binary treatment.

Ycv

A vector of the unit-level continuous outcome.

data

A data frame containing the variables of interest.

ngates

The number of gates to be used in the GATE function.

...

Additional arguments to be passed to the user-defined function.

The Consistency Test for Grouped Average Treatment Effects (GATEs) in Randomized Experiments

Description

Tests whether treatment effects are nondecreasing across score groups.

Usage

consist.test(D, tau, Y, ngates = 5, nsim = 10000, centered = TRUE)

Arguments

D

Binary treatment indicator (0 or 1).

tau

Continuous score vector.

Y

Outcome vector.

ngates

Number of groups (at least 2).

nsim

Number of Monte Carlo draws (at least 2).

centered

Whether to center outcomes before estimation.

Details

Uses consist.test for inference.

Value

A list with test statistic stat and p-value pval.

Author(s)

Michael Lingzhi Li, Technology and Operations Management, Harvard Business School mili@hbs.edu, https://www.michaellz.com/;

References

Imai and Li (2022). “Statistical Inference for Heterogeneous Treatment Effects Discovered by Generic Machine Learning in Randomized Experiments”,

Examples

D = c(1,0,1,0,1,0,1,0)
tau = c(0,0.1,0.2,0.3,0.4,0.5,0.6,0.7)
Y = c(4,5,0,2,4,1,-4,3)
consisttestlist <- consist.test(D,tau,Y,ngates=2)
consisttestlist$stat
consisttestlist$pval

The Consistency Test for Grouped Average Treatment Effects (GATEs) under Cross Validation in Randomized Experiments

Description

Tests whether treatment effects are nondecreasing across cross-validated score groups.

Usage

consistcv.test(D, tau, Y, ind, ngates = 5, nsim = 10000, centered = TRUE)

Arguments

D

Binary treatment indicator (0 or 1).

tau

A matrix of scores with one column per fold. Column i contains predictions for all observations from a model trained without fold i.

Y

Outcome vector.

ind

Integer validation-fold labels starting at 1.

ngates

Number of groups (at least 2).

nsim

Number of Monte Carlo draws (at least 2).

centered

Whether to center outcomes before estimation.

Details

Uses consistcv.test for inference.

Value

A list with test statistic stat and p-value pval.

Author(s)

Michael Lingzhi Li, Technology and Operations Management, Harvard Business School mili@hbs.edu, https://www.michaellz.com/;

References

Imai and Li (2022). “Statistical Inference for Heterogeneous Treatment Effects Discovered by Generic Machine Learning in Randomized Experiments”,

Examples

D = c(1,0,1,0,1,0,1,0)
tau = matrix(c(0,0.1,0.2,0.3,0.4,0.5,0.6,0.7,-0.5,-0.3,-0.1,0.1,0.3,0.5,0.7,0.9),nrow = 8, ncol = 2)
Y = c(4,5,0,2,4,1,-4,3)
ind = c(rep(1,4),rep(2,4))
consisttestlist <- consistcv.test(D,tau,Y,ind,ngates=2)
consisttestlist$stat
consisttestlist$pval

Evaluate Heterogeneous Treatment Effects

Description

Evaluate Heterogeneous Treatment Effects

Usage

estimate_hte(
  treatment,
  form,
  data,
  algorithms,
  n_folds = 5,
  split_ratio = 0,
  ngates = 5,
  preProcess = NULL,
  weights = NULL,
  trControl = caret::trainControl(method = "none"),
  tuneGrid = NULL,
  tuneLength = ifelse(trControl$method == "none", 1, 3),
  user_model = NULL,
  SL_library = NULL,
  meta_learner = "slearner",
  ...
)

Arguments

treatment

Treatment variable

form

a formula object that takes the form y ~ D + x1 + x2 + ....

data

A data frame that contains the outcome y and the treatment D.

algorithms

List of machine learning algorithms to be used.

n_folds

Number of cross-validation folds. Default is 5.

split_ratio

Split ratio between train and test set under sample splitting. Default is 0.

ngates

The number of groups to separate the data into. The groups are determined by tau. Default is 5.

preProcess

caret parameter

weights

caret parameter

trControl

caret parameter

tuneGrid

caret parameter

tuneLength

caret parameter

user_model

A user-defined function to estimate heterogeneous treatment effects.

SL_library

A list of machine learning algorithms to be used in the super learner.

meta_learner

The type of meta-learner to use (e.g., "slearner", "tlearner"). Default is "slearner".

...

Additional arguments passed to caret::train.

Value

An object of hte class

Evaluate Heterogeneous Treatment Effects

Description

Evaluate Heterogeneous Treatment Effects

Usage

evaluate_hte(fit, ...)

Arguments

fit

Fitted model. Usually an output from estimate_hte.

...

Additional arguments passed to the function.

Value

An object of hte class

The Heterogeneity Test for Grouped Average Treatment Effects (GATEs) in Randomized Experiments

Description

Tests whether treatment effects are equal across score groups.

Usage

het.test(D, tau, Y, ngates = 5, centered = TRUE)

Arguments

D

Binary treatment indicator (0 or 1).

tau

Continuous score vector.

Y

Outcome vector.

ngates

Number of groups (at least 2).

centered

Whether to center outcomes before estimation.

Details

Uses het.test for inference.

Value

A list with test statistic stat and p-value pval.

Author(s)

Michael Lingzhi Li, Technology and Operations Management, Harvard Business School mili@hbs.edu, https://www.michaellz.com/;

References

Imai and Li (2022). “Statistical Inference for Heterogeneous Treatment Effects Discovered by Generic Machine Learning in Randomized Experiments”,

Examples

D = c(1,0,1,0,1,0,1,0)
tau = c(0,0.1,0.2,0.3,0.4,0.5,0.6,0.7)
Y = c(4,5,0,2,4,1,-4,3)
hettestlist <- het.test(D,tau,Y,ngates=2)
hettestlist$stat
hettestlist$pval

The Heterogeneity Test for Grouped Average Treatment Effects (GATEs) under Cross Validation in Randomized Experiments

Description

Tests whether treatment effects are equal across cross-validated score groups.

Usage

hetcv.test(D, tau, Y, ind, ngates = 5, centered = TRUE)

Arguments

D

Binary treatment indicator (0 or 1).

tau

A matrix of scores with one column per fold. Column i contains predictions for all observations from a model trained without fold i.

Y

Outcome vector.

ind

Integer validation-fold labels starting at 1.

ngates

Number of groups (at least 2).

centered

Whether to center outcomes before estimation.

Details

Uses hetcv.test for inference.

Value

A list with test statistic stat and p-value pval.

Author(s)

Michael Lingzhi Li, Technology and Operations Management, Harvard Business School mili@hbs.edu, https://www.michaellz.com/;

References

Imai and Li (2022). “Statistical Inference for Heterogeneous Treatment Effects Discovered by Generic Machine Learning in Randomized Experiments”,

Examples

D = c(1,0,1,0,1,0,1,0)
tau = matrix(c(0,0.1,0.2,0.3,0.4,0.5,0.6,0.7,-0.5,-0.3,-0.1,0.1,0.3,0.5,0.7,0.9),nrow = 8, ncol = 2)
Y = c(4,5,0,2,4,1,-4,3)
ind = c(rep(1,4),rep(2,4))
hettestlist <- hetcv.test(D,tau,Y,ind,ngates=2)
hettestlist$stat
hettestlist$pval

Plot the GATE estimate

Description

Plot the GATE estimate

Usage

## S3 method for class 'hte'
plot(x, ...)

Arguments

x

An table object. This is typically an output of evaluate_hte() function.

...

Further arguments passed to the function.

Value

A plot of ggplot2 object.

Plot Confidence Intervals

Description

A generic function to plot uniform and pointwise confidence intervals for HTE objects.

Usage

plot_CI(x, ...)

Arguments

x

An object for which a plot is desired.

...

Further arguments passed to methods.

Value

A ggplot2 object displaying uniform and pointwise confidence intervals for heterogeneous treatment effects.

Plot the uniform confidence interval

Description

Plot the uniform confidence interval

Usage

## S3 method for class 'hte'
plot_CI(x, alpha = 0.05, ...)

Arguments

x

An object of evaluate_hte() class. This is typically an output of evaluate_hte() function.

alpha

Significance level. Default is 0.05.

...

Further arguments passed to the function.

Value

A plot of ggplot2 object.

Print

Description

Print

Usage

## S3 method for class 'summary.hte'
print(x, ...)

Arguments

x

An object of summary.hte class. This is typically an output of summary.hte() function.

...

Other parameters. Currently not supported.

Value

No return value, called for side effects (prints summary tables to console).

Print

Description

Print

Usage

## S3 method for class 'summary.test_hte'
print(x, ...)

Arguments

x

An object of summary.test_hte class. This is typically an output of summary.test_hte() function.

...

Other parameters.

Value

No return value, called for side effects (prints test results to console).

Summarize Heterogeneity and Consistency Tests

Description

Summarize Heterogeneity and Consistency Tests

Usage

## S3 method for class 'hte'
summary(object, ...)

Arguments

object

An object of evaluate_hte class (typically an output of evaluate_hte() function).

...

Other parameters.

Value

An object of class summary.hte, which is a list containing:

GATE

A tibble with group average treatment effect estimates, including columns: group, algorithm, estimate, std.deviation, lower, upper, z.score, and p.value.

URATE

A tibble with uplift rate estimates for exceptional responders, including columns: algorithm, estimate, std.deviation, conf.low.uniform, z.score, and p.value. Returns NULL when cross-validation is used.

Summarize Heterogeneity and Consistency Tests

Description

Summarize Heterogeneity and Consistency Tests

Usage

## S3 method for class 'test_hte'
summary(object, ...)

Arguments

object

An object of test_hte class (typically an output of test_hte() function).

...

Other parameters.

Value

An object of class summary.test_hte, which is a list containing:

Consistency

A tibble with consistency test results, including columns: algorithm, statistic, and p.value (for sample splitting).

Heterogeneity

A tibble with heterogeneity test results, including columns: algorithm, statistic, and p.value (for sample splitting).

Consistency_cv

A tibble with consistency test results for cross-validation.

Heterogeneity_cv

A tibble with heterogeneity test results for cross-validation.

Note: The output contains either the first two or last two elements depending on whether cross-validation was used.

Conduct hypothesis tests

Description

Conduct hypothesis tests

Usage

test_itr(model, nsim = 1000, centered = TRUE, ...)

Arguments

model

Fitted model. Usually an output from evaluate_hte.

nsim

Number of Monte Carlo simulations used to simulate the null distributions. Default is 1000.

centered

Whether to center outcomes before testing.

...

Further arguments passed to the function.

Value

An object of test_itr class