Title:	Inference on Predicted Data
Version:	0.4.0
Description:	Performs valid statistical inference on predicted data (IPD) using recent methods, where for a subset of the data, the outcomes have been predicted by an algorithm. Provides a wrapper function with specified defaults for the type of model and method to be used for estimation and inference. Further provides methods for tidying and summarizing results. Salerno et al., (2025) <doi:10.1093/bioinformatics/btaf055>.
License:	MIT + file LICENSE
URL:	https://github.com/ipd-tools/ipd, https://ipd-tools.github.io/ipd/
BugReports:	https://github.com/ipd-tools/ipd/issues
Depends:	R (≥ 4.4.0)
Imports:	methods, generics, BiocGenerics, caret, gam, ranger, splines, stats, MASS, randomForest, tibble
Suggests:	knitr, patchwork, rmarkdown, spelling, testthat (≥ 3.0.0), tidyverse, xfun (≥ 0.51), BiocStyle, BiocManager
VignetteBuilder:	knitr
biocViews:	Software, StatisticalMethod
Config/testthat/edition:	3
Encoding:	UTF-8
RoxygenNote:	7.3.3
Language:	en-US
NeedsCompilation:	no
Packaged:	2026-03-06 14:28:36 UTC; ssalerno
Author:	Stephen Salerno [aut, cre, cph], Jiacheng Miao [aut], Awan Afiaz [aut], Kentaro Hoffman [aut], Jesse Gronsbell [aut], Jianhui Gao [aut], David Cheng [aut], Anna Neufeld [aut], Qiongshi Lu [aut], Tyler H McCormick [aut], Jeffrey T Leek [aut]
Maintainer:	Stephen Salerno <ssalerno@fredhutch.org>
Repository:	CRAN
Date/Publication:	2026-03-06 14:50:07 UTC

ipd: Inference on Predicted Data

Description

Performs valid statistical inference on predicted data (IPD) using recent methods, where for a subset of the data, the outcomes have been predicted by an algorithm. Provides a wrapper function with specified defaults for the type of model and method to be used for estimation and inference. Further provides methods for tidying and summarizing results. Salerno et al., (2025) doi:10.1093/bioinformatics/btaf055.

The ipd package provides tools for statistical modeling and inference when a significant portion of the outcome data is predicted by AI/ML algorithms. It implements several state-of-the-art methods for inference on predicted data (IPD), offering a user-friendly interface to facilitate their use in real-world applications.

Details

This package is particularly useful in scenarios where predicted values (e.g., from machine learning models) are used as proxies for unobserved outcomes, which can introduce biases in estimation and inference. The ipd package integrates methods designed to address these challenges.

Features

• Multiple IPD methods: ⁠Chen and Chen⁠, PDC, PostPI, PPI, ⁠PPI++⁠, and PSPA currently.

• Flexible wrapper functions for ease of use.

• Custom methods for model inspection and evaluation.

• Seamless integration with common data structures in R.

• Comprehensive documentation and examples.

Key Functions

• ipd: Main wrapper function which implements various methods for inference on predicted data for a specified model/outcome type (e.g., mean estimation, linear regression).

• simdat: Simulates data for demonstrating the use of the various IPD methods.

• print.ipd: Prints a brief summary of the IPD method/model combination.

• summary.ipd: Summarizes the results of fitted IPD models.

• tidy.ipd: Tidies the IPD method/model fit into a data frame.

• glance.ipd: Glances at the IPD method/model fit, returning a one-row summary.

• augment.ipd: Augments the data used for an IPD method/model fit with additional information about each observation.

Documentation

The package includes detailed documentation for each function, including usage examples. A vignette is also provided to guide users through common workflows and applications of the package.

References

For details on the statistical methods implemented in this package, please refer to the vignette.

Author(s)

Maintainer: Stephen Salerno ssalerno@fredhutch.org (ORCID) [copyright holder]

Authors:

• Jiacheng Miao jmiao24@wisc.edu

• Awan Afiaz aafiaz@uw.edu

• Kentaro Hoffman khoffm3@uw.edu

• Jesse Gronsbell j.gronsbell@utoronto.ca

• Jianhui Gao jianhui.gao@mail.utoronto.ca

• David Cheng dcheng@mgh.harvard.edu

• Anna Neufeld acn2@williams.edu

• Qiongshi Lu qlu@biostat.wisc.edu

• Tyler H McCormick tylermc@uw.edu

• Jeffrey T Leek jtleek@fredhutch.org

See Also

Useful links:

• https://github.com/ipd-tools/ipd

• https://ipd-tools.github.io/ipd/

• Report bugs at https://github.com/ipd-tools/ipd/issues

Examples

#-- Generate Example Data

set.seed(12345)

dat <- simdat(n = c(300, 300, 300), effect = 1, sigma_Y = 1)

head(dat)

formula <- Y - f ~ X1

#-- PostPI Analytic Correction (Wang et al., 2020)

fit_postpi1 <- ipd(formula,
  method = "postpi_analytic", model = "ols",
  data = dat, label = "set_label"
)

#-- PostPI Bootstrap Correction (Wang et al., 2020)

nboot <- 200

fit_postpi2 <- ipd(formula,
  method = "postpi_boot", model = "ols",
  data = dat, label = "set_label", nboot = nboot
)

#-- PPI (Angelopoulos et al., 2023)

fit_ppi <- ipd(formula,
  method = "ppi", model = "ols",
  data = dat, label = "set_label"
)

#-- PPI++ (Angelopoulos et al., 2023)

fit_plusplus <- ipd(formula,
  method = "ppi_plusplus", model = "ols",
  data = dat, label = "set_label"
)

#-- PSPA (Miao et al., 2023)

fit_pspa <- ipd(formula,
  method = "pspa", model = "ols",
  data = dat, label = "set_label"
)

#-- Chen and Chen (Gronsbell et al., 2026)

fit_chen <- ipd(formula,
  method = "chen", model = "ols",
  data = dat, label = "set_label"
)

#-- Prediction Decorrelated Inference (Gan et al., 2024)

fit_chen <- ipd(formula,
  method = "pdc", model = "ols",
  data = dat, label = "set_label"
)

#-- Print the Model

print(fit_postpi1)

#-- Summarize the Model

summ_fit_postpi1 <- summary(fit_postpi1)

#-- Print the Model Summary

print(summ_fit_postpi1)

#-- Tidy the Model Output

tidy(fit_postpi1)

#-- Get a One-Row Summary of the Model

glance(fit_postpi1)

#-- Augment the Original Data with Fitted Values and Residuals

augmented_df <- augment(fit_postpi1)

head(augmented_df)

Build design matrices and outcome vectors

Description

Build design matrices and outcome vectors

Usage

.build_design(formula, data_l, data_u, intercept = TRUE, na_action = "na.fail")

Arguments

Value

list(X_l, Y_l, f_l, X_u, f_u)

Drop unused factor levels and report which were removed

Description

Drop unused factor levels and report which were removed

Usage

.drop_unused_levels(data, factor_vars)

Arguments

Value

data.frame with unused levels dropped

Validate and split input data

Description

Validate and split input data

Usage

.parse_inputs(data, label = NULL, unlabeled_data = NULL, na_action = "na.fail")

Arguments

Value

A list with components data_l and data_u (data.frames)

Warn on differing factor levels between labeled and unlabeled data

Description

Warn on differing factor levels between labeled and unlabeled data

Usage

.warn_differing_levels(data_l, data_u, factor_vars)

Arguments

Value

Invisibly returns NULL. Messages are printed for each variable whose levels differ.

Calculation of the matrix A based on single dataset

Description

A function for the calculation of the matrix A based on single dataset

Usage

A(
  X,
  Y,
  quant = NA,
  theta,
  method = c("ols", "quantile", "mean", "logistic", "poisson")
)

Arguments

Value

matrix A based on single dataset

Variance-covariance matrix of the estimation equation

Description

Sigma_cal function for variance-covariance matrix of the estimation equation

Usage

Sigma_cal(
  X_l,
  X_u,
  Y_l,
  f_l,
  f_u,
  w,
  theta,
  quant = NA,
  method = c("ols", "quantile", "mean", "logistic", "poisson")
)

Arguments

Value

variance-covariance matrix of the estimation equation

Augment data from an ipd fit

Description

Augment data from an ipd fit

Usage

## S3 method for class 'ipd'
augment(x, data = x@data_u, ...)

Arguments

Value

The data.frame with columns .fitted and .resid.

Examples

dat <- simdat()

fit <- ipd(Y - f ~ X1, method = "pspa", model = "ols",

    data = dat, label = "set_label")

augmented_df <- augment(fit)

head(augmented_df)

Estimate PPI++ Power Tuning Parameter

Description

Calculates the optimal value of lhat for the prediction-powered confidence interval for GLMs.

Usage

calc_lhat_glm(
  grads,
  grads_hat,
  grads_hat_unlabeled,
  inv_hessian,
  coord = NULL,
  clip = FALSE
)

Arguments

Value

(float): Optimal value of lhat in [0,1].

Chen & Chen Logistic

Description

Helper function for Chen & Chen logistic regression estimation.

Usage

chen_logistic(X_l, Y_l, f_l, X_u, f_u, intercept = TRUE)

Arguments

Details

Another look at statistical inference with machine learning-imputed data (Gronsbell et al., 2026) doi:10.48550/arXiv.2411.19908

Value

(list): A list containing the following:

est

(vector): vector of Chen & Chen logistic regression coefficient estimates.

se

(vector): vector of standard errors of the coefficients.

Examples

dat <- simdat(model = "logistic")

form <- Y - f ~ X1

X_l <- model.matrix(form, data = dat[dat$set_label == "labeled", ])

Y_l <- dat[dat$set_label == "labeled", all.vars(form)[1]] |>
  matrix(ncol = 1)

f_l <- dat[dat$set_label == "labeled", all.vars(form)[2]] |>
  matrix(ncol = 1)

X_u <- model.matrix(form, data = dat[dat$set_label == "unlabeled", ])

f_u <- dat[dat$set_label == "unlabeled", all.vars(form)[2]] |>
  matrix(ncol = 1)

chen_logistic(X_l, Y_l, f_l, X_u, f_u, intercept = TRUE)

Chen & Chen OLS

Description

Helper function for Chen & Chen OLS estimation

Usage

chen_ols(X_l, Y_l, f_l, X_u, f_u, intercept = TRUE)

Arguments

Details

Another look at statistical inference with machine learning-imputed data (Gronsbell et al., 2026) doi:10.48550/arXiv.2411.19908

Value

(list): A list containing the following:

est

(vector): vector of Chen & Chen OLS regression coefficient estimates.

se

(vector): vector of standard errors of the coefficients.

Examples

dat <- simdat(model = "ols")

form <- Y - f ~ X1

X_l <- model.matrix(form, data = dat[dat$set_label == "labeled", ])

Y_l <- dat[dat$set_label == "labeled", all.vars(form)[1]] |>
  matrix(ncol = 1)

f_l <- dat[dat$set_label == "labeled", all.vars(form)[2]] |>
  matrix(ncol = 1)

X_u <- model.matrix(form, data = dat[dat$set_label == "unlabeled", ])

f_u <- dat[dat$set_label == "unlabeled", all.vars(form)[2]] |>
  matrix(ncol = 1)

chen_ols(X_l, Y_l, f_l, X_u, f_u, intercept = TRUE)

Chen & Chen Poisson

Description

Helper function for Chen & Chen Poisson regression estimation.

Usage

chen_poisson(X_l, Y_l, f_l, X_u, f_u, intercept = TRUE)

Arguments

Details

Another look at statistical inference with machine learning-imputed data (Gronsbell et al., 2026) doi:10.48550/arXiv.2411.19908

Value

(list): A list containing the following:

est

(vector): vector of Chen & Chen Poisson regression coefficient estimates.

se

(vector): vector of standard errors of the coefficients.

Examples

dat <- simdat(model = "poisson")

form <- Y - f ~ X1

X_l <- model.matrix(form, data = dat[dat$set_label == "labeled", ])

Y_l <- dat[dat$set_label == "labeled", all.vars(form)[1]] |>
  matrix(ncol = 1)

f_l <- dat[dat$set_label == "labeled", all.vars(form)[2]] |>
  matrix(ncol = 1)

X_u <- model.matrix(form, data = dat[dat$set_label == "unlabeled", ])

f_u <- dat[dat$set_label == "unlabeled", all.vars(form)[2]] |>
  matrix(ncol = 1)

chen_poisson(X_l, Y_l, f_l, X_u, f_u, intercept = TRUE)

Empirical CDF of the Data

Description

Computes the empirical CDF of the data.

Usage

compute_cdf(Y, grid, w = NULL)

Arguments

Value

(list): Empirical CDF and its standard deviation at the specified grid points.

Empirical CDF Difference

Description

Computes the difference between the empirical CDFs of the data and the predictions.

Usage

compute_cdf_diff(Y, f, grid, w = NULL)

Arguments

Value

(list): Difference between the empirical CDFs of the data and the predictions and its standard deviation at the specified grid points.

Initial estimation

Description

est_ini function for initial estimation

Usage

est_ini(
  X,
  Y,
  quant = NA,
  method = c("ols", "quantile", "mean", "logistic", "poisson")
)

Arguments

Value

initial estimator

Glance at an ipd fit

Description

Glance at an ipd fit

Usage

## S3 method for class 'ipd'
glance(x, ...)

Arguments

Value

A one-row tibble summarizing the fit.

Examples

dat <- simdat()

fit <- ipd(Y - f ~ X1, method = "pspa", model = "ols",

    data = dat, label = "set_label")

glance(fit)

Inference on Predicted Data (ipd)

Description

The main wrapper function to conduct ipd using various methods and models, and returns a list of fitted model components.

Usage

ipd(
  formula,
  method,
  model,
  data,
  label = NULL,
  unlabeled_data = NULL,
  intercept = TRUE,
  alpha = 0.05,
  alternative = "two-sided",
  na_action = "na.fail",
  ...
)

Arguments

Details

1. Formula:

The ipd function uses one formula argument that specifies both the calibrating model (e.g., PostPI "relationship model", PPI "rectifier" model) and the inferential model. These separate models will be created internally based on the specific method called.

2. Data:

The data can be specified in two ways:

1. Single data argument (data) containing a stacked data.frame and a label identifier (label).

2. Two data arguments, one for the labeled data (data) and one for the unlabeled data (unlabeled_data).

For option (1), provide one data argument (data) which contains a stacked data.frame with both the unlabeled and labeled data and a label argument that specifies the column identifying the labeled versus the unlabeled observations in the stacked data.frame (e.g., label = "set_label" if the column "set_label" in the stacked data denotes which set an observation belongs to).

NOTE: Labeled data identifiers can be:

String

"l", "lab", "label", "labeled", "labelled", "tst", "test", "true"

Logical

TRUE

Factor

Non-reference category (i.e., binary 1)

Unlabeled data identifiers can be:

String

"u", "unlab", "unlabeled", "unlabelled", "val", "validation", "false"

Logical

FALSE

Factor

Non-reference category (i.e., binary 0)

For option (2), provide separate data arguments for the labeled data set (data) and the unlabeled data set (unlabeled_data). If the second argument is provided, the function ignores the label identifier and assumes the data provided are not stacked.

NOTE: Not all columns in data or unlabeled_data may be used unless explicitly referenced in the formula argument or in the label argument (if the data are passed as one stacked data frame).

3. Method:

Use the method argument to specify the fitting method:

"chen"

Gronsbell et al. (2026) Chen and Chen Correction

"pdc"

Gan et al. (2024) Prediction Decorrelated Inference

"postpi_analytic"

Wang et al. (2020) Post-Prediction Inference (PostPI) Analytic Correction

"postpi_boot"

Wang et al. (2020) Post-Prediction Inference (PostPI) Bootstrap Correction

"ppi"

Angelopoulos et al. (2023) Prediction-Powered Inference (PPI)

"ppi_a"

Gronsbell et al. (2025) PPI "All" Correction

"ppi_plusplus"

Angelopoulos et al. (2023) PPI++

"pspa"

Miao et al. (2023) Assumption-Lean and Data-Adaptive Post-Prediction Inference (PSPA)

4. Model:

Use the model argument to specify the type of downstream inferential model or parameter to be estimated:

"mean"

Mean value of a continuous outcome

"quantile"

qth quantile of a continuous outcome

"ols"

Linear regression coefficients for a continuous outcome

"logistic"

Logistic regression coefficients for a binary outcome

"poisson"

Poisson regression coefficients for a count outcome

The ipd wrapper function will concatenate the method and model arguments to identify the required helper function, following the naming convention "method_model".

5. Auxiliary Arguments:

The wrapper function will take method-specific auxiliary arguments (e.g., q for the quantile estimation models) and pass them to the helper function through the "..." with specified defaults for simplicity.

6. Other Arguments:

All other arguments that relate to all methods (e.g., alpha, ci.type), or other method-specific arguments, will have defaults.

Value

a summary of model output.

An S4 object of class IPD with the following slots:

coefficients

Named numeric vector of estimated parameters.

se

Named numeric vector of standard errors.

ci

A matrix of confidence intervals, with columns lower and upper.

coefTable

A data.frame summarizing Estimate, Std. Error, z-value, and Pr(>|z|) (glm-style).

fit

The raw output list returned by the method-specific helper function.

formula

The formula used for fitting the IPD model.

data_l

The labeled data.frame used in the analysis.

data_u

The unlabeled data.frame used in the analysis.

method

A character string indicating which IPD method was applied.

model

A character string indicating the downstream inferential model.

intercept

A logical indicating whether an intercept was included.

Examples

#-- Generate Example Data

dat <- simdat(n = c(300, 300, 300), effect = 1, sigma_Y = 1)

head(dat)

formula <- Y - f ~ X1

#-- Chen and Chen Correction (Gronsbell et al., 2026)

ipd(formula,
  method = "chen", model = "ols",
  data = dat, label = "set_label"
)

#-- Prediction Decorrelated Inference (Gan et al., 2024)

ipd(formula,
  method = "chen", model = "ols",
  data = dat, label = "set_label"
)

#-- PostPI Analytic Correction (Wang et al., 2020)

ipd(formula,
  method = "postpi_analytic", model = "ols",
  data = dat, label = "set_label"
)

#-- PostPI Bootstrap Correction (Wang et al., 2020)

nboot <- 200

ipd(formula,
  method = "postpi_boot", model = "ols",
  data = dat, label = "set_label", nboot = nboot
)

#-- PPI (Angelopoulos et al., 2023)

ipd(formula,
  method = "ppi", model = "ols",
  data = dat, label = "set_label"
)

#-- PPI "All" (Gronsbell et al., 2025)

ipd(formula,
  method = "ppi_a", model = "ols",
  data = dat, label = "set_label"
)

#-- PPI++ (Angelopoulos et al., 2023)

ipd(formula,
  method = "ppi_plusplus", model = "ols",
  data = dat, label = "set_label"
)

#-- PSPA (Miao et al., 2023)

ipd(formula,
  method = "pspa", model = "ols",
  data = dat, label = "set_label"
)

ipd: S4 class for inference on predicted data results

Description

ipd: S4 class for inference on predicted data results

Slots

coefficients

Numeric vector of parameter estimates.

se

Numeric vector of standard errors.

ci

Numeric matrix of confidence intervals.

coefTable

Data frame summarizing Estimate, Std. Error, z value, and Pr(>|z|).

fit

The raw list returned by the helper function.

formula

The formula used (class "formula").

data_l

The labeled data (data.frame).

data_u

The unlabeled data (data.frame).

method

Character; which IPD method was used.

model

Character; which downstream model was fitted.

intercept

Logical; was an intercept included?

Hessians of the link function

Description

link_Hessian function for Hessians of the link function

Usage

link_Hessian(t, method = c("logistic", "poisson"))

Arguments

Value

Hessians of the link function

Gradient of the link function

Description

link_grad function for gradient of the link function

Usage

link_grad(t, method = c("ols", "logistic", "poisson"))

Arguments

Value

gradient of the link function

Log1p Exponential

Description

Computes the natural logarithm of 1 plus the exponential of the input, to handle large inputs.

Usage

log1pexp(x)

Arguments

Value

(vector): A numeric vector where each element is the result of log(1 + exp(x)).

Logistic Regression Gradient and Hessian

Description

Computes the statistics needed for the logstic regression-based prediction-powered inference.

Usage

logistic_get_stats(
  est,
  X_l,
  Y_l,
  f_l,
  X_u,
  f_u,
  w_l = NULL,
  w_u = NULL,
  use_u = TRUE
)

Arguments

Value

(list): A list containing the following:

grads

(matrix): n x p matrix gradient of the loss function with respect to the coefficients.

grads_hat

(matrix): n x p matrix gradient of the loss function with respect to the coefficients, evaluated using the labeled predictions.

grads_hat_unlabeled

(matrix): N x p matrix gradient of the loss function with respect to the coefficients, evaluated using the unlabeled predictions.

inv_hessian

(matrix): p x p matrix inverse Hessian of the loss function with respect to the coefficients.

Sample expectation of psi

Description

mean_psi function for sample expectation of psi

Usage

mean_psi(
  X,
  Y,
  theta,
  quant = NA,
  method = c("ols", "quantile", "mean", "logistic", "poisson")
)

Arguments

Value

sample expectation of psi

Sample expectation of PSPA psi

Description

mean_psi_pop function for sample expectation of PSPA psi

Usage

mean_psi_pop(
  X_l,
  X_u,
  Y_l,
  f_l,
  f_u,
  w,
  theta,
  quant = NA,
  method = c("ols", "quantile", "mean", "logistic", "poisson")
)

Arguments

Value

sample expectation of PSPA psi

Ordinary Least Squares

Description

Computes the ordinary least squares coefficients.

Usage

ols(X, Y, return_se = FALSE)

Arguments

Value

(list): A list containing the following:

theta

(vector): p-vector of ordinary least squares estimates of the coefficients.

se

(vector): If return_se == TRUE, return the p-vector of standard errors of the coefficients.

OLS Gradient and Hessian

Description

Computes the statistics needed for the OLS-based prediction-powered inference.

Usage

ols_get_stats(
  est,
  X_l,
  Y_l,
  f_l,
  X_u,
  f_u,
  w_l = NULL,
  w_u = NULL,
  use_u = TRUE
)

Arguments

Value

(list): A list containing the following:

grads

(matrix): n x p matrix gradient of the loss function with respect to the coefficients.

grads_hat

(matrix): n x p matrix gradient of the loss function with respect to the coefficients, evaluated using the labeled predictions.

grads_hat_unlabeled

(matrix): N x p matrix gradient of the loss function with respect to the coefficients, evaluated using the unlabeled predictions.

inv_hessian

(matrix): p x p matrix inverse Hessian of the loss function with respect to the coefficients.

One-step update for obtaining estimator

Description

optim_est function for One-step update for obtaining estimator

Usage

optim_est(
  X_l,
  X_u,
  Y_l,
  f_l,
  f_u,
  w,
  theta,
  quant = NA,
  method = c("ols", "quantile", "mean", "logistic", "poisson")
)

Arguments

Value

estimator

One-step update for obtaining the weight vector

Description

optim_weights function for One-step update for obtaining estimator

Usage

optim_weights(
  X_l,
  X_u,
  Y_l,
  f_l,
  f_u,
  w,
  theta,
  quant = NA,
  method = c("ols", "quantile", "mean", "logistic", "poisson")
)

Arguments

Value

weights

PDC Logistic

Description

Helper function for PDC logistic regression estimation.

Usage

pdc_logistic(X_l, Y_l, f_l, X_u, f_u, intercept = TRUE)

Arguments

Details

Prediction de-correlated inference: A safe approach for post-prediction inference (Gan et al., 2024) doi:10.1111/anzs.12429

Value

(list): A list containing the following:

est

(vector): vector of PDC logistic regression coefficient estimates.

se

(vector): vector of standard errors of the coefficients.

Examples

dat <- simdat(model = "logistic")

form <- Y - f ~ X1

X_l <- model.matrix(form, data = dat[dat$set_label == "labeled", ])

Y_l <- dat[dat$set_label == "labeled", all.vars(form)[1]] |>
  matrix(ncol = 1)

f_l <- dat[dat$set_label == "labeled", all.vars(form)[2]] |>
  matrix(ncol = 1)

X_u <- model.matrix(form, data = dat[dat$set_label == "unlabeled", ])

f_u <- dat[dat$set_label == "unlabeled", all.vars(form)[2]] |>
  matrix(ncol = 1)

pdc_logistic(X_l, Y_l, f_l, X_u, f_u, intercept = TRUE)

PDC OLS

Description

Helper function for PDC OLS estimation.

Usage

pdc_ols(X_l, Y_l, f_l, X_u, f_u, intercept = TRUE)

Arguments

Details

Prediction de-correlated inference: A safe approach for post-prediction inference (Gan et al., 2024) doi:10.1111/anzs.12429

Value

(list): A list containing the following:

est

(vector): vector of PDC OLS regression coefficient estimates.

se

(vector): vector of standard errors of the coefficients.

Examples

dat <- simdat(model = "ols")

form <- Y - f ~ X1

X_l <- model.matrix(form, data = dat[dat$set_label == "labeled", ])

Y_l <- dat[dat$set_label == "labeled", all.vars(form)[1]] |>
  matrix(ncol = 1)

f_l <- dat[dat$set_label == "labeled", all.vars(form)[2]] |>
  matrix(ncol = 1)

X_u <- model.matrix(form, data = dat[dat$set_label == "unlabeled", ])

f_u <- dat[dat$set_label == "unlabeled", all.vars(form)[2]] |>
  matrix(ncol = 1)

pdc_ols(X_l, Y_l, f_l, X_u, f_u, intercept = TRUE)

PDC Poisson

Description

Helper function for PDC Poisson regression estimation.

Usage

pdc_poisson(X_l, Y_l, f_l, X_u, f_u, intercept = TRUE)

Arguments

Details

Prediction de-correlated inference: A safe approach for post-prediction inference (Gan et al., 2024) doi:10.1111/anzs.12429

Value

(list): A list containing the following:

est

(vector): vector of PDC Poisson regression coefficient estimates.

se

(vector): vector of standard errors of the coefficients.

Examples

dat <- simdat(model = "poisson")

form <- Y - f ~ X1

X_l <- model.matrix(form, data = dat[dat$set_label == "labeled", ])

Y_l <- dat[dat$set_label == "labeled", all.vars(form)[1]] |>
  matrix(ncol = 1)

f_l <- dat[dat$set_label == "labeled", all.vars(form)[2]] |>
  matrix(ncol = 1)

X_u <- model.matrix(form, data = dat[dat$set_label == "unlabeled", ])

f_u <- dat[dat$set_label == "unlabeled", all.vars(form)[2]] |>
  matrix(ncol = 1)

pdc_poisson(X_l, Y_l, f_l, X_u, f_u, intercept = TRUE)

PostPI OLS (Analytic Correction)

Description

Helper function for PostPI OLS estimation (analytic correction)

Usage

postpi_analytic_ols(X_l, Y_l, f_l, X_u, f_u, original = FALSE)

Arguments

Details

Methods for correcting inference based on outcomes predicted by machine learning (Wang et al., 2020) doi:10.1073/pnas.2001238117

Value

A list of outputs: estimate of the inference model parameters and corresponding standard error estimate.

Examples

dat <- simdat(model = "ols")

form <- Y - f ~ X1

X_l <- model.matrix(form, data = dat[dat$set_label == "labeled", ])

Y_l <- dat[dat$set_label == "labeled", all.vars(form)[1]] |>
  matrix(ncol = 1)

f_l <- dat[dat$set_label == "labeled", all.vars(form)[2]] |>
  matrix(ncol = 1)

X_u <- model.matrix(form, data = dat[dat$set_label == "unlabeled", ])

f_u <- dat[dat$set_label == "unlabeled", all.vars(form)[2]] |>
  matrix(ncol = 1)

postpi_analytic_ols(X_l, Y_l, f_l, X_u, f_u)

PostPI Logistic Regression (Bootstrap Correction)

Description

Helper function for PostPI logistic regression (bootstrap correction)

Usage

postpi_boot_logistic(X_l, Y_l, f_l, X_u, f_u, nboot = 100, se_type = "par")

Arguments

Details

Methods for correcting inference based on outcomes predicted by machine learning (Wang et al., 2020) doi:10.1073/pnas.2001238117

Value

A list of outputs: estimate of inference model parameters and corresponding standard error based on both parametric and non-parametric bootstrap methods.

Examples

dat <- simdat(model = "logistic")

form <- Y - f ~ X1

X_l <- model.matrix(form, data = dat[dat$set_label == "labeled", ])

Y_l <- dat[dat$set_label == "labeled", all.vars(form)[1]] |>
  matrix(ncol = 1)

f_l <- dat[dat$set_label == "labeled", all.vars(form)[2]] |>
  matrix(ncol = 1)

X_u <- model.matrix(form, data = dat[dat$set_label == "unlabeled", ])

f_u <- dat[dat$set_label == "unlabeled", all.vars(form)[2]] |>
  matrix(ncol = 1)

postpi_boot_logistic(X_l, Y_l, f_l, X_u, f_u, nboot = 200)

PostPI OLS (Bootstrap Correction)

Description

Helper function for PostPI OLS estimation (bootstrap correction)

Usage

postpi_boot_ols(
  X_l,
  Y_l,
  f_l,
  X_u,
  f_u,
  nboot = 100,
  se_type = "par",
  rel_func = "lm",
  scale_se = TRUE,
  n_t = Inf
)

Arguments

Details

Methods for correcting inference based on outcomes predicted by machine learning (Wang et al., 2020) doi:10.1073/pnas.2001238117

Value

A list of outputs: estimate of inference model parameters and corresponding standard error based on both parametric and non-parametric bootstrap methods.

Examples

dat <- simdat(model = "ols")

form <- Y - f ~ X1

X_l <- model.matrix(form, data = dat[dat$set_label == "labeled", ])

Y_l <- dat[dat$set_label == "labeled", all.vars(form)[1]] |>

  matrix(ncol = 1)

f_l <- dat[dat$set_label == "labeled", all.vars(form)[2]] |>

  matrix(ncol = 1)

X_u <- model.matrix(form, data = dat[dat$set_label == "unlabeled", ])

f_u <- dat[dat$set_label == "unlabeled", all.vars(form)[2]] |>

  matrix(ncol = 1)

postpi_boot_ols(X_l, Y_l, f_l, X_u, f_u, nboot = 200)

PPI "All" OLS

Description

Helper function for PPI "All" for OLS estimation

Usage

ppi_a_ols(X_l, Y_l, f_l, X_u, f_u, w_l = NULL, w_u = NULL)

Arguments

Details

Another look at statistical inference with machine learning-imputed data (Gronsbell et al., 2026) doi:10.48550/arXiv.2411.19908

Value

(list): A list containing the following:

est

(vector): vector of PPI OLS regression coefficient estimates.

se

(vector): vector of standard errors of the coefficients.

rectifier_est

(vector): vector of the rectifier OLS regression coefficient estimates.

Examples

dat <- simdat()

form <- Y - f ~ X1

X_l <- model.matrix(form, data = dat[dat$set_label == "labeled", ])

Y_l <- dat[dat$set_label == "labeled", all.vars(form)[1]] |>

  matrix(ncol = 1)

f_l <- dat[dat$set_label == "labeled", all.vars(form)[2]] |>

  matrix(ncol = 1)

X_u <- model.matrix(form, data = dat[dat$set_label == "unlabeled", ])

f_u <- dat[dat$set_label == "unlabeled", all.vars(form)[2]] |>

  matrix(ncol = 1)

ppi_a_ols(X_l, Y_l, f_l, X_u, f_u)

PPI Logistic Regression

Description

Helper function for PPI logistic regression

Usage

ppi_logistic(X_l, Y_l, f_l, X_u, f_u, opts = NULL)

Arguments

Details

Prediction Powered Inference (Angelopoulos et al., 2023) doi:10.1126/science.adi6000

Value

(list): A list containing the following:

est

(vector): vector of PPI logistic regression coefficient estimates.

se

(vector): vector of standard errors of the coefficients.

rectifier_est

(vector): vector of the rectifier logistic regression coefficient estimates.

var_u

(matrix): covariance matrix for the gradients in the unlabeled data.

var_l

(matrix): covariance matrix for the gradients in the labeled data.

grads

(matrix): matrix of gradients for the labeled data.

grads_hat_unlabeled

(matrix): matrix of predicted gradients for the unlabeled data.

grads_hat

(matrix): matrix of predicted gradients for the labeled data.

inv_hessian

(matrix): inverse Hessian matrix.

Examples

dat <- simdat(model = "logistic")

form <- Y - f ~ X1

X_l <- model.matrix(form, data = dat[dat$set_label == "labeled", ])

Y_l <- dat[dat$set_label == "labeled", all.vars(form)[1]] |>

  matrix(ncol = 1)

f_l <- dat[dat$set_label == "labeled", all.vars(form)[2]] |>

  matrix(ncol = 1)

X_u <- model.matrix(form, data = dat[dat$set_label == "unlabeled", ])

f_u <- dat[dat$set_label == "unlabeled", all.vars(form)[2]] |>

  matrix(ncol = 1)

ppi_logistic(X_l, Y_l, f_l, X_u, f_u)

PPI Mean Estimation

Description

Helper function for PPI mean estimation

Usage

ppi_mean(Y_l, f_l, f_u, alpha = 0.05, alternative = "two-sided")

Arguments

Details

Prediction Powered Inference (Angelopoulos et al., 2023) doi:10.1126/science.adi6000

Value

tuple: Lower and upper bounds of the prediction-powered confidence interval for the mean.

Examples

dat <- simdat(model = "mean")

form <- Y - f ~ 1

Y_l <- dat[dat$set_label == "labeled", all.vars(form)[1]] |>
  matrix(ncol = 1)

f_l <- dat[dat$set_label == "labeled", all.vars(form)[2]] |>
  matrix(ncol = 1)

f_u <- dat[dat$set_label == "unlabeled", all.vars(form)[2]] |>
  matrix(ncol = 1)

ppi_mean(Y_l, f_l, f_u)

PPI OLS

Description

Helper function for prediction-powered inference for OLS estimation

Usage

ppi_ols(X_l, Y_l, f_l, X_u, f_u, w_l = NULL, w_u = NULL)

Arguments

Details

Prediction Powered Inference (Angelopoulos et al., 2023) doi:10.1126/science.adi6000

Value

(list): A list containing the following:

est

(vector): vector of PPI OLS regression coefficient estimates.

se

(vector): vector of standard errors of the coefficients.

rectifier_est

(vector): vector of the rectifier OLS regression coefficient estimates.

Examples

dat <- simdat()

form <- Y - f ~ X1

X_l <- model.matrix(form, data = dat[dat$set_label == "labeled", ])

Y_l <- dat[dat$set_label == "labeled", all.vars(form)[1]] |>

  matrix(ncol = 1)

f_l <- dat[dat$set_label == "labeled", all.vars(form)[2]] |>

  matrix(ncol = 1)

X_u <- model.matrix(form, data = dat[dat$set_label == "unlabeled", ])

f_u <- dat[dat$set_label == "unlabeled", all.vars(form)[2]] |>

  matrix(ncol = 1)

ppi_ols(X_l, Y_l, f_l, X_u, f_u)

PPI++ Logistic Regression

Description

Helper function for PPI++ logistic regression

Usage

ppi_plusplus_logistic(
  X_l,
  Y_l,
  f_l,
  X_u,
  f_u,
  lhat = NULL,
  coord = NULL,
  opts = NULL,
  w_l = NULL,
  w_u = NULL
)

Arguments

Details

PPI++: Efficient Prediction Powered Inference (Angelopoulos et al., 2023) doi:10.48550/arXiv.2311.01453

Value

(list): A list containing the following:

est

(vector): vector of PPI++ logistic regression coefficient estimates.

se

(vector): vector of standard errors of the coefficients.

lambda

(float): estimated power-tuning parameter.

rectifier_est

(vector): vector of the rectifier logistic regression coefficient estimates.

var_u

(matrix): covariance matrix for the gradients in the unlabeled data.

var_l

(matrix): covariance matrix for the gradients in the labeled data.

grads

(matrix): matrix of gradients for the labeled data.

grads_hat_unlabeled

(matrix): matrix of predicted gradients for the unlabeled data.

grads_hat

(matrix): matrix of predicted gradients for the labeled data.

inv_hessian

(matrix): inverse Hessian matrix.

Examples

dat <- simdat(model = "logistic")

form <- Y - f ~ X1

X_l <- model.matrix(form, data = dat[dat$set_label == "labeled", ])

Y_l <- dat[dat$set_label == "labeled", all.vars(form)[1]] |>

  matrix(ncol = 1)

f_l <- dat[dat$set_label == "labeled", all.vars(form)[2]] |>

  matrix(ncol = 1)

X_u <- model.matrix(form, data = dat[dat$set_label == "unlabeled", ])

f_u <- dat[dat$set_label == "unlabeled", all.vars(form)[2]] |>

  matrix(ncol = 1)

ppi_plusplus_logistic(X_l, Y_l, f_l, X_u, f_u)

PPI++ Logistic Regression (Point Estimate)

Description

Helper function for PPI++ logistic regression (point estimate)

Usage

ppi_plusplus_logistic_est(
  X_l,
  Y_l,
  f_l,
  X_u,
  f_u,
  lhat = NULL,
  coord = NULL,
  opts = NULL,
  w_l = NULL,
  w_u = NULL
)

Arguments

Details

PPI++: Efficient Prediction Powered Inference (Angelopoulos et al., 2023) doi:10.48550/arXiv.2311.01453

Value

(vector): vector of prediction-powered point estimates of the logistic regression coefficients.

Examples

dat <- simdat(model = "logistic")

form <- Y - f ~ X1

X_l <- model.matrix(form, data = dat[dat$set_label == "labeled", ])

Y_l <- dat[dat$set_label == "labeled", all.vars(form)[1]] |>

  matrix(ncol = 1)

f_l <- dat[dat$set_label == "labeled", all.vars(form)[2]] |>

  matrix(ncol = 1)

X_u <- model.matrix(form, data = dat[dat$set_label == "unlabeled", ])

f_u <- dat[dat$set_label == "unlabeled", all.vars(form)[2]] |>

  matrix(ncol = 1)

ppi_plusplus_logistic_est(X_l, Y_l, f_l, X_u, f_u)

PPI++ Mean Estimation

Description

Helper function for PPI++ mean estimation

Usage

ppi_plusplus_mean(
  Y_l,
  f_l,
  f_u,
  alpha = 0.05,
  alternative = "two-sided",
  lhat = NULL,
  coord = NULL,
  w_l = NULL,
  w_u = NULL
)

Arguments

Details

PPI++: Efficient Prediction Powered Inference (Angelopoulos et al., 2023) doi:10.48550/arXiv.2311.01453

Value

tuple: Lower and upper bounds of the prediction-powered confidence interval for the mean.

Examples

dat <- simdat(model = "mean")

form <- Y - f ~ 1

Y_l <- dat[dat$set_label == "labeled", all.vars(form)[1]] |>

  matrix(ncol = 1)

f_l <- dat[dat$set_label == "labeled", all.vars(form)[2]] |>

  matrix(ncol = 1)

f_u <- dat[dat$set_label == "unlabeled", all.vars(form)[2]] |>

  matrix(ncol = 1)

ppi_plusplus_mean(Y_l, f_l, f_u)

PPI++ Mean Estimation (Point Estimate)

Description

Helper function for PPI++ mean estimation (point estimate)

Usage

ppi_plusplus_mean_est(
  Y_l,
  f_l,
  f_u,
  lhat = NULL,
  coord = NULL,
  w_l = NULL,
  w_u = NULL
)

Arguments

Details

PPI++: Efficient Prediction Powered Inference (Angelopoulos et al., 2023) doi:10.48550/arXiv.2311.01453

Value

float or ndarray: Prediction-powered point estimate of the mean.

Examples

dat <- simdat(model = "mean")

form <- Y - f ~ 1

Y_l <- dat[dat$set_label == "labeled", all.vars(form)[1]] |>

  matrix(ncol = 1)

f_l <- dat[dat$set_label == "labeled", all.vars(form)[2]] |>

  matrix(ncol = 1)

f_u <- dat[dat$set_label == "unlabeled", all.vars(form)[2]] |>

  matrix(ncol = 1)

ppi_plusplus_mean_est(Y_l, f_l, f_u)

PPI++ OLS

Description

Helper function for PPI++ OLS estimation

Usage

ppi_plusplus_ols(
  X_l,
  Y_l,
  f_l,
  X_u,
  f_u,
  lhat = NULL,
  coord = NULL,
  w_l = NULL,
  w_u = NULL
)

Arguments

Details

PPI++: Efficient Prediction Powered Inference (Angelopoulos et al., 2023) doi:10.48550/arXiv.2311.01453

Value

(list): A list containing the following:

est

(vector): vector of PPI++ OLS regression coefficient estimates.

se

(vector): vector of standard errors of the coefficients.

lambda

(float): estimated power-tuning parameter.

rectifier_est

(vector): vector of the rectifier OLS regression coefficient estimates.

var_u

(matrix): covariance matrix for the gradients in the unlabeled data.

var_l

(matrix): covariance matrix for the gradients in the labeled data.

grads

(matrix): matrix of gradients for the labeled data.

grads_hat_unlabeled

(matrix): matrix of predicted gradients for the unlabeled data.

grads_hat

(matrix): matrix of predicted gradients for the labeled data.

inv_hessian

(matrix): inverse Hessian matrix.

Examples

dat <- simdat(model = "ols")

form <- Y - f ~ X1

X_l <- model.matrix(form, data = dat[dat$set_label == "labeled", ])

Y_l <- dat[dat$set_label == "labeled", all.vars(form)[1]] |>

  matrix(ncol = 1)

f_l <- dat[dat$set_label == "labeled", all.vars(form)[2]] |>

  matrix(ncol = 1)

X_u <- model.matrix(form, data = dat[dat$set_label == "unlabeled", ])

f_u <- dat[dat$set_label == "unlabeled", all.vars(form)[2]] |>

  matrix(ncol = 1)

ppi_plusplus_ols(X_l, Y_l, f_l, X_u, f_u)

PPI++ OLS (Point Estimate)

Description

Helper function for PPI++ OLS estimation (point estimate)

Usage

ppi_plusplus_ols_est(
  X_l,
  Y_l,
  f_l,
  X_u,
  f_u,
  lhat = NULL,
  coord = NULL,
  w_l = NULL,
  w_u = NULL
)

Arguments

Details

PPI++: Efficient Prediction Powered Inference (Angelopoulos et al., 2023) doi:10.48550/arXiv.2311.01453

Value

(vector): vector of prediction-powered point estimates of the OLS coefficients.

Examples

dat <- simdat(model = "ols")

form <- Y - f ~ X1

X_l <- model.matrix(form, data = dat[dat$set_label == "labeled", ])

Y_l <- dat[dat$set_label == "labeled", all.vars(form)[1]] |>

  matrix(ncol = 1)

f_l <- dat[dat$set_label == "labeled", all.vars(form)[2]] |>

  matrix(ncol = 1)

X_u <- model.matrix(form, data = dat[dat$set_label == "unlabeled", ])

f_u <- dat[dat$set_label == "unlabeled", all.vars(form)[2]] |>

  matrix(ncol = 1)

ppi_plusplus_ols_est(X_l, Y_l, f_l, X_u, f_u)

PPI++ Quantile Estimation

Description

Helper function for PPI++ quantile estimation

Usage

ppi_plusplus_quantile(
  Y_l,
  f_l,
  f_u,
  q,
  alpha = 0.05,
  exact_grid = FALSE,
  w_l = NULL,
  w_u = NULL
)

Arguments

Details

PPI++: Efficient Prediction Powered Inference (Angelopoulos et al., 2023) doi:10.48550/arXiv.2311.01453

Value

tuple: Lower and upper bounds of the prediction-powered confidence interval for the quantile.

Examples

dat <- simdat(model = "quantile")

form <- Y - f ~ X1

Y_l <- dat[dat$set_label == "labeled", all.vars(form)[1]] |>

  matrix(ncol = 1)

f_l <- dat[dat$set_label == "labeled", all.vars(form)[2]] |>

  matrix(ncol = 1)

f_u <- dat[dat$set_label == "unlabeled", all.vars(form)[2]] |>

  matrix(ncol = 1)

ppi_plusplus_quantile(Y_l, f_l, f_u, q = 0.5)

PPI++ Quantile Estimation (Point Estimate)

Description

Helper function for PPI++ quantile estimation (point estimate)

Usage

ppi_plusplus_quantile_est(
  Y_l,
  f_l,
  f_u,
  q,
  exact_grid = FALSE,
  w_l = NULL,
  w_u = NULL
)

Arguments

Details

PPI++: Efficient Prediction Powered Inference (Angelopoulos et al., 2023) doi:10.48550/arXiv.2311.01453

Value

(float): Prediction-powered point estimate of the quantile.

Examples

dat <- simdat(model = "quantile")

form <- Y - f ~ 1

Y_l <- dat[dat$set_label == "labeled", all.vars(form)[1]] |>

  matrix(ncol = 1)

f_l <- dat[dat$set_label == "labeled", all.vars(form)[2]] |>

  matrix(ncol = 1)

f_u <- dat[dat$set_label == "unlabeled", all.vars(form)[2]] |>

  matrix(ncol = 1)

ppi_plusplus_quantile_est(Y_l, f_l, f_u, q = 0.5)

PPI Quantile Estimation

Description

Helper function for PPI quantile estimation

Usage

ppi_quantile(Y_l, f_l, f_u, q, alpha = 0.05, exact_grid = FALSE)

Arguments

Details

Prediction Powered Inference (Angelopoulos et al., 2023) doi:10.1126/science.adi6000

Value

tuple: Lower and upper bounds of the prediction-powered confidence interval for the quantile.

Examples

dat <- simdat(model = "quantile")

form <- Y - f ~ X1

Y_l <- dat[dat$set_label == "labeled", all.vars(form)[1]] |>

  matrix(ncol = 1)

f_l <- dat[dat$set_label == "labeled", all.vars(form)[2]] |>

  matrix(ncol = 1)

f_u <- dat[dat$set_label == "unlabeled", all.vars(form)[2]] |>

  matrix(ncol = 1)

ppi_quantile(Y_l, f_l, f_u, q = 0.5)

Print ipd fit

Description

Print ipd fit

Usage

## S3 method for class 'ipd'
print(x, ...)

Arguments

Value

Invisibly returns x.

Print summary.ipd

Description

Print summary.ipd

Usage

## S3 method for class 'summary.ipd'
print(x, ...)

Arguments

Value

Invisibly returns x.

Estimating equation

Description

psi function for estimating equation

Usage

psi(
  X,
  Y,
  theta,
  quant = NA,
  method = c("ols", "quantile", "mean", "logistic", "poisson")
)

Arguments

Value

estimating equation

PSPA Logistic Regression

Description

Helper function for PSPA logistic regression

Usage

pspa_logistic(X_l, Y_l, f_l, X_u, f_u, weights = NA, alpha = 0.05)

Arguments

Details

Post-prediction adaptive inference (Miao et al. 2023) doi:10.48550/arXiv.2311.14220

Value

A list of outputs: estimate of inference model parameters and corresponding standard error.

Examples

dat <- simdat(model = "logistic")

form <- Y - f ~ X1

X_l <- model.matrix(form, data = dat[dat$set_label == "labeled", ])

Y_l <- dat[dat$set_label == "labeled", all.vars(form)[1]] |>
  matrix(ncol = 1)

f_l <- dat[dat$set_label == "labeled", all.vars(form)[2]] |>
  matrix(ncol = 1)

X_u <- model.matrix(form, data = dat[dat$set_label == "unlabeled", ])

f_u <- dat[dat$set_label == "unlabeled", all.vars(form)[2]] |>
  matrix(ncol = 1)

pspa_logistic(X_l, Y_l, f_l, X_u, f_u)

PSPA Mean Estimation

Description

Helper function for PSPA mean estimation

Usage

pspa_mean(Y_l, f_l, f_u, weights = NA, alpha = 0.05)

Arguments

Details

Post-prediction adaptive inference (Miao et al., 2023) doi:10.48550/arXiv.2311.14220

Value

A list of outputs: estimate of inference model parameters and corresponding standard error.

Examples

dat <- simdat(model = "mean")

form <- Y - f ~ 1

Y_l <- dat[dat$set_label == "labeled", all.vars(form)[1]] |>
  matrix(ncol = 1)

f_l <- dat[dat$set_label == "labeled", all.vars(form)[2]] |>
  matrix(ncol = 1)

f_u <- dat[dat$set_label == "unlabeled", all.vars(form)[2]] |>
  matrix(ncol = 1)

pspa_mean(Y_l = Y_l, f_l = f_l, f_u = f_u)

PSPA OLS Estimation

Description

Helper function for PSPA OLS for linear regression

Usage

pspa_ols(X_l, Y_l, f_l, X_u, f_u, weights = NA, alpha = 0.05)

Arguments

Details

Post-prediction adaptive inference (Miao et al., 2023) doi:10.48550/arXiv.2311.14220

Value

A list of outputs: estimate of inference model parameters and corresponding standard error.

Examples

dat <- simdat(model = "ols")

form <- Y - f ~ X1

X_l <- model.matrix(form, data = dat[dat$set_label == "labeled", ])

Y_l <- dat[dat$set_label == "labeled", all.vars(form)[1]] |>
  matrix(ncol = 1)

f_l <- dat[dat$set_label == "labeled", all.vars(form)[2]] |>
  matrix(ncol = 1)

X_u <- model.matrix(form, data = dat[dat$set_label == "unlabeled", ])

f_u <- dat[dat$set_label == "unlabeled", all.vars(form)[2]] |>
  matrix(ncol = 1)

pspa_ols(X_l, Y_l, f_l, X_u, f_u)

PSPA Poisson Regression

Description

Helper function for PSPA Poisson regression

Usage

pspa_poisson(X_l, Y_l, f_l, X_u, f_u, weights = NA, alpha = 0.05)

Arguments

Details

Post-prediction adaptive inference (Miao et al., 2023) doi:10.48550/arXiv.2311.14220

Value

A list of outputs: estimate of inference model parameters and corresponding standard error.

Examples

dat <- simdat(model = "poisson")

form <- Y - f ~ X1

X_l <- model.matrix(form, data = dat[dat$set_label == "labeled", ])

Y_l <- dat[dat$set_label == "labeled", all.vars(form)[1]] |>
  matrix(ncol = 1)

f_l <- dat[dat$set_label == "labeled", all.vars(form)[2]] |>
  matrix(ncol = 1)

X_u <- model.matrix(form, data = dat[dat$set_label == "unlabeled", ])

f_u <- dat[dat$set_label == "unlabeled", all.vars(form)[2]] |>
  matrix(ncol = 1)

pspa_poisson(X_l, Y_l, f_l, X_u, f_u)

PSPA Quantile Estimation

Description

Helper function for PSPA quantile estimation

Usage

pspa_quantile(Y_l, f_l, f_u, q, weights = NA, alpha = 0.05)

Arguments

Details

Post-prediction adaptive inference (Miao et al., 2023) doi:10.48550/arXiv.2311.14220

Value

A list of outputs: estimate of inference model parameters and corresponding standard error.

Examples

dat <- simdat(model = "quantile")

form <- Y - f ~ 1

Y_l <- dat[dat$set_label == "labeled", all.vars(form)[1]] |> matrix(ncol = 1)

f_l <- dat[dat$set_label == "labeled", all.vars(form)[2]] |> matrix(ncol = 1)

f_u <- dat[dat$set_label == "unlabeled", all.vars(form)[2]] |>
  matrix(ncol = 1)

pspa_quantile(Y_l = Y_l, f_l = f_l, f_u = f_u, q = 0.5)

PSPA M-Estimation for ML-predicted labels

Description

pspa_y function conducts post-prediction M-Estimation.

Usage

pspa_y(
  X_l = NA,
  X_u = NA,
  Y_l,
  f_l,
  f_u,
  alpha = 0.05,
  weights = NA,
  quant = NA,
  intercept = FALSE,
  method
)

Arguments

Value

A summary table presenting point estimates, standard error, confidence intervals (1 - alpha), P-values, and weights.

Rectified CDF

Description

Computes the rectified CDF of the data.

Usage

rectified_cdf(Y_l, f_l, f_u, grid, w_l = NULL, w_u = NULL)

Arguments

Value

(vector): Rectified CDF of the data at the specified grid points.

Rectified P-Value

Description

Computes a rectified p-value.

Usage

rectified_p_value(
  rectifier,
  rectifier_std,
  imputed_mean,
  imputed_std,
  null = 0,
  alternative = "two-sided"
)

Arguments

Value

(float or vector): The rectified p-value.

tidy re-exported from generics packages

Description

These objects are imported from other packages. Follow the links below to see their documentation.

generics

augment, glance, tidy

Value

A wrapper for the tidy generic. See tidy for details.

A wrapper for the glance generic. See glance for details.

A wrapper for the augment generic. See augment for details.

See Also

tidy

glance

augment

Examples

dat <- simdat()

fit <- ipd(Y - f ~ X1, method = "pspa", model = "ols",

    data = dat, label = "set_label")

tidy(fit)


dat <- simdat()

fit <- ipd(Y - f ~ X1, method = "pspa", model = "ols",

    data = dat, label = "set_label")

glance(fit)


dat <- simdat()

fit <- ipd(Y - f ~ X1, method = "pspa", model = "ols",

    data = dat, label = "set_label")

augmented_df <- augment(fit)

head(augmented_df)

Show an ipd object

Description

Display a concise summary of an ipd S4 object, including method, model, formula, and a glm-style coefficient table.

Usage

## S4 method for signature 'ipd'
show(object)

Arguments

Value

Invisibly returns object after printing.

Simulate the data for testing the functions

Description

sim_data_y for simulation with ML-predicted Y

Usage

sim_data_y(r = 0.9, binary = FALSE)

Arguments

Value

simulated data

Data generation function for various underlying models

Description

Data generation function for various underlying models

Usage

simdat(
  n = c(300, 300, 300),
  effect = 1,
  sigma_Y = 1,
  model = "ols",
  shift = 0,
  scale = 1
)

Arguments

Details

The simdat function generates three datasets consisting of independent realizations of Y (for model = "mean" or "quantile"), or \{Y, \boldsymbol{X}\} (for model = "ols", "logistic", or "poisson"): a training dataset of size n_t, a labeled dataset of size n_l, and an unlabeled dataset of size n_u. These sizes are specified by the argument n.

NOTE: In the unlabeled data subset, outcome data are still generated to facilitate a benchmark for comparison with an "oracle" model that uses the true Y^{\mathcal{U}} values for estimation and inference.

Generating Data

For "mean" and "quantile", we simulate a continuous outcome, Y \in \mathbb{R}, with mean given by the effect argument and error variance given by the sigma_y argument.

For "ols", "logistic", or "poisson" models, predictor data, \boldsymbol{X} \in \mathbb{R}^4 are simulated such that the ith observation follows a standard multivariate normal distribution with a zero mean vector and identity covariance matrix:

\boldsymbol{X_i} = (X_{i1}, X_{i2}, X_{i3}, X_{i4}) \sim \mathcal{N}_4(\boldsymbol{0}, \boldsymbol{I}).

For "ols", a continuous outcome Y \in \mathbb{R} is simulated to depend on X_1 through a linear term with the effect size specified by the effect argument, while the other predictors, \boldsymbol{X} \setminus X_1, have nonlinear effects:

Y_i = effect \times Z_{i1} + \frac{1}{2} Z_{i2}^2 + \frac{1}{3} Z_{i3}^3 + \frac{1}{4} Z_{i4}^2 + \varepsilon_y,

and \varepsilon_y \sim \mathcal{N}(0, sigma_y), where the sigma_y argument specifies the error variance.

For "logistic", we simulate:

\Pr(Y_i = 1 \mid \boldsymbol{X}) = logit^{-1}(effect \times Z_{i1} + \frac{1}{2} Z_{i2}^2 + \frac{1}{3} Z_{i3}^3 + \frac{1}{4} Z_{i4}^2 + \varepsilon_y)

and generate:

Y_i \sim Bern[1, \Pr(Y_i = 1 \mid \boldsymbol{X})]

where \varepsilon_y \sim \mathcal{N}(0, sigma\_y).

For "poisson", we simulate:

\lambda_Y = exp(effect \times Z_{i1} + \frac{1}{2} Z_{i2}^2 + \frac{1}{3} Z_{i3}^3 + \frac{1}{4} Z_{i4}^2 + \varepsilon_y)

and generate:

Y_i \sim Poisson(\lambda_Y)

Generating Predictions

To generate predicted outcomes for "mean" and "quantile", we simulate a continuous variable with mean given by the empirical mean of the training data and error variance given by the sigma_y argument.

For "ols", we fit a generalized additive model (GAM) on the simulated training dataset and calculate predictions for the labeled and unlabeled datasets as deterministic functions of \boldsymbol{X}. Specifically, we fit the following GAM:

Y^{\mathcal{T}} = s_0 + s_1(X_1^{\mathcal{T}}) + s_2(X_2^{\mathcal{T}}) + s_3(X_3^{\mathcal{T}}) + s_4(X_4^{\mathcal{T}}) + \varepsilon_p,

where \mathcal{T} denotes the training dataset, s_0 is an intercept term, and s_1(\cdot), s_2(\cdot), s_3(\cdot), and s_4(\cdot) are smoothing spline functions for X_1, X_2, X_3, and X_4, respectively, with three target equivalent degrees of freedom. Residual error is modeled as \varepsilon_p.

Predictions for labeled and unlabeled datasets are calculated as:

f(\boldsymbol{X}^{\mathcal{L}\cup\mathcal{U}}) = \hat{s}_0 + \hat{s}_1(X_1^{\mathcal{L}\cup\mathcal{U}}) + \hat{s}_2(X_2^{\mathcal{L}\cup\mathcal{U}}) + \hat{s}_3(X_3^{\mathcal{L}\cup\mathcal{U}}) + \hat{s}_4(X_4^{\mathcal{L}\cup\mathcal{U}}),

where \hat{s}_0, \hat{s}_1, \hat{s}_2, \hat{s}_3, and \hat{s}_4 are estimates of s_0, s_1, s_2, s_3, and s_4, respectively.

NOTE: For continuous outcomes, we provide optional arguments shift and scale to further apply a location shift and scaling factor, respectively, to the predicted outcomes. These default to shift = 0 and scale = 1, i.e., no location shift or scaling.

For "logistic", we train k-nearest neighbors (k-NN) classifiers on the simulated training dataset for values of k ranging from 1 to 10. The optimal k is chosen via cross-validation, minimizing the misclassification error on the validation folds. Predictions for the labeled and unlabeled datasets are obtained by applying the k-NN classifier with the optimal k to \boldsymbol{X}.

Specifically, for each observation in the labeled and unlabeled datasets:

\hat{Y} = \operatorname{argmax}_c \sum_{i \in \mathcal{N}_k} I(Y_i = c),

where \mathcal{N}_k represents the set of k nearest neighbors in the training dataset, c indexes the possible classes (0 or 1), and I(\cdot) is an indicator function.

For "poisson", we fit a generalized linear model (GLM) with a log link function to the simulated training dataset. The model is of the form:

\log(\mu^{\mathcal{T}}) = \gamma_0 + \gamma_1 X_1^{\mathcal{T}} + \gamma_2 X_2^{\mathcal{T}} + \gamma_3 X_3^{\mathcal{T}} + \gamma_4 X_4^{\mathcal{T}},

where \mu^{\mathcal{T}} is the expected count for the response variable in the training dataset, \gamma_0 is the intercept, and \gamma_1, \gamma_2, \gamma_3, and \gamma_4 are the regression coefficients for the predictors X_1, X_2, X_3, and X_4, respectively.

Predictions for the labeled and unlabeled datasets are calculated as:

\hat{\mu}^{\mathcal{L} \cup \mathcal{U}} = \exp(\hat{\gamma}_0 + \hat{\gamma}_1 X_1^{\mathcal{L} \cup \mathcal{U}} + \hat{\gamma}_2 X_2^{\mathcal{L} \cup \mathcal{U}} + \hat{\gamma}_3 X_3^{\mathcal{L} \cup \mathcal{U}} + \hat{\gamma}_4 X_4^{\mathcal{L} \cup \mathcal{U}}),

where \hat{\gamma}_0, \hat{\gamma}_1, \hat{\gamma}_2, \hat{\gamma}_3, and \hat{\gamma}_4 are the estimated coefficients.

Value

A data.frame containing n rows and columns corresponding to the labeled outcome (Y), the predicted outcome (f), a character variable (set_label) indicating which data set the observation belongs to (training, labeled, or unlabeled), and four independent, normally distributed predictors (X1, X2, X3, and X4), where applicable.

Examples

#-- Mean

dat_mean <- simdat(c(100, 100, 100),
  effect = 1, sigma_Y = 1,
  model = "mean"
)

head(dat_mean)

#-- Linear Regression

dat_ols <- simdat(c(100, 100, 100),
  effect = 1, sigma_Y = 1,
  model = "ols"
)

head(dat_ols)

Summarize ipd fit

Description

Summarize ipd fit

Usage

## S3 method for class 'ipd'
summary(object, ...)

Arguments

Value

An object of class summary.ipd containing:

call

The model formula.

coefficients

A glm-style table of estimates, SE, z, p.

method

Which IPD method was used.

model

Which downstream model was fitted.

intercept

Logical; whether an intercept was included.

Tidy an ipd fit

Description

Tidy an ipd fit

Usage

## S3 method for class 'ipd'
tidy(x, ...)

Arguments

Value

A tibble with columns term, estimate, std.error, conf.low, conf.high.

Examples

dat <- simdat()

fit <- ipd(Y - f ~ X1, method = "pspa", model = "ols",

    data = dat, label = "set_label")

tidy(fit)

Weighted Least Squares

Description

Computes the weighted least squares estimate of the coefficients.

Usage

wls(X, Y, w = NULL, return_se = FALSE)

Arguments

Value

(list): A list containing the following:

theta

(vector): p-vector of weighted least squares estimates of the coefficients.

se

(vector): If return_se == TRUE, return the p-vector of standard errors of the coefficients.

Normal Confidence Intervals

Description

Calculates normal confidence intervals for a given alternative at a given significance level.

Usage

zconfint_generic(mean, std_mean, alpha, alternative)

Arguments

Value

(vector): Lower and upper (1 - alpha) * 100% confidence limits.

Compute Z-Statistic and P-Value

Description

Computes the z-statistic and the corresponding p-value for a given test.

Usage

zstat_generic(value1, value2, std_diff, alternative, diff = 0)

Arguments

Value

(list): A list containing the following:

zstat

(numeric): The computed z-statistic.

pvalue

(numeric): The corresponding p-value for the test.