Precision Profiles and Functional Sensitivity

Marcello Grassi

Introduction

Analytical precision varies across the measurement range of most laboratory assays. Near the limit of detection, imprecision (expressed as coefficient of variation, CV) is typically higher, while at elevated concentrations, CV approaches an asymptotic minimum. Understanding this relationship is essential for:

This vignette demonstrates how to construct precision profiles and estimate functional sensitivity using valytics.

library(valytics)
library(ggplot2)

The Precision Profile Concept

A precision profile plots CV (or SD) against analyte concentration. For many immunoassays and clinical chemistry methods, this relationship follows a characteristic hyperbolic pattern:

\[CV = \sqrt{a^2 + \left(\frac{b}{x}\right)^2}\]

where: - \(a\) is the asymptotic CV at high concentrations (the “floor” of precision) - \(b\) is the concentration-dependent component - \(x\) is the analyte concentration

At low concentrations, the \((b/x)^2\) term dominates, causing CV to increase sharply. At high concentrations, CV approaches \(a\).

Example: High-Sensitivity Troponin Assay

We will use the troponin_precision dataset, which contains multi-level precision data for a simulated high-sensitivity cardiac troponin I assay.

data("troponin_precision")
head(troponin_precision)
#>   level level_label concentration day run replicate value
#> 1    L1 L1 (5 ng/L)             5   1   1         1  4.36
#> 2    L1 L1 (5 ng/L)             5   1   1         2  4.69
#> 3    L1 L1 (5 ng/L)             5   1   2         1  4.99
#> 4    L1 L1 (5 ng/L)             5   1   2         2  5.21
#> 5    L1 L1 (5 ng/L)             5   2   1         1  4.91
#> 6    L1 L1 (5 ng/L)             5   2   1         2  5.17

The dataset includes measurements at six concentration levels from 5 to 500 ng/L, with a nested design of 5 days × 2 runs × 2 replicates per level.

# Summarize the experimental design
table(troponin_precision$level, troponin_precision$day)
#>     
#>      1 2 3 4 5
#>   L1 4 4 4 4 4
#>   L2 4 4 4 4 4
#>   L3 4 4 4 4 4
#>   L4 4 4 4 4 4
#>   L5 4 4 4 4 4
#>   L6 4 4 4 4 4

Step 1: Precision Study Analysis

First, we analyze precision at each concentration level using precision_study(). The sample argument indicates that each level should be analyzed separately.

prec <- precision_study(
 data = troponin_precision,
 value = "value",
 sample = "level",
 day = "day",
 run = "run"
)

print(prec)
#> 
#> Precision Study Analysis
#> --------------------------------------------- 
#> n = 120 observations
#> Design: day/run/replicate (inferred)
#> Estimation: ANOVA (Method of Moments)
#> CI method: Satterthwaite, 95% CI
#> 
#> Samples: 6 concentration levels
#> (Showing results for first sample; use $by_sample for all)
#> 
#> Precision Estimates:
#> --------------------------------------------- 
#>   Repeatability:       SD = 0.265  (CV = 5.33%)
#>                        95% CI: [0.185, 0.464]
#>   Between-run:         SD = 0.235  (CV = 4.74%)
#>                        95% CI: [0.147, 0.577]
#>   Between-day:         SD = 0.000  (CV = 0.00%)
#>                        95% CI: [0.000, 0.000]
#>   Within-laboratory precision: SD = 0.354  (CV = 7.14%)
#>                        95% CI: [0.260, 0.556]

The results show variance components and precision estimates. For precision profiles, we are primarily interested in within-laboratory precision (or repeatability) at each concentration level.

We can examine results for individual levels:

# Access results for each concentration level
names(prec$by_sample)
#> [1] "L1" "L2" "L3" "L4" "L5" "L6"

# Example: Level 1 (5 ng/L)
prec$by_sample$L1$precision
#>                       measure        sd   cv_pct  ci_lower  ci_upper
#> 1               Repeatability 0.2646507 5.334624 0.1849160 0.4644444
#> 2                 Between-run 0.2353083 4.743163 0.1468813 0.5771204
#> 3                 Between-day 0.0000000 0.000000 0.0000000 0.0000000
#> 4 Within-laboratory precision 0.3541327 7.138334 0.2598699 0.5557666
#>   cv_ci_lower cv_ci_upper
#> 1    3.727393    9.361911
#> 2    2.960720   11.633148
#> 3    0.000000    0.000000
#> 4    5.238257   11.202713

Step 2: Generate Precision Profile

Now we generate the precision profile by modeling CV as a function of concentration:

profile <- precision_profile(
 prec,
 model = "hyperbolic",
 cv_targets = c(10, 20)
)

print(profile)
#> 
#> Precision Profile Analysis
#> ---------------------------------------- 
#> n = 6 concentration levels
#> Concentration range: 4.96 to 501 (100.89-fold)
#> 
#> Model: Hyperbolic
#>   CV = sqrt(3.037^2 + (32.169/x)^2)
#> 
#> Parameters:
#>   a = 3.037
#>   b = 32.169
#> 
#> Fit Quality:
#>   R-squared = 0.955
#>   RMSE = 0.322
#> 
#> Functional Sensitivity:
#>   CV = 10%: concentration = 3.376
#>   CV = 20%: concentration = 1.627

The output shows: - Model parameters: The fitted values of \(a\) (asymptotic CV) and \(b\) - Fit quality: R² and RMSE indicating how well the model fits the data - Functional sensitivity: The concentration at which target CVs are achieved

Understanding the Model Parameters

# Extract model parameters
a <- profile$model$parameters["a"]
b <- profile$model$parameters["b"]

cat(sprintf("Asymptotic CV (a): %.2f%%\n", a))
#> Asymptotic CV (a): 3.04%
cat(sprintf("Concentration component (b): %.2f\n", b))
#> Concentration component (b): 32.17
cat(sprintf("\nModel equation: %s\n", profile$model$equation))
#> 
#> Model equation: CV = sqrt(3.037^2 + (32.169/x)^2)

The asymptotic CV tells us the best precision achievable at high concentrations. In this case, ~3% CV represents the analytical “floor” of the assay.

Visualizing the Precision Profile

plot(profile)
Precision profile showing CV versus concentration with fitted hyperbolic model.

Precision profile showing CV versus concentration with fitted hyperbolic model.

The plot displays: - Points: Observed CV values at each concentration level - Blue curve: Fitted hyperbolic model - Dashed lines: Target CV thresholds (10% and 20%) - Dotted lines: Concentrations at which targets are achieved (functional sensitivity)

For a logarithmic concentration scale (often more informative across wide ranges):

plot(profile, log_x = TRUE)
Precision profile with logarithmic concentration scale.

Precision profile with logarithmic concentration scale.

Functional Sensitivity

Functional sensitivity is the lowest concentration at which a measurement procedure achieves a specified level of precision. Common thresholds include:

profile$functional_sensitivity
#>   cv_target concentration ci_lower ci_upper achievable
#> 1        10      3.376304       NA       NA       TRUE
#> 2        20      1.627302       NA       NA       TRUE

In this example: - At 10% CV, the functional sensitivity is approximately 3.4 ng/L - At 20% CV, the functional sensitivity is approximately 1.6 ng/L

Unachievable Targets

If a target CV is below the asymptotic minimum (\(a\)), it cannot be achieved at any concentration:

# Try a target CV of 2% (below asymptotic ~3%)
profile_strict <- precision_profile(
 prec,
 cv_targets = c(2, 5, 10)
)

profile_strict$functional_sensitivity
#>   cv_target concentration ci_lower ci_upper achievable
#> 1         2            NA       NA       NA      FALSE
#> 2         5      8.098367       NA       NA       TRUE
#> 3        10      3.376304       NA       NA       TRUE

The 2% CV target is marked as “not achievable” because it is below the assay’s asymptotic CV floor.

Bootstrap Confidence Intervals

For uncertainty quantification on functional sensitivity estimates, use bootstrap resampling:

# Note: This takes longer to run
profile_boot <- precision_profile(
 prec,
 cv_targets = c(10, 20),
 bootstrap = TRUE,
 boot_n = 999
)

profile_boot$functional_sensitivity

The confidence intervals help assess the precision of the functional sensitivity estimates.

Alternative: Linear Model

For some assays, a simpler linear model may be adequate:

\[CV = a + \frac{b}{x}\]

profile_linear <- precision_profile(
 prec,
 model = "linear",
 cv_targets = c(10, 20)
)

print(profile_linear)
#> 
#> Precision Profile Analysis
#> ---------------------------------------- 
#> n = 6 concentration levels
#> Concentration range: 4.96 to 501 (100.89-fold)
#> 
#> Model: Linear
#>   CV = 2.695 + 21.173/x
#> 
#> Parameters:
#>   a = 2.695
#>   b = 21.173
#> 
#> Fit Quality:
#>   R-squared = 0.953
#>   RMSE = 0.330
#> 
#> Functional Sensitivity:
#>   CV = 10%: concentration = 2.899
#>   CV = 20%: concentration = 1.224

Compare model fits:

cat("Hyperbolic model R²:", round(profile$fit_quality$r_squared, 4), "\n")
#> Hyperbolic model R²: 0.9551
cat("Linear model R²:", round(profile_linear$fit_quality$r_squared, 4), "\n")
#> Linear model R²: 0.9528

The hyperbolic model typically provides better fit because it correctly captures the asymptotic behavior at high concentrations.

Detailed Summary

The summary() function provides comprehensive output:

summary(profile)

This includes: - Fitted values at each concentration level - Residual analysis - Complete interpretation of functional sensitivity results

Working with Data Frames Directly

If you have pre-calculated CV values (e.g., from external sources), you can create a precision profile directly from a data frame:

# Create a data frame with concentration and CV values
cv_data <- data.frame(
 concentration = c(5, 10, 25, 50, 100, 500),
 cv_pct = c(5.8, 4.2, 3.5, 3.2, 3.1, 3.0)
)

profile_df <- precision_profile(
 cv_data,
 concentration = "concentration",
 cv = "cv_pct"
)

print(profile_df)
#> 
#> Precision Profile Analysis
#> ---------------------------------------- 
#> n = 6 concentration levels
#> Concentration range: 5 to 500 (100.00-fold)
#> 
#> Model: Hyperbolic
#>   CV = sqrt(3.173^2 + (24.732/x)^2)
#> 
#> Parameters:
#>   a = 3.173
#>   b = 24.732
#> 
#> Fit Quality:
#>   R-squared = 0.982
#>   RMSE = 0.132
#> 
#> Functional Sensitivity:
#>   CV = 10%: concentration = 2.608
#>   CV = 20%: concentration = 1.252

Clinical Application Example

For high-sensitivity cardiac troponin assays, regulatory and clinical guidelines suggest:

  1. Functional sensitivity should be ≤ 50% of the 99th percentile URL
  1. CV at the 99th percentile should be ≤ 10%

Let us check our simulated assay against these criteria:

# Assume 99th percentile URL = 20 ng/L
url_99th <- 20

# Check functional sensitivity at 10% CV
fs_10pct <- profile$functional_sensitivity$concentration[
 profile$functional_sensitivity$cv_target == 10
]

cat(sprintf("99th percentile URL: %.0f ng/L\n", url_99th))
#> 99th percentile URL: 20 ng/L
cat(sprintf("Functional sensitivity (10%% CV): %.1f ng/L\n", fs_10pct))
#> Functional sensitivity (10% CV): 3.4 ng/L
cat(sprintf("Ratio (FS / URL): %.2f\n", fs_10pct / url_99th))
#> Ratio (FS / URL): 0.17

if (fs_10pct <= 0.5 * url_99th) {
 cat("\n✓ Meets criterion: FS ≤ 50% of 99th percentile URL\n")
} else {
 cat("\n✗ Does not meet criterion: FS > 50% of 99th percentile URL\n")
}
#> 
#> ✓ Meets criterion: FS ≤ 50% of 99th percentile URL

Recommendations for Precision Profile Studies

Based on established methodology:

  1. Minimum concentration levels: At least 4-5 levels spanning the measurement range
  2. Concentration span: Should cover at least 2 orders of magnitude when possible
  3. Precision study design: Minimum 5 days with multiple runs per day
  4. Include levels near clinical decision points: 99th percentile, diagnostic thresholds

Summary

The precision profile workflow in valytics:

  1. Collect data at multiple concentration levels using a nested precision design
  2. Run precision_study() with sample argument to analyze each level
  3. Generate precision_profile() to model CV-concentration relationship
  4. Interpret functional sensitivity relative to clinical requirements
  5. Visualize with plot() to communicate findings
# Complete workflow
data("troponin_precision")

# Step 1: Precision study
prec <- precision_study(
 data = troponin_precision,
 value = "value",
 sample = "level",
 day = "day",
 run = "run"
)

# Step 2: Precision profile
profile <- precision_profile(
 prec,
 model = "hyperbolic",
 cv_targets = c(10, 20)
)

# Step 3: Interpret and visualize
summary(profile)
plot(profile, log_x = TRUE)

References

Armbruster DA, Pry T (2008). Limit of blank, limit of detection and limit of quantitation. Clinical Biochemist Reviews, 29(Suppl 1):S49-S52.

Apple FS, et al. (2017). Cardiac Troponin Assays: Guide to Understanding Analytical Characteristics and Their Impact on Clinical Care. Clinical Chemistry, 63(1):73-81.

Chesher D (2008). Evaluating assay precision. Clinical Biochemist Reviews, 29(Suppl 1):S23-S26.