test_repeated()/test_repeated_long() no
longer error with “missing value where TRUE/FALSE needed” on R-devel.
summary() on an aovlist fit walks every
Error() stratum, including the degenerate intercept-only
stratum (a single fitted value), and a recent change to
stats:::summary.aov()’s perfect-fit check can produce
NA there. We now summarize each stratum independently and
skip any that fail, since the intercept-only stratum was never used for
the reported test.Closed remaining gaps where an assumption was labeled
"not checked" because no diagnostic had been implemented
(as opposed to being genuinely uncheckable, e.g. too few observations,
which is unchanged). Every assumption status now reflects either a real
check or an honest “not applicable”/“not enough data” label - never an
unimplemented one.
car is now a hard dependency (Imports, not
Suggests): regression homoscedasticity (Breusch-Pagan),
multicollinearity (VIF), and ANOVA/factorial variance homogeneity
(Levene’s test) no longer fall back to "not checked" when
car isn’t installed, and
test_factorial(type = 2/3) no longer errors for the same
reason.test_repeated()/test_repeated_long() with 3+
conditions, via Mauchly’s test (stats::mauchly.test() on
the wide-format multivariate model, X = ~1) - verified to
match mauchly.test() and rstatix::anova_test()
exactly. Reports “not applicable” for exactly two conditions, where
sphericity isn’t defined.test_one_sample() and test_paired() (used to
justify the Wilcoxon signed-rank test), via the closed-form Cabilio
& Masaro (1996) test - verified to match
lawstat::symmetry.test(option = "CM", boot = FALSE)
exactly.test_correlation(method = "pearson"), via a quadratic-term
F-test (anova() comparing y ~ x to
y ~ x + I(x^2)) - equivalent to Ramsey’s RESET test with
a single quadratic regressor term, verified to match
lmtest::resettest(power = 2, type = "regressor")
exactly.test_correlation_matrix() now applies a
Benjamini-Hochberg correction to its pairwise p-values by default
(p.adj column in the correlation table), instead of
reporting pairwise p-values with no correction applied.sample_size_ordinal(method = "whitehead").
Unlike every other formula in the sample-size module, the
proportional-odds formula had not been checked against a published
worked example or an external reference implementation (only against
itself), and the input convention for probs (which arm’s
category distribution to supply) is ambiguous across secondary sources
describing Whitehead (1993). sample_size_ordinal() now
implements only Noether’s method, which was independently verified. It
no longer takes method, probs, or
odds_ratio arguments.Second of two planned display-ergonomics changes (the first added
workflow-specific print titles and field labels). Regression
coefficients, Cox hazard ratios, the diagnostic-accuracy table, the
ROC/Youden threshold, the ICC(1)/(2)/(3) comparison table,
correlation-method/pairwise-correlation tables, factorial ANOVA tables,
and every tidy post-hoc result (repeated measures, repeated categorical,
multinomial, paired categorical) were computed but never shown in
print()/summary() console output - only
reachable via x$alternative_tests/x$posthoc.
They now print directly.
R/workflow-display.R’s registry gained a
table/table_label field per applicable
workflow, resolved by workflow_meta() into a small data
frame when the referenced field is already tidy (a
data.frame/ tibble) or a base
table/matrix (auto-converted, with generic
Var1/Var2/Freq column names
relabeled using the same outcome/group labels already resolved for that
workflow, rather than left meaningless).tf_table() helper in R/print.R prints
these tables, applying the same
format_stat()/format_p() conventions already
used elsewhere (p-value columns get “<0.001” instead of a misleading
truncated “0.00”; integer count columns are formatted without decimal
places).groups/repeated‘s omnibus-ANOVA post-hoc
(posthoc_groups(), R/posthoc.R) wraps a raw
TukeyHSD/pairwise.htest object rather than a
tidy data frame; it’s silently skipped rather than crashing or dumping a
raw object, matching prior behavior. (repeated’s own
separate paired-comparison post-hoc, from
paired_posthoc_numeric(), is already tidy and
renders normally - only groups’ omnibus-ANOVA path is
affected.) Formatting an arbitrary htest/TukeyHSD object nicely is a
larger follow-up, not attempted here.data.frame printing wraps onto a second,
visually misaligned block of rows once the default 80-character console
width is exceeded. tf_table() now temporarily widens
options(width = ...) for the duration of the print call
rather than truncating labels or letting rows wrap. The first version of
this width calculation didn’t account for NA values (which
produce an NA width via plain
max()/sum(), in turn making
options(width = NA) error) - caught by the existing
assumption- structure test suite hitting a real
car::Anova() aliased-coefficients edge case with an
all-NA ANOVA row, not by a synthetic test written after the
fact.Testing: devtools::test() 409/409 passing, including new assertions pinning table rendering for six representative workflows (regression, Cox, diagnostic, ICC’s no-wrap behavior, agreement’s relabeled columns, and the groups-skips/repeated-renders post-hoc distinction). devtools::check(): 0 errors, 0 warnings, 1 environment-only NOTE, including vignette rebuild (console-output examples updated automatically).
print.testflow() and
print.summary.testflow() now show a title specific to the
workflow that actually ran (e.g. “Linear Regression”, “Cox Proportional
Hazards Regression”, “Intraclass Correlation”) instead of the same
generic “Statistical test workflow” for every single
test_*() result. The now-redundant separate
Design: field (which always duplicated the same information
less legibly) is dropped.Outcome:/Group: console field labels
are now workflow-appropriate where Group: previously held
something other than a comparison group: “Predictors” for
test_linear_regression()/test_logistic_regression()/
test_cox(), “Reference” for test_diagnostic(),
“Rater 1”/“Rater 2” for test_agreement(), “Time” in place
of “Outcome” for the two survival workflows, and so on.R/workflow-display.R centralizes this
per-workflow display metadata in one registry keyed by
x$workflow, rather than threading new parameters through
every new_testflow() call site across
R/test_*.R (none of those files changed).
sample_size-class objects are unaffected: their generic
“Sample size planning” title already fits every call, since they share
one purpose unlike the widely varying test_*()
workflows.x$alternative_tests/x$posthoc.sample_size_bioequivalence(): two one-sided test
(TOST) bioequivalence planning on the log scale, for
crossover or parallel designs.
method = "iterative_tost" (default) searches via
stats::uniroot() for the smallest n achieving the exact
TOST power, verified to recover the documented closed-form special case
at GMR=1 exactly and to match an independently computed
uniroot() search off-center;
method = "normal_approx" offers the closed-form
approximation for comparison.sample_size_precision(): sample size for a target
confidence- interval half-width instead of power against an effect size,
covering one- and two-sample continuous and binary designs plus
log-odds-ratio precision.sample_size_cluster_adjust(): design-effect
adjustment (DE = 1+(m-1)rho, plus an unequal-cluster-size
approximation) for converting an individually randomized sample size to
a cluster-randomized one. Like
sample_size_adjust_dropout(), it returns a plain integer,
not a sample_size object.sample_size_ordinal() gained
method = c("noether", "whitehead"); Whitehead’s
proportional-odds formula is included with an explicit caveat (in both
the documentation and the returned assumptions) that, unlike every other
formula in this release, it has not been validated against a published
worked example and is sensitive to the anticipated category-probability
distribution.sample_size_survival() gained optional
accrual_duration/follow_up arguments: when
supplied with survival_a/survival_b, event
probabilities use a closed-form uniform-accrual adjustment instead of
assuming every subject is followed for the full study duration. Verified
to reduce to the existing flat conversion as
accrual_duration -> 0, and to always require more
subjects than instantaneous accrual for the same event target.ss_assurance in the original formula spec)
remains explicitly out of scope: it is simulation-based rather than
closed-form and was flagged in the spec itself as “a later module, not
v1.”test_diagnostic(): sensitivity, specificity,
positive/negative predictive values, and positive/negative likelihood
ratios from a 2x2 table against a gold-standard reference, each with a
confidence interval (exact/Clopper-Pearson for the proportions, the
closed-form log-scale interval of Simel, Samsa & Matchar (1991) for
the likelihood ratios). The primary test compares overall accuracy to
the no-information rate, the same convention
caret::confusionMatrix() uses.test_roc(): AUC computed from the Mann-Whitney U
statistic (the same identity already used for the package’s
rank-biserial correlation), with the closed-form Hanley & McNeil
(1982) confidence interval and the Youden’s-J-optimal threshold,
avoiding a pROC dependency.test_agreement(): Cohen’s kappa with the Fleiss,
Cohen & Everitt
psych::cohen.kappa()’s confidence interval exactly), not
the simpler approximation that understates variance under uneven
category marginals.test_icc(): ICC(1,1), ICC(2,1), and ICC(3,1) via
Shrout & Fleiss
irr::icc() exactly for all three forms). ICC(2,1) (two-way
random, absolute agreement) is reported as the primary/effect-size
estimate, following the reliability-study recommendation of Koo & Li
(2016).psych/irr
during development (not shipped as dependencies).test_icc(): the
ICC(2,1) F-test (testing whether ICC exceeds 0) was accidentally using
the same Satterthwaite coefficients as the confidence interval, which
are evaluated at different points (the null value 0 for the test, the
estimated ICC for the interval) and must not be shared - this silently
produced a wildly wrong p-value (0.48 instead of <0.001) for data
with strong measured reliability.safe_tidy_htest() (used by most test_*()
workflows) left stray name attributes on numeric columns inherited from
the underlying htest object (e.g. binom.test()$statistic
carrying the name "number of successes"), which could trip
up all.equal()-based comparisons against independently
computed reference values despite the numbers being identical.test_survival(): Kaplan-Meier estimation and the
log-rank test
(survival::survfit()/survival::survdiff()) for
a two-group time-to- event comparison, with a companion univariate Cox
hazard ratio (and its confidence interval) as the effect size.test_cox(): Cox proportional hazards regression
(survival::coxph()), with the overall likelihood-ratio test
as the primary result, per-term hazard ratios, a proportional-hazards
assumption check via the Schoenfeld residual test
(survival::cox.zph()), and the concordance index as the
effect size.survival is now a hard dependency
(Imports, not Suggests): it ships with every
standard R installation as a “Recommended” package, so
requireNamespace()-guarding it added complexity without
meaningfully widening compatibility. Surv() is re-exported
from testflow so it’s available after
library(testflow) alone.print.testflow() and make_report() hid an
effect size’s estimate entirely whenever its magnitude label was
NA, even when the estimate itself was available. Both now
show the estimate unconditionally and append the magnitude label only
when one is assigned.test_linear_regression(): multiple linear
regression via stats::lm(), with residual normality
(Shapiro-Wilk), homoscedasticity (Breusch-Pagan), and multicollinearity
(variance inflation factor) assumption checks, the overall model F-test
as the primary result, the per-term coefficient table, and R squared /
adjusted R squared as the effect size.test_logistic_regression(): logistic regression
via stats::glm(family = binomial), with multicollinearity
and influential- observation (Cook’s distance) assumption checks, the
likelihood-ratio test against the intercept-only model as the primary
result, coefficients on both the log-odds and odds-ratio scale, and
McFadden’s pseudo R squared as the effect size.vignettes/statistical-test-workflows.Rmd and
vignettes/effect-size-formulas.Rmd, and every reported
statistic is pinned in tests against
summary(lm())/summary(glm()) computed
independently.An exhaustive correctness review of every test_*()
workflow (run against base-R references and the package’s own documented
formulas) found and fixed the following:
test_two_groups() reporting Cohen’s d (and
rank-biserial correlation) with the opposite sign from the primary
t-test/Wilcoxon test it’s attached to, whenever group labels weren’t
already alphabetically ordered in the source data. The shared
assert_two_groups() helper now derives group order from
levels(as.factor(...)), matching what
t.test/wilcox.test’s formula interface uses
internally, instead of first-appearance order in the raw data.test_repeated() computing the wrong F-statistic,
error degrees of freedom, and p-value for the parametric
(repeated-measures ANOVA) branch: the subject id column was
never coerced to a factor before
aov(y ~ time + Error(id/time)), so Error()
built a spurious extra stratum instead of the correct
id:time error term. This affected the default
path (auto-generated subject ids are always integer), and the
test suite’s own reference computation reproduced the same bug rather
than catching it.test_categorical() recommending Fisher’s exact
test while simultaneously showing an “Expected cell counts: acceptable”
assumption panel for the same table, because the recommendation used an
any-cell-below-threshold rule while the displayed panel used a different
(Cochran 80%-of-cells) rule. Both now use the same
fisher_threshold rule, so the panel explains the
recommendation actually made.test_correlation_matrix() silently doing listwise
deletion across all selected variables before computing
pairwise correlations, which made
use = "pairwise.complete.obs" a no-op, discarded usable
rows, and contradicted its own “Pairwise complete observations are used”
assumption message. Each pairwise correlation and test now uses its own
pairwise-complete rows.test_factorial()’s type
parameter, which defaulted to 2 but was documented as “a
placeholder for future car integration” and always computed Type I
(sequential) sums of squares regardless of its value. For unbalanced
factorial designs, Type I vs. Type II vs. Type III sums of squares can
give substantially different p-values;
type = 2/3 now dispatch to
car::Anova() (Type III with the required sum-to-zero
contrasts), and the reported eta squared is computed from the same sums
of squares as the reported test rather than always from Type I.test_multinomial()’s pairwise post-hoc binomial
tests not applying a multiple-comparison correction, unlike every other
post-hoc helper in the package (BH by default).
p.adj/p.adjust.method columns are now
included.posthoc_groups()’s Welch/Kruskal-Wallis
follow-up tests, which were called “Games-Howell-style” and “Dunn-style”
but are plain BH-adjusted pairwise Welch t-tests and pairwise Wilcoxon
rank-sum tests respectively, not the named procedures (which use a
studentized-range or pooled-rank-variance correction). Labels now
describe what’s actually run.test_one_sample() and
test_correlation() labeling unchecked assumptions (Wilcoxon
symmetry, Pearson linearity) as "acceptable" with no
diagnostic actually performed. Both now use the honest
"warning"/ "not checked" status already used
for the identical situation elsewhere in the package
(e.g. test_paired()’s symmetry check).sample_size_survival() understating total sample
size by a factor of 2 whenever
survival_a/survival_b were supplied: the
events-to-N conversion now correctly applies
N_total = 2 * D_total / (qA + qB) instead of
D_total / (qA + qB). This affected superiority,
non-inferiority, and equivalence planning alike.sample_size_binary(objective = "equivalence")
silently ignoring the allocation ratio when
p1 == p2, which produced incorrect per-group sizes for
unequal allocation designs.plot(x, type = "curve") and
plot(x, type = "both") erroring for every
sample_size object with curve data: a local tibble column
named x shadowed the x (sample-size object)
argument during construction.sample_size() and its helpers and into
the function documentation (?sample_size_continuous, etc.),
consistent with normal R package conventions. summary(),
print(), and as_tibble() output no longer
carries a formula/reference field.design/objective/method
combination across sample_size_continuous(),
sample_size_binary(), sample_size_survival(),
and sample_size_ordinal(), each paired with its formula and
a runnable example.test_outliers() on the same
test_*(formula, data, ...) calling convention used by the
other workflow helpers.plot() S3
path via plot.testflow().print().testflow and summary output with
cli-based colors, section headings, field labels, and
assumption bullets.testflow objects and summaries. Set
options(testflow.cli_colors = FALSE) to disable them, or
options(testflow.cli_colors = TRUE) to force colors in
scripts.sumtab() example to the README.sumtab() for formula-driven descriptive summary
tables with optional automatic p-value selection.0.8.0.First public release.
test_two_groups(outcome ~ group, data = data)test_groups(outcome ~ group, data = data)test_categorical(x ~ y, data = data)test_correlation(y ~ x, data = data)test_paired(after ~ before, data = data)test_factorial(outcome ~ factor1 * factor2, data = data)as_tibble() includes H0, confidence interval, effect
size name, effect size estimate, effect size magnitude, and
decision.inst/doc.