Perform closed graphical multiple comparison procedures
Source:R/graph_test_closure.R
graph_test_closure.Rd
Closed graphical multiple comparison procedures, or graphical multiple comparison procedures based on the closure, generate the closure based on a graph consisting of all intersection hypotheses. It tests each intersection hypothesis and rejects an individual hypothesis if all intersection hypotheses involving it have been rejected. An intersection hypothesis represents the parameter space where individual null hypotheses involved are true simultaneously.
For a graphical multiple comparison procedure with $m$ hypotheses, there are
\(2^m-1\) intersection hypotheses. For each intersection hypothesis, a test
type could be chosen to determine how to reject the intersection hypothesis.
Current choices of test types include Bonferroni, Simes and parametric. This
implementation offers a more general framework covering Bretz et al. (2011),
Lu (2016), and Xi et al. (2017). See vignette("closed-testing")
for more
illustration of closed test procedures and interpretation of their outputs.
Arguments
- graph
An initial graph as returned by
graph_create()
.- p
A numeric vector of p-values (unadjusted, raw), whose values should be between 0 & 1. The length should match the number of hypotheses in
graph
.- alpha
A numeric value of the overall significance level, which should be between 0 & 1. The default is 0.025 for one-sided hypothesis testing problems; another common choice is 0.05 for two-sided hypothesis testing problems. Note when parametric tests are used, only one-sided tests are supported.
- test_groups
A list of numeric vectors specifying hypotheses to test together. Grouping is needed to correctly perform Simes and parametric tests.
- test_types
A character vector of test types to apply to each test group. This is needed to correctly perform Simes and parametric tests. The length should match the number of elements in
test_groups
.- test_corr
(Optional) A list of numeric correlation matrices. Each entry in the list should correspond to each test group. For a test group using Bonferroni or Simes tests, its corresponding entry in
test_corr
should beNA
. For a test group using parametric tests, its corresponding entry intest_corr
should be a numeric correlation matrix specifying the correlation between test statistics for hypotheses in this test group. The length should match the number of elements intest_groups
.- verbose
A logical scalar specifying whether the details of the adjusted p-value calculations should be included in results. When
verbose = TRUE
, adjusted p-values are provided for each intersection hypothesis. The default isverbose = FALSE
.- test_values
A logical scalar specifying whether adjusted significance levels should be provided for each hypothesis. When
test_values = TRUE
, it provides an equivalent way of performing graphical multiple comparison procedures by comparing each p-value with its significance level. If the p-value of a hypothesis is less than or equal to its significance level, the hypothesis is rejected. The default istest_values = FALSE
.
Value
A graph_report
object with a list of 4 elements:
inputs
- Input parameters, which is a list of:graph
- Initial graph,p
- (Unadjusted or raw) p-values,alpha
- Overall significance level,test_groups
- Groups of hypotheses for different types of tests,test_types
- Different types of tests,test_corr
- Correlation matrices for parametric tests.
outputs
- Output parameters, which is a list of:adjusted_p
- Adjusted p-values,rejected
- Rejected hypotheses,graph
- Updated graph after deleting all rejected hypotheses.
details
- Verbose outputs with adjusted p-values for intersection hypotheses, ifverbose = TRUE
.test_values
- Adjusted significance levels, iftest_values = TRUE
.
Details for test specification
Test specification includes three components: test_groups
, test_types
,
and test_corr
. Alignment among entries in these components is important
for correct implementation. There are two ways to provide test specification.
The first approach is the "unnamed" approach, which assumes that all 3
components are ordered the same way, i.e., the $n$-th element of test_types
and test_corr
should apply to the $n$-th group in test_groups
. The
second "named" approach uses the name of each element of each component to
connect the element of test_types
and test_corr
with the correct element
of test_groups
. Consistency should be ensured for correct implementation.
References
Bretz, F., Posch, M., Glimm, E., Klinglmueller, F., Maurer, W., and Rohmeyer, K. (2011). Graphical approaches for multiple comparison procedures using weighted Bonferroni, Simes, or parametric tests. Biometrical Journal, 53(6), 894-913.
Lu, K. (2016). Graphical approaches using a Bonferroni mixture of weighted Simes tests. Statistics in Medicine, 35(22), 4041-4055.
Xi, D., Glimm, E., Maurer, W., and Bretz, F. (2017). A unified framework for weighted parametric multiple test procedures. Biometrical Journal, 59(5), 918-931.
See also
graph_test_shortcut()
for shortcut graphical multiple comparison
procedures.
Examples
# A graphical multiple comparison procedure with two primary hypotheses
# (H1 and H2) and two secondary hypotheses (H3 and H4)
# See Figure 4 in Bretz et al. (2011).
hypotheses <- c(0.5, 0.5, 0, 0)
delta <- 0.5
transitions <- rbind(
c(0, delta, 1 - delta, 0),
c(delta, 0, 0, 1 - delta),
c(0, 1, 0, 0),
c(1, 0, 0, 0)
)
g <- graph_create(hypotheses, transitions)
p <- c(0.018, 0.01, 0.105, 0.006)
alpha <- 0.025
# Closed graphical multiple comparison procedure using Bonferroni tests
# Same results as `graph_test_shortcut(g, p, alpha)`
graph_test_closure(g, p, alpha)
#>
#> Test parameters ($inputs) ------------------------------------------------------
#> Initial graph
#>
#> --- Hypothesis weights ---
#> H1: 0.5
#> H2: 0.5
#> H3: 0.0
#> H4: 0.0
#>
#> --- Transition weights ---
#> H1 H2 H3 H4
#> H1 0.0 0.5 0.5 0.0
#> H2 0.5 0.0 0.0 0.5
#> H3 0.0 1.0 0.0 0.0
#> H4 1.0 0.0 0.0 0.0
#>
#> Alpha = 0.025
#>
#> H1 H2 H3 H4
#> Unadjusted p-values: 0.018 0.010 0.105 0.006
#>
#> Test types
#> bonferroni: (H1, H2, H3, H4)
#>
#> Test summary ($outputs) --------------------------------------------------------
#> Hypothesis Adj. P-value Reject
#> H1 0.024 TRUE
#> H2 0.020 TRUE
#> H3 0.105 FALSE
#> H4 0.024 TRUE
#>
#> Final updated graph after removing rejected hypotheses
#>
#> --- Hypothesis weights ---
#> H1: NA
#> H2: NA
#> H3: 1
#> H4: NA
#>
#> --- Transition weights ---
#> H1 H2 H3 H4
#> H1 NA NA NA NA
#> H2 NA NA NA NA
#> H3 NA NA 0 NA
#> H4 NA NA NA NA
#>
# Closed graphical multiple comparison procedure using parametric tests for
# H1 and H2, and Bonferroni tests for H3 and H4
set.seed(1234)
corr_list <- list(matrix(c(1, 0.5, 0.5, 1), nrow = 2), NA)
graph_test_closure(
graph = g,
p = p,
alpha = alpha,
test_groups = list(1:2, 3:4),
test_types = c("parametric", "bonferroni"),
test_corr = corr_list
)
#>
#> Test parameters ($inputs) ------------------------------------------------------
#> Initial graph
#>
#> --- Hypothesis weights ---
#> H1: 0.5
#> H2: 0.5
#> H3: 0.0
#> H4: 0.0
#>
#> --- Transition weights ---
#> H1 H2 H3 H4
#> H1 0.0 0.5 0.5 0.0
#> H2 0.5 0.0 0.0 0.5
#> H3 0.0 1.0 0.0 0.0
#> H4 1.0 0.0 0.0 0.0
#>
#> Alpha = 0.025
#>
#> H1 H2 H3 H4
#> Unadjusted p-values: 0.018 0.010 0.105 0.006
#>
#> Correlation matrix: H1 H2
#> H1 1.0 0.5
#> H2 0.5 1.0
#>
#> Test types
#> parametric: (H1, H2)
#> bonferroni: (H3, H4)
#>
#> Test summary ($outputs) --------------------------------------------------------
#> Hypothesis Adj. P-value Reject
#> H1 0.02400 TRUE
#> H2 0.01871 TRUE
#> H3 0.10500 FALSE
#> H4 0.02400 TRUE
#>
#> Final updated graph after removing rejected hypotheses
#>
#> --- Hypothesis weights ---
#> H1: NA
#> H2: NA
#> H3: 1
#> H4: NA
#>
#> --- Transition weights ---
#> H1 H2 H3 H4
#> H1 NA NA NA NA
#> H2 NA NA NA NA
#> H3 NA NA 0 NA
#> H4 NA NA NA NA
#>
# The "named" approach to obtain the same results
# Note that "group2" appears before "group1" in `test_groups`
set.seed(1234)
corr_list <- list(group1 = matrix(c(1, 0.5, 0.5, 1), nrow = 2), group2 = NA)
graph_test_closure(
graph = g,
p = p,
alpha = alpha,
test_groups = list(group1 = 1:2, group2 = 3:4),
test_types = c(group2 = "bonferroni", group1 = "parametric"),
test_corr = corr_list
)
#>
#> Test parameters ($inputs) ------------------------------------------------------
#> Initial graph
#>
#> --- Hypothesis weights ---
#> H1: 0.5
#> H2: 0.5
#> H3: 0.0
#> H4: 0.0
#>
#> --- Transition weights ---
#> H1 H2 H3 H4
#> H1 0.0 0.5 0.5 0.0
#> H2 0.5 0.0 0.0 0.5
#> H3 0.0 1.0 0.0 0.0
#> H4 1.0 0.0 0.0 0.0
#>
#> Alpha = 0.025
#>
#> H1 H2 H3 H4
#> Unadjusted p-values: 0.018 0.010 0.105 0.006
#>
#> Correlation matrix: H1 H2
#> H1 1.0 0.5
#> H2 0.5 1.0
#>
#> Test types
#> parametric: (H1, H2)
#> bonferroni: (H3, H4)
#>
#> Test summary ($outputs) --------------------------------------------------------
#> Hypothesis Adj. P-value Reject
#> H1 0.02400 TRUE
#> H2 0.01871 TRUE
#> H3 0.10500 FALSE
#> H4 0.02400 TRUE
#>
#> Final updated graph after removing rejected hypotheses
#>
#> --- Hypothesis weights ---
#> H1: NA
#> H2: NA
#> H3: 1
#> H4: NA
#>
#> --- Transition weights ---
#> H1 H2 H3 H4
#> H1 NA NA NA NA
#> H2 NA NA NA NA
#> H3 NA NA 0 NA
#> H4 NA NA NA NA
#>
# Closed graphical multiple comparison procedure using parametric tests for
# H1 and H2, and Simes tests for H3 and H4
set.seed(1234)
graph_test_closure(
graph = g,
p = p,
alpha = alpha,
test_groups = list(group1 = 1:2, group2 = 3:4),
test_types = c(group1 = "parametric", group2 = "simes"),
test_corr = corr_list
)
#>
#> Test parameters ($inputs) ------------------------------------------------------
#> Initial graph
#>
#> --- Hypothesis weights ---
#> H1: 0.5
#> H2: 0.5
#> H3: 0.0
#> H4: 0.0
#>
#> --- Transition weights ---
#> H1 H2 H3 H4
#> H1 0.0 0.5 0.5 0.0
#> H2 0.5 0.0 0.0 0.5
#> H3 0.0 1.0 0.0 0.0
#> H4 1.0 0.0 0.0 0.0
#>
#> Alpha = 0.025
#>
#> H1 H2 H3 H4
#> Unadjusted p-values: 0.018 0.010 0.105 0.006
#>
#> Correlation matrix: H1 H2
#> H1 1.0 0.5
#> H2 0.5 1.0
#>
#> Test types
#> parametric: (H1, H2)
#> simes: (H3, H4)
#>
#> Test summary ($outputs) --------------------------------------------------------
#> Hypothesis Adj. P-value Reject
#> H1 0.02400 TRUE
#> H2 0.01871 TRUE
#> H3 0.10500 FALSE
#> H4 0.02400 TRUE
#>
#> Final updated graph after removing rejected hypotheses
#>
#> --- Hypothesis weights ---
#> H1: NA
#> H2: NA
#> H3: 1
#> H4: NA
#>
#> --- Transition weights ---
#> H1 H2 H3 H4
#> H1 NA NA NA NA
#> H2 NA NA NA NA
#> H3 NA NA 0 NA
#> H4 NA NA NA NA
#>