To improve the readability of the code, we provide the glossary to serve as an educational document to grow people’s understanding of the graphical approach to multiple comparison procedures. Most terms are inputs or outputs of exported functions and some are used only in the internal code. If any definition could be clarified or improved, please submit an issue to the GitHub repository.
Entity | Definition | Aliases | Variable(s) |
---|---|---|---|
Graph | A set of nodes and directed edges representing a graphical multiple comparison procedure. Each node correspond to hypothesis and each edge corresponds to transition. | graph |
|
|
Remove a hypothesis from a graph, and update the graph according to algorithm 1 of Bretz et al. (2011). This is an operation on a graph. | updated_graph |
|
|
Under a given graph, testing
strategy, and alpha , a hypothesis is rejected if
its p-value is sufficiently small, which is determined
by the graphical multiple comparison procedure. |
||
Hypothesis | A node with weight in a graph. Each node represents
a null hypothesis, associated with a hypothesis
weights. The corresponding significance level is the
weight times alpha . |
weight, hypothesis weight | hypotheses |
Terms associated with hypothesis get their variable names: hypothesis name, and number of hypotheses. |
hyp_names , num_hyps
|
||
Transition | A directed edge with weight in a graph. Each edge defines the proportion of the hypothesis weight to be propagated from the origin hypothesis to the end hypothesis, when the origin is rejected. | edge, transition weight | transitions |
Intersection hypothesis | An intersection hypothesis is an intersection of multiple null hypotheses, which means that all associated null hypotheses are true. Plural often implies all intersections of all subsets of hypotheses. | intersection, sub-graph(s), closure | intersections |
Weighting strategy | The set of all intersections and their hypothesis weights according to Algorithm 1 in Bretz et al. (2011). | intersection weights, closure weights | weighting_strategy |
Adjusted weight |
The hypothesis weight, adjusted according to a multiple comparison procedure:
|
adjusted_weights |
|
P-value | A p-value before multiplicity adjustment. Also could be called as an unadjusted p-value or a raw p-value. | p |
|
Ordered p-value | A p-value sorted from the smallest to the largest. They are mainly used to perform Simes tests. | ordered_p |
|
Adjusted p-value | A p-value that has been adjusted according to a
multiple comparison procedure. A hypothesis may be rejected if its
adjusted p-value is less than or equal to
alpha . |
adjusted_p |
|
Significance level | A threshold chosen to test a null hypothesis, which may be rejected
if its p-value is less than or equal to its
significance level. The overall significance level to
test all hypotheses is alpha . |
alpha |
|
Test type | A specification of which testing type to use for an intersection hypothesis - Bonferroni, Simes, and parametric are currently supported. | tests | test_types |
Test group | A partition of all null hypotheses in a graph specifying which hypotheses should be tested together using a test type. | groups |
groups , test_groups
|
Testing strategy | Test types and test groups combined with a graph. | multiple comparison procedure | |
Marginal power | The power to reject a null hypothesis at the
significance level alpha (without
multiplicity adjustment). |
marginal_power |
|
|
Specification of correlations between test statistics for
hypotheses. The correlation for testing
test_corr is used to perform parametric tests. The
correlation for simulation test_sim is used to simulate
p-values from the alternative hypotheses for to assess
power, under assumptions. |
corr , test_corr ,
sim_corr
|
|
Power | With a given graph, testing
strategy, alpha , and the underlying distribution
of test statistics under the alternative hypotheses, the estimated
likelihood that a particular success criterion is
met. |
probability of achieving success criterion | power_* |
|
Specification of the success criterion, which could be a combination of null hypotheses. Being success means that the combination of null hypotheses has been rejected. | sim_success |
Bretz, Frank, Martin Posch, Ekkehard Glimm, Florian Klinglmueller, Willi
Maurer, and Kornelius Rohmeyer. 2011. “Graphical Approaches for
Multiple Comparison Procedures Using Weighted Bonferroni, Simes, or
Parametric Tests.” Biometrical Journal 53 (6): 894–913.
https://onlinelibrary.wiley.com/doi/10.1002/bimj.201000239.