Rejection Rates

Get all rejection rates of all tests stored in a power4test object or other supported objects.

Usage

rejection_rates(object, ...)

# Default S3 method
rejection_rates(object, ...)

# S3 method for class 'power4test'
rejection_rates(
  object,
  all_columns = FALSE,
  ci = TRUE,
  level = 0.95,
  se = FALSE,
  collapse = c("none", "all_sig", "at_least_one_sig", "at_least_k_sig"),
  at_least_k = 1,
  ...
)

# S3 method for class 'power4test_by_es'
rejection_rates(
  object,
  all_columns = FALSE,
  ci = TRUE,
  level = 0.95,
  se = FALSE,
  ...
)

# S3 method for class 'power4test_by_n'
rejection_rates(
  object,
  all_columns = FALSE,
  ci = TRUE,
  level = 0.95,
  se = FALSE,
  ...
)

# S3 method for class 'rejection_rates_df'
print(x, digits = 3, annotation = TRUE, abbreviate_col_names = TRUE, ...)

Arguments

object: The object from which the rejection rates are to be computed, such as a power4test object, a power4test_by_n object, or a power4test_by_es object.
...: Optional arguments. For the print method, these arguments will be passed to the print method of data.frame objects print.data.frame(). Not used by other methods.
all_columns: If TRUE, all columns stored by a test will be extracted. Default is FALSE and only essential columns related to power will be printed.
ci: If TRUE, confidence intervals for the rejection rates (column reject or sig) will be computed. Normal approximation is used in forming the confidence intervals.
level: The level of confidence for the confidence intervals, if ci is TRUE. Default is .95, denoting 95%.
se: If TRUE, standard errors for the rejection rates (column reject or sig) will be computed. Normal approximation is used to compute the standard errors.
collapse: Whether a single decision (significant vs. not significant) is made across all tests for a test that consists of several tests (e.g., the tests of several parameters). If "none", tests will be summarized individually. If "all_sig", then the set of tests is considered significant if all individual tests are significant. If "at_least_one_sig", then the set of tests is considered significant if at least one of the tests is significant. If "at_least_k_sig", then the set of tests is considered significant if at least k tests are significant, k set by the argument at_least_k.
at_least_k: Used by collapse, the number of tests required to be significant for the set of tests to be considered significant.
x: The rejection_rates_df object to be printed.
digits: The number of digits to be printed after the decimal.
annotation: Logical. Whether additional notes will be printed.
abbreviate_col_names: Logical. Whether some column names will be abbreviated.

Value

The rejection_rates method returns a rejection_rates_df object, with a print method.

If the input (object) is a power4test object, the rejection_rates_df object is a data-frame like object with the number of rows equal to the number of tests. Note that some tests, such as the test by test_parameters(), conduct one test for each parameter. Each such test is counted as one test.

The data frame has at least these columns:

test: The name of the test.
label: The label for each test, or "Test" if a test only does one test (e.g., test_indirect_effect()).
pvalid: The proportion of valid tests across all replications.
reject: The rejection rate for each test. If the null hypothesis is false, then this is the power.

The rejection_rates method for power4test_by_es objects returns an object of the class rejection_rates_df_by_es, which is a subclass of rejection_rates_df. It is a data frame which is similar to the output of rejection_rates(), with two columns added for the effect size (pop_es_name and pop_es_values) for each test.

The rejection_rates method for power4test_by_n objects returns an object of the class rejection_rates_df_by_n, which is a subclass of rejection_rates_df. It is a data frame which is similar to the output of a power4test object, with a column n added for the sample size for each test.

The print method of a rejection_rates_df object returns the object invisibly. It is called for its side-effect.

Details

For a power4test object, rejection_rates loops over the tests stored in a power4test object and retrieves the rejection rate of each test.

The rejection_rates method for power4test_by_es objects is used to compute the rejection rates from a power4test_by_es object, with effect sizes added to the output.

The rejection_rates method for power4test_by_n objects is used to compute the rejection rates, with sample sizes added to the output.

Examples


# Specify the population model

model_simple_med <-
"
m ~ x
y ~ m + x
"

# Specify the effect sizes (population parameter values)

model_simple_med_es <-
"
y ~ m: l
m ~ x: m
y ~ x: n
"

# Generate some datasets to check the model

sim_only <- power4test(nrep = 4,
                       model = model_simple_med,
                       pop_es = model_simple_med_es,
                       n = 100,
                       R = 50,
                       ci_type = "boot",
                       fit_model_args = list(fit_function = "lm"),
                       do_the_test = FALSE,
                       iseed = 1234)
#> Simulate the data:
#> Fit the model(s):
#> Generate bootstrap estimates:

# Do the test 'test_indirect_effect' on each datasets

test_out <- power4test(object = sim_only,
                       test_fun = test_indirect_effect,
                       test_args = list(x = "x",
                                        m = "m",
                                        y = "y",
                                        boot_ci = TRUE,
                                        mc_ci = FALSE))
#> Do the test: test_indirect: x->m->y 

# Do the test 'test_parameters' on each datasets
# and add the results to 'test_out'

test_out <- power4test(object = test_out,
                       test_fun = test_parameters)
#> Do the test: test_parameters: CIs  

# Compute and print the rejection rates for stored tests

rejection_rates(test_out)
#>                     test test_label    est   p.v reject r.cilo r.cihi
#> 1 test_indirect: x->m->y       Test  0.149 1.000  1.000  1.000  1.000
#> 2  test_parameters: CIs         m~x  0.317 1.000  1.000  1.000  1.000
#> 3  test_parameters: CIs         y~m  0.475 1.000  1.000  1.000  1.000
#> 4  test_parameters: CIs         y~x -0.080 1.000  0.000  0.000  0.000
#> Notes:
#> - p.v: The proportion of valid replications.
#> - est: The mean of the estimates in a test across replications.
#> - reject: The proportion of 'significant' replications, that is, the
#>   rejection rate. If the null hypothesis is true, this is the Type I
#>   error rate. If the null hypothesis is false, this is the power.
#> - r.cilo,r.cihi: The confidence interval of the rejection rate, based
#>   on normal approximation.
#> - Refer to the tests for the meanings of other columns.

# See the help pages of power4test_by_n() and power4test_by_es()
# for other examples.

Usage

Arguments

Value

Details

See also

Examples