Appendix I — chapter3.analysis.statistical_tests
chapter3.analysis.statistical_tests
Provides functions to compute statistical tests to determine the significance of the obtained results.
Functions
| Name | Description |
|---|---|
| is_parametric_data | Determines if the results in the reports follow a parametric or a non-parametric distribution. |
| pairwise_n_comparision | Computes pairwise tests for each value of n. |
| statistical_comparison |
is_parametric_data
chapter3.analysis.statistical_tests.is_parametric_data(reports, models, sources)
Determines if the results in the reports follow a parametric or a non-parametric distribution.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
reports |
pandas.DataFrame |
Model reports. | required |
models |
list[libs.chapter3.model.Models] |
List with the models. | required |
sources |
list[libs.chapter3.model.Source] |
List with the data sources. | required |
Returns
| Type | Description |
|---|---|
pandas.DataFrame |
DataFrame indicating if the results from a model+source are parametric (True) or not (False). |
pairwise_n_comparision
chapter3.analysis.statistical_tests.pairwise_n_comparision(data, filters, alternative='two-sided', stars=False, parametric=False)
Computes pairwise tests for each value of n.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
data |
pandas.DataFrame |
Model reports. | required |
filters |
str | Filter to apply to the model reports. See: libs.chapter3.model.Filter |
required |
alternative |
str | Hypothesis to test. One of: ‘two-sided’, ‘less’ or ‘greater’. | 'two-sided' |
stars |
boolean | Replace p-values under 0.05 by stars. ‘’ when 0.01<p-value<0.05; ’’ when 0.001<p-value<0.01; ’’ when p-value<0.001; | False |
parametric |
boolean | Compute parametric or non-parametric tests. | False |
Returns
| Type | Description |
|---|---|
pandas.DataFrame |
DataFrame containing the pairwise tests. |
statistical_comparison
chapter3.analysis.statistical_tests.statistical_comparison(reports, metric_filter, focus_on, groups, alternative='two-sided')
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
reports |
pandas.DataFrame |
Model reports. | required |
metric_filter |
tuple[str, str] | Metric filter to apply to the model reports. | required |
focus_on |
list[str] | Items being compared. List items are one of libs.chapter3.model.Source or libs.chapter3.model.Model. |
required |
groups |
list[str] | Each group where focus_on items are compared. List items are one of libs.chapter3.model.Source or libs.chapter3.model.Model. |
required |
alternative |
str | Hypothesis to test. One of: ‘two-sided’, ‘less’ or ‘greater’. | 'two-sided' |
Returns
| Type | Description |
|---|---|
pandas.DataFrame |
DataFrame containing groups comparisons. |