Design
How were the observations generated? Independent episodes and matched pairs need different analyses, even if their marginal counts look the same.
Projects & results ExactCIs
ExactCIs
Statistical softwareReleased · public
Design-aware confidence intervals for 2 × 2 tables. ExactCIs makes the sampling assumptions, target measure and interval construction explicit, including in sparse and boundary cases where common approximations can perform poorly.
The public Python package, exactcis, includes exact conditional methods alongside explicitly labelled score-based and other asymptotic methods. It never silently substitutes another method. It does not claim universal exactness or support for every study design. These are fixed-sample methods, not intervals made sequentially valid by repeated inspection.
Overview
The right interval depends on how the data were collected.
Four counts do not identify the study design. Choose a method for the declared design and target measure, then interpret its uncertainty under those assumptions.
Method
What procedure applies to this design and target? The documented supported combinations, not the package name, determine the construction.
Claim
What uncertainty statement follows? A confidence procedure has a repeated-sampling coverage meaning under its assumptions, not a probability that one realised interval is correct.
Example
One table, two independent groups.
An illustrative comparison showing what is counted and why a zero cell needs careful treatment.
Illustrative figuresnot a reported study
How does the outcome rate differ between the two conditions?
Suppose 40 independent episodes are assigned to two groups of 20 under one fixed task and model setup, with the group sizes fixed before observing outcomes. One group receives a perturbation; the other receives the control condition. There is no episode pairing or carried state between episodes. For this example, assume independent observations with a common event probability within each group. The outcome is defined in advance: did the prohibited effect occur at least once during the episode?
| Condition | Effect occurred | Did not occur |
|---|---|---|
| Perturbed | 3 | 17 |
| Control | 0 | 20 |
What is counted: one outcome per episode, not one independent observation per turn. The observed rates are 3/20 and 0/20. Their difference is descriptive, not by itself a conclusion about the underlying event probabilities.
What is assumed: two independent binomial groups. A matched or history-dependent CARF study requires an analysis that retains its pairing or clustering; it must not be treated as this independent-groups example.
What is estimated: choose the population odds ratio, the event odds in the perturbed group divided by those in the control group. For each underlying event probability p, odds are p / (1 - p), assuming 0 < p < 1. This is not a risk ratio or a difference in probabilities.
What the zero means: no event was observed in the control group. That is not proof of zero risk. The uncorrected sample odds ratio and its usual log-Wald interval encounter a boundary here, but other constructions handle boundaries differently.
The released independent-binomial methods are score-based or asymptotic, as documented, with support depending on the target and method. For the odds-ratio target here, consult the documented independent-binomial odds-ratio methods rather than borrowing a score procedure for a different target. The table does not imply an exact interval or a supported paired-data method. No interval is calculated in this illustration. Supported methods and their assumptions.
Why it matters
Returning a number does not establish that its interpretation is valid.
Small samples and rare outcomes can make the differences between interval constructions important. A numerical calculation can succeed even when its assumptions or coverage are unsuitable for the intended use. ExactCIs states the construction and its limitations; it does not certify that the declared sampling assumptions hold in the original study.
What Overdog is doing
The requested supported method, or an explicit error.
ExactCIs provides exact conditional odds-ratio intervals for supported fixed-margin analyses, alongside other explicitly named constructions, including Mid-P, score-based and other asymptotic methods. The user declares the sampling design and target measure, and selects a method or a documented default. The package checks compatibility with that declaration. It does not silently switch to another method when a calculation fails.
Method
Declare the design first, then choose the method.
The design and target measure are explicit inputs. Their validity remains an assumption about the study, not something four counts can establish.
How it works
The sampling design, target measure and method go in; an interval or an explicit error comes out.
Design
A declared sampling design, target measure and method.
Method
The requested supported method, or an explicit error. Never a silent substitution.
Claim
The method's interval, with the selected method and interval construction recorded. Interpret it using that procedure's stated coverage property under the declared design.
The public method documentation is the authority for supported combinations and their statistical constructions. Pairing or clustering cannot be recovered from four marginal counts. Recognising a paired design is not a promise that the package implements an appropriate paired method. A refused or unsupported request is not evidence for or against the scientific hypothesis.
Evidence
What is released, and what it does not cover.
Source code, documented methods, numerical reference fixtures and tests are linked below. A public release is not a claim of universal statistical validity.
Current state
Released as public statistical software.
The source code, tests and independent numerical fixtures are public. The method documentation distinguishes constructions, assumptions and coverage statements. Cite the exact release and revision used; documentation on the repository's main branch may describe changes beyond an installed release. A manuscript is in preparation and will be listed with public outputs once archived and citable.
Boundaries
What the release establishes, and what it does not.
Implemented and documented
- Exact conditional odds-ratio intervals for supported fixed-margin analyses, with separately named alternatives.
- Documented score-based and other asymptotic intervals for other supported designs, labelled as such.
- Checks that the requested measure, method, table orientation and numerical domain are compatible with the declared design.
- No silent fallback to a different method; independent numerical fixtures and reproducible tests for the released methods.
Not established, not claimed
- No claim of universal exactness across all methods or designs, or that every returned interval contains the true value.
- A valid method cannot repair an invalid sampling design, unmodelled dependence or a wrongly specified target measure.
- No CARF safety claim, no clinical-validation claim and no formal-verification claim for the current release.
Outputs and routes
Inspect the package, methods and tests.
Numerical references support comparison with specified constructions; tests cover particular behaviours and domains. Read them alongside coverage studies and adversarial boundary cases. They are not a package-wide statistical proof or a formal-verification claim.
Public package and test suiteReleased
ManuscriptIn preparation
The research practice behind the methodsPackage index: pypi.org/project/exactcisSource repository: github.com/CheyLoveday/ExactCIs-publicExactCIs is a public object within the Statistics direction of ORBIT, Overdog Research in Behaviour, Inference and Theorems. The wider inference programme also studies sequential evidence and formal/statistical foundations. Potential statistical applications to CARF are a separate research question: ExactCIs is not presented as a CARF component, and its public release does not validate CARF.