Levels of Rigor

Formally proven, automated proof checked mathematical theorems.

Conclusions follow from premises with absolute certainty in all possible universes. "a=b ∧ b=c → a=c"

Traditionally proven mathematical theorems.

"The monster group is the largest sporadic simple group"

Scientific laws: testable, repeatable, precise predictions validated by experiments by different organizations without counterexample.

"The gravitational force between two objects is ((m1×m2)/r^2) × 6.67430×10^−11 m^3/(kg s^2)"

Probabilistic findings: large, preregistered, randomized, triple blinded, controlled trials.

"Vaccinated individuals were 95% less likely to get infected with COVID-19 in a large RCT" ...and then not so much later

Cohort Studies. Non-randomized, preregistered controlled trials.

"We followed children who passed the marshmallow test in pre-K and compared outcomes with those in the same class who didn't."

1 Absolute Certainty 2 Certain 3 Practically Certain 4 Very Likely 5 Likely

The line of direct testing. Below here lies historical reconstruction; not repeatable, not re-observable, and not fully testable.

Computer forensic analysis.

"This system was compromised at 10:00 AM on May 2." ...unless the event log is forged or clock was wrong or...

Historical - primary written accounts, physical evidence

"Several general's memoirs document the order to attack" ...at least the manuscripts the victors allowed to survive do

Case-series (and poor quality cohort and case-control studies)

"We found four people with a lump on their heads and note they all played basketball"

Extrapolation, apples-to-oranges comparisons.

"In rabbits this works. Maybe in humans too."

Untestable theories. Opinion. Wishful thinking.

"I want to believe X but X seems extraordinarily unlikely, so imagine there are practically infinite universes and we are in the one that by chance..."

6 More likely than not 7 Believable 8 Plausible 9 Speculative 10 Imaginative