Formalization: A Minimax Bracket for Average Treatment Effect Estimation with Discrete Adjustment and Bounded Heterogeneity
The complete Lean development behind this paper — every definition, lemma, and theorem of its module, including helpers the paper text never cites. Identifiers link within this page, into the Causalean library, or out to the official Mathlib docs.
Basic 51 declarations
One observed record (X,A,Y) in the real-outcome experiment.
instance instMeasurableSpaceObs
Observed records carry the measurable structure generated by their covariate, treatment, and outcome coordinates.
Definition (Lean source)
instance {d : ℕ} : MeasurableSpace (Obs d) :=
comap (fun o : Obs d => (o.x, o.a, o.y)) inferInstance
One full-data record (X,A,Y(0),Y(1),Y).
instance instMeasurableSpaceFullObs
Full-data records carry the measurable structure generated by the covariate, treatment, both potential outcomes, and the observed outcome.
Definition (Lean source)
instance {d : ℕ} : MeasurableSpace (FullObs d) :=
comap
(fun z : FullObs d => (z.x, z.a, z.y0, z.y1, z.y)) inferInstance
def observed
Projection from full data to the observed record.
Definition (Lean source)
def FullObs.observed {d : ℕ} (z : FullObs d) : Obs d :=
⟨z.x, z.a, z.y⟩ -- @realizes O(O=(X,A,Y))
def realMass
Real mass of an event under a measure.
Definition (Lean source)
noncomputable def realMass {α : Type*} [MeasurableSpace α]
(μ : Measure α) (s : Set α) : ℝ := (μ s).toReal
A real-outcome finite-cell law, including the primitive cell parametrization and a full-data coupling. The identities below pin every auxiliary to the law.
Definition (Lean source)
parameters
d :
ℕ
fields
observedLaw :
Measure (Obs d)
observed_isProbability :
IsProbabilityMeasure observedLaw
full_isProbability :
IsProbabilityMeasure fullLaw
cellMass :
Fin d → ℝ
propensity :
Fin d → ℝ
@realizes mu_{ak}(arm-cell outcome law carrier) @realizes a(Bool index)
outcome_isProbability :
∀ a k, IsProbabilityMeasure (outcomeLaw a k)
@realizes mu_{ak}(conditional outcome law is a probability law)
observed_margin :
Measure.map FullObs.observed fullLaw = observedLaw
@realizes P_O(observed margin of P)
cellMass_eq :
∀ k, cellMass k = realMass observedLaw {o | o.x = k}
@realizes p_k(p_k=P(X=k))
cellMass_range :
∀ k, cellMass k ∈ Icc (0 : ℝ) 1
@realizes p_k(range [0,1])
propensity_range :
∀ k, propensity k ∈ Icc (0 : ℝ) 1
@realizes pi_k(range [0,1])
arm_outcome_factorization :
∀ a k s
if
@realizes pi_k(P(A=1|X=k))
outcomeMean_eq :
∀ a k, outcomeMean a k = ∫ y, y ∂outcomeLaw a k
def cellEffect
Cell treatment effect tau_k = mu_1k - mu_0k.
Definition (Lean source)
def cellEffect {d : ℕ} (P : RealLaw d) (k : Fin d) : ℝ :=
P.outcomeMean true k - P.outcomeMean false k
def rawAteFormula
The finite-cell g-formula attached to an arbitrary real law. This auxiliary is used while the identified model classes are being assembled.
Definition (Lean source)
noncomputable def rawAteFormula {d : ℕ} (P : RealLaw d) : ℝ :=
∑ k : Fin d, P.cellMass k * cellEffect P k
def causalATE
The causal full-data average E[Y(1)-Y(0)].
Definition (Lean source)
noncomputable def causalATE {d : ℕ} (P : RealLaw d) : ℝ :=
∫ z, z.y1 - z.y0 ∂P.fullLaw
def cellDeviation
Cell deviation from the average treatment effect.
Definition (Lean source)
noncomputable def cellDeviation {d : ℕ} (P : RealLaw d) (k : Fin d) : ℝ :=
cellEffect P k - rawAteFormula P
Standing range condition for the known outcome scale.
Definition (Lean source)
def OutcomeScale (M : ℝ) : Prop :=
1 ≤ M
def Consistency
Y = Y(A) almost surely under the full-data law.
Definition (Lean source)
def Consistency {d : ℕ} (P : RealLaw d) : Prop :=
P.fullLaw {z | z.y ≠ if z.a then z.y1 else z.y0} = 0
def ConditionalExchangeability
Atomic-cell formulation of (Y(0),Y(1)) conditionally independent of A given X, valid for arbitrary measurable potential-outcome events.
Definition (Lean source)
def ConditionalExchangeability {d : ℕ} (P : RealLaw d) : Prop :=
∀ k a s0 s1, MeasurableSet s0 → MeasurableSet s1 →
P.fullLaw {z | z.x = k ∧ z.a = a ∧ z.y0 ∈ s0 ∧ z.y1 ∈ s1} *
P.fullLaw {z | z.x = k} =
P.fullLaw {z | z.x = k ∧ z.a = a} *
P.fullLaw {z | z.x = k ∧ z.y0 ∈ s0 ∧ z.y1 ∈ s1}
def Overlap
Positive-mass cells have propensity in [epsilon,1-epsilon].
def MeanNormalization
Conditional means lie in [-M/2,M/2] on positive-mass cells.
Definition (Lean source)
def MeanNormalization {d : ℕ} (M : ℝ) (P : RealLaw d) : Prop :=
∀ a k, 0 < P.cellMass k → |P.outcomeMean a k| ≤ M / 2
def SecondCentralMoment
The conditional second central moment is bounded by M^2; outcomes themselves need not be bounded.
Definition (Lean source)
def SecondCentralMoment {d : ℕ} (M : ℝ) (P : RealLaw d) : Prop :=
∀ a k, 0 < P.cellMass k →
Integrable (fun y => (y - P.outcomeMean a k) ^ 2) (P.outcomeLaw a k) ∧
∫ y, (y - P.outcomeMean a k) ^ 2 ∂P.outcomeLaw a k ≤ M ^ 2
theorem outcome_integrable_of_second_moment Lemma outcome_integrable_of_second_moment in the paper ↗
If conditional second moments are finite and bounded and the stated condition on the cell holds, a finite conditional second central moment implies the first-moment integrability needed by the finite-cell g-formula.
Formal statement
hypotheses
conclusion
Integrable (fun y : ℝ => y) (P.outcomeLaw a k)
Proof (Lean source)
lemma outcome_integrable_of_second_moment {d : ℕ} {M : ℝ}
(P : RealLaw d) (hsecond : SecondCentralMoment M P)
(a : Bool) (k : Fin d) (hk : 0 < P.cellMass k) :
Integrable (fun y : ℝ => y) (P.outcomeLaw a k) := by
let _ : IsProbabilityMeasure (P.outcomeLaw a k) :=
P.outcome_isProbability a k
have hsq := (hsecond a k hk).1
have hcenter_meas : AEStronglyMeasurable
(fun y : ℝ => y - P.outcomeMean a k) (P.outcomeLaw a k) :=
(measurable_id.sub measurable_const).aestronglyMeasurable
have hcenterLp : MemLp (fun y : ℝ => y - P.outcomeMean a k) 2
(P.outcomeLaw a k) :=
(memLp_two_iff_integrable_sq hcenter_meas).2 hsq
have hcenter : Integrable (fun y : ℝ => y - P.outcomeMean a k)
(P.outcomeLaw a k) := hcenterLp.integrable (by norm_num)
refine (hcenter.add (integrable_const (P.outcomeMean a k))).congr ?_
filter_upwards with y
simp only [Pi.add_apply]
ring
def ApproximateHomogeneity
Every supported cell effect is within sigma*M of the ATE.
Definition (Lean source)
def ApproximateHomogeneity {d : ℕ} (M sigma : ℝ) (P : RealLaw d) : Prop :=
∀ k, 0 < P.cellMass k → |cellDeviation P k| ≤ sigma * M
The causal-identification domain for the finite-cell functional. It records exactly consistency, conditional exchangeability, overlap, and the supported-cell first-moment integrability presupposed by the g-formula.
Definition (Lean source)
parameters
fields
epsilon_pos :
0 < epsilon
epsilon_lt_half :
epsilon < 1 / 2
consistency :
Consistency law
exchangeability :
overlap :
Overlap epsilon law
outcome_integrable :
∀ a k
if
0 < law.cellMass k
then
Integrable (fun y : ℝ => y) (law.outcomeLaw a k)
def ateFunctional
The identified ATE functional on the causal model domain. Its value is the finite-cell g-formula; ate_identification proves that this equals the causal expectation E[Y(1)-Y(0)].
Definition (Lean source)
noncomputable def ateFunctional {d : ℕ} {epsilon : ℝ}
(P : RealLaw d) (_hP : CausalIdentificationDomain d epsilon P) : ℝ :=
rawAteFormula P
structure ModelClass
The approximately homogeneous real-outcome model class.
Definition (Lean source)
parameters
d :
ℕ
epsilon M sigma :
ℝ
fields
law :
RealLaw d
epsilon_pos :
0 < epsilon
epsilon_lt_half :
epsilon < 1 / 2
M_ge_one :
1 ≤ M
sigma_nonneg :
0 ≤ sigma
sigma_le_two :
sigma ≤ 2
consistency :
Consistency law
exchangeability :
overlap :
Overlap epsilon law
mean_normalization :
MeanNormalization M law
second_moment :
SecondCentralMoment M law
homogeneity :
ApproximateHomogeneity M sigma law
structure UnrestrictedClass
The corresponding model without a heterogeneity-radius restriction.
Definition (Lean source)
parameters
d :
ℕ
epsilon M :
ℝ
fields
law :
RealLaw d
epsilon_pos :
0 < epsilon
epsilon_lt_half :
epsilon < 1 / 2
M_ge_one :
1 ≤ M
consistency :
Consistency law
exchangeability :
overlap :
Overlap epsilon law
mean_normalization :
MeanNormalization M law
second_moment :
SecondCentralMoment M law
theorem identifiedLaw
The identified-law view carried by every approximately homogeneous model.
Formal statement
hypotheses
conclusion
CausalIdentificationDomain d epsilon P.law
Proof (Lean source)
theorem ModelClass.identifiedLaw {d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) :
CausalIdentificationDomain d epsilon P.law :=
{ epsilon_pos := P.epsilon_pos
epsilon_lt_half := P.epsilon_lt_half
consistency := P.consistency
exchangeability := P.exchangeability
overlap := P.overlap
outcome_integrable :=
outcome_integrable_of_second_moment P.law P.second_moment }
theorem identifiedLaw
The identified-law view carried by every unrestricted model.
Formal statement
hypotheses
conclusion
CausalIdentificationDomain d epsilon P.law
Proof (Lean source)
theorem UnrestrictedClass.identifiedLaw {d : ℕ} {epsilon M : ℝ}
(P : UnrestrictedClass d epsilon M) :
CausalIdentificationDomain d epsilon P.law :=
{ epsilon_pos := P.epsilon_pos
epsilon_lt_half := P.epsilon_lt_half
consistency := P.consistency
exchangeability := P.exchangeability
overlap := P.overlap
outcome_integrable :=
outcome_integrable_of_second_moment P.law P.second_moment }
def productLaw
The n-fold observed product law generated by a one-unit real law.
instance instIsProbabilityMeasureForallFinObsProductLaw
Every finite product of the observed-data law is a probability measure.
Definition (Lean source)
instance {d n : ℕ} (P : RealLaw d) : IsProbabilityMeasure (productLaw n P) := by
unfold productLaw
infer_instance
def sampleExperiment
The induced experiment of all observed n-sample laws from the model class.
Definition (Lean source)
noncomputable def sampleExperiment (n d : ℕ) (epsilon M sigma : ℝ) :
Set (Measure (Fin n → Obs d)) :=
{Q | ∃ P : ModelClass d epsilon M sigma, Q = productLaw n P.law}
def Estimator
Total estimators with measurable output clipped to [-M,M].
Definition (Lean source)
def Estimator (n d : ℕ) (M : ℝ) :=
{f : (Fin n → Obs d) → ℝ // Measurable f ∧ ∀ x, f x ∈ Icc (-M) M}
Mean-squared error under the model's product sample law.
Definition (Lean source)
noncomputable def mse {n d : ℕ} (P : RealLaw d)
(est : (Fin n → Obs d) → ℝ) : ℝ :=
∫ x, (est x - rawAteFormula P) ^ 2 ∂productLaw n P
def worstCaseMSE
Worst-case risk of an estimator over the specified model class.
Definition (Lean source)
noncomputable def worstCaseMSE (n d : ℕ) (epsilon M sigma : ℝ)
(est : (Fin n → Obs d) → ℝ) : ℝ :=
⨆ P : ModelClass d epsilon M sigma, mse P.law est
Minimax MSE over all total measurable [-M,M]-valued estimators.
Definition (Lean source)
noncomputable def minimaxRisk (n d : ℕ) (epsilon M sigma : ℝ) : ℝ :=
⨅ est : Estimator n d M, worstCaseMSE n d epsilon M sigma est.1
The logarithmic scale log(e n).
Polynomial-estimator component d^2/(n^2 log(en)^2).
Definition (Lean source)
noncomputable def polynomialComponent (n d : ℕ) : ℝ :=
(d : ℝ) ^ 2 / ((n : ℝ) ^ 2 * logEN n ^ 2)
def collisionComponent
Collision-estimator component sigma^2+d/n^2.
Definition (Lean source)
noncomputable def collisionComponent (n d : ℕ) (sigma : ℝ) : ℝ :=
sigma ^ 2 + (d : ℝ) / (n : ℝ) ^ 2
The paper's upper frontier rate.
Definition (Lean source)
noncomputable def frontierRate (n d : ℕ) (sigma : ℝ) : ℝ :=
1 / (n : ℝ) + min 1 (min (polynomialComponent n d) (collisionComponent n d sigma))
theorem frontierRate_pos
If the sample is nonempty, on its declared positive-sample domain, the frontier rate is positive.
Formal statement
Proof (Lean source)
lemma frontierRate_pos {n d : ℕ} {sigma : ℝ} (hn : 0 < n) :
0 < frontierRate n d sigma := by
have hn_real : 0 < (n : ℝ) := by exact_mod_cast hn
have hpoly : 0 ≤ polynomialComponent n d := by
unfold polynomialComponent
positivity
have hcollision : 0 ≤ collisionComponent n d sigma := by
unfold collisionComponent
positivity
unfold frontierRate
have hminimum : 0 ≤ min 1
(min (polynomialComponent n d) (collisionComponent n d sigma)) :=
le_min zero_le_one (le_min hpoly hcollision)
positivity
The proved capped converse benchmark.
Definition (Lean source)
noncomputable def converseRate (n d : ℕ) (sigma : ℝ) : ℝ :=
1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2) +
sigma ^ 2 * min 1 (polynomialComponent n d)
def armCount
Number of observations in arm a and cell k.
def cellCount
Total number of observations in cell k.
def collisionDenominatorKernel
Pairwise collision-denominator kernel: the two records share a cell and belong to opposite treatment arms.
Definition (Lean source)
def collisionDenominatorKernel {n d : ℕ} (sample : Fin n → Obs d)
(i j : Fin n) : Bool :=
decide ((sample i).x = (sample j).x ∧ (sample i).a ≠ (sample j).a)
def usableCell
Whether both treatment arms occur in a cell.
def usableTotal
Total occupancy among cells containing both treatment arms.
Definition (Lean source)
def usableTotal {n d : ℕ} (sample : Fin n → Obs d) : ℕ :=
∑ k : Fin d, if usableCell sample k then cellCount sample k else 0
def armSum
Sum of observed outcomes in arm a and cell k.
def empiricalArmMean
Indicator-totalized empirical arm mean.
def collisionWeight
Population collision weight for cell k.
Definition (Lean source)
def collisionWeight {d : ℕ} (P : RealLaw d) (k : Fin d) : ℝ :=
2 * P.cellMass k ^ 2 * P.propensity k * (1 - P.propensity k)
def totalCollisionWeight
Total collision weight on the positive-overlap model.
Definition (Lean source)
noncomputable def totalCollisionWeight {d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) : ℝ :=
∑ k : Fin d, collisionWeight P.law k
theorem sum_cellMass_eq_one
The finite collection of cell masses partitions the observed probability law.
Proof (Lean source)
-- @node: sum_cellMass_eq_one
lemma sum_cellMass_eq_one {d : ℕ} (P : RealLaw d) :
∑ k : Fin d, P.cellMass k = 1 := by
classical
have htuple : Measurable (fun o : Obs d => (o.x, o.a, o.y)) := by
rw [measurable_iff_comap_le]
rfl
have hx : Measurable (fun o : Obs d => o.x) := measurable_fst.comp htuple
have hsum := MeasureTheory.sum_measureReal_preimage_singleton
(μ := P.observedLaw) (Finset.univ : Finset (Fin d))
(f := fun o : Obs d => o.x)
(hf := by
intro k hk
exact (measurableSet_singleton k).preimage hx)
(h := by
intro k hk
exact measure_ne_top P.observedLaw _)
rw [Finset.sum_congr rfl (fun k _ => P.cellMass_eq k)]
convert hsum using 1 <;> simp [Measure.real, realMass, preimage]
theorem totalCollisionWeight_pos
If the alphabet is nonempty, a nonempty positive-overlap model has strictly positive collision weight.
Formal statement
hypotheses
conclusion
0 < totalCollisionWeight P
Proof (Lean source)
-- @node: totalCollisionWeight_pos
lemma totalCollisionWeight_pos {d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (_hd : 0 < d) :
0 < totalCollisionWeight P := by
classical
have hmass_nonneg (k : Fin d) : 0 ≤ P.law.cellMass k :=
(P.law.cellMass_range k).1
obtain ⟨k, hkpos⟩ : ∃ k : Fin d, 0 < P.law.cellMass k := by
by_contra h
simp only [not_exists, not_lt] at h
have hzero : ∀ k : Fin d, P.law.cellMass k = 0 := by
intro k
exact le_antisymm (h k) (hmass_nonneg k)
have := sum_cellMass_eq_one P.law
simp [hzero] at this
obtain ⟨hpi_lower, hpi_upper⟩ := P.overlap k hkpos
have hpi_pos : 0 < P.law.propensity k :=
lt_of_lt_of_le P.epsilon_pos hpi_lower
have hone_sub_pi_pos : 0 < 1 - P.law.propensity k := by
linarith [P.epsilon_pos]
have hweight_pos : 0 < collisionWeight P.law k := by
unfold collisionWeight
exact mul_pos
(mul_pos (mul_pos (by norm_num) (sq_pos_of_pos hkpos)) hpi_pos)
hone_sub_pi_pos
have hweight_nonneg (l : Fin d) : 0 ≤ collisionWeight P.law l := by
unfold collisionWeight
obtain ⟨hpi0, hpi1⟩ := P.law.propensity_range l
exact mul_nonneg
(mul_nonneg (mul_nonneg (by norm_num) (sq_nonneg _)) hpi0)
(sub_nonneg.mpr hpi1)
unfold totalCollisionWeight
exact Finset.sum_pos' (fun l _ => hweight_nonneg l)
⟨k, Finset.mem_univ k, hweight_pos⟩
theorem sum_cellMass_le_one
The primitive cell masses of a real law sum to at most one.
Proof (Lean source)
lemma sum_cellMass_le_one {d : ℕ} (P : RealLaw d) :
∑ k : Fin d, P.cellMass k ≤ 1 := by
have htuple : Measurable (fun o : Obs d => (o.x, o.a, o.y)) := by
rw [measurable_iff_comap_le]
rfl
have hx : Measurable (fun o : Obs d => o.x) := measurable_fst.comp htuple
have hmeas : ∀ k : Fin d, MeasurableSet {o : Obs d | o.x = k} := by
intro k
exact (measurableSet_singleton k).preimage hx
have hdisj : (↑(Finset.univ : Finset (Fin d)) : Set (Fin d)).Pairwise
(onFun (AEDisjoint P.observedLaw) (fun k => {o : Obs d | o.x = k})) := by
intro i hi j hj hij
apply Disjoint.aedisjoint
exact Set.disjoint_left.2 (by
intro o hoi hoj
exact hij (hoi.symm.trans hoj))
have hsum_enn :
(∑ k : Fin d, P.observedLaw {o : Obs d | o.x = k}) ≤ P.observedLaw univ :=
sum_measure_le_measure_univ (fun k _ => (hmeas k).nullMeasurableSet) hdisj
rw [measure_univ] at hsum_enn
have hfinite : ∀ k ∈ (Finset.univ : Finset (Fin d)),
P.observedLaw {o : Obs d | o.x = k} ≠ ∞ := by
intro k hk
exact measure_ne_top P.observedLaw _
rw [Finset.sum_congr rfl (fun k _ => P.cellMass_eq k)]
change (∑ k : Fin d, (P.observedLaw {o : Obs d | o.x = k}).toReal) ≤ 1
rw [← ENNReal.toReal_sum hfinite]
rw [← ENNReal.toReal_one]
exact ENNReal.toReal_mono ENNReal.one_ne_top hsum_enn
Mean normalization bounds every cell effect and the ATE, so radius 2 is equivalent to the unrestricted class.
Formal statement
hypotheses
d :
ℕ
epsilon M :
ℝ
conclusion
(∀ P : UnrestrictedClass d epsilon M, (∀ k, 0 < P.law.cellMass k → |cellEffect P.law k| ≤ M) ∧ |rawAteFormula P.law| ≤ M ∧ (∀ k, 0 < P.law.cellMass k → |cellDeviation P.law k| ≤ 2 * M)) ∧
(∀ P : UnrestrictedClass d epsilon M, ∃ Q : ModelClass d epsilon M 2, Q.law = P.law) ∧
(∀ Q : ModelClass d epsilon M 2, ∃ P : UnrestrictedClass d epsilon M, P.law = Q.law)
Proof (Lean source)
lemma scale_sanity {d : ℕ} {epsilon M : ℝ} :
(∀ P : UnrestrictedClass d epsilon M,
(∀ k, 0 < P.law.cellMass k → |cellEffect P.law k| ≤ M) ∧
|rawAteFormula P.law| ≤ M ∧
(∀ k, 0 < P.law.cellMass k → |cellDeviation P.law k| ≤ 2 * M)) ∧
-- @realizes delta_k(range [-2M,2M] on positive-mass cells)
(∀ P : UnrestrictedClass d epsilon M,
∃ Q : ModelClass d epsilon M 2, Q.law = P.law) ∧
(∀ Q : ModelClass d epsilon M 2,
∃ P : UnrestrictedClass d epsilon M, P.law = Q.law) := by
have bounds : ∀ P : UnrestrictedClass d epsilon M,
(∀ k, 0 < P.law.cellMass k → |cellEffect P.law k| ≤ M) ∧
|rawAteFormula P.law| ≤ M ∧
(∀ k, 0 < P.law.cellMass k → |cellDeviation P.law k| ≤ 2 * M) := by
intro P
have hM : 0 ≤ M := le_trans zero_le_one P.M_ge_one
have hcell : ∀ k, 0 < P.law.cellMass k → |cellEffect P.law k| ≤ M := by
intro k hk
unfold cellEffect
calc
|P.law.outcomeMean true k - P.law.outcomeMean false k| ≤
|P.law.outcomeMean true k| + |P.law.outcomeMean false k| := abs_sub _ _
_ ≤ M / 2 + M / 2 := add_le_add
(P.mean_normalization true k hk) (P.mean_normalization false k hk)
_ = M := by ring
have hterm : ∀ k : Fin d,
|P.law.cellMass k * cellEffect P.law k| ≤ P.law.cellMass k * M := by
intro k
rcases eq_or_lt_of_le (P.law.cellMass_range k).1 with hk | hk
· simp [← hk]
· rw [abs_mul, abs_of_nonneg (le_of_lt hk)]
exact mul_le_mul_of_nonneg_left (hcell k hk) (le_of_lt hk)
have hate : |rawAteFormula P.law| ≤ M := by
unfold rawAteFormula
calc
|∑ k : Fin d, P.law.cellMass k * cellEffect P.law k| ≤
∑ k : Fin d, |P.law.cellMass k * cellEffect P.law k| :=
Finset.abs_sum_le_sum_abs _ _
_ ≤ ∑ k : Fin d, P.law.cellMass k * M :=
Finset.sum_le_sum (fun k _ => hterm k)
_ = (∑ k : Fin d, P.law.cellMass k) * M := by rw [Finset.sum_mul]
_ ≤ 1 * M := mul_le_mul_of_nonneg_right (sum_cellMass_le_one P.law) hM
_ = M := one_mul M
refine ⟨hcell, hate, ?_⟩
intro k hk
unfold cellDeviation
calc
|cellEffect P.law k - rawAteFormula P.law| ≤
|cellEffect P.law k| + |rawAteFormula P.law| := abs_sub _ _
_ ≤ M + M := add_le_add (hcell k hk) hate
_ = 2 * M := by ring
refine ⟨bounds, ?_, ?_⟩
· intro P
refine ⟨{
law := P.law
epsilon_pos := P.epsilon_pos
epsilon_lt_half := P.epsilon_lt_half
M_ge_one := P.M_ge_one
sigma_nonneg := by norm_num
sigma_le_two := by norm_num
consistency := P.consistency
exchangeability := P.exchangeability
overlap := P.overlap
mean_normalization := P.mean_normalization
second_moment := P.second_moment
homogeneity := ?_ }, rfl⟩
exact (bounds P).2.2
· intro Q
exact ⟨{
law := Q.law
epsilon_pos := Q.epsilon_pos
epsilon_lt_half := Q.epsilon_lt_half
M_ge_one := Q.M_ge_one
consistency := Q.consistency
exchangeability := Q.exchangeability
overlap := Q.overlap
mean_normalization := Q.mean_normalization
second_moment := Q.second_moment }, rfl⟩
Helpers.AffineEmbedding 21 declarations
A binary full-data record used to state law-level source couplings.
Definition (Lean source)
structure BinaryFullObs (d : ℕ) where
x : Fin d
a : Bool
b0 : Bool
b1 : Bool
deriving Fintype, DecidableEq
instance instDecidableEqBinaryFullObs
Equality of binary full-data records is decidable.
Definition (Lean source)
deriving Fintype, DecidableEq
instance instFintypeBinaryFullObs
The space of binary full-data records is finite.
Definition (Lean source)
deriving Fintype, DecidableEq
instance instMeasurableSpaceBinaryFullObs
Binary full-data records carry the discrete measurable structure.
Definition (Lean source)
instance {d : ℕ} : MeasurableSpace (BinaryFullObs d) := ⊤
def observed
The observed binary record selected from a binary full-data record.
Definition (Lean source)
def BinaryFullObs.observed {d : ℕ} (z : BinaryFullObs d) :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.Obs d :=
(z.x, z.a, if z.a then z.b1 else z.b0)
def affine
Deterministic affine scaling of both binary potential outcomes.
Definition (Lean source)
noncomputable def BinaryFullObs.affine {d e : ℕ} (M : ℝ) (pad : Fin d → Fin e)
(z : BinaryFullObs d) : FullObs e :=
let scale (b : Bool) := M * ((if b then 1 else 0) - 1 / 2)
⟨pad z.x, z.a, scale z.b0, scale z.b1,
if z.a then scale z.b1 else scale z.b0⟩
A binary potential-outcome coupling has the prescribed observed source law.
Definition (Lean source)
def BinaryFullCoupling {d : ℕ}
(P : CausalSmith.Stat.DiscreteAteMinimaxLoggap.DiscreteLaw d)
(R : Measure (BinaryFullObs d)) : Prop :=
IsProbabilityMeasure R ∧
Measure.map BinaryFullObs.observed R =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P
def binaryFullLift
Lift an observed binary record to consistent binary full data by using a fixed value for the unobserved potential outcome.
Definition (Lean source)
def binaryFullLift {d : ℕ}
(z : CausalSmith.Stat.DiscreteAteMinimaxLoggap.Obs d) : BinaryFullObs d :=
if z.2.1 then ⟨z.1, true, false, z.2.2⟩
else ⟨z.1, false, z.2.2, false⟩
theorem binaryFullLift_observed
Observing the canonical full-data lift recovers the source record.
Formal statement
hypotheses
d :
ℕ
z :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.Obs d
conclusion
(binaryFullLift z).observed = z
Proof (Lean source)
lemma binaryFullLift_observed {d : ℕ}
(z : CausalSmith.Stat.DiscreteAteMinimaxLoggap.Obs d) :
(binaryFullLift z).observed = z := by
cases z with
| mk x rest =>
cases rest with
| mk a b => cases a <;> rfl
def canonicalBinaryFullCoupling
The pushforward of a binary observation law by the canonical lift is a probability coupling with exactly the prescribed observed margin.
Definition (Lean source)
noncomputable def canonicalBinaryFullCoupling {d : ℕ}
(P : CausalSmith.Stat.DiscreteAteMinimaxLoggap.DiscreteLaw d) :
Measure (BinaryFullObs d) :=
Measure.map binaryFullLift
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P)
theorem canonicalBinaryFullCoupling_spec
The canonical lifted law satisfies the binary coupling interface.
Formal statement
hypotheses
d :
ℕ
P :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.DiscreteLaw d
conclusion
Proof (Lean source)
lemma canonicalBinaryFullCoupling_spec {d : ℕ}
(P : CausalSmith.Stat.DiscreteAteMinimaxLoggap.DiscreteLaw d) :
BinaryFullCoupling P (canonicalBinaryFullCoupling P) := by
constructor
· exact Measure.isProbabilityMeasure_map (by fun_prop)
· unfold canonicalBinaryFullCoupling
rw [Measure.map_map (by fun_prop) (by fun_prop)]
convert Measure.map_id
funext z
exact binaryFullLift_observed z
def affineObserved
The deterministic affine pushforward on an observed binary record.
Definition (Lean source)
noncomputable def affineObserved {d : ℕ} (M : ℝ)
(z : CausalSmith.Stat.DiscreteAteMinimaxLoggap.Obs d) : Obs d :=
⟨z.1, z.2.1, M * ((if z.2.2 then 1 else 0) - 1 / 2)⟩
theorem observed_affine_binaryFullObs
Observing an affinely scaled full-data record is the same as affinely scaling its observed binary record.
Formal statement
hypotheses
conclusion
FullObs.observed (BinaryFullObs.affine M (fun k => k) z) = affineObserved M z.observed
Proof (Lean source)
lemma observed_affine_binaryFullObs {d : ℕ} (M : ℝ) (z : BinaryFullObs d) :
FullObs.observed (BinaryFullObs.affine M (fun k => k) z) =
affineObserved M z.observed := by
cases z with
| mk x a b0 b1 => cases a <;> rfl
theorem affineCanonicalFullLaw_observed_margin
The affine image of the canonical binary full-data coupling has the expected affinely transformed observed margin.
Formal statement
hypotheses
d :
ℕ
M :
ℝ
P :
BinLaw d
conclusion
Measure.map FullObs.observed (Measure.map (BinaryFullObs.affine M (fun k => k)) (canonicalBinaryFullCoupling P))
= Measure.map (affineObserved M) (CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P)
Proof (Lean source)
lemma affineCanonicalFullLaw_observed_margin {d : ℕ} (M : ℝ) (P : BinLaw d) :
Measure.map FullObs.observed
(Measure.map (BinaryFullObs.affine M (fun k => k))
(canonicalBinaryFullCoupling P)) =
Measure.map (affineObserved M)
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P) := by
rw [Measure.map_map (by fun_prop) (by fun_prop)]
have hc := (canonicalBinaryFullCoupling_spec P).2
rw [← hc, Measure.map_map (by fun_prop) (by fun_prop)]
congr 1
funext z
exact observed_affine_binaryFullObs M z
def binaryConditionalOutcomeLaw
The totalized binary conditional outcome law in one arm and cell.
Definition (Lean source)
noncomputable def binaryConditionalOutcomeLaw {d : ℕ} (P : BinLaw d)
(a : Bool) (k : Fin d) : Measure Bool :=
ofReal
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass P k a false /
CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P k a) •
Measure.dirac false +
ofReal
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass P k a true /
CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P k a) •
Measure.dirac true
def AffineBinaryEmbedding
One law-level affine binary-to-real pushforward. The same M-indexed map pins the observed law, every positive arm-cell conditional outcome law, and a full-data potential-outcome coupling.
Definition (Lean source)
def AffineBinaryEmbedding {d : ℕ} (M : ℝ) (P : BinLaw d) (Q : RealLaw d) : Prop :=
Q.observedLaw = Measure.map (affineObserved M)
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P) ∧
(∀ k, Q.cellMass k =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P k) ∧
(∀ k, Q.propensity k =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.propensity P k) ∧
(∀ a k, 0 < CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P k a →
Q.outcomeLaw a k = Measure.map
(fun b : Bool => M * ((if b then 1 else 0) - 1 / 2))
(binaryConditionalOutcomeLaw P a k)) ∧
(∀ a k, Q.outcomeMean a k = M *
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P a k - 1 / 2)) ∧
∃ R : Measure (BinaryFullObs d),
BinaryFullCoupling P R ∧
Q.fullLaw = Measure.map (BinaryFullObs.affine M (fun k => k)) R
The one-record affine binary-to-real observation map is measurable.
Formal statement
Proof (Lean source)
lemma measurable_affineObserved {d : ℕ} (M : ℝ) :
Measurable (affineObserved (d := d) M) := by
rw [measurable_comap_iff]
change Measurable (fun z :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.Obs d =>
(z.1, z.2.1, M * ((if z.2.2 then 1 else 0) - 1 / 2)))
fun_prop
theorem productLaw_eq_map_affineObserved_of_embedding
An affine embedding transports the whole finite sample coordinatewise.
Formal statement
hypotheses
conclusion
productLaw n Q
= Measure.map (fun sample : Fin n → CausalSmith.Stat.DiscreteAteMinimaxLoggap.Obs d => fun i => affineObserved M (sample i)) (CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P n)
Proof (Lean source)
lemma productLaw_eq_map_affineObserved_of_embedding {n d : ℕ} {M : ℝ}
{P : BinLaw d} {Q : RealLaw d} (h : AffineBinaryEmbedding M P Q) :
productLaw n Q = Measure.map
(fun sample : Fin n →
CausalSmith.Stat.DiscreteAteMinimaxLoggap.Obs d =>
fun i => affineObserved M (sample i))
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P n) := by
unfold productLaw CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw
rw [h.1]
exact (map_pi_finCoordinatewise n
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P)
(measurable_affineObserved M)).symm
theorem rawAteFormula_affine_binary
The real-outcome g-formula of an affine binary embedding is the binary weighted regression contrast multiplied by the affine slope.
Formal statement
hypotheses
conclusion
Proof (Lean source)
lemma rawAteFormula_affine_binary {d : ℕ} {M : ℝ}
{P : BinLaw d} {Q : RealLaw d} (h : AffineBinaryEmbedding M P Q) :
rawAteFormula Q = M * ∑ k : Fin d,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P k *
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P true k -
CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P false k) := by
rcases h with ⟨_hobs, hmass, _hprop, _houtcomeLaw, hmean, _hfull⟩
unfold rawAteFormula cellEffect
simp_rw [hmass, hmean]
rw [Finset.mul_sum]
apply Finset.sum_congr rfl
intro k _
ring
theorem rawAteFormula_eq_mul_binaryAte_of_embedding
If the source law satisfies overlap, under binary overlap, the preceding scaling identity is exactly scaling of the source ATE functional.
Formal statement
hypotheses
d :
ℕ
epsilon M :
ℝ
P :
BinLaw d
Q :
RealLaw d
h :
AffineBinaryEmbedding M P Q
hoverlap :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.Overlap epsilon P
conclusion
rawAteFormula Q = M * CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P
Proof (Lean source)
lemma rawAteFormula_eq_mul_binaryAte_of_embedding {d : ℕ} {epsilon M : ℝ}
{P : BinLaw d} {Q : RealLaw d} (h : AffineBinaryEmbedding M P Q)
(hoverlap : CausalSmith.Stat.DiscreteAteMinimaxLoggap.Overlap epsilon P) :
rawAteFormula Q = M *
CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P := by
rw [rawAteFormula_affine_binary h,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional_eq_weighted_regression
P hoverlap]
theorem minimaxRisk_ge_of_affine_binary_hard_family
If the outcome scale is nonnegative and the outcome scale satisfies its stated bound and the source parameter set has the stated form and the affine embedding identity holds and the source law satisfies overlap and the transported family belongs to the target model class, a genuinely hard binary family transfers to the ambient minimax problem through any affine embedding whose images have model-class witnesses.
Formal statement
hypotheses
n d :
ℕ
epsilon M sigma L :
ℝ
hM0 :
0 ≤ M
hM :
M ≠ 0
source :
Set (BinLaw d)
hsource :
∀ est : (Fin n → CausalSmith.Stat.DiscreteAteMinimaxLoggap.Obs d) → ℝ
if
Measurable est
then
∃ P ∈ source,
L
≤ CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse (CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P n) est (CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P)
Phi :
BinLaw d → RealLaw d
hembed :
∀ P, AffineBinaryEmbedding M P (Phi P)
hoverlap :
∀ P ∈ source, CausalSmith.Stat.DiscreteAteMinimaxLoggap.Overlap epsilon P
hmembership :
∀ P ∈ source, ∃ Q : ModelClass d epsilon M sigma, Q.law = Phi P
conclusion
M ^ 2 * L ≤ minimaxRisk n d epsilon M sigma
Proof (Lean source)
-- @node: minimaxRisk_ge_of_affine_binary_hard_family
lemma minimaxRisk_ge_of_affine_binary_hard_family {n d : ℕ}
{epsilon M sigma L : ℝ} (hM0 : 0 ≤ M) (hM : M ≠ 0)
(source : Set (BinLaw d))
(hsource : ∀ est : (Fin n →
CausalSmith.Stat.DiscreteAteMinimaxLoggap.Obs d) → ℝ,
Measurable est → ∃ P ∈ source,
L ≤ CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P n)
est (CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P))
(Phi : BinLaw d → RealLaw d)
(hembed : ∀ P, AffineBinaryEmbedding M P (Phi P))
(hoverlap : ∀ P ∈ source,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.Overlap epsilon P)
(hmembership : ∀ P ∈ source,
∃ Q : ModelClass d epsilon M sigma, Q.law = Phi P) :
M ^ 2 * L ≤ minimaxRisk n d epsilon M sigma := by
let est0 : Estimator n d M :=
⟨fun _ ↦ 0, measurable_const, fun _ ↦ by simp [hM0]⟩
letI : Nonempty (Estimator n d M) := ⟨est0⟩
unfold minimaxRisk
apply le_ciInf
intro est
let I := {P : BinLaw d // P ∈ source}
have htransport :=
forall_estimator_exists_sqRisk_ge_of_deterministic_affine_transport_pi
(Iota := I) (n := n)
(P := fun j => CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw j.1)
(Q := fun j => (Phi j.1).observedLaw)
(theta := fun j =>
CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional j.1)
(phi := affineObserved M) (a := M) (b := 0) (L := L)
hM (measurable_affineObserved M)
(fun j => (hembed j.1).1)
(by
intro sourceEst hmeas
obtain ⟨P, hP, hrisk⟩ := hsource sourceEst hmeas
exact ⟨⟨P, hP⟩, by
simpa [sqRisk,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw] using hrisk⟩)
est.1 est.2.1
obtain ⟨j, hj⟩ := htransport
obtain ⟨Q, hQ⟩ := hmembership j.1 j.2
have htau : rawAteFormula Q.law =
M * CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional j.1 := by
rw [hQ]
exact rawAteFormula_eq_mul_binaryAte_of_embedding
(hembed j.1) (hoverlap j.1 j.2)
have hrisk : M ^ 2 * L ≤ mse Q.law est.1 := by
rw [← htau] at hj
simpa [sqRisk, productLaw, mse, hQ] using hj
have hb : BddAbove (range (fun R : ModelClass d epsilon M sigma ↦
mse R.law est.1)) := by
refine ⟨(2 * M) ^ 2, ?_⟩
rintro _ ⟨R, rfl⟩
exact test_model_mse_le R est
exact hrisk.trans (le_ciSup hb Q)
Helpers.AffineMembership 7 declarations
theorem affineBinaryRealLaw_overlap
If the source law satisfies the stated model condition, affine outcome scaling preserves the binary source law's overlap condition.
Formal statement
hypotheses
d :
ℕ
epsilon M :
ℝ
P :
BinLaw d
hP :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.Overlap epsilon P
conclusion
Overlap epsilon (affineBinaryRealLaw M P)
Proof (Lean source)
lemma affineBinaryRealLaw_overlap {d : ℕ} {epsilon M : ℝ} {P : BinLaw d}
(hP : CausalSmith.Stat.DiscreteAteMinimaxLoggap.Overlap epsilon P) :
Overlap epsilon (affineBinaryRealLaw M P) := by
intro k hk
exact hP k hk
theorem affineBinaryRealLaw_meanNormalization
If the outcome scale satisfies its stated bound, the affine binary real-outcome law has conditional means bounded in absolute value by half the outcome scale.
Formal statement
hypotheses
d :
ℕ
M :
ℝ
P :
BinLaw d
hM :
0 ≤ M
conclusion
MeanNormalization M (affineBinaryRealLaw M P)
Proof (Lean source)
lemma affineBinaryRealLaw_meanNormalization {d : ℕ} {M : ℝ} {P : BinLaw d}
(hM : 0 ≤ M) : MeanNormalization M (affineBinaryRealLaw M P) := by
intro a k _hk
rcases CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean_mem_unitInterval P a k with
⟨hmu0, hmu1⟩
change |M *
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P a k - 1 / 2)| ≤ M / 2
rw [abs_mul, abs_of_nonneg hM]
have : |CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P a k - 1 / 2| ≤
1 / 2 := by
rw [abs_le]
constructor <;> linarith
nlinarith
theorem affineBinaryRealLaw_secondCentralMoment
If the outcome scale satisfies its stated bound, the affine binary real-outcome law has conditional second central moments bounded by the squared outcome scale.
Formal statement
hypotheses
d :
ℕ
M :
ℝ
P :
BinLaw d
hM :
0 ≤ M
conclusion
Proof (Lean source)
lemma affineBinaryRealLaw_secondCentralMoment {d : ℕ} {M : ℝ} {P : BinLaw d}
(hM : 0 ≤ M) : SecondCentralMoment M (affineBinaryRealLaw M P) := by
intro a k _hk
let scale : Bool → ℝ := fun b => M * ((if b then 1 else 0) - 1 / 2)
let mu : ℝ := M *
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P a k - 1 / 2)
have hscale : Measurable scale := by fun_prop
have hbound (b : Bool) : (scale b - mu) ^ 2 ≤ M ^ 2 := by
rcases CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean_mem_unitInterval P a k with
⟨heta0, heta1⟩
cases b
· simp only [scale, mu, Bool.false_eq_true, if_false]
ring_nf
have hs := (sq_le_sq₀ (mul_nonneg hM heta0) hM).2
(by simpa using mul_le_mul_of_nonneg_left heta1 hM)
nlinarith [hs]
· simp only [scale, mu, if_true]
ring_nf
have hs := (sq_le_sq₀ (mul_nonneg hM (sub_nonneg.mpr heta1)) hM).2
(by simpa using mul_le_mul_of_nonneg_left (sub_le_self 1 heta0) hM)
nlinarith [hs]
have hint : Integrable (fun b : Bool => (scale b - mu) ^ 2)
(binaryOutcomePMF P a k).toMeasure := by
apply Integrable.of_bound (by fun_prop) (M ^ 2)
filter_upwards [] with b
rw [Real.norm_eq_abs, abs_of_nonneg (sq_nonneg _)]
exact hbound b
change Integrable (fun y => (y - mu) ^ 2)
((map scale (binaryOutcomePMF P a k)).toMeasure) ∧
∫ y, (y - mu) ^ 2 ∂((map scale (binaryOutcomePMF P a k)).toMeasure) ≤ M ^ 2
rw [← PMF.toMeasure_map scale _ hscale]
constructor
· rw [integrable_map_measure (by fun_prop) hscale.aemeasurable]
exact hint
· rw [integral_map hscale.aemeasurable (by fun_prop)]
have hconst : Integrable (fun _ : Bool => M ^ 2)
(binaryOutcomePMF P a k).toMeasure := integrable_const _
calc
∫ b, (scale b - mu) ^ 2 ∂(binaryOutcomePMF P a k).toMeasure ≤
∫ _b : Bool, M ^ 2 ∂(binaryOutcomePMF P a k).toMeasure :=
integral_mono hint hconst hbound
_ = M ^ 2 := by simp only [integral_const, probReal_univ, one_smul]
def affineBinaryRealLaw_unrestricted
This packages an affine binary source law as a member of the unrestricted real-outcome class.
Definition (Lean source)
def affineBinaryRealLaw_unrestricted {d : ℕ} {epsilon M : ℝ} (P : BinLaw d)
(he0 : 0 < epsilon) (he1 : epsilon < 1 / 2) (hM : 1 ≤ M)
(hP : CausalSmith.Stat.DiscreteAteMinimaxLoggap.Overlap epsilon P) :
UnrestrictedClass d epsilon M where
law := affineBinaryRealLaw M P
epsilon_pos := he0
epsilon_lt_half := he1
M_ge_one := hM
consistency := affineBinaryRealLaw_consistency M P
exchangeability := affineBinaryRealLaw_exchangeability M P
overlap := affineBinaryRealLaw_overlap hP
mean_normalization := affineBinaryRealLaw_meanNormalization (le_trans zero_le_one hM)
second_moment := affineBinaryRealLaw_secondCentralMoment (le_trans zero_le_one hM)
theorem affineBinaryRealLaw_exactHomogeneity
If the source law satisfies the stated model condition, an exactly homogeneous binary source remains exactly homogeneous after affine embedding into real outcomes.
Formal statement
hypotheses
conclusion
ApproximateHomogeneity M 0 (affineBinaryRealLaw M P)
Proof (Lean source)
lemma affineBinaryRealLaw_exactHomogeneity {d : ℕ} {epsilon M : ℝ} {P : BinLaw d}
(hP : BinaryExactHomogeneous epsilon P) :
ApproximateHomogeneity M 0 (affineBinaryRealLaw M P) := by
intro k _hk
have heffect (l : Fin d) :
cellEffect (affineBinaryRealLaw M P) l =
M * (CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P true k -
CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P false k) := by
unfold cellEffect
change M * (_ - 1 / 2) - M * (_ - 1 / 2) = _
calc
M * (_ - 1 / 2) - M * (_ - 1 / 2) =
M * (CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P true l -
CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P false l) := by ring
_ = _ := congrArg (fun x : ℝ => M * x) (hP.2.2 l k)
have hraw : rawAteFormula (affineBinaryRealLaw M P) =
M * (CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P true k -
CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P false k) := by
unfold rawAteFormula
simp_rw [heffect]
rw [← Finset.sum_mul, sum_cellMass_eq_one]
simp
unfold cellDeviation
rw [heffect k, hraw]
simp
def affineBinaryRealLaw_model_zero
This packages an exactly homogeneous affine binary law as a zero-radius member of the model class.
Definition (Lean source)
def affineBinaryRealLaw_model_zero {d : ℕ} {epsilon M : ℝ} (P : BinLaw d)
(he0 : 0 < epsilon) (he1 : epsilon < 1 / 2) (hM : 1 ≤ M)
(hP : BinaryExactHomogeneous epsilon P) : ModelClass d epsilon M 0 where
law := affineBinaryRealLaw M P
epsilon_pos := he0
epsilon_lt_half := he1
M_ge_one := hM
sigma_nonneg := le_rfl
sigma_le_two := by norm_num
consistency := affineBinaryRealLaw_consistency M P
exchangeability := affineBinaryRealLaw_exchangeability M P
overlap := affineBinaryRealLaw_overlap hP.1
mean_normalization := affineBinaryRealLaw_meanNormalization (le_trans zero_le_one hM)
second_moment := affineBinaryRealLaw_secondCentralMoment (le_trans zero_le_one hM)
homogeneity := affineBinaryRealLaw_exactHomogeneity hP
theorem affineBinaryRealLaw_ne_testModelLaw
If the overlap constant is positive and the overlap constant is below one half and the outcome scale satisfies its stated bound and the heterogeneity radius is nonnegative and the heterogeneity radius is at most two, a model whose control outcome is identically zero is outside the affine binary image, since a nonzero affine scale only takes the values ±M/2.
Formal statement
hypotheses
d :
ℕ
k :
Fin d
epsilon M sigma :
ℝ
he0 :
0 < epsilon
he1 :
epsilon < 1 / 2
hM :
1 ≤ M
hs0 :
0 ≤ sigma
hs2 :
sigma ≤ 2
P :
BinLaw d
conclusion
≠ (testModelClass k epsilon M sigma 0 he0 he1 hM hs0 hs2 (by rw [abs_zero]; positivity)).law
Proof (Lean source)
lemma affineBinaryRealLaw_ne_testModelLaw {d : ℕ} (k : Fin d)
{epsilon M sigma : ℝ} (he0 : 0 < epsilon) (he1 : epsilon < 1 / 2)
(hM : 1 ≤ M) (hs0 : 0 ≤ sigma) (hs2 : sigma ≤ 2) (P : BinLaw d) :
affineBinaryRealLaw M P ≠
(testModelClass k epsilon M sigma 0 he0 he1 hM hs0 hs2
(by rw [abs_zero]; positivity)).law := by
intro heq
have hout := congrArg (fun R : RealLaw d => R.outcomeLaw false k {0}) heq
change ((map (fun b : Bool => M * ((if b then 1 else 0) - 1 / 2))
(binaryOutcomePMF P false k)).toMeasure) {0} =
testOutcomeLaw (M / 2) 0 false {0} at hout
have hscale : Measurable (fun b : Bool => M * ((if b then 1 else 0) - 1 / 2)) := by
fun_prop
rw [← PMF.toMeasure_map _ _ hscale,
Measure.map_apply hscale (measurableSet_singleton 0)] at hout
have hpre : (fun b : Bool => M * ((if b then 1 else 0) - 1 / 2)) ⁻¹' {0} = ∅ := by
have hM0 : M ≠ 0 := by linarith
ext b
cases b <;> simp [hM0] <;> norm_num
rw [hpre] at hout
simp [testOutcomeLaw] at hout
Helpers.AffineRealLaw 15 declarations
This is the two-point outcome distribution with the specified success probability and affine outcome scale.
Definition (Lean source)
noncomputable def binaryOutcomePMF {d : ℕ} (P : BinLaw d) (a : Bool) (k : Fin d) : PMF Bool :=
PMF.ofFintype (fun b => ofReal
(if b then CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P a k
else 1 - CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P a k)) (by
rw [Fintype.sum_bool]
simp only [if_true, if_false]
rw [← ENNReal.ofReal_add]
· norm_num
· exact (CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean_mem_unitInterval
P a k).1
· simpa only [Bool.false_eq_true, if_false] using
sub_nonneg.mpr
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean_mem_unitInterval
P a k).2)
theorem binaryOutcomePMF_true_toReal
the binary outcome distribution assigns real probability p to the upper affine endpoint.
Formal statement
hypotheses
conclusion
((binaryOutcomePMF P a k) true).toReal
= CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P a k
Proof (Lean source)
lemma binaryOutcomePMF_true_toReal {d : ℕ} (P : BinLaw d) (a : Bool) (k : Fin d) :
((binaryOutcomePMF P a k) true).toReal =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P a k := by
rw [binaryOutcomePMF, PMF.ofFintype_apply]
exact ENNReal.toReal_ofReal
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean_mem_unitInterval P a k).1
theorem binaryOutcomePMF_false_toReal
the binary outcome distribution assigns real probability one minus p to the lower affine endpoint.
Formal statement
hypotheses
conclusion
((binaryOutcomePMF P a k) false).toReal
= 1 - CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P a k
Proof (Lean source)
lemma binaryOutcomePMF_false_toReal {d : ℕ} (P : BinLaw d) (a : Bool) (k : Fin d) :
((binaryOutcomePMF P a k) false).toReal =
1 - CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P a k := by
rw [binaryOutcomePMF, PMF.ofFintype_apply]
rw [show (if false then
CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P a k
else 1 - CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P a k) =
1 - CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P a k from rfl]
exact ENNReal.toReal_ofReal (sub_nonneg.mpr
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean_mem_unitInterval P a k).2)
def binaryIndependentLift
This augments an observed binary record with an independently drawn missing potential outcome.
Definition (Lean source)
def binaryIndependentLift {d : ℕ} (z : BinObs d) (other : Bool) : BinaryFullObs d :=
if z.2.1 then ⟨z.1, true, other, z.2.2⟩ else ⟨z.1, false, z.2.2, other⟩
def binaryIndependentFullPMF
This is the full-data probability mass function obtained by independently imputing the missing binary potential outcome.
Definition (Lean source)
noncomputable def binaryIndependentFullPMF {d : ℕ} (P : BinLaw d) : PMF (BinaryFullObs d) :=
P.pmf.bind fun z =>
(binaryOutcomePMF P (!z.2.1) z.1).map (binaryIndependentLift z)
theorem binaryIndependentLift_observed
projecting the independent full-data lift recovers the original observed binary record.
Formal statement
Proof (Lean source)
lemma binaryIndependentLift_observed {d : ℕ} (z : BinObs d) (other : Bool) :
(binaryIndependentLift z other).observed = z := by
rcases z with ⟨k, a, y⟩
cases a <;> rfl
theorem map_binaryIndependentFullPMF_observed
projecting the independent full-data distribution gives the original observed-data law.
Formal statement
hypotheses
d :
ℕ
P :
BinLaw d
conclusion
(binaryIndependentFullPMF P).map BinaryFullObs.observed = P.pmf
Proof (Lean source)
lemma map_binaryIndependentFullPMF_observed {d : ℕ} (P : BinLaw d) :
(binaryIndependentFullPMF P).map BinaryFullObs.observed = P.pmf := by
rw [binaryIndependentFullPMF, PMF.map_bind]
simp_rw [PMF.map_comp]
have hmap (z : BinObs d) :
map (BinaryFullObs.observed ∘ binaryIndependentLift z)
(binaryOutcomePMF P (!z.2.1) z.1) = PMF.pure z := by
rw [← PMF.map_const]
congr 1
funext other
exact binaryIndependentLift_observed z other
simp_rw [hmap]
exact PMF.bind_pure _
theorem realMass_map_binaryOutcomePMF
If the specified event is measurable, the affine image of the binary outcome distribution assigns each endpoint its binary probability.
Formal statement
hypotheses
conclusion
= ∑ b : Bool, s.indicator (fun _ => ((binaryOutcomePMF P a k) b).toReal) (M * ((if b then 1 else 0) - 1 / 2))
Proof (Lean source)
lemma realMass_map_binaryOutcomePMF {d : ℕ} (M : ℝ) (P : BinLaw d)
(a : Bool) (k : Fin d) (s : Set ℝ) (hs : MeasurableSet s) :
realMass (((binaryOutcomePMF P a k).map
(fun b => M * ((if b then 1 else 0) - 1 / 2))).toMeasure) s =
∑ b : Bool, s.indicator (fun _ => ((binaryOutcomePMF P a k) b).toReal)
(M * ((if b then 1 else 0) - 1 / 2)) := by
classical
have hscale : Measurable
(fun b : Bool => M * ((if b then 1 else 0) - 1 / 2)) := by fun_prop
unfold realMass
rw [← PMF.toMeasure_map _ _ hscale]
rw [Measure.map_apply_of_aemeasurable hscale.aemeasurable hs]
rw [PMF.toMeasure_apply _ MeasurableSet.of_discrete]
simp only [indicator, tsum_fintype]
rw [ENNReal.toReal_sum]
· simp_rw [apply_ite]
simp [indicator]
· intro b
split_ifs <;> simp [PMF.apply_ne_top]
If the specified event is measurable, the affine observed-data pushforward assigns each arm-cell-outcome event the corresponding binary joint probability.
Formal statement
hypotheses
conclusion
= ∑ y : Bool, s.indicator (fun _ => CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass P k a y) (M * ((if y then 1 else 0) - 1 / 2))
Proof (Lean source)
lemma realMass_affineObserved_event {d : ℕ} (M : ℝ) (P : BinLaw d)
(k : Fin d) (a : Bool) (s : Set ℝ) (hs : MeasurableSet s) :
realMass ((P.pmf.map (affineObserved M)).toMeasure)
{o | o.x = k ∧ o.a = a ∧ o.y ∈ s} =
∑ y : Bool, s.indicator (fun _ =>
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass P k a y)
(M * ((if y then 1 else 0) - 1 / 2)) := by
classical
have htuple : Measurable (fun o : Obs d => (o.x, o.a, o.y)) := by
rw [measurable_iff_comap_le]
rfl
have hx : Measurable (fun o : Obs d => o.x) := measurable_fst.comp htuple
have ha : Measurable (fun o : Obs d => o.a) := measurable_fst.comp htuple.snd
have hy : Measurable (fun o : Obs d => o.y) := htuple.snd.snd
have hevent : MeasurableSet {o : Obs d | o.x = k ∧ o.a = a ∧ o.y ∈ s} :=
((measurableSet_singleton k).preimage hx).inter
(((measurableSet_singleton a).preimage ha).inter (hs.preimage hy))
unfold realMass
rw [← PMF.toMeasure_map _ _ (measurable_affineObserved M)]
rw [Measure.map_apply_of_aemeasurable
(measurable_affineObserved M).aemeasurable hevent]
rw [PMF.toMeasure_apply _ MeasurableSet.of_discrete]
simp only [indicator, tsum_fintype]
rw [ENNReal.toReal_sum]
· simp_rw [apply_ite]
simp only [CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass]
rw [Fintype.sum_prod_type]
rw [Finset.sum_eq_single k]
· rw [Fintype.sum_prod_type]
rw [Finset.sum_eq_single a]
· simp [affineObserved, indicator]
· intro b _ hba
simp [affineObserved, hba]
· simp
· intro l _ hlk
simp [affineObserved, hlk]
· simp
· intro z
split_ifs <;> simp [PMF.apply_ne_top]
def affineBinaryRealLaw
This construction embeds a binary-outcome observational law into a real-outcome law by affine outcome scaling and an independent full-data coupling.
Definition (Lean source)
noncomputable def affineBinaryRealLaw {d : ℕ} (M : ℝ) (P : BinLaw d) : RealLaw d where
observedLaw := (P.pmf.map (affineObserved M)).toMeasure
observed_isProbability := by infer_instance
fullLaw := ((binaryIndependentFullPMF P).map
(BinaryFullObs.affine M (fun k => k))).toMeasure
full_isProbability := by infer_instance
cellMass := CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P
propensity := CausalSmith.Stat.DiscreteAteMinimaxLoggap.propensity P
outcomeLaw := fun a k =>
((binaryOutcomePMF P a k).map
(fun b => M * ((if b then 1 else 0) - 1 / 2))).toMeasure
outcome_isProbability := by intro; infer_instance
outcomeMean := fun a k => M *
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P a k - 1 / 2)
observed_margin := by
rw [PMF.toMeasure_map]
congr 1
rw [← map_binaryIndependentFullPMF_observed P, PMF.map_comp, PMF.map_comp]
congr 1
funext z
exact observed_affine_binaryFullObs M z
fun_prop
cellMass_eq := by
intro k
have htuple : Measurable (fun o : Obs d => (o.x, o.a, o.y)) := by
rw [measurable_iff_comap_le]
rfl
have hx : Measurable (fun o : Obs d => o.x) := measurable_fst.comp htuple
have hs : MeasurableSet {o : Obs d | o.x = k} :=
(measurableSet_singleton k).preimage hx
unfold realMass
rw [← PMF.toMeasure_map (affineObserved M) P.pmf (measurable_affineObserved M)]
rw [Measure.map_apply_of_aemeasurable (measurable_affineObserved M).aemeasurable hs]
rw [PMF.toMeasure_apply P.pmf MeasurableSet.of_discrete]
simp [CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass,
affineObserved, indicator]
rw [ENNReal.toReal_sum]
· simp_rw [apply_ite]
simp only [Fintype.sum_prod_type]
symm
rw [Finset.sum_eq_single k]
· simp
· intro b _ hbk
simp [hbk]
· simp
· intro a
split_ifs <;> simp [PMF.apply_ne_top]
cellMass_range := CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass_mem_unitInterval P
propensity_range := CausalSmith.Stat.DiscreteAteMinimaxLoggap.propensity_mem_unitInterval P
arm_outcome_factorization := by
intro a k s hs
rw [realMass_map_binaryOutcomePMF M P a k s hs,
realMass_affineObserved_event M P k a s hs]
by_cases hp : 0 < CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P k
· have hfactor :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P k *
(if a then CausalSmith.Stat.DiscreteAteMinimaxLoggap.propensity P k
else 1 - CausalSmith.Stat.DiscreteAteMinimaxLoggap.propensity P k) =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P k a := by
have hpne : CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P k ≠ 0 :=
ne_of_gt hp
cases a with
| false =>
have hadd :=
CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass_add_eq_cellMass P k
simp only [Bool.false_eq_true, if_false]
rw [CausalSmith.Stat.DiscreteAteMinimaxLoggap.propensity]
field_simp [hpne]
nlinarith
| true =>
simp only [if_true]
rw [CausalSmith.Stat.DiscreteAteMinimaxLoggap.propensity]
field_simp [hpne]
rw [hfactor, Fintype.sum_bool, Fintype.sum_bool,
binaryOutcomePMF_true_toReal, binaryOutcomePMF_false_toReal]
have htrue :=
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass_true_eq_outcomeMean_mul_armMass
P k a
have hsum : CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P k a =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass P k a false +
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass P k a true := by
simp [CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass, add_comm]
simp only [if_pos True.intro, if_neg Bool.false_ne_true]
by_cases ht : M * (1 - (1 : ℝ) / 2) ∈ s
· simp only [Set.indicator_of_mem ht]
by_cases hf : M * (0 - (1 : ℝ) / 2) ∈ s
· simp only [Set.indicator_of_mem hf]
nlinarith
· simp only [Set.indicator_of_notMem hf]
nlinarith
· simp only [Set.indicator_of_notMem ht]
by_cases hf : M * (0 - (1 : ℝ) / 2) ∈ s
· simp only [Set.indicator_of_mem hf]
nlinarith
· simp only [Set.indicator_of_notMem hf]
nlinarith
· have hp0 : CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P k = 0 :=
le_antisymm (le_of_not_gt hp)
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass_mem_unitInterval P k).1
have hj (y : Bool) :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass P k a y = 0 :=
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass_eq_zero_of_cellMass_eq_zero
P k hp0 a y
simp [hp0, hj, indicator]
outcomeMean_eq := by
intro a k
have hscale : Measurable
(fun b : Bool => M * ((if b then 1 else 0) - 1 / 2)) := by fun_prop
rw [← PMF.toMeasure_map _ _ hscale]
rw [integral_map hscale.aemeasurable (by fun_prop)]
rw [PMF.integral_eq_sum]
rw [Fintype.sum_bool, binaryOutcomePMF_true_toReal,
binaryOutcomePMF_false_toReal]
simp
ring
theorem binaryOutcomePMF_toMeasure_eq_conditional
If the stated pos condition holds, the measure induced by the binary outcome distribution equals the corresponding conditional outcome law.
Formal statement
hypotheses
conclusion
Proof (Lean source)
lemma binaryOutcomePMF_toMeasure_eq_conditional {d : ℕ} (P : BinLaw d)
(a : Bool) (k : Fin d)
(hpos : 0 < CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P k a) :
(binaryOutcomePMF P a k).toMeasure = binaryConditionalOutcomeLaw P a k := by
classical
have hsum : CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P k a =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass P k a false +
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass P k a true := by
simp [CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass, add_comm]
have hfalse : 1 -
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass P k a true /
CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P k a =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass P k a false /
CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P k a := by
field_simp [ne_of_gt hpos]
linarith
ext s hs
rw [PMF.toMeasure_apply _ hs]
unfold binaryConditionalOutcomeLaw
rw [Measure.add_apply, Measure.smul_apply, Measure.smul_apply]
simp only [Measure.dirac_apply' _ hs, tsum_fintype, Fintype.sum_bool]
by_cases ht : true ∈ s <;> by_cases hf : false ∈ s <;>
simp [indicator, ht, hf, binaryOutcomePMF, PMF.ofFintype_apply,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean, hfalse, add_comm]
theorem affineBinaryRealLaw_embedding
the affine real-outcome construction satisfies the binary embedding identities for cell probabilities, propensities, and conditional means.
Formal statement
Proof (Lean source)
lemma affineBinaryRealLaw_embedding {d : ℕ} (M : ℝ) (P : BinLaw d) :
AffineBinaryEmbedding M P (affineBinaryRealLaw M P) := by
refine ⟨?_, ?_, ?_, ?_, ?_, ?_⟩
· exact (PMF.toMeasure_map (affineObserved M) P.pmf
(measurable_affineObserved M)).symm
· intro k
rfl
· intro k
rfl
· intro a k hpos
rw [← binaryOutcomePMF_toMeasure_eq_conditional P a k hpos]
exact (PMF.toMeasure_map _ _ (by fun_prop)).symm
· intro a k
rfl
· refine ⟨(binaryIndependentFullPMF P).toMeasure, ?_, ?_⟩
· constructor
· infer_instance
· rw [PMF.toMeasure_map BinaryFullObs.observed _ (measurable_of_finite _)]
exact congrArg toMeasure (map_binaryIndependentFullPMF_observed P)
· exact (PMF.toMeasure_map _ _ (by fun_prop)).symm
theorem binaryIndependentFullPMF_apply_toReal
the real probability assigned to a full-data atom factors into its observed-atom probability and the independent missing-potential probability.
Formal statement
hypotheses
conclusion
((binaryIndependentFullPMF P) z).toReal
= if z.a then CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass P z.x true z.b1 * ((binaryOutcomePMF P false z.x) z.b0).toReal else CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass P z.x false z.b0 * ((binaryOutcomePMF P true z.x) z.b1).toReal
Proof (Lean source)
lemma binaryIndependentFullPMF_apply_toReal {d : ℕ} (P : BinLaw d)
(z : BinaryFullObs d) :
((binaryIndependentFullPMF P) z).toReal =
if z.a then
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass P z.x true z.b1 *
((binaryOutcomePMF P false z.x) z.b0).toReal
else
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass P z.x false z.b0 *
((binaryOutcomePMF P true z.x) z.b1).toReal := by
rcases z with ⟨x, a, b0, b1⟩
cases a <;> cases b0 <;> cases b1 <;>
simp only [binaryIndependentFullPMF, PMF.bind_apply, PMF.map_apply,
binaryIndependentLift, CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass,
tsum_fintype, Fintype.sum_prod_type, Bool.false_eq_true, if_false,
if_true]
all_goals
rw [Finset.sum_eq_single x]
· simp [ENNReal.toReal_mul]
· intro y _ hy
simp [binaryIndependentLift, Ne.symm hy]
· simp
theorem affineBinaryRealLaw_consistency
the affine binary real law satisfies consistency.
Formal statement
Proof (Lean source)
lemma affineBinaryRealLaw_consistency {d : ℕ} (M : ℝ) (P : BinLaw d) :
Consistency (affineBinaryRealLaw M P) := by
unfold Consistency
change ((map (BinaryFullObs.affine M (fun k => k))
(binaryIndependentFullPMF P)).toMeasure) _ = 0
rw [← PMF.toMeasure_map _ _ (by fun_prop)]
have htuple : Measurable (fun z : FullObs d => (z.x, z.a, z.y0, z.y1, z.y)) := by
rw [measurable_iff_comap_le]
rfl
have ha : Measurable (fun z : FullObs d => z.a) := htuple.snd.fst
have hy0 : Measurable (fun z : FullObs d => z.y0) := htuple.snd.snd.fst
have hy1 : Measurable (fun z : FullObs d => z.y1) := htuple.snd.snd.snd.fst
have hy : Measurable (fun z : FullObs d => z.y) := htuple.snd.snd.snd.snd
have hevent : MeasurableSet
{z : FullObs d | z.y ≠ if z.a then z.y1 else z.y0} := by
have hf : Measurable (fun z : FullObs d =>
z.y - if z.a then z.y1 else z.y0) := by
apply hy.sub
exact Measurable.ite ((measurableSet_singleton true).preimage ha) hy1 hy0
have hz := (measurableSet_singleton (0 : ℝ)).compl.preimage hf
have heq : {z : FullObs d | z.y ≠ if z.a then z.y1 else z.y0} =
((fun z : FullObs d => z.y - if z.a then z.y1 else z.y0) ⁻¹' {0})ᶜ := by
ext z
simp [sub_eq_zero]
rw [heq]
exact hz
rw [Measure.map_apply_of_aemeasurable (by fun_prop) hevent]
have hempty : (BinaryFullObs.affine M (fun k => k)) ⁻¹'
{z : FullObs d | z.y ≠ if z.a then z.y1 else z.y0} = ∅ := by
ext z
rcases z with ⟨x, a, b0, b1⟩
cases a <;> simp [BinaryFullObs.affine]
rw [hempty]
simp
theorem affineBinaryRealLaw_exchangeability
The independent missing-potential-outcome augmentation makes the affine full-data law conditionally exchangeable.
Formal statement
Proof (Lean source)
lemma affineBinaryRealLaw_exchangeability {d : ℕ} (M : ℝ) (P : BinLaw d) :
ConditionalExchangeability (affineBinaryRealLaw M P) := by
unfold ConditionalExchangeability
intro k a s0 s1 hs0 hs1
change ((map (BinaryFullObs.affine M (fun k => k))
(binaryIndependentFullPMF P)).toMeasure) _ *
((map (BinaryFullObs.affine M (fun k => k))
(binaryIndependentFullPMF P)).toMeasure) _ =
((map (BinaryFullObs.affine M (fun k => k))
(binaryIndependentFullPMF P)).toMeasure) _ *
((map (BinaryFullObs.affine M (fun k => k))
(binaryIndependentFullPMF P)).toMeasure) _
have hx : MeasurableSet {z : FullObs d | z.x = k} :=
(measurableSet_singleton k).preimage measurable_full_x
have harm : MeasurableSet {z : FullObs d | z.x = k ∧ z.a = a} :=
hx.inter ((measurableSet_singleton a).preimage measurable_full_a)
have hpot : MeasurableSet
{z : FullObs d | z.x = k ∧ z.y0 ∈ s0 ∧ z.y1 ∈ s1} :=
hx.inter ((hs0.preimage measurable_full_y0).inter
(hs1.preimage measurable_full_y1))
have hjoint : MeasurableSet
{z : FullObs d | z.x = k ∧ z.a = a ∧ z.y0 ∈ s0 ∧ z.y1 ∈ s1} :=
hx.inter (((measurableSet_singleton a).preimage measurable_full_a).inter
((hs0.preimage measurable_full_y0).inter (hs1.preimage measurable_full_y1)))
have hmap (E : Set (FullObs d)) (hE : MeasurableSet E) :
((map (BinaryFullObs.affine M (fun k => k))
(binaryIndependentFullPMF P)).toMeasure) E =
(binaryIndependentFullPMF P).toMeasure
((BinaryFullObs.affine M (fun k => k)) ⁻¹' E) := by
rw [← PMF.toMeasure_map _ _ (by fun_prop)]
exact Measure.map_apply (by fun_prop) hE
rw [hmap _ hjoint, hmap _ hx, hmap _ harm, hmap _ hpot]
rw [PMF.toMeasure_apply _ MeasurableSet.of_discrete,
PMF.toMeasure_apply _ MeasurableSet.of_discrete,
PMF.toMeasure_apply _ MeasurableSet.of_discrete,
PMF.toMeasure_apply _ MeasurableSet.of_discrete]
simp only [tsum_fintype]
classical
have hsum (E : Set (BinaryFullObs d)) :
(∑ x, E.indicator (⇑(binaryIndependentFullPMF P)) x) ≠ ∞ := by
apply (ENNReal.sum_ne_top).2
intro x hx
simp only [indicator]
split
· exact (binaryIndependentFullPMF P).apply_ne_top x
· simp
rw [← ENNReal.toReal_eq_toReal_iff'
(ENNReal.mul_ne_top (hsum _) (hsum _))
(ENNReal.mul_ne_top (hsum _) (hsum _))]
simp_rw [ENNReal.toReal_mul]
have htoReal (E : Set (BinaryFullObs d)) :
(∑ x, E.indicator (⇑(binaryIndependentFullPMF P)) x).toReal =
∑ x, (E.indicator (⇑(binaryIndependentFullPMF P)) x).toReal := by
apply ENNReal.toReal_sum
intro x hx
simp only [indicator]
split
· exact (binaryIndependentFullPMF P).apply_ne_top x
· simp
simp_rw [htoReal]
simp only [indicator]
simp_rw [apply_ite, ENNReal.toReal_zero,
binaryIndependentFullPMF_apply_toReal]
let e : (Fin d × Bool × Bool × Bool) ≃ BinaryFullObs d := {
toFun := fun z => ⟨z.1, z.2.1, z.2.2.1, z.2.2.2⟩
invFun := fun z => (z.x, z.a, z.b0, z.b1)
left_inv := by intro z; rcases z with ⟨x, a, b0, b1⟩; rfl
right_inv := by intro z; rcases z with ⟨x, a, b0, b1⟩; rfl }
simp_rw [← e.sum_comp]
simp_rw [Fintype.sum_prod_type, Fintype.sum_bool]
have sum_cell (f : Fin d → ℝ) (hf : ∀ x, x ≠ k → f x = 0) :
(∑ x, f x) = f k := by
apply Finset.sum_eq_single k
· intro x hx hne
exact hf x hne
· simp
repeat' rw [sum_cell _ (by
intro x hx
simp [e, BinaryFullObs.affine, hx])]
have hrel (b : Bool) :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass P k b true =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P b k *
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass P k b false +
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass P k b true) := by
calc
_ = CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P b k *
CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P k b :=
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass_true_eq_outcomeMean_mul_armMass
P k b
_ = _ := by
rw [CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass, Fintype.sum_bool]
ring
have hrel_mul (b c y : Bool) :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass P k b true *
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass P k c y =
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P b k *
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass P k b false +
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass P k b true)) *
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass P k c y := by
exact congrArg (fun r => r *
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass P k c y) (hrel b)
cases a <;>
simp [e, BinaryFullObs.affine, binaryOutcomePMF_true_toReal,
binaryOutcomePMF_false_toReal] <;>
split_ifs <;> nlinarith [hrel false, hrel true,
hrel_mul false true false, hrel_mul false true true,
hrel_mul true false false, hrel_mul true false true]
Helpers.BinaryPadding 17 declarations
If the scalar satisfies the stated range condition, above one, the natural floor retains at least half of a nonnegative real.
Formal statement
hypotheses
x :
ℝ
hx :
1 ≤ x
conclusion
x / 2 ≤ (Nat.floor x : ℝ)
Proof (Lean source)
lemma half_le_natFloor_of_one_le {x : ℝ} (hx : 1 ≤ x) : x / 2 ≤ (Nat.floor x : ℝ) := by
by_cases hx2 : x ≤ 2
· have hfloor : 1 ≤ Nat.floor x := Nat.le_floor (n := 1) (a := x) (by simpa using hx)
have hfloorR : (1 : ℝ) ≤ Nat.floor x := by exact_mod_cast hfloor
linarith
· have hfloor := Nat.sub_one_lt_floor x
have hxlarge : 2 < x := lt_of_not_ge hx2
linarith
Include a binary observation on the first m cells into an alphabet of size d.
Definition (Lean source)
def binaryPadObs {m d : ℕ} (hmd : m ≤ d) : BinObs m → BinObs d :=
fun z ↦ (⟨z.1, lt_of_lt_of_le z.1.isLt hmd⟩, z.2.1, z.2.2)
theorem binaryPadObs_injective
If the source alphabet embeds in the target alphabet, the zero-padding map from the source observation alphabet into the larger alphabet is injective.
Formal statement
Proof (Lean source)
lemma binaryPadObs_injective {m d : ℕ} (hmd : m ≤ d) :
Injective (binaryPadObs hmd) := by
intro z w hzw
rcases z with ⟨x, a, y⟩
rcases w with ⟨x', a', y'⟩
simp only [binaryPadObs, Prod.mk.injEq] at hzw
obtain ⟨hx, ha, hy⟩ := hzw
subst a'
subst y'
have : x = x' := Fin.ext (congrArg (fun q : Fin d => q.val) hx)
subst x'
rfl
Push a binary law into the first m cells of a larger alphabet, assigning zero probability to all unused cells.
Definition (Lean source)
noncomputable def binaryPadLaw {m d : ℕ} (hmd : m ≤ d) (P : BinLaw m) : BinLaw d :=
⟨PMF.map (binaryPadObs hmd) P.pmf⟩
theorem binaryPadLaw_obsLaw
If the source alphabet embeds in the target alphabet, the observed-data law after padding is the pushforward of the source law under the padding map.
Formal statement
hypotheses
m d :
ℕ
hmd :
m ≤ d
P :
BinLaw m
conclusion
CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw (binaryPadLaw hmd P)
= Measure.map (binaryPadObs hmd) (CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P)
Proof (Lean source)
lemma binaryPadLaw_obsLaw {m d : ℕ} (hmd : m ≤ d) (P : BinLaw m) :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw (binaryPadLaw hmd P) =
Measure.map (binaryPadObs hmd)
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P) := by
exact (PMF.toMeasure_map (binaryPadObs hmd) P.pmf (measurable_of_finite _)).symm
theorem binaryPadLaw_jointMass_image
If the source alphabet embeds in the target alphabet, padding preserves joint probabilities on embedded source cells.
Formal statement
hypotheses
conclusion
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass (binaryPadLaw hmd P) ⟨k, lt_of_lt_of_le k.isLt hmd⟩ a y
= CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass P k a y
Proof (Lean source)
lemma binaryPadLaw_jointMass_image {m d : ℕ} (hmd : m ≤ d) (P : BinLaw m)
(k : Fin m) (a y : Bool) :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass (binaryPadLaw hmd P)
⟨k, lt_of_lt_of_le k.isLt hmd⟩ a y =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass P k a y := by
unfold CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass binaryPadLaw
rw [PMF.map_apply]
rw [tsum_eq_single (k, a, y)]
· simp [binaryPadObs]
· intro z hz
have hne :
(⟨k, lt_of_lt_of_le k.isLt hmd⟩, a, y) ≠ binaryPadObs hmd z := by
intro heq
exact hz ((binaryPadObs_injective hmd) (by simpa [binaryPadObs] using heq)).symm
simp [hne]
theorem binaryPadLaw_jointMass_off_image
If the source alphabet embeds in the target alphabet and the stated condition on the cell holds, padding assigns zero joint probability to every cell outside the embedded source alphabet.
Formal statement
hypotheses
conclusion
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass (binaryPadLaw hmd P) k a y = 0
Proof (Lean source)
lemma binaryPadLaw_jointMass_off_image {m d : ℕ} (hmd : m ≤ d) (P : BinLaw m)
(k : Fin d) (hk : ∀ r : Fin m, (⟨r, lt_of_lt_of_le r.isLt hmd⟩ : Fin d) ≠ k)
(a y : Bool) :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass (binaryPadLaw hmd P) k a y = 0 := by
unfold CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass binaryPadLaw
rw [PMF.map_apply]
rw [tsum_fintype]
apply (ENNReal.toReal_eq_zero_iff _).mpr
left
apply Finset.sum_eq_zero
intro z _hz
rcases z with ⟨r, b, c⟩
have hne : (k, a, y) ≠ binaryPadObs hmd (r, b, c) := by
intro h
exact hk r (congrArg fst h).symm
simp [hne]
theorem binaryPadLaw_cellMass_image
If the source alphabet embeds in the target alphabet, padding preserves cell probabilities on embedded source cells.
Formal statement
hypotheses
conclusion
CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass (binaryPadLaw hmd P) ⟨k, lt_of_lt_of_le k.isLt hmd⟩
= CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P k
Proof (Lean source)
lemma binaryPadLaw_cellMass_image {m d : ℕ} (hmd : m ≤ d) (P : BinLaw m)
(k : Fin m) :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass (binaryPadLaw hmd P)
⟨k, lt_of_lt_of_le k.isLt hmd⟩ =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P k := by
unfold CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass
simp_rw [binaryPadLaw_jointMass_image]
theorem binaryPadLaw_cellMass_off_image
If the source alphabet embeds in the target alphabet and the stated condition on the cell holds, padding assigns zero cell probability outside the embedded source alphabet.
Formal statement
hypotheses
conclusion
CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass (binaryPadLaw hmd P) k = 0
Proof (Lean source)
lemma binaryPadLaw_cellMass_off_image {m d : ℕ} (hmd : m ≤ d) (P : BinLaw m)
(k : Fin d) (hk : ∀ r : Fin m, (⟨r, lt_of_lt_of_le r.isLt hmd⟩ : Fin d) ≠ k) :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass (binaryPadLaw hmd P) k = 0 := by
unfold CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass
simp_rw [binaryPadLaw_jointMass_off_image hmd P k hk]
simp
theorem binaryPadLaw_armMass_image
If the source alphabet embeds in the target alphabet, padding preserves arm-specific cell probabilities on embedded source cells.
Formal statement
hypotheses
conclusion
CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass (binaryPadLaw hmd P) ⟨k, lt_of_lt_of_le k.isLt hmd⟩ a
= CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P k a
Proof (Lean source)
lemma binaryPadLaw_armMass_image {m d : ℕ} (hmd : m ≤ d) (P : BinLaw m)
(k : Fin m) (a : Bool) :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass (binaryPadLaw hmd P)
⟨k, lt_of_lt_of_le k.isLt hmd⟩ a =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P k a := by
unfold CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass
simp_rw [binaryPadLaw_jointMass_image]
theorem binaryPadLaw_propensity_image
If the source alphabet embeds in the target alphabet, padding preserves propensities on embedded source cells.
Formal statement
hypotheses
conclusion
CausalSmith.Stat.DiscreteAteMinimaxLoggap.propensity (binaryPadLaw hmd P) ⟨k, lt_of_lt_of_le k.isLt hmd⟩
= CausalSmith.Stat.DiscreteAteMinimaxLoggap.propensity P k
Proof (Lean source)
lemma binaryPadLaw_propensity_image {m d : ℕ} (hmd : m ≤ d) (P : BinLaw m)
(k : Fin m) :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.propensity (binaryPadLaw hmd P)
⟨k, lt_of_lt_of_le k.isLt hmd⟩ =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.propensity P k := by
unfold CausalSmith.Stat.DiscreteAteMinimaxLoggap.propensity
rw [binaryPadLaw_armMass_image, binaryPadLaw_cellMass_image]
theorem binaryPadLaw_outcomeMean_image
If the source alphabet embeds in the target alphabet, padding preserves conditional outcome means on embedded source cells.
Formal statement
hypotheses
conclusion
CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean (binaryPadLaw hmd P) a ⟨k, lt_of_lt_of_le k.isLt hmd⟩
= CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P a k
Proof (Lean source)
lemma binaryPadLaw_outcomeMean_image {m d : ℕ} (hmd : m ≤ d) (P : BinLaw m)
(k : Fin m) (a : Bool) :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean (binaryPadLaw hmd P) a
⟨k, lt_of_lt_of_le k.isLt hmd⟩ =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P a k := by
unfold CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean
rw [binaryPadLaw_jointMass_image, binaryPadLaw_armMass_image]
theorem binaryPadLaw_overlap
If the source alphabet embeds in the target alphabet and the source law satisfies the stated model condition, the binary pad law preserves the overlap condition.
Formal statement
hypotheses
m d :
ℕ
hmd :
m ≤ d
epsilon :
ℝ
P :
BinLaw m
hP :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.Overlap epsilon P
conclusion
CausalSmith.Stat.DiscreteAteMinimaxLoggap.Overlap epsilon (binaryPadLaw hmd P)
Proof (Lean source)
lemma binaryPadLaw_overlap {m d : ℕ} (hmd : m ≤ d) {epsilon : ℝ} {P : BinLaw m}
(hP : CausalSmith.Stat.DiscreteAteMinimaxLoggap.Overlap epsilon P) :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.Overlap epsilon (binaryPadLaw hmd P) := by
intro k hk
by_cases himage : ∃ r : Fin m, (⟨r, lt_of_lt_of_le r.isLt hmd⟩ : Fin d) = k
· obtain ⟨r, rfl⟩ := himage
rw [binaryPadLaw_propensity_image]
apply hP r
simpa [binaryPadLaw_cellMass_image] using hk
· have hoff : CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass
(binaryPadLaw hmd P) k = 0 := by
unfold CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass
simp_rw [binaryPadLaw_jointMass_off_image hmd P k (by simpa using himage)]
simp
linarith
theorem binaryPadLaw_ateFunctional
If the source alphabet embeds in the target alphabet and the source law satisfies the stated model condition, zero-mass padding preserves the binary average-treatment-effect functional.
Formal statement
hypotheses
m d :
ℕ
hmd :
m ≤ d
epsilon :
ℝ
P :
BinLaw m
hP :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.Overlap epsilon P
conclusion
CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional (binaryPadLaw hmd P)
= CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P
Proof (Lean source)
lemma binaryPadLaw_ateFunctional {m d : ℕ} (hmd : m ≤ d) {epsilon : ℝ} {P : BinLaw m}
(hP : CausalSmith.Stat.DiscreteAteMinimaxLoggap.Overlap epsilon P) :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional (binaryPadLaw hmd P) =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P := by
rw [CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional_eq_weighted_regression _
(binaryPadLaw_overlap hmd hP),
CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional_eq_weighted_regression _ hP]
classical
let f : Fin m → Fin d := fun r ↦ ⟨r, lt_of_lt_of_le r.isLt hmd⟩
let g : Fin d → ℝ := fun k ↦
CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass (binaryPadLaw hmd P) k *
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean (binaryPadLaw hmd P) true k -
CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean (binaryPadLaw hmd P) false k)
calc
∑ k : Fin d, g k = ∑ k ∈ image f univ, g k := by
symm
apply Finset.sum_subset
· simp
· intro k _hk hkimage
have himage : ¬ ∃ r : Fin m, f r = k := by simpa using hkimage
have hoff : CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass
(binaryPadLaw hmd P) k = 0 := by
unfold CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass
simp_rw [binaryPadLaw_jointMass_off_image hmd P k (by simpa [f] using himage)]
simp
simp [g, hoff]
_ = ∑ r : Fin m, g (f r) := by
exact Finset.sum_image (fun _ _ _ _ h =>
Fin.ext (congrArg (fun q : Fin d => q.val) h))
_ = ∑ k : Fin m,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P k *
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P true k -
CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P false k) := by
apply Finset.sum_congr rfl
intro k _hk
simp [g, f, binaryPadLaw_cellMass_image, binaryPadLaw_outcomeMean_image]
theorem affineBinaryPaddedLaw_exactHomogeneity
If the source alphabet embeds in the target alphabet and the stated condition on the source size or matching order holds and the source law satisfies the stated model condition, padding by zero-mass cells preserves exact treatment-effect homogeneity in the support-qualified real-outcome model.
Formal statement
hypotheses
conclusion
Proof (Lean source)
lemma affineBinaryPaddedLaw_exactHomogeneity {m d : ℕ} (hmd : m ≤ d) (hm : 0 < m)
{epsilon M : ℝ} {P : BinLaw m} (hP : BinaryExactHomogeneous epsilon P) :
ApproximateHomogeneity M 0 (affineBinaryRealLaw M (binaryPadLaw hmd P)) := by
have hate (r : Fin m) :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P true r -
CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P false r := by
rw [CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional_eq_weighted_regression P hP.1]
simp_rw [hP.2.1, hP.2.2 _ r]
rw [Finset.sum_const, Finset.card_univ, Fintype.card_fin]
have hmR : (m : ℝ) ≠ 0 := by exact_mod_cast hm.ne'
simp [nsmul_eq_mul, hmR]
intro k hk
have himage : ∃ r : Fin m, (⟨r, lt_of_lt_of_le r.isLt hmd⟩ : Fin d) = k := by
by_contra hnot
have hoff := binaryPadLaw_cellMass_off_image hmd P k (by simpa using hnot)
have hmass : (affineBinaryRealLaw M (binaryPadLaw hmd P)).cellMass k = 0 := by
change CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass (binaryPadLaw hmd P) k = 0
exact hoff
linarith
obtain ⟨r, rfl⟩ := himage
have hembed := affineBinaryRealLaw_embedding M (binaryPadLaw hmd P)
have hraw : rawAteFormula (affineBinaryRealLaw M (binaryPadLaw hmd P)) =
M * CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P := by
rw [rawAteFormula_eq_mul_binaryAte_of_embedding hembed (binaryPadLaw_overlap hmd hP.1),
binaryPadLaw_ateFunctional hmd hP.1]
unfold cellDeviation cellEffect
change |M * (_ - 1 / 2) - M * (_ - 1 / 2) - _| ≤ 0 * M
rw [binaryPadLaw_outcomeMean_image, binaryPadLaw_outcomeMean_image, hraw, hate r]
ring_nf
simp
def affineBinaryPaddedLaw_model
The padded affine image of an exact binary source law belongs to every nonnegative-radius ambient class.
Definition (Lean source)
noncomputable def affineBinaryPaddedLaw_model {m d : ℕ} (hmd : m ≤ d) (hm : 0 < m)
{epsilon M sigma : ℝ} (P : BinLaw m)
(he0 : 0 < epsilon) (he1 : epsilon < 1 / 2) (hM : 1 ≤ M)
(hs0 : 0 ≤ sigma) (hs2 : sigma ≤ 2) (hP : BinaryExactHomogeneous epsilon P) :
ModelClass d epsilon M sigma where
law := affineBinaryRealLaw M (binaryPadLaw hmd P)
epsilon_pos := he0
epsilon_lt_half := he1
M_ge_one := hM
sigma_nonneg := hs0
sigma_le_two := hs2
consistency := affineBinaryRealLaw_consistency M (binaryPadLaw hmd P)
exchangeability := affineBinaryRealLaw_exchangeability M (binaryPadLaw hmd P)
overlap := affineBinaryRealLaw_overlap (binaryPadLaw_overlap hmd hP.1)
mean_normalization := affineBinaryRealLaw_meanNormalization (le_trans zero_le_one hM)
second_moment := affineBinaryRealLaw_secondCentralMoment (le_trans zero_le_one hM)
homogeneity := by
intro k hk
have hz := affineBinaryPaddedLaw_exactHomogeneity hmd hm hP k hk
rw [zero_mul] at hz
have hnonneg : 0 ≤ sigma * M := mul_nonneg hs0 (le_trans zero_le_one hM)
exact hz.trans hnonneg
theorem minimaxRisk_ge_of_padded_exact_hard_family
If the source alphabet embeds in the target alphabet and the stated condition on the source size or matching order holds and the overlap constant is positive and the overlap constant is below one half and the outcome scale satisfies its stated bound and the heterogeneity radius is nonnegative and the heterogeneity radius is at most two and the source parameter set has the stated form, a hard exact binary family on m cells transfers to the real-outcome problem on any larger alphabet by zero-mass padding followed by affine scaling.
Formal statement
hypotheses
n m d :
ℕ
epsilon M sigma L :
ℝ
hmd :
m ≤ d
hm :
0 < m
he0 :
0 < epsilon
he1 :
epsilon < 1 / 2
hM :
1 ≤ M
hs0 :
0 ≤ sigma
hs2 :
sigma ≤ 2
hsource :
∀ est : (Fin n → BinObs m) → ℝ
if
Measurable est
then
∃ P : BinaryExactLaw n m epsilon,
L
≤ CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse (CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n) est (CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1)
conclusion
M ^ 2 * L ≤ minimaxRisk n d epsilon M sigma
Proof (Lean source)
lemma minimaxRisk_ge_of_padded_exact_hard_family {n m d : ℕ}
{epsilon M sigma L : ℝ} (hmd : m ≤ d) (hm : 0 < m)
(he0 : 0 < epsilon) (he1 : epsilon < 1 / 2)
(hM : 1 ≤ M) (hs0 : 0 ≤ sigma) (hs2 : sigma ≤ 2)
(hsource : ∀ est : (Fin n → BinObs m) → ℝ, Measurable est →
∃ P : BinaryExactLaw n m epsilon,
L ≤ CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n)
est (CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1)) :
M ^ 2 * L ≤ minimaxRisk n d epsilon M sigma := by
have hM0 : 0 ≤ M := le_trans zero_le_one hM
have hMne : M ≠ 0 := ne_of_gt (lt_of_lt_of_le zero_lt_one hM)
let est0 : Estimator n d M :=
⟨fun _ ↦ 0, measurable_const, fun _ ↦ by simp [hM0]⟩
letI : Nonempty (Estimator n d M) := ⟨est0⟩
unfold minimaxRisk
apply le_ciInf
intro est
let Phi : BinaryExactLaw n m epsilon → RealLaw d := fun P ↦
affineBinaryRealLaw M (binaryPadLaw hmd P.1)
let phi : BinObs m → Obs d := fun z ↦ affineObserved M (binaryPadObs hmd z)
have hphi : Measurable phi :=
(measurable_affineObserved M).comp (measurable_of_finite _)
have hobs (P : BinaryExactLaw n m epsilon) :
(Phi P).observedLaw = Measure.map phi
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P.1) := by
change (affineBinaryRealLaw M (binaryPadLaw hmd P.1)).observedLaw = _
rw [(affineBinaryRealLaw_embedding M (binaryPadLaw hmd P.1)).1,
binaryPadLaw_obsLaw, Measure.map_map]
· rfl
· exact measurable_affineObserved M
· exact measurable_of_finite _
have htransport :=
forall_estimator_exists_sqRisk_ge_of_deterministic_affine_transport_pi
(Iota := BinaryExactLaw n m epsilon) (n := n)
(P := fun j ↦ CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw j.1)
(Q := fun j ↦ (Phi j).observedLaw)
(theta := fun j ↦ CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional j.1)
(phi := phi) (a := M) (b := 0) (L := L)
hMne hphi hobs
(by
intro sourceEst hmeas
obtain ⟨P, hrisk⟩ := hsource sourceEst hmeas
exact ⟨P, by
simpa [sqRisk,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw] using hrisk⟩)
est.1 est.2.1
obtain ⟨P, hP⟩ := htransport
let Q : ModelClass d epsilon M sigma :=
affineBinaryPaddedLaw_model hmd hm P.1 he0 he1 hM hs0 hs2 P.2
have hQ : Q.law = Phi P := rfl
have htau : rawAteFormula Q.law =
M * CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1 := by
change rawAteFormula (affineBinaryRealLaw M (binaryPadLaw hmd P.1)) = _
rw [rawAteFormula_eq_mul_binaryAte_of_embedding
(affineBinaryRealLaw_embedding M (binaryPadLaw hmd P.1))
(binaryPadLaw_overlap hmd P.2.1), binaryPadLaw_ateFunctional hmd P.2.1]
have hrisk : M ^ 2 * L ≤ mse Q.law est.1 := by
rw [← htau] at hP
rw [hQ] at hP
rw [hQ]
simpa [sqRisk, productLaw, mse, Phi] using hP
have hb : BddAbove (range (fun R : ModelClass d epsilon M sigma ↦
mse R.law est.1)) := by
refine ⟨(2 * M) ^ 2, ?_⟩
rintro _ ⟨R, rfl⟩
exact test_model_mse_le R est
exact hrisk.trans (le_ciSup hb Q)
Helpers.CitedGates 8 declarations
The uniform-mass, exactly homogeneous binary source class.
Definition (Lean source)
def BinaryExactHomogeneous {d : ℕ} (epsilon : ℝ) (P : BinLaw d) : Prop :=
CausalSmith.Stat.DiscreteAteMinimaxLoggap.Overlap epsilon P ∧
(∀ k, CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P k = 1 / (d : ℝ)) ∧
(∀ k l,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P true k -
CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P false k =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P true l -
CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P false l)
def BinaryExactLaw
Binary laws in the cited exact-homogeneity source experiment.
Definition (Lean source)
def BinaryExactLaw (_n d : ℕ) (epsilon : ℝ) :=
{P : BinLaw d // BinaryExactHomogeneous epsilon P}
def binaryExactMinimaxRisk
Exact-homogeneity source minimax risk.
Definition (Lean source)
noncomputable def binaryExactMinimaxRisk (n d : ℕ) (epsilon : ℝ) : ℝ :=
⨅ est : {f : (Fin n → BinObs d) → ℝ // Measurable f},
⨆ P : BinaryExactLaw n d epsilon,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n)
est.1 (CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1)
Cited gate (Zeng, Balakrishnan, Han, and Kennedy, 2024, revised 2026). Source: arXiv:2405.00118v3, Theorem 4 and Appendix C.8. The fixed-sample uniform-mass exactly homogeneous binary experiment has minimax risk at least a constant times 1/n+d/n^2 throughout its source range.
Definition (Lean source)
def ZengBinaryExactHomogeneityLower (epsilon : ℝ) : Prop :=
0 < epsilon ∧ epsilon < 1 / 2 →
∃ a_epsilon b_epsilon : ℝ, ∃ N_epsilon : ℕ,
0 < a_epsilon ∧ 0 < b_epsilon ∧
∀ n d : ℕ, 0 < d → N_epsilon ≤ n →
(d : ℝ) ≤ b_epsilon * (n : ℝ) ^ 2 →
a_epsilon * (1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2) ≤
binaryExactMinimaxRisk n d epsilon
Re-export of the cited fixed-sample one-arm lower-bound interface from the binary source development (Zeng et al., arXiv:2405.00118v3, Theorem 2, Appendix C.5, Lemma 4, and Appendix D.2).
Definition (Lean source)
abbrev ZengOneArmMinimaxLower :=
CausalSmith.Stat.DiscreteAteMinimaxLoggap.ZengOneArmMinimaxLower
Cited gate (Zeng, Balakrishnan, Han, and Kennedy, 2024, revised 2026). Source: arXiv:2405.00118v3, Lemma 1 and Appendix C.7, equation (26) and the reciprocal-occupancy display. The constants are quantified before the model parameters and hence depend only on epsilon.
Definition (Lean source)
def ZengUsableOccupancyReciprocal (epsilon : ℝ) : Prop :=
∃ b_epsilon B_epsilon : ℝ,
0 < b_epsilon ∧ 0 < B_epsilon ∧
∀ n d : ℕ, ∀ M sigma : ℝ, ∀ P : ModelClass d epsilon M sigma,
realMass (productLaw n P.law) {s | usableTotal s = 0} ≤
2 * exp (-b_epsilon * (n : ℝ) ^ 2 / (max n d : ℕ)) ∧
(∫ s, (if 0 < usableTotal s then (1 : ℝ) / usableTotal s else 0)
∂productLaw n P.law) ≤
B_epsilon * ((max n d : ℕ) / (n : ℝ) ^ 2 +
exp (-b_epsilon * (n : ℝ) ^ 2 / (max n d : ℕ)))
sigma_bin is the actual maximal binary treatment-effect heterogeneity, not merely an arbitrary envelope.
Definition (Lean source)
def BinaryMaximalHeterogeneity {d : ℕ} (P : BinLaw d)
(sigma_bin : ℝ) : Prop :=
0 ≤ sigma_bin ∧
(∀ k,
|(CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P true k -
CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P false k) -
CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P| ≤ sigma_bin) ∧
∃ k,
|(CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P true k -
CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P false k) -
CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P| = sigma_bin
Cited gate (Zeng, Balakrishnan, Han, and Kennedy, 2024, revised 2026). Source: arXiv:2405.00118v3, Theorem 3 on page 11 and Appendix C.7. This is the published bias/variance guarantee for their equation-(13) estimator.
Definition (Lean source)
def ZengBinaryCollisionUpper (epsilon : ℝ) : Prop :=
0 < epsilon ∧ epsilon < 1 / 2 →
∃ C_epsilon b_epsilon : ℝ, 0 < C_epsilon ∧ 0 < b_epsilon ∧
∀ n d : ℕ, 0 < d → ∀ P : BinLaw d, ∀ sigma_bin : ℝ,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.Overlap epsilon P →
BinaryMaximalHeterogeneity P sigma_bin →
|(∫ s, sourceCollisionEstimator s
∂CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P n) -
CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P| ≤
sigma_bin + 2 * exp (-b_epsilon * (n : ℝ) ^ 2 / (max n d : ℕ)) ∧
variance sourceCollisionEstimator
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P n) ≤
C_epsilon * (sigma_bin ^ 2 + (d : ℝ) / (n : ℝ) ^ 2 + 1 / (n : ℝ))
Helpers.ConcreteExactHandle 10 declarations This module packages the canonical full-data coupling and the deterministic zero-padding identities used by the exact half of the least-favorable handle.
Concrete exact-family handle facts
This module packages the canonical full-data coupling and the deterministic zero-padding identities used by the exact half of the least-favorable handle.
def binaryPadFullObs
Include a binary full-data record on the first m cells into an ambient alphabet of size d.
Definition (Lean source)
def binaryPadFullObs {m d : ℕ} (hmd : m ≤ d) :
BinaryFullObs m → BinaryFullObs d :=
fun z => ⟨⟨z.x, lt_of_lt_of_le z.x.isLt hmd⟩, z.a, z.b0, z.b1⟩
theorem binaryOutcomePMF_binaryPadLaw
If the source alphabet embeds in the target alphabet, padding a binary law leaves its conditional outcome PMFs unchanged on the embedded coordinates.
Formal statement
hypotheses
conclusion
binaryOutcomePMF (binaryPadLaw hmd P) a ⟨k, lt_of_lt_of_le k.isLt hmd⟩
= binaryOutcomePMF P a k
Proof (Lean source)
lemma binaryOutcomePMF_binaryPadLaw {m d : ℕ} (hmd : m ≤ d)
(P : BinLaw m) (a : Bool) (k : Fin m) :
binaryOutcomePMF (binaryPadLaw hmd P) a
⟨k, lt_of_lt_of_le k.isLt hmd⟩ =
binaryOutcomePMF P a k := by
apply PMF.ext
intro b
simp only [binaryOutcomePMF, PMF.ofFintype_apply]
rw [binaryPadLaw_outcomeMean_image]
theorem binaryPadFullObs_independentLift
If the source alphabet embeds in the target alphabet, padding commutes with the independent full-data lift of an observation.
Formal statement
hypotheses
conclusion
binaryPadFullObs hmd (binaryIndependentLift z other)
= binaryIndependentLift (binaryPadObs hmd z) other
Proof (Lean source)
lemma binaryPadFullObs_independentLift {m d : ℕ} (hmd : m ≤ d)
(z : BinObs m) (other : Bool) :
binaryPadFullObs hmd (binaryIndependentLift z other) =
binaryIndependentLift (binaryPadObs hmd z) other := by
rcases z with ⟨k, a, y⟩
cases a <;> rfl
theorem binaryIndependentFullPMF_binaryPadLaw
If the source alphabet embeds in the target alphabet, the canonical independent binary full-data coupling commutes with zero-mass cell padding.
Formal statement
hypotheses
m d :
ℕ
hmd :
m ≤ d
P :
BinLaw m
conclusion
= binaryIndependentFullPMF (binaryPadLaw hmd P)
Proof (Lean source)
lemma binaryIndependentFullPMF_binaryPadLaw {m d : ℕ} (hmd : m ≤ d)
(P : BinLaw m) :
map (binaryPadFullObs hmd) (binaryIndependentFullPMF P) =
binaryIndependentFullPMF (binaryPadLaw hmd P) := by
unfold binaryIndependentFullPMF
rw [PMF.map_bind, binaryPadLaw, PMF.bind_map]
apply congrArg (bind P.pmf)
funext z
rw [PMF.map_comp]
simp only [Function.comp_apply]
change map (binaryPadFullObs hmd ∘ binaryIndependentLift z)
(binaryOutcomePMF P (!z.2.1) z.1) =
map (binaryIndependentLift (binaryPadObs hmd z))
(binaryOutcomePMF (binaryPadLaw hmd P) (!z.2.1)
(binaryPadObs hmd z).1)
rw [show (binaryPadObs hmd z).1 =
⟨z.1, lt_of_lt_of_le z.1.isLt hmd⟩ from rfl,
binaryOutcomePMF_binaryPadLaw hmd P (!z.2.1) z.1]
congr 1
funext other
exact binaryPadFullObs_independentLift hmd z other
theorem binaryIndependentFullCoupling_spec
The independent-potential-outcome lift used by affineBinaryRealLaw is a full-data coupling of its binary observed law.
Formal statement
hypotheses
d :
ℕ
P :
BinLaw d
conclusion
Proof (Lean source)
lemma binaryIndependentFullCoupling_spec {d : ℕ} (P : BinLaw d) :
BinaryFullCoupling P (binaryIndependentFullPMF P).toMeasure := by
constructor
· infer_instance
· rw [PMF.toMeasure_map BinaryFullObs.observed _ (measurable_of_finite _)]
exact congrArg toMeasure (map_binaryIndependentFullPMF_observed P)
theorem affineBinaryPadded_fullLaw
If the source alphabet embeds in the target alphabet, the full-data law of the padded affine embedding is exactly the affine pushforward of the canonical independent binary coupling.
Formal statement
hypotheses
m d :
ℕ
hmd :
m ≤ d
M :
ℝ
P :
BinLaw m
conclusion
(affineBinaryRealLaw M (binaryPadLaw hmd P)).fullLaw
= Measure.map (BinaryFullObs.affine M (fun k : Fin m => ⟨k, lt_of_lt_of_le k.isLt hmd⟩)) (binaryIndependentFullPMF P).toMeasure
Proof (Lean source)
lemma affineBinaryPadded_fullLaw {m d : ℕ} (hmd : m ≤ d) (M : ℝ)
(P : BinLaw m) :
(affineBinaryRealLaw M (binaryPadLaw hmd P)).fullLaw =
Measure.map (BinaryFullObs.affine M
(fun k : Fin m => ⟨k, lt_of_lt_of_le k.isLt hmd⟩))
(binaryIndependentFullPMF P).toMeasure := by
change
(map (BinaryFullObs.affine M (fun k : Fin d => k))
(binaryIndependentFullPMF (binaryPadLaw hmd P))).toMeasure = _
rw [← binaryIndependentFullPMF_binaryPadLaw hmd P, PMF.map_comp]
have hfun :
BinaryFullObs.affine M (fun k : Fin d => k) ∘ binaryPadFullObs hmd =
BinaryFullObs.affine M
(fun k : Fin m => ⟨k, lt_of_lt_of_le k.isLt hmd⟩) := by
funext z
rfl
rw [hfun]
exact (PMF.toMeasure_map _ _ (by fun_prop)).symm
theorem affineBinaryPadded_observedLaw
If the source alphabet embeds in the target alphabet, the observed law of a padded affine source is the deterministic coordinatewise padding-and-scaling pushforward of the source observation.
Formal statement
hypotheses
m d :
ℕ
hmd :
m ≤ d
M :
ℝ
P :
BinLaw m
conclusion
(affineBinaryRealLaw M (binaryPadLaw hmd P)).observedLaw
= Measure.map (fun z : BinObs m => affineObserved M (binaryPadObs hmd z)) (CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P)
Proof (Lean source)
lemma affineBinaryPadded_observedLaw {m d : ℕ} (hmd : m ≤ d) (M : ℝ)
(P : BinLaw m) :
(affineBinaryRealLaw M (binaryPadLaw hmd P)).observedLaw =
Measure.map (fun z : BinObs m => affineObserved M (binaryPadObs hmd z))
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P) := by
rw [(affineBinaryRealLaw_embedding M (binaryPadLaw hmd P)).1,
binaryPadLaw_obsLaw, Measure.map_map]
· rfl
· exact measurable_affineObserved M
· exact measurable_of_finite _
theorem affineBinaryPadded_cellMass_image
If the source alphabet embeds in the target alphabet, padding and affine outcome scaling preserve every source cell mass on the embedded coordinates.
Formal statement
hypotheses
conclusion
(affineBinaryRealLaw M (binaryPadLaw hmd P)).cellMass ⟨k, lt_of_lt_of_le k.isLt hmd⟩
= CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P k
Proof (Lean source)
lemma affineBinaryPadded_cellMass_image {m d : ℕ} (hmd : m ≤ d) (M : ℝ)
(P : BinLaw m) (k : Fin m) :
(affineBinaryRealLaw M (binaryPadLaw hmd P)).cellMass
⟨k, lt_of_lt_of_le k.isLt hmd⟩ =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P k := by
exact binaryPadLaw_cellMass_image hmd P k
theorem affineBinaryPadded_cellMass_off_image
If the source alphabet embeds in the target alphabet and the stated condition on the cell holds, every ambient coordinate outside the padded source image has zero mass.
Formal statement
hypotheses
conclusion
(affineBinaryRealLaw M (binaryPadLaw hmd P)).cellMass k = 0
Proof (Lean source)
lemma affineBinaryPadded_cellMass_off_image {m d : ℕ} (hmd : m ≤ d) (M : ℝ)
(P : BinLaw m) (k : Fin d)
(hk : ∀ r : Fin m, (⟨r, lt_of_lt_of_le r.isLt hmd⟩ : Fin d) ≠ k) :
(affineBinaryRealLaw M (binaryPadLaw hmd P)).cellMass k = 0 := by
exact binaryPadLaw_cellMass_off_image hmd P k hk
theorem affineBinaryPadded_rawAteFormula
If the source alphabet embeds in the target alphabet and the source law satisfies the stated model condition, the exact padded affine embedding multiplies the binary ATE by M.
Formal statement
hypotheses
m d :
ℕ
hmd :
m ≤ d
epsilon M :
ℝ
P :
BinLaw m
hP :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.Overlap epsilon P
conclusion
= M * CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P
Proof (Lean source)
lemma affineBinaryPadded_rawAteFormula {m d : ℕ} (hmd : m ≤ d)
{epsilon M : ℝ} (P : BinLaw m)
(hP : CausalSmith.Stat.DiscreteAteMinimaxLoggap.Overlap epsilon P) :
rawAteFormula (affineBinaryRealLaw M (binaryPadLaw hmd P)) =
M * CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P := by
rw [rawAteFormula_eq_mul_binaryAte_of_embedding
(affineBinaryRealLaw_embedding M (binaryPadLaw hmd P))
(binaryPadLaw_overlap hmd hP),
binaryPadLaw_ateFunctional hmd hP]
Helpers.ConcreteHandleCertificates 4 declarations This module packages ambient membership and coupling certificates for handles whose fields are the canonical padded binary constructions.
Concrete least-favorable handle certificates
This module packages ambient membership and coupling certificates for handles whose fields are the canonical padded binary constructions.
theorem exactEmbeddingMembership_of_affinePadded
If the cap satisfies the exact-embedding bound and the padded dimension satisfies the cap bound and the overlap constant is positive and the overlap constant is below one half and the outcome scale satisfies its stated bound and the specified embedding certificate holds, a handle using the canonical padded affine exact embedding has the required radius-zero ambient membership certificate.
Formal statement
hypotheses
n d :
ℕ
epsilon M sigma :
ℝ
H :
LeastFavorableHandle n d epsilon M sigma
hcap :
0 < H.exactCap
hcapd :
H.exactCap ≤ d
he0 :
0 < epsilon
he1 :
epsilon < 1 / 2
hM :
1 ≤ M
hembedding :
∀ P, H.exactEmbedding P = affineBinaryRealLaw M (binaryPadLaw hcapd P.1)
conclusion
Proof (Lean source)
lemma exactEmbeddingMembership_of_affinePadded {n d : ℕ}
{epsilon M sigma : ℝ} (H : LeastFavorableHandle n d epsilon M sigma)
(hcap : 0 < H.exactCap) (hcapd : H.exactCap ≤ d)
(he0 : 0 < epsilon) (he1 : epsilon < 1 / 2) (hM : 1 ≤ M)
(hembedding : ∀ P, H.exactEmbedding P =
affineBinaryRealLaw M (binaryPadLaw hcapd P.1)) :
ExactEmbeddingMembership H := by
intro P _hP
refine ⟨affineBinaryPaddedLaw_model hcapd hcap P.1 he0 he1 hM
(by norm_num) (by norm_num) P.2, ?_⟩
exact (hembedding P).symm
theorem radialEmbeddingMembership_of_contractedPadded
If the padded dimension satisfies the cap bound and the overlap constant is positive and the overlap constant is below one half and the outcome scale satisfies its stated bound and the heterogeneity radius is nonnegative and the heterogeneity radius is at most two and the specified embedding certificate holds, a handle using the padded Bernoulli-contracted affine embedding has the required radius-indexed ambient membership certificate.
Formal statement
hypotheses
n d :
ℕ
epsilon M sigma :
ℝ
H :
LeastFavorableHandle n d epsilon M sigma
hcapd :
H.radialCap ≤ d
he0 :
0 < epsilon
he1 :
epsilon < 1 / 2
hM :
1 ≤ M
hs0 :
0 ≤ sigma
hs2 :
sigma ≤ 2
hembedding :
∀ P,
H.radialEmbedding P
conclusion
Proof (Lean source)
lemma radialEmbeddingMembership_of_contractedPadded {n d : ℕ}
{epsilon M sigma : ℝ} (H : LeastFavorableHandle n d epsilon M sigma)
(hcapd : H.radialCap ≤ d)
(he0 : 0 < epsilon) (he1 : epsilon < 1 / 2) (hM : 1 ≤ M)
(hs0 : 0 ≤ sigma) (hs2 : sigma ≤ 2)
(hembedding : ∀ P, H.radialEmbedding P =
affineBinaryRealLaw M
(binaryPadLaw hcapd (radialContractedBinaryLaw P.1 sigma hs0 hs2))) :
RadialEmbeddingMembership H := by
intro P _hP
refine ⟨radialPaddedAffineLaw_model hcapd P he0 he1 hM hs0 hs2, ?_⟩
exact (hembedding P).symm
theorem exactCouplingCertificate_of_independentFullPMF
If the specified full-data coupling is available, the canonical independent full-data lift packages the exact coupling certificate of a concrete handle.
Formal statement
hypotheses
n d :
ℕ
epsilon M sigma :
ℝ
H :
LeastFavorableHandle n d epsilon M sigma
hcoupling :
∀ P, H.exactCoupling P = (binaryIndependentFullPMF P.1).toMeasure
conclusion
Proof (Lean source)
lemma exactCouplingCertificate_of_independentFullPMF {n d : ℕ}
{epsilon M sigma : ℝ} (H : LeastFavorableHandle n d epsilon M sigma)
(hcoupling : ∀ P, H.exactCoupling P =
(binaryIndependentFullPMF P.1).toMeasure) :
ExactCouplingCertificate H := by
intro P _hP
rw [hcoupling P]
exact binaryIndependentFullCoupling_spec P.1
theorem radialCouplingCertificate_of_independentFullPMF
If the specified full-data coupling is available, the same canonical independent full-data lift packages the radial source coupling certificate of a concrete handle.
Formal statement
hypotheses
n d :
ℕ
epsilon M sigma :
ℝ
H :
LeastFavorableHandle n d epsilon M sigma
hcoupling :
∀ P, H.radialCoupling P = (binaryIndependentFullPMF P.1).toMeasure
conclusion
Proof (Lean source)
lemma radialCouplingCertificate_of_independentFullPMF {n d : ℕ}
{epsilon M sigma : ℝ} (H : LeastFavorableHandle n d epsilon M sigma)
(hcoupling : ∀ P, H.radialCoupling P =
(binaryIndependentFullPMF P.1).toMeasure) :
RadialCouplingCertificate H := by
intro P _hP
rw [hcoupling P]
exact binaryIndependentFullCoupling_spec P.1
Helpers.ConcreteRadialHandle 3 declarations This module identifies the observed experiment generated by the padded Bernoulli contraction and records the mass identities needed by the concrete least-favorable handle.
Concrete radial-family handle facts
This module identifies the observed experiment generated by the padded Bernoulli contraction and records the mass identities needed by the concrete least-favorable handle.
theorem radialPaddedAffine_observedLaw
If the source alphabet embeds in the target alphabet and the heterogeneity radius is nonnegative and the heterogeneity radius is at most two, padding, Bernoulli contraction, and affine outcome scaling agree exactly with applying the common observed-data Markov kernel to the source law.
Formal statement
hypotheses
m d :
ℕ
hmd :
m ≤ d
M sigma :
ℝ
hs0 :
0 ≤ sigma
hs2 :
sigma ≤ 2
P :
BinLaw m
conclusion
= bernoulliContractionObservedKernel (fun k : Fin m => ⟨k, lt_of_lt_of_le k.isLt hmd⟩) M sigma ∘ₘ CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P
Proof (Lean source)
lemma radialPaddedAffine_observedLaw {m d : ℕ} (hmd : m ≤ d) (M sigma : ℝ)
(hs0 : 0 ≤ sigma) (hs2 : sigma ≤ 2) (P : BinLaw m) :
(affineBinaryRealLaw M
(binaryPadLaw hmd (radialContractedBinaryLaw P sigma hs0 hs2))).observedLaw =
bernoulliContractionObservedKernel
(fun k : Fin m => ⟨k, lt_of_lt_of_le k.isLt hmd⟩) M sigma ∘ₘ
CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P := by
rw [(affineBinaryRealLaw_embedding M _).1, binaryPadLaw_obsLaw]
ext E hE
rw [Measure.map_apply (measurable_affineObserved M) hE,
Measure.map_apply (measurable_of_finite _)
(hE.preimage (measurable_affineObserved M)),
Measure.bind_apply hE (Kernel.aemeasurable _)]
unfold bernoulliContractionObservedKernel
rw [CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw]
rw [radialContractedBinaryLaw, PMF.toMeasure_bind_apply
(p := P.pmf)
(f := fun z => (radialContractionPMF sigma z.2.2 hs0 hs2).map
(fun b' => (⟨z.1, z.2.1, b'⟩ : BinObs m)))
(s := binaryPadObs hmd ⁻¹' affineObserved M ⁻¹' E)
((hE.preimage (measurable_affineObserved M)).preimage
(measurable_of_finite _))]
rw [tsum_fintype, lintegral_fintype]
classical
apply Finset.sum_congr rfl
intro z _hz
rw [show CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P {z} = P.pmf z by
exact PMF.toMeasure_apply_singleton P.pmf z (MeasurableSet.singleton z)]
have hinner :
((radialContractionPMF sigma z.2.2 hs0 hs2).map
(fun b' => (⟨z.1, z.2.1, b'⟩ : BinObs m))).toMeasure
(binaryPadObs hmd ⁻¹' affineObserved M ⁻¹' E) =
(bernoulliContractionObservedKernel
(fun k : Fin m => ⟨k, lt_of_lt_of_le k.isLt hmd⟩) M sigma z) E := by
change _ = (Measure.map
(fun r : ℝ => (⟨⟨z.1, lt_of_lt_of_le z.1.isLt hmd⟩,
z.2.1, M * (r - 1 / 2)⟩ : Obs d))
(bernoulliLaw
(bernoulliContractionSuccess sigma z.2.2))) E
have hout : Measurable (fun r : ℝ =>
(⟨⟨z.1, lt_of_lt_of_le z.1.isLt hmd⟩,
z.2.1, M * (r - 1 / 2)⟩ : Obs d)) := by
rw [measurable_comap_iff]
change Measurable (fun r : ℝ =>
((⟨z.1, lt_of_lt_of_le z.1.isLt hmd⟩ : Fin d),
z.2.1, M * (r - 1 / 2)))
fun_prop
rw [Measure.map_apply hout hE]
rw [PMF.toMeasure_map_apply
(f := fun b' => (⟨z.1, z.2.1, b'⟩ : BinObs m))
(p := radialContractionPMF sigma z.2.2 hs0 hs2)
(s := binaryPadObs hmd ⁻¹' affineObserved M ⁻¹' E)
(measurable_of_finite _)
((hE.preimage (measurable_affineObserved M)).preimage
(measurable_of_finite _))]
have hleft : MeasurableSet
((fun b' => (⟨z.1, z.2.1, b'⟩ : BinObs m)) ⁻¹'
binaryPadObs hmd ⁻¹' affineObserved M ⁻¹' E) :=
((hE.preimage (measurable_affineObserved M)).preimage
(measurable_of_finite _)).preimage (measurable_of_finite _)
rw [PMF.toMeasure_apply
(p := radialContractionPMF sigma z.2.2 hs0 hs2) hleft,
tsum_fintype, Fintype.sum_bool]
unfold bernoulliLaw
rw [Measure.add_apply, Measure.smul_apply, Measure.smul_apply]
simp only [Measure.dirac_apply, smul_eq_mul]
by_cases h1 : (fun r : ℝ =>
(⟨⟨z.1, lt_of_lt_of_le z.1.isLt hmd⟩,
z.2.1, M * (r - 1 / 2)⟩ : Obs d)) 1 ∈ E <;>
by_cases h0 : (fun r : ℝ =>
(⟨⟨z.1, lt_of_lt_of_le z.1.isLt hmd⟩,
z.2.1, M * (r - 1 / 2)⟩ : Obs d)) 0 ∈ E <;>
simp [Set.indicator_apply, binaryPadObs, affineObserved, h1, h0,
radialContractionPMF, PMF.ofFintype_apply]
rw [hinner, mul_comm]
rfl
theorem radialPaddedAffine_cellMass_image
If the source alphabet embeds in the target alphabet and the heterogeneity radius is nonnegative and the heterogeneity radius is at most two, the concrete radial embedding preserves every positive source coordinate's cell mass.
Formal statement
hypotheses
conclusion
(affineBinaryRealLaw M (binaryPadLaw hmd (radialContractedBinaryLaw P sigma hs0 hs2))).cellMass ⟨k, lt_of_lt_of_le k.isLt hmd⟩
= CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P k
Proof (Lean source)
lemma radialPaddedAffine_cellMass_image {m d : ℕ} (hmd : m ≤ d)
(M sigma : ℝ) (hs0 : 0 ≤ sigma) (hs2 : sigma ≤ 2)
(P : BinLaw m) (k : Fin m) :
(affineBinaryRealLaw M
(binaryPadLaw hmd (radialContractedBinaryLaw P sigma hs0 hs2))).cellMass
⟨k, lt_of_lt_of_le k.isLt hmd⟩ =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P k := by
rw [affineBinaryPadded_cellMass_image, radialContractedBinaryLaw_cellMass]
theorem radialPaddedAffine_cellMass_off_image
If the source alphabet embeds in the target alphabet and the heterogeneity radius is nonnegative and the heterogeneity radius is at most two and the stated condition on the cell holds, every ambient coordinate outside the radial padding image has zero mass.
Formal statement
hypotheses
conclusion
= 0
Proof (Lean source)
lemma radialPaddedAffine_cellMass_off_image {m d : ℕ} (hmd : m ≤ d)
(M sigma : ℝ) (hs0 : 0 ≤ sigma) (hs2 : sigma ≤ 2)
(P : BinLaw m) (k : Fin d)
(hk : ∀ r : Fin m, (⟨r, lt_of_lt_of_le r.isLt hmd⟩ : Fin d) ≠ k) :
(affineBinaryRealLaw M
(binaryPadLaw hmd (radialContractedBinaryLaw P sigma hs0 hs2))).cellMass k = 0 := by
exact affineBinaryPadded_cellMass_off_image hmd M _ k hk
Helpers.Estimators 25 declarations
def clip
Clipping to a closed real interval.
def collisionEstimator
The occupancy-weighted arm-mean estimator, with every empirical ratio indicator-totalized and with a zero fallback when no usable cell exists.
Definition (Lean source)
noncomputable def collisionEstimator {n d : ℕ} (M : ℝ)
(sample : Fin n → Obs d) : ℝ :=
clip (-M) M <|
if 0 < usableTotal sample then
(∑ k : Fin d,
if usableCell sample k then
(cellCount sample k : ℝ) *
(empiricalArmMean sample true k - empiricalArmMean sample false k)
else 0) / usableTotal sample
else 0
def inPilot
Whether an index belongs to the pilot half of the sample.
def pilotCount
Pilot occupancy of one cell.
Definition (Lean source)
def pilotCount {n d : ℕ} (sample : Fin n → Obs d) (k : Fin d) : ℕ :=
(Finset.univ.filter fun i => inPilot i ∧ (sample i).x = k).card
def estimationBlockSize
Size of the estimation block I_1.
Definition (Lean source)
def estimationBlockSize (n : ℕ) : ℕ := n - n / 2
def estimationArmCount
Estimation-block arm/cell count.
Definition (Lean source)
def estimationArmCount {n d : ℕ} (sample : Fin n → Obs d)
(a : Bool) (k : Fin d) : ℕ :=
(Finset.univ.filter fun i => !(inPilot i) ∧
(sample i).x = k ∧ (sample i).a = a).card
def estimationCellCount
Estimation-block cell count.
Definition (Lean source)
def estimationCellCount {n d : ℕ} (sample : Fin n → Obs d) (k : Fin d) : ℕ :=
estimationArmCount sample false k + estimationArmCount sample true k
def estimationArmSum
Estimation-block outcome sum.
def estimationArmMean
Indicator-totalized estimation-block arm mean.
Definition (Lean source)
noncomputable def estimationArmMean {n d : ℕ} (sample : Fin n → Obs d)
(a : Bool) (k : Fin d) : ℝ :=
if 0 < estimationArmCount sample a k then
estimationArmSum sample a k / estimationArmCount sample a k
else 0
def shiftedCoefficient
Fixed shifted-Chebyshev coefficient from the paper.
Definition (Lean source)
noncomputable def shiftedCoefficient (K j : ℕ) : ℝ :=
(-1 : ℝ) ^ j * 2 ^ (2 * j + 3) / (K * (K + j + 2) : ℕ) *
choose (K + j + 2) (2 * j + 4)
def orderedMarkedFactorial
Ordered distinct-index falling-factorial statistic with exactly one real mark.
Definition (Lean source)
noncomputable def orderedMarkedFactorial {n d : ℕ} (M : ℝ)
(sample : Fin n → Obs d) (k : Fin d) (a : Bool) (j : ℕ) : ℝ :=
(estimationArmSum sample a k / M) *
((estimationArmCount sample a k - 1).descFactorial j : ℝ) *
(estimationCellCount sample k - (j + 1) : ℕ) /
(estimationBlockSize n).descFactorial (j + 2)
def lightPolynomialTerm
The light-cell polynomial contribution.
Definition (Lean source)
noncomputable def lightPolynomialTerm {n d : ℕ} (M B : ℝ) (K : ℕ)
(sample : Fin n → Obs d) (k : Fin d) : ℝ :=
∑ j ∈ range (K - 1),
shiftedCoefficient K j / B ^ (j + 1) *
(orderedMarkedFactorial M sample k true j -
orderedMarkedFactorial M sample k false j)
def polynomialAlpha0
Calibrated constant multiplying log(en) in the polynomial degree.
def polynomialDegree
Actual shifted-Chebyshev degree used by the polynomial program.
Definition (Lean source)
def heavyEmpiricalTerm
Heavy-cell empirical contribution, normalized by the outcome scale.
Definition (Lean source)
noncomputable def heavyEmpiricalTerm {n d : ℕ} (M : ℝ)
(sample : Fin n → Obs d) (k : Fin d) : ℝ :=
(estimationCellCount sample k : ℝ) / estimationBlockSize n *
((estimationArmMean sample true k - estimationArmMean sample false k) / M)
def rawPolyEstimator
The uncalibrated split-sample heavy/light signed one-mark program. This companion is exposed only so the polynomial-upper lemma can certify its cutoff parameters before the public handle is formed.
Definition (Lean source)
noncomputable def rawPolyEstimator (N : ℕ) (rho : ℝ) {n d : ℕ} (M : ℝ)
(sample : Fin n → Obs d) : ℝ :=
if N ≤ n ∧ (d : ℝ) ≤ rho * n * logEN n then
let K := polynomialDegree n
let B := 4096 * logEN n / (n - n / 2 : ℕ)
M * clip (-1) 1
(∑ k : Fin d,
if 256 * logEN n < pilotCount sample k then
heavyEmpiricalTerm M sample k
else lightPolynomialTerm M B K sample k)
else 0
Dependent calibration data for the polynomial estimator: a cutoff, a positive active-range multiplier, and the active-branch degree certificate.
Definition (Lean source)
def PolynomialHandle :=
{p : ℕ × ℝ //
0 < p.2 ∧
∀ n d : ℕ, 0 < n → 0 < d →
p.1 ≤ n ∧ (d : ℝ) ≤ p.2 * n * logEN n → 2 ≤ polynomialDegree n}
The handle-indexed pilot-split heavy/light signed one-mark estimator. Its body is definitionally the complete raw program, including clipping and the exact zero fallback outside the calibrated range.
Definition (Lean source)
noncomputable def polyEstimator (handle : PolynomialHandle)
{n d : ℕ} (M : ℝ) (sample : Fin n → Obs d) : ℝ :=
rawPolyEstimator handle.N handle.rho M sample
theorem measurable_lightPolynomialTerm
A light-cell signed polynomial contribution is measurable.
Formal statement
hypotheses
conclusion
Proof (Lean source)
lemma measurable_lightPolynomialTerm {n d : ℕ} (M B : ℝ) (K : ℕ)
(k : Fin d) :
Measurable (fun s : Fin n → Obs d => lightPolynomialTerm M B K s k) := by
unfold lightPolynomialTerm
apply measurable_sum
intro j _
exact measurable_const.mul
((measurable_orderedMarkedFactorial M k true j).sub
(measurable_orderedMarkedFactorial M k false j))
theorem measurable_heavyEmpiricalTerm
A heavy-cell empirical contribution is measurable.
Formal statement
hypotheses
conclusion
Proof (Lean source)
lemma measurable_heavyEmpiricalTerm {n d : ℕ} (M : ℝ) (k : Fin d) :
Measurable (fun s : Fin n → Obs d => heavyEmpiricalTerm M s k) := by
unfold heavyEmpiricalTerm
exact (((measurable_of_countable (fun q : ℕ => (q : ℝ))).comp
(measurable_estimationCellCount k)).div measurable_const).mul
(((measurable_estimationArmMean true k).sub
(measurable_estimationArmMean false k)).div measurable_const)
theorem collisionEstimator_admissible
If the outcome scale satisfies its stated bound, the collision construction is measurable and lies in the estimator range.
Formal statement
hypotheses
n d :
ℕ
M :
ℝ
hM :
0 ≤ M
conclusion
Measurable (collisionEstimator (n := n) (d := d) M) ∧
∀ s : Fin n → Obs d, collisionEstimator (n := n) (d := d) M s ∈ Icc (-M) M
Proof (Lean source)
-- @node: collisionEstimator_admissible
lemma collisionEstimator_admissible {n d : ℕ} {M : ℝ} (hM : 0 ≤ M) :
Measurable (collisionEstimator (n := n) (d := d) M) ∧
∀ s : Fin n → Obs d,
collisionEstimator (n := n) (d := d) M s ∈ Icc (-M) M := by
constructor
· unfold collisionEstimator clip
apply Measurable.max measurable_const
apply Measurable.min measurable_const
apply Measurable.ite
· exact measurable_usableTotal measurableSet_Ioi
· apply Measurable.div
· apply measurable_sum
intro k _
apply Measurable.ite
· convert measurable_usableCell k (measurableSet_singleton true) using 1
ext s
simp
· exact ((measurable_of_countable (fun q : ℕ => (q : ℝ))).comp
(measurable_cellCount k)).mul
((measurable_empiricalArmMean true k).sub
(measurable_empiricalArmMean false k))
· exact measurable_const
· exact (measurable_of_countable (fun q : ℕ => (q : ℝ))).comp
measurable_usableTotal
· exact measurable_const
· intro s
unfold collisionEstimator clip
constructor
· exact le_max_left _ _
· rw [max_le_iff]
exact ⟨by linarith, min_le_left _ _⟩
If the outcome scale satisfies its stated bound, the polynomial construction is measurable and lies in the estimator range.
Formal statement
hypotheses
n d N :
ℕ
M rho :
ℝ
hM :
0 ≤ M
conclusion
Measurable (rawPolyEstimator (n := n) (d := d) N rho M) ∧
∀ s : Fin n → Obs d, rawPolyEstimator (n := n) (d := d) N rho M s ∈ Icc (-M) M
Proof (Lean source)
lemma polyEstimator_admissible {n d N : ℕ} {M rho : ℝ} (hM : 0 ≤ M) :
Measurable (rawPolyEstimator (n := n) (d := d) N rho M) ∧
∀ s : Fin n → Obs d,
rawPolyEstimator (n := n) (d := d) N rho M s ∈ Icc (-M) M := by
constructor
· unfold rawPolyEstimator
by_cases hbranch : N ≤ n ∧ (d : ℝ) ≤ rho * n * logEN n
· simp_rw [if_pos hbranch]
apply const_mul
unfold clip
apply Measurable.max measurable_const
apply Measurable.min measurable_const
apply measurable_sum
intro k _
apply Measurable.ite
· exact (((measurable_of_countable (fun q : ℕ => (q : ℝ))).comp
(measurable_pilotCount k)) measurableSet_Ioi)
· exact measurable_heavyEmpiricalTerm M k
· exact measurable_lightPolynomialTerm M
(4096 * logEN n / (n - n / 2 : ℕ)) (polynomialDegree n) k
· simp_rw [if_neg hbranch]
exact measurable_const
· intro s
by_cases hbranch : N ≤ n ∧ (d : ℝ) ≤ rho * n * logEN n
· unfold rawPolyEstimator
rw [if_pos hbranch]
unfold clip
constructor
· have hclip_lower : (-1 : ℝ) ≤ max (-1) (min 1
(∑ k : Fin d, if 256 * logEN n < pilotCount s k then
heavyEmpiricalTerm M s k else
lightPolynomialTerm M (4096 * logEN n / (n - n / 2 : ℕ))
(polynomialDegree n) s k)) := le_max_left _ _
nlinarith
· have hclip_upper : max (-1) (min 1
(∑ k : Fin d, if 256 * logEN n < pilotCount s k then
heavyEmpiricalTerm M s k else
lightPolynomialTerm M (4096 * logEN n / (n - n / 2 : ℕ))
(polynomialDegree n) s k)) ≤ 1 := by
rw [max_le_iff]
exact ⟨by norm_num, min_le_left _ _⟩
nlinarith
· unfold rawPolyEstimator
rw [if_neg hbranch]
exact ⟨by linarith, hM⟩
The collision construction packaged as a member of the estimator space.
Definition (Lean source)
noncomputable def collisionEstimatorElement {n d : ℕ} {M : ℝ} (hM : 1 ≤ M) :
Estimator n d M :=
⟨collisionEstimator M, collisionEstimator_admissible (le_trans zero_le_one hM)⟩
The polynomial handle packaged as an admissible estimator.
Definition (Lean source)
noncomputable def polyEstimatorElement {M : ℝ} {n d : ℕ}
(handle : PolynomialHandle) (hM : 1 ≤ M) :
Estimator n d M :=
⟨polyEstimator handle M, by
change Measurable
(rawPolyEstimator (n := n) (d := d) handle.N handle.rho M) ∧
∀ s : Fin n → Obs d,
rawPolyEstimator handle.N handle.rho M s ∈ Icc (-M) M
exact polyEstimator_admissible (n := n) (d := d)
(N := handle.N) (rho := handle.rho) (le_trans zero_le_one hM)⟩
Deterministic three-branch known-radius selector on two admissible estimators. The result itself remains in the same measurable [-M,M]-valued space.
Definition (Lean source)
noncomputable def totalSelector {n d : ℕ} {M : ℝ} (hM : 1 ≤ M) (sigma : ℝ)
(poly collision : Estimator n d M) : Estimator n d M :=
if collisionComponent n d sigma ≤ min 1 (polynomialComponent n d) then collision
else if polynomialComponent n d < min 1 (collisionComponent n d sigma) then poly
else
⟨fun _ => 0, measurable_const, fun _ =>
⟨by linarith [hM], by linarith [hM]⟩⟩
Helpers.ExactHardFamily 1 declarations
theorem binaryExactMinimaxRisk_hard_family_of_lt Lemma binaryExactMinimaxRisk_hard_family_of_lt in the paper ↗
If the sample is nonempty and the alphabet is nonempty and the overlap constant is positive and the overlap constant is below one half and the logarithmic scale satisfies its stated bound, any level strictly below the exact binary minimax risk is attained as a lower bound against every measurable estimator by some exact source law.
Formal statement
hypotheses
n d :
ℕ
epsilon L :
ℝ
_hn :
0 < n
hd :
0 < d
he0 :
0 < epsilon
he1 :
epsilon < 1 / 2
hL :
L < binaryExactMinimaxRisk n d epsilon
conclusion
∀ est : (Fin n → BinObs d) → ℝ
if
Measurable est
then
∃ P : BinaryExactLaw n d epsilon,
L
≤ CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse (CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n) est (CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1)
Proof (Lean source)
-- @node: binaryExactMinimaxRisk_hard_family_of_lt
lemma binaryExactMinimaxRisk_hard_family_of_lt {n d : ℕ} {epsilon L : ℝ}
(_hn : 0 < n) (hd : 0 < d) (he0 : 0 < epsilon) (he1 : epsilon < 1 / 2)
(hL : L < binaryExactMinimaxRisk n d epsilon) :
∀ est : (Fin n → BinObs d) → ℝ, Measurable est →
∃ P : BinaryExactLaw n d epsilon,
L ≤ CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n)
est (CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1) := by
letI : Nonempty (Fin d) := Fin.pos_iff_nonempty.mp hd
obtain ⟨P0, _htau0, _hv0, _hP0⟩ :=
endpoint_null_exact (n := n) hd he0 he1
letI : Nonempty (BinaryExactLaw n d epsilon) := ⟨P0⟩
intro est hest
let est' : {f : (Fin n → BinObs d) → ℝ // Measurable f} := ⟨est, hest⟩
have hb : BddAbove (range (fun P : BinaryExactLaw n d epsilon ↦
CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n)
est (CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1))) := by
refine ⟨((∑ sample : Fin n → BinObs d, |est sample|) + 1) ^ 2, ?_⟩
rintro _ ⟨P, rfl⟩
exact CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse_le_estimator_abs_sum_bound
P.1 P.2.1 est
have hbelow : BddBelow (range (fun e : {f : (Fin n → BinObs d) → ℝ //
Measurable f} ↦ ⨆ P : BinaryExactLaw n d epsilon,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n)
e.1 (CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1))) := by
refine ⟨0, ?_⟩
rintro _ ⟨e, rfl⟩
have hbe : BddAbove (range (fun P : BinaryExactLaw n d epsilon ↦
CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n)
e.1 (CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1))) := by
refine ⟨((∑ sample : Fin n → BinObs d, |e.1 sample|) + 1) ^ 2, ?_⟩
rintro _ ⟨P, rfl⟩
exact CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse_le_estimator_abs_sum_bound
P.1 P.2.1 e.1
have hmse0 : 0 ≤ CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P0.1 n)
e.1 (CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P0.1) := by
unfold CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse
exact integral_nonneg fun _ => sq_nonneg _
exact hmse0.trans (le_ciSup hbe P0)
have hinf : binaryExactMinimaxRisk n d epsilon ≤
⨆ P : BinaryExactLaw n d epsilon,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n)
est (CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1) := by
unfold binaryExactMinimaxRisk
exact ciInf_le hbelow est'
have hsup : L < ⨆ P : BinaryExactLaw n d epsilon,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n)
est (CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1) :=
hL.trans_le hinf
obtain ⟨P, hLP⟩ := (lt_ciSup_iff hb).mp hsup
exact ⟨P, hLP.le⟩
Helpers.ExactHomogeneityLower 4 declarations This module discharges the fixed-sample exact-homogeneity source interface used by the heterogeneity-frontier lower transfer.
Exact-homogeneity binary lower bound
This module discharges the fixed-sample exact-homogeneity source interface used by the heterogeneity-frontier lower transfer. The parametric part is obtained from the accepted endpoint two-point experiment. The collision part uses the symmetric finite Rademacher mixture from Zeng--Balakrishnan--Han--Kennedy, Theorem 4 and Appendix C.8.
If the alphabet is nonempty and the overlap constant is positive and the overlap constant is below one half, the null endpoint distribution belongs to the exact homogeneous binary model.
Formal statement
hypotheses
Proof (Lean source)
lemma endpoint_null_exact {n d : ℕ} [Nonempty (Fin d)]
(hd : 0 < d) {epsilon : ℝ}
(he0 : 0 < epsilon) (he1 : epsilon < 1 / 2) :
∃ P : BinaryExactLaw n d epsilon,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1 = 0 ∧
∃ hv : ValidDGP
(mC (C := Fin d) (1 / 2))
(gNull (C := Fin d) (1 / 2) (1 / 2)),
P.1 = CausalSmith.Stat.DiscreteAteMinimaxLoggap.endpointParametricLaw hv := by
let hv := validDGP_null
(C := Fin d) (m₀ := (1 / 2 : ℝ)) (g₀ := (1 / 2 : ℝ)) (g₁ := (1 / 2 : ℝ))
(by norm_num) (by norm_num) (by norm_num) (by norm_num) (by norm_num) (by norm_num)
let P : BinLaw d :=
CausalSmith.Stat.DiscreteAteMinimaxLoggap.endpointParametricLaw hv
have hmass (k : Fin d) :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P k = 1 / (d : ℝ) := by
have hcard : (card (Fin d) : ℝ) ≠ 0 :=
Nat.cast_ne_zero.mpr Fintype.card_ne_zero
simp [P, CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.endpointParametricLaw_jointMass,
obsReal,
mC,
gNull]
ring
have hover : CausalSmith.Stat.DiscreteAteMinimaxLoggap.Overlap epsilon P := by
intro k hk
have hhalf :=
CausalSmith.Stat.DiscreteAteMinimaxLoggap.endpointParametricLaw_overlap hv k hk
constructor <;> linarith
have heffect (k l : Fin d) :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P true k -
CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P false k =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P true l -
CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P false l := by
have hcard : (card (Fin d) : ℝ) ≠ 0 :=
Nat.cast_ne_zero.mpr Fintype.card_ne_zero
simp [P, CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.endpointParametricLaw_jointMass,
obsReal,
mC,
gNull]
refine ⟨⟨P, hover, hmass, heffect⟩,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.endpointParametricLaw_ate hv, hv, rfl⟩
theorem binaryExactMinimaxRisk_parametric_lower
If the sample is nonempty and the alphabet is nonempty and the overlap constant is positive and the overlap constant is below one half, the exact-homogeneity binary source class contains the standard randomized Bernoulli two-point experiment, hence retains the parametric 1/n lower term.
Formal statement
hypotheses
n d :
ℕ
epsilon :
ℝ
hn :
0 < n
hd :
0 < d
he0 :
0 < epsilon
he1 :
epsilon < 1 / 2
conclusion
1 / (100 * (n : ℝ)) ≤ binaryExactMinimaxRisk n d epsilon
Proof (Lean source)
lemma binaryExactMinimaxRisk_parametric_lower {n d : ℕ} {epsilon : ℝ}
(hn : 0 < n) (hd : 0 < d) (he0 : 0 < epsilon) (he1 : epsilon < 1 / 2) :
1 / (100 * (n : ℝ)) ≤ binaryExactMinimaxRisk n d epsilon := by
letI : Nonempty (Fin d) := Fin.pos_iff_nonempty.mp hd
obtain ⟨P0, htau0, hv0, hP0⟩ := endpoint_null_exact (n := n) hd he0 he1
obtain ⟨delta, hdelta, hdeltasq, P1, htau1, hv1, hP1⟩ :=
endpoint_pert_exact hn hd he0 he1
have hreg : (n : ℝ) *
((1 / 2 : ℝ) * delta ^ 2 / ((1 / 2 : ℝ) * (1 - 1 / 2))) ≤ log 2 := by
rw [hdeltasq]
have hlog : (8 / 25 : ℝ) ≤ log 2 :=
le_trans (by norm_num) (le_of_lt Real.log_two_gt_d9)
convert hlog using 1 <;> field_simp <;> ring
have htv : tvDist
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P0.1 n)
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P1.1 n) ≤ 1 / 2 := by
rw [hP0, hP1]
simpa [CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.endpointParametricLaw,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.endpointParametricPertLaw,
productLaw,
obsLaw] using
(tvDist_productLaw_le_half
hv0 hv1 (by norm_num) (by norm_num) (by norm_num) (by norm_num)
(by norm_num) (by norm_num) hreg)
have h := binaryExactMinimaxRisk_two_point P0 P1 hdelta.le htau0 htau1 htv
rw [hdeltasq] at h
convert h using 1 <;> field_simp <;> ring
If the sample is nonempty and the alphabet is nonempty and the overlap constant is positive and the overlap constant is below one half and the stated dn condition holds, the exact-homogeneity binary minimax risk obeys the collision-regime lower bound.
Formal statement
hypotheses
n d :
ℕ
epsilon :
ℝ
hn :
0 < n
hd :
0 < d
he0 :
0 < epsilon
he1 :
epsilon < 1 / 2
hdn :
(d : ℝ) ≤ (n : ℝ) ^ 2
conclusion
((min (1 / 2 : ℝ) (8 * (1 / 2 - epsilon) ^ 2)) ^ 2 / 64) * ((d : ℝ) / (n : ℝ) ^ 2)
≤ binaryExactMinimaxRisk n d epsilon
Proof (Lean source)
lemma binaryExactMinimaxRisk_collision_lower {n d : ℕ} {epsilon : ℝ}
(hn : 0 < n) (hd : 0 < d) (he0 : 0 < epsilon) (he1 : epsilon < 1 / 2)
(hdn : (d : ℝ) ≤ (n : ℝ) ^ 2) :
((min (1 / 2 : ℝ) (8 * (1 / 2 - epsilon) ^ 2)) ^ 2 / 64) *
((d : ℝ) / (n : ℝ) ^ 2) ≤ binaryExactMinimaxRisk n d epsilon := by
letI : Nonempty (Fin d) := Fin.pos_iff_nonempty.mp hd
letI : NeZero d := ⟨Nat.ne_of_gt hd⟩
let eta : ℝ := 1 / 2 - epsilon
let u : ℝ := min (1 / 2 : ℝ) (8 * eta ^ 2)
let q : ℝ := u * sqrt d / n
let r : ℝ := sqrt (q / 8)
have hnR : (0 : ℝ) < n := by exact_mod_cast hn
have hdR : (0 : ℝ) < d := by exact_mod_cast hd
have heta : 0 < eta := by dsimp [eta]; linarith
have hu0 : 0 < u := by dsimp [u]; positivity
have huhalf : u ≤ 1 / 2 := min_le_left _ _
have hueta : u ≤ 8 * eta ^ 2 := min_le_right _ _
have hsqrt : 0 < sqrt (d : ℝ) := Real.sqrt_pos.2 hdR
have hsqrt_le : sqrt (d : ℝ) ≤ (n : ℝ) := by
have hs := Real.sqrt_le_sqrt hdn
rw [Real.sqrt_sq_eq_abs, abs_of_pos hnR] at hs
exact hs
have hq0 : 0 ≤ q := by dsimp [q]; positivity
have hqU : q ≤ u := by
dsimp [q]
have hratio : sqrt (d : ℝ) / (n : ℝ) ≤ 1 :=
(div_le_one hnR).2 hsqrt_le
calc
u * sqrt (d : ℝ) / (n : ℝ) =
u * (sqrt (d : ℝ) / (n : ℝ)) := by ring
_ ≤ u * 1 := mul_le_mul_of_nonneg_left hratio hu0.le
_ = u := mul_one u
have hqhalf : q ≤ 1 / 2 := hqU.trans huhalf
have hqpos : 0 < q := by dsimp [q]; positivity
have hr0 : 0 ≤ r := Real.sqrt_nonneg _
have hrsq : r ^ 2 = q / 8 := by
dsimp [r]
exact Real.sq_sqrt (by positivity)
have hrquarter : r ≤ 1 / 4 := by
nlinarith [hrsq, sq_nonneg (r - 1 / 4)]
have hrE : r ≤ 1 / 2 - epsilon := by
change r ≤ eta
have hqeta : q / 8 ≤ eta ^ 2 := by nlinarith [hqU, hueta]
nlinarith [hrsq, sq_nonneg (r - eta)]
have hrhalf : r < 1 / 2 := by linarith
have hrq : r + q / 4 ≤ 1 / 2 := by nlinarith [hrquarter, hqhalf]
let Gamma : ℝ := q - q ^ 2 / 2 + q ^ 3 / 8
have hGamma0 : 0 ≤ Gamma := by
have hbracket : 0 ≤ 1 - q / 2 + q ^ 2 / 8 := by
nlinarith [sq_nonneg (q - 2)]
dsimp [Gamma]
nlinarith [mul_nonneg hq0 hbracket]
have hGammaQ : Gamma ≤ q := by
have hmul : q ^ 2 * (q - 4) ≤ 0 :=
mul_nonpos_of_nonneg_of_nonpos (sq_nonneg q) (by linarith)
dsimp [Gamma]
nlinarith
have hGamma1 : Gamma ≤ 1 := hGammaQ.trans (hqhalf.trans (by norm_num))
have hqsq : q ^ 2 = u ^ 2 * (d : ℝ) / (n : ℝ) ^ 2 := by
dsimp [q]
calc
(u * sqrt (d : ℝ) / (n : ℝ)) ^ 2 =
u ^ 2 * (sqrt (d : ℝ)) ^ 2 / (n : ℝ) ^ 2 := by ring
_ = u ^ 2 * (d : ℝ) / (n : ℝ) ^ 2 := by rw [Real.sq_sqrt hdR.le]
have hreg : 2 * (n : ℝ) ^ 2 * (Gamma / 2) ^ 2 ≤
(d : ℝ) * log 2 := by
have hGsq : Gamma ^ 2 ≤ q ^ 2 := by nlinarith [hGamma0, hGammaQ, hq0]
have hlog : (1 / 8 : ℝ) ≤ log 2 :=
le_trans (by norm_num) (le_of_lt Real.log_two_gt_d9)
have hbase : (n : ℝ) ^ 2 * q ^ 2 / 2 = u ^ 2 * (d : ℝ) / 2 := by
rw [hqsq]
field_simp [hnR.ne']
have huSq : u ^ 2 ≤ (1 / 2 : ℝ) ^ 2 := by nlinarith [hu0.le, huhalf]
calc
2 * (n : ℝ) ^ 2 * (Gamma / 2) ^ 2
= (n : ℝ) ^ 2 * Gamma ^ 2 / 2 := by ring
_ ≤ (n : ℝ) ^ 2 * q ^ 2 / 2 := by
exact div_le_div_of_nonneg_right
(mul_le_mul_of_nonneg_left hGsq (sq_nonneg (n : ℝ))) (by norm_num)
_ = u ^ 2 * (d : ℝ) / 2 := hbase
_ ≤ (d : ℝ) / 8 := by
have hm := mul_le_mul_of_nonneg_right huSq hdR.le
nlinarith
_ ≤ (d : ℝ) * log 2 := by
have hm := mul_le_mul_of_nonneg_left hlog hdR.le
simpa [div_eq_mul_inv] using hm
have htv := exact_tvDist_le_half (d := d) (n := n) hq0 hr0 hrhalf.le hrq hrsq
hGamma0 hGamma1 (by simpa [Gamma] using hreg)
have hlower := binaryExactMinimaxRisk_collision_aux (n := n) (d := d)
he0 he1 hq0 hr0 hrE hrhalf hrq htv
rw [hqsq] at hlower
change (u ^ 2 / 64) * ((d : ℝ) / (n : ℝ) ^ 2) ≤ _
have halg : (u ^ 2 / 64) * ((d : ℝ) / (n : ℝ) ^ 2) =
u ^ 2 * (d : ℝ) / (n : ℝ) ^ 2 / 64 := by ring
exact halg.trans_le hlower
Zeng--Balakrishnan--Han--Kennedy's exact-homogeneity lower bound, in the fixed finite-sample interface used by the frontier theorem.
Formal statement
Proof (Lean source)
theorem zengBinaryExactHomogeneityLower (epsilon : ℝ) :
ZengBinaryExactHomogeneityLower epsilon := by
intro he
rcases he with ⟨he0, he1⟩
let u : ℝ := min (1 / 2 : ℝ) (8 * (1 / 2 - epsilon) ^ 2)
let a : ℝ := min (1 / 200 : ℝ) (u ^ 2 / 128)
refine ⟨a, 1, 1, ?_, by norm_num, ?_⟩
· have hu : 0 < u := by dsimp [u]; positivity
dsimp [a]
positivity
· intro n d hd hn _hdn
have hnpos : 0 < n := lt_of_lt_of_le Nat.zero_lt_one hn
have hnR : (0 : ℝ) < n := by exact_mod_cast hnpos
have hdR : (0 : ℝ) < d := by exact_mod_cast hd
have hdnSq : (d : ℝ) ≤ (n : ℝ) ^ 2 := by simpa using _hdn
have hpar := binaryExactMinimaxRisk_parametric_lower
(n := n) (d := d) hnpos hd he0 he1
have hcol := binaryExactMinimaxRisk_collision_lower
(n := n) (d := d) hnpos hd he0 he1 hdnSq
have hu : 0 < u := by dsimp [u]; positivity
have ha0 : 0 ≤ a := by dsimp [a]; positivity
by_cases hsmall : d ≤ n
· have hsmallR : (d : ℝ) ≤ n := by exact_mod_cast hsmall
have hyx : (d : ℝ) / (n : ℝ) ^ 2 ≤ 1 / (n : ℝ) := by
apply (div_le_iff₀ (sq_pos_of_pos hnR)).2
field_simp [hnR.ne']
nlinarith
have hsum : 1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2 ≤
2 * (1 / (n : ℝ)) := by linarith
have ha : a ≤ 1 / 200 := by dsimp [a]; exact min_le_left _ _
calc
a * (1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2)
≤ (1 / 200 : ℝ) * (2 * (1 / (n : ℝ))) :=
mul_le_mul ha hsum (by positivity) (by norm_num)
_ = 1 / (100 * (n : ℝ)) := by field_simp [hnR.ne']; ring
_ ≤ binaryExactMinimaxRisk n d epsilon := hpar
· have hlarge : n ≤ d := le_of_lt (Nat.lt_of_not_ge hsmall)
have hlargeR : (n : ℝ) ≤ d := by exact_mod_cast hlarge
have hxy : 1 / (n : ℝ) ≤ (d : ℝ) / (n : ℝ) ^ 2 := by
apply (le_div_iff₀ (sq_pos_of_pos hnR)).2
field_simp [hnR.ne']
nlinarith
have hsum : 1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2 ≤
2 * ((d : ℝ) / (n : ℝ) ^ 2) := by linarith
have ha : a ≤ u ^ 2 / 128 := by dsimp [a]; exact min_le_right _ _
calc
a * (1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2)
≤ (u ^ 2 / 128) *
(2 * ((d : ℝ) / (n : ℝ) ^ 2)) :=
mul_le_mul ha hsum (by positivity) (by positivity)
_ = (u ^ 2 / 64) * ((d : ℝ) / (n : ℝ) ^ 2) := by ring
_ ≤ binaryExactMinimaxRisk n d epsilon := by simpa [u] using hcol
Helpers.FactorialCovariance 32 declarations
def allBlockOrderedMarkedFactorial
The paper's unsplit all-block statistic: all m observations participate, and normalization is by (m)_{j+2} rather than the estimation-half factorial.
Definition (Lean source)
noncomputable def allBlockOrderedMarkedFactorial {m d : ℕ} (M : ℝ)
(sample : Fin m → Obs d) (k : Fin d) (a : Bool) (j : ℕ) : ℝ :=
(∑ idx : Fin (j + 2) → Fin m,
if Injective idx then
((sample (idx 0)).y / M) *
(if (sample (idx 0)).x = k ∧ (sample (idx 0)).a = a then 1 else 0) *
(if (sample (idx 1)).x = k then 1 else 0) *
(∏ q : Fin (j + 2),
if 2 ≤ q.val then
if (sample (idx q)).x = k ∧ (sample (idx q)).a = a then 1 else 0
else 1)
else 0) / m.descFactorial (j + 2)
def allBlockLightPolynomialTerm
One all-block light-cell polynomial contribution.
Definition (Lean source)
noncomputable def allBlockLightPolynomialTerm {m d : ℕ} (M B : ℝ) (K : ℕ)
(sample : Fin m → Obs d) (k : Fin d) : ℝ :=
∑ j ∈ range (K - 1),
shiftedCoefficient K j / B ^ (j + 1) *
(allBlockOrderedMarkedFactorial M sample k true j -
allBlockOrderedMarkedFactorial M sample k false j)
def allBlockMarkedPolynomialSum
Sum of the unsplit all-block statistics over deterministic light cells.
Definition (Lean source)
noncomputable def allBlockMarkedPolynomialSum {m d : ℕ} (M B : ℝ) (K : ℕ)
(S : Finset (Fin d)) (sample : Fin m → Obs d) : ℝ :=
∑ k ∈ S, allBlockLightPolynomialTerm M B K sample k
def markedFactorialCoordinate
Coordinate factors whose ordered product is one all-block marked factorial kernel: the zeroth coordinate carries the outcome mark, the first only selects the cell, and all later coordinates select the arm and cell.
Definition (Lean source)
noncomputable def markedFactorialCoordinate {d : ℕ} (M : ℝ) (k : Fin d)
(a : Bool) (j : ℕ) (q : Fin (j + 2)) (o : Obs d) : ℝ :=
if q.val = 0 then
(o.y / M) * (if o.x = k ∧ o.a = a then 1 else 0)
else if q.val = 1 then
if o.x = k then 1 else 0
else if o.x = k ∧ o.a = a then 1 else 0
theorem prod_markedFactorialCoordinate
Multiplying the coordinate factors recovers the paper's displayed marked kernel, including its redundant later-coordinate product.
Formal statement
hypotheses
conclusion
(∏ q, markedFactorialCoordinate M k a j q (z q))
= ((z 0).y / M) * (if (z 0).x = k ∧ (z 0).a = a then 1 else 0) * (if (z 1).x = k then 1 else 0) * (∏ q : Fin (j + 2), if 2 ≤ q.val then if (z q).x = k ∧ (z q).a = a then 1 else 0 else 1)
Proof (Lean source)
lemma prod_markedFactorialCoordinate {d : ℕ} (M : ℝ) (k : Fin d)
(a : Bool) (j : ℕ) (z : Fin (j + 2) → Obs d) :
(∏ q, markedFactorialCoordinate M k a j q (z q)) =
((z 0).y / M) * (if (z 0).x = k ∧ (z 0).a = a then 1 else 0) *
(if (z 1).x = k then 1 else 0) *
(∏ q : Fin (j + 2),
if 2 ≤ q.val then
if (z q).x = k ∧ (z q).a = a then 1 else 0
else 1) := by
classical
simp [markedFactorialCoordinate, Fin.prod_univ_succ]
by_cases h0 : (z 0).x = k ∧ (z 0).a = a <;>
by_cases h1 : (z 1).x = k <;> simp [h0, h1]
theorem measurable_markedFactorialCoordinate
Every coordinate of the marked factorial kernel is measurable. This is the regularity input needed by the generic mixed-order covariance expansion.
Formal statement
hypotheses
conclusion
Measurable (markedFactorialCoordinate M k a j q)
Proof (Lean source)
lemma measurable_markedFactorialCoordinate {d : ℕ} (M : ℝ) (k : Fin d)
(a : Bool) (j : ℕ) (q : Fin (j + 2)) :
Measurable (markedFactorialCoordinate M k a j q) := by
have htuple : Measurable (fun o : Obs d => (o.x, o.a, o.y)) :=
measurable_iff_comap_le.mpr le_rfl
have hx : Measurable (fun o : Obs d => o.x) := measurable_fst.comp htuple
have ha : Measurable (fun o : Obs d => o.a) :=
measurable_fst.comp (measurable_snd.comp htuple)
have hy : Measurable (fun o : Obs d => o.y) :=
measurable_snd.comp (measurable_snd.comp htuple)
unfold markedFactorialCoordinate
split
· apply mul (hy.div measurable_const)
apply Measurable.ite
· simpa only [preimage, Set.mem_singleton_iff, Prod.mk.injEq] using
((measurableSet_singleton (k, a)).preimage (prodMk hx ha))
· exact measurable_const
· exact measurable_const
· split
· apply Measurable.ite
· exact (measurableSet_singleton k).preimage hx
· exact measurable_const
· exact measurable_const
· apply Measurable.ite
· simpa only [preimage, Set.mem_singleton_iff, Prod.mk.injEq] using
((measurableSet_singleton (k, a)).preimage (prodMk hx ha))
· exact measurable_const
· exact measurable_const
theorem measurable_orderedProductKernel_markedFactorialCoordinate
The full coordinate product defining one marked factorial kernel is measurable under the finite product measurable space.
Formal statement
hypotheses
conclusion
Proof (Lean source)
lemma measurable_orderedProductKernel_markedFactorialCoordinate {d : ℕ}
(M : ℝ) (k : Fin d) (a : Bool) (j : ℕ) :
Measurable (orderedProductKernel
(markedFactorialCoordinate M k a j)) := by
unfold orderedProductKernel
apply Finset.measurable_prod
intro q _
exact (measurable_markedFactorialCoordinate M k a j q).comp
(measurable_pi_apply q)
theorem measurable_mergedProductKernel_markedFactorialCoordinate
Every partial-matching merge of two marked factorial kernels is measurable, including the possible collision of their two real-valued marks.
Formal statement
hypotheses
conclusion
Measurable (mergedProductKernel (markedFactorialCoordinate M k a j) (markedFactorialCoordinate M l b r) N)
Proof (Lean source)
lemma measurable_mergedProductKernel_markedFactorialCoordinate {d : ℕ}
(M : ℝ) (k l : Fin d) (a b : Bool) (j r : ℕ)
(N : PartialMatching (j + 2) (r + 2)) :
Measurable (mergedProductKernel
(markedFactorialCoordinate M k a j)
(markedFactorialCoordinate M l b r) N) := by
unfold mergedProductKernel
apply mul
· apply Finset.measurable_prod
intro q _
exact (measurable_markedFactorialCoordinate M k a j q).comp
(measurable_pi_apply (N.leftInjection q))
· apply Finset.measurable_prod
intro q _
exact (measurable_markedFactorialCoordinate M l b r q).comp
(measurable_pi_apply (N.rightInjection q))
theorem markedFactorialCoordinate_eq_zero_of_cell_ne
If the observation belongs to a different cell, every coordinate factor vanishes away from its designated cell.
Formal statement
hypotheses
conclusion
markedFactorialCoordinate M k a j q o = 0
Proof (Lean source)
lemma markedFactorialCoordinate_eq_zero_of_cell_ne {d : ℕ} (M : ℝ)
(k : Fin d) (a : Bool) (j : ℕ) (q : Fin (j + 2)) (o : Obs d)
(ho : o.x ≠ k) : markedFactorialCoordinate M k a j q o = 0 := by
simp [markedFactorialCoordinate, ho]
theorem mergedProductKernel_markedFactorialCoordinate_eq_zero_of_cell_ne
If the sampling budget satisfies the stated lower bound and the two cells are distinct, a positive-size matching between statistics from different cells has zero merged kernel: a matched observation cannot satisfy both cell selectors.
Formal statement
hypotheses
d :
ℕ
M :
ℝ
k l :
Fin d
a b :
j r :
ℕ
N :
PartialMatching (j + 2) (r + 2)
hN :
0 < N.size
hkl :
k ≠ l
z :
N.MergedIndex → Obs d
conclusion
= 0
Proof (Lean source)
lemma mergedProductKernel_markedFactorialCoordinate_eq_zero_of_cell_ne
{d : ℕ} (M : ℝ) (k l : Fin d) (a b : Bool) (j r : ℕ)
(N : PartialMatching (j + 2) (r + 2))
(hN : 0 < N.size) (hkl : k ≠ l)
(z : N.MergedIndex → Obs d) :
Causalean.Stat.mergedProductKernel
(markedFactorialCoordinate M k a j)
(markedFactorialCoordinate M l b r) N z = 0 := by
classical
obtain ⟨i, hi⟩ := Finset.card_pos.mp
(by simpa [size] using hN)
let j' : Fin (r + 2) := (N.equiv ⟨i, hi⟩).1
have hj' : j' ∈ N.right := (N.equiv ⟨i, hi⟩).2
have hinj : N.rightInjection j' = N.leftInjection i := by
simp only [rightInjection, hj', dite_true,
leftInjection]
congr
simpa [j'] using N.equiv.symm_apply_apply ⟨i, hi⟩
unfold Causalean.Stat.mergedProductKernel
by_cases hx : (z (N.leftInjection i)).x = k
· have hxl : (z (N.rightInjection j')).x ≠ l := by
rw [hinj]
exact fun h => hkl (hx.symm.trans h)
rw [Finset.prod_eq_zero (Finset.mem_univ j')
(markedFactorialCoordinate_eq_zero_of_cell_ne M l b r j'
(z (N.rightInjection j')) hxl), mul_zero]
· rw [Finset.prod_eq_zero (Finset.mem_univ i)
(markedFactorialCoordinate_eq_zero_of_cell_ne M k a j i
(z (N.leftInjection i)) hx), zero_mul]
theorem mergedProductMoment_markedFactorialCoordinate_eq_zero_of_cell_ne
If the sampling budget satisfies the stated lower bound and the two cells are distinct, consequently every positive-overlap merged moment from two different cells is exactly zero.
Formal statement
hypotheses
d :
ℕ
P0 :
Measure (Obs d)
M :
ℝ
k l :
Fin d
a b :
j r :
ℕ
N :
PartialMatching (j + 2) (r + 2)
hN :
0 < N.size
hkl :
k ≠ l
conclusion
= 0
Proof (Lean source)
lemma mergedProductMoment_markedFactorialCoordinate_eq_zero_of_cell_ne
{d : ℕ} (P0 : Measure (Obs d)) (M : ℝ) (k l : Fin d)
(a b : Bool) (j r : ℕ)
(N : PartialMatching (j + 2) (r + 2))
(hN : 0 < N.size) (hkl : k ≠ l) :
mergedProductMoment P0
(markedFactorialCoordinate M k a j)
(markedFactorialCoordinate M l b r) N = 0 := by
unfold mergedProductMoment
apply integral_eq_zero_of_ae
filter_upwards with z
exact mergedProductKernel_markedFactorialCoordinate_eq_zero_of_cell_ne
M k l a b j r N hN hkl z
theorem allBlockOrderedMarkedFactorial_eq_normalizedOrderedProductStatistic
The paper's all-block marked factorial statistic is exactly the generic normalized ordered-product statistic evaluated on the finite sample prefix.
Formal statement
hypotheses
Ω :
Type*
m d :
ℕ
μ :
Measure Ω
P0 :
Measure (Obs d)
S0 :
IIDSample Ω (Obs d) μ P0
M :
ℝ
k :
Fin d
a :
j :
ℕ
ω :
Ω
conclusion
allBlockOrderedMarkedFactorial M (fun i : Fin m => S0.Z i ω) k a j
= normalizedOrderedProductStatistic S0 (markedFactorialCoordinate M k a j) m ω
Proof (Lean source)
lemma allBlockOrderedMarkedFactorial_eq_normalizedOrderedProductStatistic
{Ω : Type*} [MeasurableSpace Ω] {m d : ℕ} {μ : Measure Ω}
{P0 : Measure (Obs d)} (S0 : IIDSample Ω (Obs d) μ P0)
(M : ℝ) (k : Fin d) (a : Bool) (j : ℕ) (ω : Ω) :
allBlockOrderedMarkedFactorial M (fun i : Fin m => S0.Z i ω) k a j =
normalizedOrderedProductStatistic S0
(markedFactorialCoordinate M k a j) m ω := by
classical
unfold allBlockOrderedMarkedFactorial
normalizedOrderedProductStatistic
normalizedFiniteKernelStatistic
orderedProductKernel
finiteInjectiveTuples
simp only [Fintype.card_fin]
rw [div_eq_mul_inv, mul_comm]
congr 1
rw [Finset.sum_filter]
simp_rw [prod_markedFactorialCoordinate]
congr 1
ext
simp
theorem observed_arm_cell_measure_eq_zero_of_cellMass_eq_zero
If the stated condition on the cell holds, a zero-mass cell makes each of its observed arm-cell events null. This discharges the unsupported-cell branch of the marked moment audit without requesting conditional moments where the model deliberately supplies none.
Formal statement
Proof (Lean source)
lemma observed_arm_cell_measure_eq_zero_of_cellMass_eq_zero {d : ℕ}
(P : RealLaw d) (a : Bool) (k : Fin d) (hk : P.cellMass k = 0) :
P.observedLaw {o : Obs d | o.x = k ∧ o.a = a} = 0 := by
have hfactor := P.arm_outcome_factorization a k Set.univ MeasurableSet.univ
let _ : IsProbabilityMeasure (P.outcomeLaw a k) := P.outcome_isProbability a k
have hout : realMass (P.outcomeLaw a k) Set.univ = 1 := by
simp [realMass]
rw [hout, mul_one, hk, zero_mul] at hfactor
have hreal : realMass P.observedLaw {o : Obs d | o.x = k ∧ o.a = a} = 0 := by
simpa using hfactor.symm
rw [realMass, ENNReal.toReal_eq_zero_iff] at hreal
exact hreal.resolve_right (measure_ne_top _ _)
theorem observed_arm_cell_outcome_measure
Restricting the observed law to one arm and cell and then projecting the outcome gives its conditional outcome law scaled by the arm-cell mass.
Formal statement
Proof (Lean source)
lemma observed_arm_cell_outcome_measure {d : ℕ}
(P : RealLaw d) (a : Bool) (k : Fin d) :
Measure.map (fun o : Obs d => o.y)
(P.observedLaw.restrict {o : Obs d | o.x = k ∧ o.a = a}) =
ofReal
(P.cellMass k * (if a then P.propensity k else 1 - P.propensity k)) •
P.outcomeLaw a k := by
let _ : IsProbabilityMeasure (P.outcomeLaw a k) := P.outcome_isProbability a k
have hc : 0 ≤ P.cellMass k *
(if a then P.propensity k else 1 - P.propensity k) :=
mul_nonneg (P.cellMass_range k).1 (by
split <;> simp_all [(P.propensity_range k).1, (P.propensity_range k).2])
apply Measure.ext
intro s hs
have htuple : Measurable (fun o : Obs d => (o.x, o.a, o.y)) :=
measurable_iff_comap_le.mpr le_rfl
have hy : Measurable (fun o : Obs d => o.y) :=
measurable_snd.comp (measurable_snd.comp htuple)
have hxa : MeasurableSet {o : Obs d | o.x = k ∧ o.a = a} := by
have hx : Measurable (fun o : Obs d => o.x) := measurable_fst.comp htuple
have ha : Measurable (fun o : Obs d => o.a) :=
measurable_fst.comp (measurable_snd.comp htuple)
simpa only [preimage, Set.mem_singleton_iff, Prod.mk.injEq] using
(measurableSet_singleton (k, a)).preimage (prodMk hx ha)
rw [Measure.map_apply hy hs, Measure.restrict_apply (hy hs)]
simp only [Measure.smul_apply, preimage]
have hfac := P.arm_outcome_factorization a k s hs
apply (ENNReal.toReal_eq_toReal_iff'
(measure_ne_top _ _)
(ENNReal.mul_ne_top ENNReal.ofReal_ne_top (measure_ne_top _ _))).mp
rw [ENNReal.toReal_mul, ENNReal.toReal_ofReal hc]
simpa [realMass, Set.inter_def, and_assoc, and_left_comm, and_comm] using hfac.symm
theorem outcome_second_moment_le_five_fourths
If the stated condition on the cell holds, on every positive-mass arm and cell, the central-moment and mean envelopes imply integrability of the raw squared outcome and the bound E[Y²] ≤ 5M²/4. This is the paper's moment audit before normalizing the unique real mark.
Formal statement
hypotheses
conclusion
Integrable (fun y : ℝ => y ^ 2) (P.law.outcomeLaw a k) ∧
∫ y, y ^ 2 ∂P.law.outcomeLaw a k ≤ (5 / 4 : ℝ) * M ^ 2
Proof (Lean source)
lemma outcome_second_moment_le_five_fourths {d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (a : Bool) (k : Fin d)
(hk : 0 < P.law.cellMass k) :
Integrable (fun y : ℝ => y ^ 2) (P.law.outcomeLaw a k) ∧
∫ y, y ^ 2 ∂P.law.outcomeLaw a k ≤ (5 / 4 : ℝ) * M ^ 2 := by
let _ : IsProbabilityMeasure (P.law.outcomeLaw a k) :=
P.law.outcome_isProbability a k
have hc := (P.second_moment a k hk).1
have hcL2 : MemLp (fun y : ℝ => y - P.law.outcomeMean a k) 2
(P.law.outcomeLaw a k) :=
(memLp_two_iff_integrable_sq (by fun_prop)).2 hc
have hyL2 : MemLp (fun y : ℝ => y) 2 (P.law.outcomeLaw a k) := by
convert hcL2.add (memLp_const (P.law.outcomeMean a k)) using 1
ext y
simp
have hy2 := hyL2.integrable_sq
refine ⟨hy2, ?_⟩
have hmean : |P.law.outcomeMean a k| ≤ M / 2 :=
P.mean_normalization a k hk
have hmean_sq : P.law.outcomeMean a k ^ 2 ≤ M ^ 2 / 4 := by
have hM : 0 ≤ M / 2 := by nlinarith [P.M_ge_one]
calc
P.law.outcomeMean a k ^ 2 = |P.law.outcomeMean a k| ^ 2 := by
rw [sq_abs]
_ ≤ (M / 2) ^ 2 := pow_le_pow_left₀ (abs_nonneg _) hmean 2
_ = M ^ 2 / 4 := by ring
have hcenter := (P.second_moment a k hk).2
have hcenter_eq :
(∫ y, (y - P.law.outcomeMean a k) ^ 2 ∂P.law.outcomeLaw a k) =
(∫ y, y ^ 2 ∂P.law.outcomeLaw a k) - P.law.outcomeMean a k ^ 2 := by
have hy : Integrable (fun y : ℝ => y) (P.law.outcomeLaw a k) :=
hyL2.integrable (by norm_num)
have hlin : Integrable (fun y : ℝ =>
(2 * P.law.outcomeMean a k) * y) (P.law.outcomeLaw a k) :=
hy.const_mul _
calc
(∫ y, (y - P.law.outcomeMean a k) ^ 2 ∂P.law.outcomeLaw a k) =
∫ y, (y ^ 2 - (2 * P.law.outcomeMean a k) * y) +
P.law.outcomeMean a k ^ 2 ∂P.law.outcomeLaw a k := by
congr 1
funext y
ring
_ = (∫ y, y ^ 2 - (2 * P.law.outcomeMean a k) * y
∂P.law.outcomeLaw a k) + P.law.outcomeMean a k ^ 2 := by
simpa [integral_const, probReal_univ] using
(integral_add (hy2.sub hlin)
(integrable_const (P.law.outcomeMean a k ^ 2)))
_ = ((∫ y, y ^ 2 ∂P.law.outcomeLaw a k) -
(2 * P.law.outcomeMean a k) *
(∫ y, y ∂P.law.outcomeLaw a k)) +
P.law.outcomeMean a k ^ 2 := by
rw [integral_sub hy2 hlin, integral_const_mul]
_ = (∫ y, y ^ 2 ∂P.law.outcomeLaw a k) -
P.law.outcomeMean a k ^ 2 := by
rw [← P.law.outcomeMean_eq]
ring
rw [hcenter_eq] at hcenter
nlinarith
theorem normalized_outcome_second_moment_le_five_fourths
If the stated condition on the cell holds, dividing by the model scale turns the raw outcome-moment audit into the dimensionless bound E[(Y/M)²] ≤ 5/4 used for a collision of two marks.
Formal statement
hypotheses
conclusion
Integrable (fun y : ℝ => (y / M) ^ 2) (P.law.outcomeLaw a k) ∧
∫ y, (y / M) ^ 2 ∂P.law.outcomeLaw a k ≤ (5 / 4 : ℝ)
Proof (Lean source)
lemma normalized_outcome_second_moment_le_five_fourths
{d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (a : Bool) (k : Fin d)
(hk : 0 < P.law.cellMass k) :
Integrable (fun y : ℝ => (y / M) ^ 2) (P.law.outcomeLaw a k) ∧
∫ y, (y / M) ^ 2 ∂P.law.outcomeLaw a k ≤ (5 / 4 : ℝ) := by
have h := outcome_second_moment_le_five_fourths P a k hk
have hM : 0 < M := lt_of_lt_of_le (by norm_num) P.M_ge_one
have hM2 : 0 < M ^ 2 := sq_pos_of_pos hM
have heq : (fun y : ℝ => (y / M) ^ 2) = fun y => y ^ 2 / M ^ 2 := by
funext y
ring
rw [heq]
refine ⟨h.1.div_const _, ?_⟩
have heq' : (fun y : ℝ => y ^ 2 / M ^ 2) =
fun y => (M ^ 2)⁻¹ * y ^ 2 := by
funext y
rw [inv_mul_eq_div]
rw [heq', integral_const_mul, inv_mul_eq_div]
apply (div_le_iff₀ hM2).2
nlinarith [h.2]
theorem integrable_observed_normalized_outcome_sq
The normalized squared outcome is integrable under the full observed law. This packages the finite arm-cell partition needed whenever two marked coordinates collide in a partial matching.
Formal statement
hypotheses
conclusion
Integrable (fun o : Obs d => (o.y / M) ^ 2) P.law.observedLaw
Proof (Lean source)
lemma integrable_observed_normalized_outcome_sq
{d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) :
Integrable (fun o : Obs d => (o.y / M) ^ 2) P.law.observedLaw := by
have htuple : Measurable (fun o : Obs d => (o.x, o.a, o.y)) :=
measurable_iff_comap_le.mpr le_rfl
have hy : Measurable (fun o : Obs d => o.y) :=
measurable_snd.comp (measurable_snd.comp htuple)
have hcell : ∀ k : Fin d, ∀ a : Bool,
Integrable
({o : Obs d | o.x = k ∧ o.a = a}.indicator
(fun o => (o.y / M) ^ 2)) P.law.observedLaw := by
intro k a
have hxa : MeasurableSet {o : Obs d | o.x = k ∧ o.a = a} := by
have hx : Measurable (fun o : Obs d => o.x) := measurable_fst.comp htuple
have ha : Measurable (fun o : Obs d => o.a) :=
measurable_fst.comp (measurable_snd.comp htuple)
simpa only [preimage, Set.mem_singleton_iff, Prod.mk.injEq] using
(measurableSet_singleton (k, a)).preimage (prodMk hx ha)
by_cases hk : P.law.cellMass k = 0
· have hnull := observed_arm_cell_measure_eq_zero_of_cellMass_eq_zero
P.law a k hk
have hzero : Integrable (fun _ : Obs d => (0 : ℝ)) P.law.observedLaw :=
integrable_const 0
refine hzero.congr (ae_iff.mpr ?_)
apply measure_mono_null _ hnull
intro o ho
by_contra hout
change ¬(o.x = k ∧ o.a = a) at hout
have hz : {z : Obs d | z.x = k ∧ z.a = a}.indicator
(fun z => (z.y / M) ^ 2) o = 0 := by
simp [indicator, hout]
exact ho hz.symm
· have hkpos : 0 < P.law.cellMass k :=
lt_of_le_of_ne (P.law.cellMass_range k).1 (Ne.symm hk)
have hc := (normalized_outcome_second_moment_le_five_fourths P a k hkpos).1
have hc' := hc.smul_measure (c := ofReal
(P.law.cellMass k *
(if a then P.law.propensity k else 1 - P.law.propensity k)))
ENNReal.ofReal_ne_top
rw [← observed_arm_cell_outcome_measure P.law a k] at hc'
have hrestrict : Integrable (fun o : Obs d => (o.y / M) ^ 2)
(P.law.observedLaw.restrict {o | o.x = k ∧ o.a = a}) :=
(MeasureTheory.integrable_map_measure
(((measurable_id.div measurable_const).pow_const 2).aestronglyMeasurable)
hy.aemeasurable).mp hc'
exact (integrable_indicator_iff hxa).2 hrestrict
have hsum : Integrable
(fun o : Obs d => ∑ k : Fin d, ∑ a : Bool,
{z : Obs d | z.x = k ∧ z.a = a}.indicator
(fun z => (z.y / M) ^ 2) o) P.law.observedLaw :=
integrable_finset_sum univ fun k _ =>
integrable_finset_sum univ fun a _ => hcell k a
convert hsum using 1
funext o
rw [Finset.sum_eq_single o.x]
· cases o.a <;> simp [indicator]
· intro k _ hk
have hx : ¬o.x = k := fun h => hk h.symm
simp [indicator, hx]
· simp
theorem memLp_two_observed_normalized_outcome
The normalized observed outcome itself is square-integrable.
Formal statement
hypotheses
conclusion
MemLp (fun o : Obs d => o.y / M) 2 P.law.observedLaw
Proof (Lean source)
lemma memLp_two_observed_normalized_outcome
{d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) :
MemLp (fun o : Obs d => o.y / M) 2 P.law.observedLaw := by
have htuple : Measurable (fun o : Obs d => (o.x, o.a, o.y)) :=
measurable_iff_comap_le.mpr le_rfl
have hy : Measurable (fun o : Obs d => o.y) :=
measurable_snd.comp (measurable_snd.comp htuple)
exact (memLp_two_iff_integrable_sq
(hy.div measurable_const).aestronglyMeasurable).2
(integrable_observed_normalized_outcome_sq P)
theorem markedFactorialCoordinate_norm_le_mark
Selector coordinates have norm at most one, while the unique marked coordinate has norm at most the normalized outcome magnitude.
Formal statement
Proof (Lean source)
lemma markedFactorialCoordinate_norm_le_mark {d : ℕ} (M : ℝ)
(k : Fin d) (a : Bool) (j : ℕ) (q : Fin (j + 2)) (o : Obs d) :
‖markedFactorialCoordinate M k a j q o‖ ≤
if q.val = 0 then |o.y / M| else 1 := by
by_cases hq : q.val = 0
· by_cases ho : o.x = k ∧ o.a = a
· simp only [markedFactorialCoordinate, hq, if_pos, ho, mul_one,
Real.norm_eq_abs, norm_div]
simp [abs_div]
· simp [markedFactorialCoordinate, hq, ho]
· rw [if_neg hq, markedFactorialCoordinate, if_neg hq]
split <;> split <;> norm_num
theorem normalized_outcome_mean_abs_le_half
If the stated condition on the cell holds, the normalized outcome mark is integrable in each supported arm-cell law, and its conditional mean has absolute value at most one half.
Formal statement
hypotheses
conclusion
Integrable (fun y : ℝ => y / M) (P.law.outcomeLaw a k) ∧
|∫ y, y / M ∂P.law.outcomeLaw a k| ≤ (1 / 2 : ℝ)
Proof (Lean source)
lemma normalized_outcome_mean_abs_le_half
{d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (a : Bool) (k : Fin d)
(hk : 0 < P.law.cellMass k) :
Integrable (fun y : ℝ => y / M) (P.law.outcomeLaw a k) ∧
|∫ y, y / M ∂P.law.outcomeLaw a k| ≤ (1 / 2 : ℝ) := by
let _ : IsProbabilityMeasure (P.law.outcomeLaw a k) :=
P.law.outcome_isProbability a k
have h2 := outcome_second_moment_le_five_fourths P a k hk
have hyL2 : MemLp (fun y : ℝ => y) 2 (P.law.outcomeLaw a k) :=
(memLp_two_iff_integrable_sq (by fun_prop)).2 h2.1
have hy : Integrable (fun y : ℝ => y) (P.law.outcomeLaw a k) :=
hyL2.integrable (by norm_num)
have hM : 0 < M := lt_of_lt_of_le (by norm_num) P.M_ge_one
have heq : (fun y : ℝ => y / M) = fun y => M⁻¹ * y := by
funext y
rw [inv_mul_eq_div]
rw [heq]
refine ⟨hy.const_mul _, ?_⟩
rw [integral_const_mul, ← P.law.outcomeMean_eq, abs_mul, abs_inv,
abs_of_pos hM]
rw [← div_eq_inv_mul]
apply (div_le_iff₀ hM).2
simpa [div_eq_mul_inv, mul_comm] using P.mean_normalization a k hk
theorem integrable_markedFactorialCoordinate
Every individual marked-factorial coordinate is integrable under the observed law; the unique outcome coordinate uses the arm-cell transport.
Formal statement
hypotheses
conclusion
Integrable (markedFactorialCoordinate M k a j q) P.law.observedLaw
Proof (Lean source)
lemma integrable_markedFactorialCoordinate {d : ℕ}
{epsilon M sigma : ℝ} (P : ModelClass d epsilon M sigma)
(k : Fin d) (a : Bool) (j : ℕ) (q : Fin (j + 2)) :
Integrable (markedFactorialCoordinate M k a j q) P.law.observedLaw := by
have htuple : Measurable (fun o : Obs d => (o.x, o.a, o.y)) :=
measurable_iff_comap_le.mpr le_rfl
have hy : Measurable (fun o : Obs d => o.y) :=
measurable_snd.comp (measurable_snd.comp htuple)
have hxa : MeasurableSet {o : Obs d | o.x = k ∧ o.a = a} := by
have hx : Measurable (fun o : Obs d => o.x) := measurable_fst.comp htuple
have ha : Measurable (fun o : Obs d => o.a) :=
measurable_fst.comp (measurable_snd.comp htuple)
simpa only [preimage, Set.mem_singleton_iff, Prod.mk.injEq] using
(measurableSet_singleton (k, a)).preimage (prodMk hx ha)
by_cases hq : q.val = 0
· by_cases hk : P.law.cellMass k = 0
· have hnull := observed_arm_cell_measure_eq_zero_of_cellMass_eq_zero
P.law a k hk
have hzero : Integrable (fun _ : Obs d => (0 : ℝ)) P.law.observedLaw :=
integrable_const 0
refine hzero.congr (ae_iff.mpr ?_)
apply measure_mono_null _ hnull
intro o ho
by_contra hout
change ¬(o.x = k ∧ o.a = a) at hout
have hz : markedFactorialCoordinate M k a j q o = 0 := by
simp [markedFactorialCoordinate, hq, hout]
exact ho hz.symm
· have hkpos : 0 < P.law.cellMass k :=
lt_of_le_of_ne (P.law.cellMass_range k).1 (Ne.symm hk)
have hc := (normalized_outcome_mean_abs_le_half P a k hkpos).1
have hc' := hc.smul_measure (c := ofReal
(P.law.cellMass k *
(if a then P.law.propensity k else 1 - P.law.propensity k)))
ENNReal.ofReal_ne_top
rw [← observed_arm_cell_outcome_measure P.law a k] at hc'
have hrestrict : Integrable (fun o : Obs d => o.y / M)
(P.law.observedLaw.restrict {o | o.x = k ∧ o.a = a}) :=
(MeasureTheory.integrable_map_measure
((measurable_id.div measurable_const).aestronglyMeasurable)
hy.aemeasurable).mp hc'
have hind : Integrable
({o : Obs d | o.x = k ∧ o.a = a}.indicator (fun o => o.y / M))
P.law.observedLaw :=
(integrable_indicator_iff hxa).2 hrestrict
apply hind.congr
filter_upwards with o
simp [markedFactorialCoordinate, hq, indicator, mul_ite]
· have hbound : ∀ o, ‖markedFactorialCoordinate M k a j q o‖ ≤ 1 := by
intro o
rw [markedFactorialCoordinate, if_neg hq]
split <;> split <;> norm_num
exact Integrable.of_bound
(measurable_markedFactorialCoordinate M k a j q).aestronglyMeasurable 1
(Filter.Eventually.of_forall hbound)
theorem memLp_two_markedFactorialCoordinate
Every marked coordinate is square-integrable. The marked position uses the observed normalized second-moment audit; every selector-only position is uniformly bounded by one.
Formal statement
hypotheses
conclusion
MemLp (markedFactorialCoordinate M k a j q) 2 P.law.observedLaw
Proof (Lean source)
lemma memLp_two_markedFactorialCoordinate {d : ℕ}
{epsilon M sigma : ℝ} (P : ModelClass d epsilon M sigma)
(k : Fin d) (a : Bool) (j : ℕ) (q : Fin (j + 2)) :
MemLp (markedFactorialCoordinate M k a j q) 2 P.law.observedLaw := by
have hmeas : AEStronglyMeasurable (markedFactorialCoordinate M k a j q)
P.law.observedLaw :=
(measurable_markedFactorialCoordinate M k a j q).aestronglyMeasurable
by_cases hq : q.val = 0
· have htuple : Measurable (fun o : Obs d => (o.x, o.a, o.y)) :=
measurable_iff_comap_le.mpr le_rfl
have hx : Measurable (fun o : Obs d => o.x) := measurable_fst.comp htuple
have ha : Measurable (fun o : Obs d => o.a) :=
measurable_fst.comp (measurable_snd.comp htuple)
have hxa : MeasurableSet {o : Obs d | o.x = k ∧ o.a = a} := by
simpa only [preimage, Set.mem_singleton_iff, Prod.mk.injEq] using
(measurableSet_singleton (k, a)).preimage (prodMk hx ha)
apply (memLp_two_iff_integrable_sq hmeas).2
have hi := (integrable_observed_normalized_outcome_sq P).indicator hxa
apply hi.congr
filter_upwards with o
by_cases ho : o.x = k ∧ o.a = a <;>
simp [markedFactorialCoordinate, hq, indicator, ho]
· apply MemLp.of_bound hmeas 1
filter_upwards with o
rw [markedFactorialCoordinate, if_neg hq]
split <;> split <;> norm_num
theorem integrable_orderedProductKernel_markedFactorialCoordinate
A whole marked factorial kernel is integrable under the corresponding finite product law.
Formal statement
hypotheses
conclusion
Integrable (orderedProductKernel (markedFactorialCoordinate M k a j)) (Measure.pi fun _ : Fin (j + 2) => P.law.observedLaw)
Proof (Lean source)
lemma integrable_orderedProductKernel_markedFactorialCoordinate {d : ℕ}
{epsilon M sigma : ℝ} (P : ModelClass d epsilon M sigma)
(k : Fin d) (a : Bool) (j : ℕ) :
Integrable (orderedProductKernel
(markedFactorialCoordinate M k a j))
(Measure.pi fun _ : Fin (j + 2) => P.law.observedLaw) := by
exact Integrable.fintype_prod fun q =>
integrable_markedFactorialCoordinate P k a j q
theorem integrable_mergedProductKernel_markedFactorialCoordinate
Every partial-matching merge is integrable. After bounded selectors are discarded, the only possible unbounded factor is the product of the two marked coordinates; distinct marks factor across product coordinates, while a collision is controlled by the observed second moment.
Formal statement
hypotheses
d :
ℕ
epsilon M sigma :
ℝ
P :
ModelClass d epsilon M sigma
k l :
Fin d
a b :
j r :
ℕ
N :
PartialMatching (j + 2) (r + 2)
conclusion
Integrable (mergedProductKernel (markedFactorialCoordinate M k a j) (markedFactorialCoordinate M l b r) N) (Measure.pi fun _ : N.MergedIndex => P.law.observedLaw)
Proof (Lean source)
lemma integrable_mergedProductKernel_markedFactorialCoordinate {d : ℕ}
{epsilon M sigma : ℝ} (P : ModelClass d epsilon M sigma)
(k l : Fin d) (a b : Bool) (j r : ℕ)
(N : PartialMatching (j + 2) (r + 2)) :
Integrable (mergedProductKernel
(markedFactorialCoordinate M k a j)
(markedFactorialCoordinate M l b r) N)
(Measure.pi fun _ : N.MergedIndex => P.law.observedLaw) := by
classical
let uL : N.MergedIndex := N.leftInjection 0
let uR : N.MergedIndex := N.rightInjection 0
let v : Obs d → ℝ := fun o => o.y / M
let phi : N.MergedIndex → Obs d → ℝ := fun u o =>
(if u = uL then |v o| else 1) * (if u = uR then |v o| else 1)
have hv2 : Integrable (fun o : Obs d => (v o) ^ 2) P.law.observedLaw := by
simpa [v] using integrable_observed_normalized_outcome_sq P
have hv : Integrable (fun o : Obs d => |v o|) P.law.observedLaw := by
exact (memLp_two_observed_normalized_outcome P).integrable (by norm_num) |>.norm
have hphi : ∀ u : N.MergedIndex, Integrable (phi u) P.law.observedLaw := by
intro u
by_cases huL : u = uL
· by_cases huR : u = uR
· have hLR : uL = uR := huL.symm.trans huR
simpa [phi, huL, huR, hLR, pow_two] using hv2
· have hLR : uL ≠ uR := fun h => huR (huL.trans h)
simpa [phi, huL, huR, hLR] using hv
· by_cases huR : u = uR
· have hRL : uR ≠ uL := fun h => huL (huR.trans h)
simpa [phi, huL, huR, hRL] using hv
· simpa [phi, huL, huR] using (integrable_const (1 : ℝ))
have hdom : Integrable (fun z : N.MergedIndex → Obs d => ∏ u, phi u (z u))
(Measure.pi fun _ : N.MergedIndex => P.law.observedLaw) :=
Integrable.fintype_prod hphi
apply hdom.mono'
· exact (measurable_mergedProductKernel_markedFactorialCoordinate
M k l a b j r N).aestronglyMeasurable
· filter_upwards with z
unfold mergedProductKernel
rw [norm_mul, norm_prod, norm_prod]
have hleft : (∏ q : Fin (j + 2),
‖markedFactorialCoordinate M k a j q (z (N.leftInjection q))‖) ≤
|v (z uL)| := by
calc
_ ≤ ∏ q : Fin (j + 2), if q.val = 0 then
|v (z (N.leftInjection q))| else 1 := by
apply Finset.prod_le_prod
· intro q _
positivity
· intro q _
simpa [v] using markedFactorialCoordinate_norm_le_mark M k a j q
(z (N.leftInjection q))
_ = |v (z uL)| := by
simp [uL]
have hright : (∏ q : Fin (r + 2),
‖markedFactorialCoordinate M l b r q (z (N.rightInjection q))‖) ≤
|v (z uR)| := by
calc
_ ≤ ∏ q : Fin (r + 2), if q.val = 0 then
|v (z (N.rightInjection q))| else 1 := by
apply Finset.prod_le_prod
· intro q _
positivity
· intro q _
simpa [v] using markedFactorialCoordinate_norm_le_mark M l b r q
(z (N.rightInjection q))
_ = |v (z uR)| := by
simp [uR]
calc
_ ≤ |v (z uL)| * |v (z uR)| :=
mul_le_mul hleft hright (by positivity) (by positivity)
_ = ∏ u, phi u (z u) := by
simp only [phi, Finset.prod_mul_distrib]
simp [uL, uR]
theorem memLp_two_orderedProductKernel_markedFactorialCoordinate
The ordered marked kernel is square-integrable under its finite product law, obtained by factoring its square coordinatewise.
Formal statement
hypotheses
conclusion
MemLp (orderedProductKernel (markedFactorialCoordinate M k a j)) 2 (Measure.pi fun _ : Fin (j + 2) => P.law.observedLaw)
Proof (Lean source)
lemma memLp_two_orderedProductKernel_markedFactorialCoordinate {d : ℕ}
{epsilon M sigma : ℝ} (P : ModelClass d epsilon M sigma)
(k : Fin d) (a : Bool) (j : ℕ) :
MemLp (orderedProductKernel
(markedFactorialCoordinate M k a j)) 2
(Measure.pi fun _ : Fin (j + 2) => P.law.observedLaw) := by
have hmeas : AEStronglyMeasurable
(orderedProductKernel
(markedFactorialCoordinate M k a j))
(Measure.pi fun _ : Fin (j + 2) => P.law.observedLaw) :=
(measurable_orderedProductKernel_markedFactorialCoordinate M k a j).aestronglyMeasurable
apply (memLp_two_iff_integrable_sq hmeas).2
have hcoord : ∀ q : Fin (j + 2),
Integrable (fun o => (markedFactorialCoordinate M k a j q o) ^ 2)
P.law.observedLaw := by
intro q
exact (memLp_two_iff_integrable_sq
(measurable_markedFactorialCoordinate M k a j q).aestronglyMeasurable).1
(memLp_two_markedFactorialCoordinate P k a j q)
have hprod := Integrable.fintype_prod hcoord
convert hprod using 1
funext z
simp [orderedProductKernel, Finset.prod_pow]
theorem partialMatching_count_le_order_power Lemma partialMatching_count_le_order_power in the paper ↗
If the second factorial order is admissible and the stated size condition holds, if both coordinate families have order at most K, the number of size-h partial matchings is at most K^(2h) / h!. This is the matching-count factor used when the covariance expansion is summed by overlap size.
Formal statement
hypotheses
r s h K :
ℕ
hr :
r ≤ K
hs :
s ≤ K
conclusion
((partialMatchingsOfSize r s h).card : ℝ) ≤ (K : ℝ) ^ (2 * h) / h.factorial
Proof (Lean source)
lemma partialMatching_count_le_order_power {r s h K : ℕ}
(hr : r ≤ K) (hs : s ≤ K) :
((partialMatchingsOfSize r s h).card : ℝ) ≤
(K : ℝ) ^ (2 * h) / h.factorial := by
rw [card_partialMatchingsOfSize]
have hf : (0 : ℝ) < h.factorial := by positivity
have hrb : (Nat.choose r h : ℝ) ≤ (r : ℝ) ^ h / h.factorial :=
Nat.choose_le_pow_div h r
have hsb : (Nat.choose s h : ℝ) ≤ (s : ℝ) ^ h / h.factorial :=
Nat.choose_le_pow_div h s
have hr0 : (0 : ℝ) ≤ Nat.choose r h := by positivity
have hs0 : (0 : ℝ) ≤ Nat.choose s h := by positivity
have hprod : (Nat.choose r h : ℝ) * Nat.choose s h ≤
((r : ℝ) ^ h / h.factorial) * ((s : ℝ) ^ h / h.factorial) :=
mul_le_mul hrb hsb hs0 (by positivity)
have hrp : (r : ℝ) ^ h ≤ (K : ℝ) ^ h :=
pow_le_pow_left₀ (by positivity) (by exact_mod_cast hr) h
have hsp : (s : ℝ) ^ h ≤ (K : ℝ) ^ h :=
pow_le_pow_left₀ (by positivity) (by exact_mod_cast hs) h
calc
((Nat.choose r h * Nat.choose s h * h.factorial : ℕ) : ℝ) =
(Nat.choose r h : ℝ) * Nat.choose s h * h.factorial := by
push_cast
ring
_ ≤ (((r : ℝ) ^ h / h.factorial) *
((s : ℝ) ^ h / h.factorial)) * h.factorial :=
mul_le_mul_of_nonneg_right hprod hf.le
_ = (r : ℝ) ^ h * (s : ℝ) ^ h / h.factorial := by field_simp
_ ≤ (K : ℝ) ^ h * (K : ℝ) ^ h / h.factorial := by
exact div_le_div_of_nonneg_right
(mul_le_mul hrp hsp (by positivity) (by positivity)) hf.le
_ = (K : ℝ) ^ (2 * h) / h.factorial := by rw [two_mul, pow_add]
If the first factorial order is admissible and the second factorial order is admissible and the stated condition on the source size or matching order holds and the sampling budget satisfies the stated lower bound, the generic mixed-order normalization bound specialized to the marked factorial orders occurring in the paper.
Formal statement
hypotheses
m K j r h :
ℕ
PartialMatching (j + 2) (r + 2)
hj :
j + 2 ≤ K
hr :
r + 2 ≤ K
hm :
4 * (K + 2) ^ 2 ≤ m
hN :
N ∈ partialMatchingsOfSize (j + 2) (r + 2) h
conclusion
matchingNormalization m N ≤ exp 1 / (m : ℝ) ^ h
Proof (Lean source)
lemma marked_matchingNormalization_le {m K j r h : ℕ}
{N : PartialMatching (j + 2) (r + 2)}
(hj : j + 2 ≤ K) (hr : r + 2 ≤ K) (hm : 4 * (K + 2) ^ 2 ≤ m)
(hN : N ∈ partialMatchingsOfSize (j + 2) (r + 2) h) :
matchingNormalization m N ≤
exp 1 / (m : ℝ) ^ h := by
exact matchingNormalization_le
(R := K + 2) (hj.trans (by omega)) (hr.trans (by omega)) hm hN
theorem marked_emptyMatchingNormalization_sub_one_le Lemma marked_emptyMatchingNormalization_sub_one_le in the paper ↗
If the first factorial order is admissible and the second factorial order is admissible and the stated condition on the source size or matching order holds, the disjoint-tuple correction for two marked factorial orders is at most 2(K+2)²/m, uniformly over the polynomial degrees.
Formal statement
hypotheses
m K j r :
ℕ
hj :
j + 2 ≤ K
hr :
r + 2 ≤ K
hm :
4 * (K + 2) ^ 2 ≤ m
conclusion
Proof (Lean source)
lemma marked_emptyMatchingNormalization_sub_one_le {m K j r : ℕ}
(hj : j + 2 ≤ K) (hr : r + 2 ≤ K) (hm : 4 * (K + 2) ^ 2 ≤ m) :
|matchingNormalization m
(empty (j + 2) (r + 2)) - 1| ≤
2 * ((K + 2 : ℕ) : ℝ) ^ 2 / m := by
exact emptyMatchingNormalization_sub_one_le
(R := K + 2) (hj.trans (by omega)) (hr.trans (by omega)) hm
theorem marked_matchingNormalization_sum_le Lemma marked_matchingNormalization_sum_le in the paper ↗
If the first factorial order is admissible and the second factorial order is admissible and the stated condition on the source size or matching order holds, summing the normalization over all size-h overlaps costs at most the paper's K^(2h)/h! matching count times the generic exp(1)/m^h ratio.
Formal statement
hypotheses
m K j r h :
ℕ
hj :
j + 2 ≤ K
hr :
r + 2 ≤ K
hm :
4 * (K + 2) ^ 2 ≤ m
conclusion
∑ N ∈ partialMatchingsOfSize (j + 2) (r + 2) h, matchingNormalization m N
≤ ((K : ℝ) ^ (2 * h) / h.factorial) * (exp 1 / (m : ℝ) ^ h)
Proof (Lean source)
lemma marked_matchingNormalization_sum_le {m K j r h : ℕ}
(hj : j + 2 ≤ K) (hr : r + 2 ≤ K) (hm : 4 * (K + 2) ^ 2 ≤ m) :
∑ N ∈ partialMatchingsOfSize (j + 2) (r + 2) h,
matchingNormalization m N ≤
((K : ℝ) ^ (2 * h) / h.factorial) *
(exp 1 / (m : ℝ) ^ h) := by
calc
∑ N ∈ partialMatchingsOfSize (j + 2) (r + 2) h,
matchingNormalization m N ≤
∑ _N ∈ partialMatchingsOfSize (j + 2) (r + 2) h,
(exp 1 / (m : ℝ) ^ h) := by
apply Finset.sum_le_sum
intro N hN
exact marked_matchingNormalization_le hj hr hm hN
_ = ((partialMatchingsOfSize (j + 2) (r + 2) h).card : ℝ) *
(exp 1 / (m : ℝ) ^ h) := by
simp [mul_comm]
_ ≤ ((K : ℝ) ^ (2 * h) / h.factorial) *
(exp 1 / (m : ℝ) ^ h) := by
exact mul_le_mul_of_nonneg_right
(partialMatching_count_le_order_power hj hr)
(by positivity)
theorem measurable_allBlockOrderedMarkedFactorial
The all-block version of each marked factorial statistic is measurable on the finite product sample space.
Formal statement
hypotheses
conclusion
Proof (Lean source)
lemma measurable_allBlockOrderedMarkedFactorial {m d : ℕ} (M : ℝ)
(k : Fin d) (a : Bool) (j : ℕ) :
Measurable (fun s : Fin m → Obs d =>
allBlockOrderedMarkedFactorial M s k a j) := by
unfold allBlockOrderedMarkedFactorial
apply Measurable.div
· apply Finset.measurable_sum
intro idx _
apply Measurable.ite
· by_cases hidx : Injective idx
· simpa [hidx] using (measurableSet_univ :
MeasurableSet (Set.univ : Set (Fin m → Obs d)))
· simpa [hidx] using (measurableSet_empty :
MeasurableSet (∅ : Set (Fin m → Obs d)))
· have hmap : Measurable (fun s : Fin m → Obs d =>
fun q : Fin (j + 2) => s (idx q)) :=
measurable_pi_lambda _ fun q => measurable_pi_apply (idx q)
convert
(measurable_orderedProductKernel_markedFactorialCoordinate M k a j).comp
hmap using 1
funext s
exact (prod_markedFactorialCoordinate M k a j
(fun q => s (idx q))).symm
· exact measurable_const
· exact measurable_const
theorem measurable_allBlockMarkedPolynomialSum Lemma measurable_allBlockMarkedPolynomialSum in the paper ↗
Finite summation preserves measurability of the complete deterministic light-cell statistic.
Formal statement
hypotheses
conclusion
Measurable (allBlockMarkedPolynomialSum (m := m) M B K S)
Proof (Lean source)
lemma measurable_allBlockMarkedPolynomialSum {m d : ℕ} (M B : ℝ) (K : ℕ)
(S : Finset (Fin d)) :
Measurable (allBlockMarkedPolynomialSum (m := m) M B K S) := by
unfold allBlockMarkedPolynomialSum allBlockLightPolynomialTerm
apply measurable_sum
intro k _
apply measurable_sum
intro j _
exact measurable_const.mul
((measurable_allBlockOrderedMarkedFactorial M k true j).sub
(measurable_allBlockOrderedMarkedFactorial M k false j))
theorem variance_allBlockMarkedPolynomialSum_eq_infinitePi
The variance under the finite product law can be evaluated on the canonical infinite-product IID realization by restricting it to the first m coordinates.
Formal statement
hypotheses
conclusion
= variance (fun ω : ℕ → Obs d => allBlockMarkedPolynomialSum M B K S (fun i : Fin m => ω i)) (Measure.infinitePi fun _ : ℕ => P0)
Proof (Lean source)
lemma variance_allBlockMarkedPolynomialSum_eq_infinitePi {m d : ℕ}
(P0 : Measure (Obs d)) [IsProbabilityMeasure P0]
(M B : ℝ) (K : ℕ) (S : Finset (Fin d)) :
variance (allBlockMarkedPolynomialSum (m := m) M B K S)
(Measure.pi fun _ : Fin m => P0) =
variance (fun ω : ℕ → Obs d =>
allBlockMarkedPolynomialSum M B K S (fun i : Fin m => ω i))
(Measure.infinitePi fun _ : ℕ => P0) := by
let S0 := iidSample_infinitePi P0
have hpush := Causalean.Stat.iidSample_finN_pushforward S0 m
have hprefix : Measurable (fun ω : ℕ → Obs d => fun i : Fin m => ω i) :=
iidSample_finN_measurable S0 m
dsimp [S0, iidSample_infinitePi] at hpush hprefix
rw [← hpush]
simpa [Function.comp_def] using
(variance_map
(measurable_allBlockMarkedPolynomialSum (m := m) M B K S).aemeasurable
hprefix.aemeasurable)
Helpers.FactorialCovarianceAssembly 6 declarations
If the two cells are distinct and the first factorial order is admissible and the second factorial order is admissible and the stated condition on the source size or matching order holds, for different cells every positive partial matching is incompatible, so only the disjoint-tuple normalization correction remains.
Formal statement
hypotheses
d m K :
ℕ
epsilon M sigma :
ℝ
P :
ModelClass d epsilon M sigma
k l :
Fin d
hkl :
k ≠ l
a b :
j r :
ℕ
hj :
j + 2 ≤ K
hr :
r + 2 ≤ K
hm :
4 * (K + 2) ^ 2 ≤ m
conclusion
|centeredCrossMoment (Measure.infinitePi fun _ : ℕ => P.law.observedLaw) (fun ω : ℕ → Obs d => allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) k a j) (fun ω : ℕ → Obs d => allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) l b r)|
≤ (2 * ((K + 2 : ℕ) : ℝ) ^ 2 / m) * P.law.cellMass k ^ (j + 2) * P.law.cellMass l ^ (r + 2)
Proof (Lean source)
lemma centeredCrossMoment_allBlockOrderedMarkedFactorial_cell_ne_abs_le
{d m K : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (k l : Fin d) (hkl : k ≠ l)
(a b : Bool) (j r : ℕ) (hj : j + 2 ≤ K) (hr : r + 2 ≤ K)
(hm : 4 * (K + 2) ^ 2 ≤ m) :
|centeredCrossMoment
(Measure.infinitePi fun _ : ℕ => P.law.observedLaw)
(fun ω : ℕ → Obs d =>
allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) k a j)
(fun ω : ℕ → Obs d =>
allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) l b r)| ≤
(2 * ((K + 2 : ℕ) : ℝ) ^ 2 / m) *
P.law.cellMass k ^ (j + 2) * P.law.cellMass l ^ (r + 2) := by
classical
rw [centeredCrossMoment_allBlockOrderedMarkedFactorial P k l a b j r hj hr hm]
have hpositive :
(∑ h ∈ (range (min (j + 2) (r + 2) + 1)).filter
(fun h => 0 < h),
∑ N ∈ partialMatchingsOfSize (j + 2) (r + 2) h,
matchingNormalization m N *
mergedProductMoment P.law.observedLaw
(markedFactorialCoordinate M k a j)
(markedFactorialCoordinate M l b r) N) = 0 := by
apply Finset.sum_eq_zero
intro h hh
apply Finset.sum_eq_zero
intro N hN
have hhpos : 0 < h := (Finset.mem_filter.mp hh).2
have hsize : N.size = h := by
simpa [partialMatchingsOfSize] using hN
rw [mergedProductMoment_markedFactorialCoordinate_eq_zero_of_cell_ne
P.law.observedLaw M k l a b j r N (hsize.symm ▸ hhpos) hkl, mul_zero]
rw [hpositive, add_zero, abs_mul, abs_mul]
have hnorm := marked_emptyMatchingNormalization_sub_one_le hj hr hm
have hkmean := orderedProductMean_markedFactorialCoordinate_abs_le P a k j
have hlmean := orderedProductMean_markedFactorialCoordinate_abs_le P b l r
have hmpos : (0 : ℝ) < m := by
exact_mod_cast (lt_of_lt_of_le (by positivity : 0 < 4 * (K + 2) ^ 2) hm)
have hnormR : 0 ≤ 2 * ((K + 2 : ℕ) : ℝ) ^ 2 / m := by positivity
have hk0 := P.law.cellMass_range k |>.1
have hfactor : 0 ≤
(2 * ((K + 2 : ℕ) : ℝ) ^ 2 / m) *
P.law.cellMass k ^ (j + 2) :=
mul_nonneg hnormR (pow_nonneg hk0 _)
exact mul_le_mul
(mul_le_mul hnorm hkmean (abs_nonneg _) hnormR) hlmean
(abs_nonneg _) hfactor
theorem weighted_centeredCrossMoment_cell_ne_sum_le
If the two cells are distinct and the polynomial or elbow parameter satisfies its stated bound and the outcome bound is positive and the stated condition on the cell holds and the stated l condition holds and the stated condition on the source size or matching order holds, summing the different-cell correction over both arms and all polynomial degrees gives the required K²/m cross-cell scale.
Formal statement
hypotheses
d m K :
ℕ
epsilon M sigma B :
ℝ
P :
ModelClass d epsilon M sigma
k l :
Fin d
hkl :
k ≠ l
hK :
2 ≤ K
hB :
0 < B
hk :
P.law.cellMass k ≤ B / 4
hl :
P.law.cellMass l ≤ B / 4
hm :
4 * (K + 2) ^ 2 ≤ m
conclusion
∑ a : Bool, ∑ b : Bool, ∑ j ∈ range (K - 1), ∑ r ∈ range (K - 1), |shiftedCoefficient K j / B ^ (j + 1)| * |shiftedCoefficient K r / B ^ (r + 1)| * |centeredCrossMoment (Measure.infinitePi fun _ : ℕ => P.law.observedLaw) (fun ω : ℕ → Obs d => allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) k a j) (fun ω : ℕ → Obs d => allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) l b r)|
≤ 32 * (K : ℝ) ^ 2 / m * (B * (6 : ℝ) ^ K) ^ 2
Proof (Lean source)
lemma weighted_centeredCrossMoment_cell_ne_sum_le
{d m K : ℕ} {epsilon M sigma B : ℝ}
(P : ModelClass d epsilon M sigma) (k l : Fin d) (hkl : k ≠ l)
(hK : 2 ≤ K) (hB : 0 < B)
(hk : P.law.cellMass k ≤ B / 4)
(hl : P.law.cellMass l ≤ B / 4)
(hm : 4 * (K + 2) ^ 2 ≤ m) :
∑ a : Bool, ∑ b : Bool,
∑ j ∈ range (K - 1), ∑ r ∈ range (K - 1),
|shiftedCoefficient K j / B ^ (j + 1)| *
|shiftedCoefficient K r / B ^ (r + 1)| *
|centeredCrossMoment
(Measure.infinitePi fun _ : ℕ => P.law.observedLaw)
(fun ω : ℕ → Obs d =>
allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) k a j)
(fun ω : ℕ → Obs d =>
allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) l b r)| ≤
32 * (K : ℝ) ^ 2 / m * (B * (6 : ℝ) ^ K) ^ 2 := by
classical
let q : ℝ := 2 * ((K + 2 : ℕ) : ℝ) ^ 2 / m
let A : ℕ → ℝ := fun j =>
|shiftedCoefficient K j / B ^ (j + 1)| *
P.law.cellMass k ^ (j + 2)
let D : ℕ → ℝ := fun r =>
|shiftedCoefficient K r / B ^ (r + 1)| *
P.law.cellMass l ^ (r + 2)
have hterm (a b : Bool) (j r : ℕ)
(hj : j ∈ range (K - 1)) (hr : r ∈ range (K - 1)) :
|shiftedCoefficient K j / B ^ (j + 1)| *
|shiftedCoefficient K r / B ^ (r + 1)| *
|centeredCrossMoment
(Measure.infinitePi fun _ : ℕ => P.law.observedLaw)
(fun ω : ℕ → Obs d =>
allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) k a j)
(fun ω : ℕ → Obs d =>
allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) l b r)| ≤
q * A j * D r := by
have hjK : j + 2 ≤ K := by
have := Finset.mem_range.mp hj
omega
have hrK : r + 2 ≤ K := by
have := Finset.mem_range.mp hr
omega
have hc := centeredCrossMoment_allBlockOrderedMarkedFactorial_cell_ne_abs_le
P k l hkl a b j r hjK hrK hm
dsimp only [q, A, D]
calc
_ ≤ |shiftedCoefficient K j / B ^ (j + 1)| *
|shiftedCoefficient K r / B ^ (r + 1)| *
((2 * ((K + 2 : ℕ) : ℝ) ^ 2 / m) *
P.law.cellMass k ^ (j + 2) *
P.law.cellMass l ^ (r + 2)) := by
gcongr
_ = _ := by ring
have hkSum : ∑ j ∈ range (K - 1), A j ≤ B * (6 : ℝ) ^ K := by
exact shiftedCoefficient_weighted_cellMass_sum_le
(show 0 < K by omega) hB (P.law.cellMass_range k).1 hk
have hlSum : ∑ r ∈ range (K - 1), D r ≤ B * (6 : ℝ) ^ K := by
exact shiftedCoefficient_weighted_cellMass_sum_le
(show 0 < K by omega) hB (P.law.cellMass_range l).1 hl
have hmpos : (0 : ℝ) < m := by
exact_mod_cast (lt_of_lt_of_le (by positivity : 0 < 4 * (K + 2) ^ 2) hm)
have hq0 : 0 ≤ q := by dsimp [q]; positivity
have hA0 : ∀ j, 0 ≤ A j := fun j => by
exact mul_nonneg (abs_nonneg _) (pow_nonneg (P.law.cellMass_range k).1 _)
have hD0 : ∀ r, 0 ≤ D r := fun r => by
exact mul_nonneg (abs_nonneg _) (pow_nonneg (P.law.cellMass_range l).1 _)
have hAsum0 : 0 ≤ ∑ j ∈ range (K - 1), A j :=
sum_nonneg fun j _ => hA0 j
have hDsum0 : 0 ≤ ∑ r ∈ range (K - 1), D r :=
sum_nonneg fun r _ => hD0 r
have hsum :
(∑ j ∈ range (K - 1), ∑ r ∈ range (K - 1),
q * A j * D r) =
q * (∑ j ∈ range (K - 1), A j) *
(∑ r ∈ range (K - 1), D r) := by
calc
_ = ∑ j ∈ range (K - 1),
(q * A j) * (∑ r ∈ range (K - 1), D r) := by
apply Finset.sum_congr rfl
intro j hj
exact (Finset.mul_sum (s := range (K - 1))
(f := D) (q * A j)).symm
_ = (∑ j ∈ range (K - 1), q * A j) *
(∑ r ∈ range (K - 1), D r) := by
exact (Finset.sum_mul (s := range (K - 1))
(f := fun j => q * A j)
(∑ r ∈ range (K - 1), D r)).symm
_ = _ := by
congr 1
exact (Finset.mul_sum (s := range (K - 1)) (f := A) q).symm
calc
_ ≤ ∑ _a : Bool, ∑ _b : Bool,
∑ j ∈ range (K - 1), ∑ r ∈ range (K - 1),
q * A j * D r := by
apply Finset.sum_le_sum
intro a ha
apply Finset.sum_le_sum
intro b hb
apply Finset.sum_le_sum
intro j hj
apply Finset.sum_le_sum
intro r hr
exact hterm a b j r hj hr
_ = 4 * q * (∑ j ∈ range (K - 1), A j) *
(∑ r ∈ range (K - 1), D r) := by
simp only [hsum, Finset.sum_const]
rw [show Finset.univ.card = 2 by decide]
norm_num
ring
_ ≤ 4 * q * (B * (6 : ℝ) ^ K) * (B * (6 : ℝ) ^ K) := by
have hR0 : 0 ≤ B * (6 : ℝ) ^ K := by positivity
exact mul_le_mul
(mul_le_mul_of_nonneg_left hkSum (mul_nonneg (by positivity) hq0))
hlSum hDsum0 (mul_nonneg (mul_nonneg (by positivity) hq0) hR0)
_ ≤ 32 * (K : ℝ) ^ 2 / m * (B * (6 : ℝ) ^ K) ^ 2 := by
dsimp [q]
have hK2 : ((K + 2 : ℕ) : ℝ) ^ 2 ≤ 4 * (K : ℝ) ^ 2 := by
push_cast
have hKreal : (2 : ℝ) ≤ K := by exact_mod_cast hK
nlinarith
have hcoef :
4 * (2 * ((K + 2 : ℕ) : ℝ) ^ 2 / m) ≤
32 * (K : ℝ) ^ 2 / m := by
rw [show 4 * (2 * ((K + 2 : ℕ) : ℝ) ^ 2 / (m : ℝ)) =
(8 * ((K + 2 : ℕ) : ℝ) ^ 2) / m by ring]
apply (div_le_div_iff_of_pos_right hmpos).2
nlinarith
rw [pow_two (B * (6 : ℝ) ^ K)]
calc
4 * (2 * ((K + 2 : ℕ) : ℝ) ^ 2 / m) *
(B * 6 ^ K) * (B * 6 ^ K) =
(4 * (2 * ((K + 2 : ℕ) : ℝ) ^ 2 / m)) *
((B * 6 ^ K) * (B * 6 ^ K)) := by ring
_ ≤ (32 * (K : ℝ) ^ 2 / m) *
((B * 6 ^ K) * (B * 6 ^ K)) :=
mul_le_mul_of_nonneg_right hcoef (mul_self_nonneg _)
theorem memLp_allBlockOrderedMarkedFactorial
every all-block ordered marked factorial statistic has a finite second moment.
Formal statement
hypotheses
conclusion
MemLp (fun ω : ℕ → Obs d => allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) k a j) 2 (Measure.infinitePi fun _ : ℕ => P.law.observedLaw)
Proof (Lean source)
lemma memLp_allBlockOrderedMarkedFactorial {d m : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (k : Fin d) (a : Bool) (j : ℕ) :
MemLp (fun ω : ℕ → Obs d =>
allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) k a j) 2
(Measure.infinitePi fun _ : ℕ => P.law.observedLaw) := by
let S0 := Causalean.Stat.iidSample_infinitePi P.law.observedLaw
have hk : MemLp (orderedProductKernel
(markedFactorialCoordinate M k a j)) 2
(Measure.pi fun _ : Fin (j + 2) => P.law.observedLaw) :=
memLp_two_orderedProductKernel_markedFactorialCoordinate P k a j
have hsum := S0.memLp_injectiveTuples_sum
(measurable_orderedProductKernel_markedFactorialCoordinate M k a j)
hk.integrable_sq m
have hnorm : MemLp (normalizedOrderedProductStatistic S0
(markedFactorialCoordinate M k a j) m) 2
(Measure.infinitePi fun _ : ℕ => P.law.observedLaw) := by
unfold normalizedOrderedProductStatistic
normalizedFiniteKernelStatistic
refine (hsum.const_mul ((m.descFactorial (j + 2) : ℝ)⁻¹)).ae_eq ?_
filter_upwards with ω
have ht : finiteInjectiveTuples (Fin (j + 2)) m =
injectiveTuples (j + 2) m := by
ext t
simp [finiteInjectiveTuples,
injectiveTuples]
rw [Fintype.card_fin, ht]
convert hnorm using 1
funext ω
exact allBlockOrderedMarkedFactorial_eq_normalizedOrderedProductStatistic
(m := m) S0 M k a j ω
theorem memLp_allBlockLightPolynomialTerm
every all-block light polynomial term has a finite second moment.
Formal statement
hypotheses
conclusion
MemLp (fun ω : ℕ → Obs d => allBlockLightPolynomialTerm M B K (fun i : Fin m => ω i) k) 2 (Measure.infinitePi fun _ : ℕ => P.law.observedLaw)
Proof (Lean source)
lemma memLp_allBlockLightPolynomialTerm {d m K : ℕ} {epsilon M sigma B : ℝ}
(P : ModelClass d epsilon M sigma) (k : Fin d) :
MemLp (fun ω : ℕ → Obs d =>
allBlockLightPolynomialTerm M B K (fun i : Fin m => ω i) k) 2
(Measure.infinitePi fun _ : ℕ => P.law.observedLaw) := by
unfold allBlockLightPolynomialTerm
apply memLp_finset_sum
intro j hj
exact ((memLp_allBlockOrderedMarkedFactorial P k true j).sub (memLp_allBlockOrderedMarkedFactorial P k false j)).const_mul _
theorem covariance_allBlockLightPolynomialTerm_abs_le
the absolute covariance between two all-block light polynomial terms satisfies the stated bound.
Formal statement
hypotheses
conclusion
|covariance (fun ω : ℕ → Obs d => allBlockLightPolynomialTerm M B K (fun i : Fin m => ω i) k) (fun ω : ℕ → Obs d => allBlockLightPolynomialTerm M B K (fun i : Fin m => ω i) l) (Measure.infinitePi fun _ : ℕ => P.law.observedLaw)|
≤ ∑ a : Bool, ∑ b : Bool, ∑ j ∈ range (K - 1), ∑ r ∈ range (K - 1), |shiftedCoefficient K j / B ^ (j + 1)| * |shiftedCoefficient K r / B ^ (r + 1)| * |centeredCrossMoment (Measure.infinitePi fun _ : ℕ => P.law.observedLaw) (fun ω : ℕ → Obs d => allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) k a j) (fun ω : ℕ → Obs d => allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) l b r)|
Proof (Lean source)
lemma covariance_allBlockLightPolynomialTerm_abs_le {d m K : ℕ} {epsilon M sigma B : ℝ}
(P : ModelClass d epsilon M sigma) (k l : Fin d) :
|covariance
(fun ω : ℕ → Obs d =>
allBlockLightPolynomialTerm M B K (fun i : Fin m => ω i) k)
(fun ω : ℕ → Obs d =>
allBlockLightPolynomialTerm M B K (fun i : Fin m => ω i) l)
(Measure.infinitePi fun _ : ℕ => P.law.observedLaw)| ≤
∑ a : Bool, ∑ b : Bool,
∑ j ∈ range (K - 1), ∑ r ∈ range (K - 1),
|shiftedCoefficient K j / B ^ (j + 1)| *
|shiftedCoefficient K r / B ^ (r + 1)| *
|centeredCrossMoment
(Measure.infinitePi fun _ : ℕ => P.law.observedLaw)
(fun ω : ℕ → Obs d =>
allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) k a j)
(fun ω : ℕ → Obs d =>
allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) l b r)| := by
classical
let μ := Measure.infinitePi fun _ : ℕ => P.law.observedLaw
let U := fun (k : Fin d) (a : Bool) (j : ℕ) (ω : ℕ → Obs d) =>
allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) k a j
let c := fun j : ℕ => shiftedCoefficient K j / B ^ (j + 1)
have hU (q : Fin d) (a : Bool) (j : ℕ) : MemLp (U q a j) 2 μ :=
memLp_allBlockOrderedMarkedFactorial P q a j
have hsum (q : Fin d) :
(fun ω : ℕ → Obs d =>
allBlockLightPolynomialTerm M B K (fun i : Fin m => ω i) q) =
fun ω => ∑ j ∈ range (K - 1), c j * (U q true j ω - U q false j ω) := by
rfl
rw [hsum k, hsum l]
have hterm (q : Fin d) (j : ℕ) :
MemLp (fun ω => c j * (U q true j ω - U q false j ω)) 2 μ :=
((hU q true j).sub (hU q false j)).const_mul _
rw [covariance_fun_sum_fun_sum'
(fun j _ => hterm k j) (fun r _ => hterm l r)]
calc
_ ≤ ∑ j ∈ range (K - 1), ∑ r ∈ range (K - 1),
|covariance (fun ω => c j * (U k true j ω - U k false j ω))
(fun ω => c r * (U l true r ω - U l false r ω)) μ| :=
Finset.abs_sum_le_sum_abs _ _ |>.trans
(Finset.sum_le_sum fun j hj => Finset.abs_sum_le_sum_abs _ _)
_ ≤ ∑ j ∈ range (K - 1), ∑ r ∈ range (K - 1),
∑ a : Bool, ∑ b : Bool,
|shiftedCoefficient K j / B ^ (j + 1)| *
|shiftedCoefficient K r / B ^ (r + 1)| *
|centeredCrossMoment
(Measure.infinitePi fun _ : ℕ => P.law.observedLaw)
(fun ω : ℕ → Obs d =>
allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) k a j)
(fun ω : ℕ → Obs d =>
allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) l b r)| := by
apply Finset.sum_le_sum
intro j hj
apply Finset.sum_le_sum
intro r hr
rw [covariance_const_mul_left, covariance_const_mul_right,
abs_mul, abs_mul]
have hsub := covariance_sub_sub (hU k true j) (hU k false j)
(hU l true r) (hU l false r)
change |c j| * (|c r| *
|covariance (U k true j - U k false j)
(U l true r - U l false r) μ|) ≤ _
rw [hsub]
have hfour :
|covariance (U k true j) (U l true r) μ -
covariance (U k true j) (U l false r) μ -
covariance (U k false j) (U l true r) μ +
covariance (U k false j) (U l false r) μ| ≤
∑ a : Bool, ∑ b : Bool,
|centeredCrossMoment μ (U k a j) (U l b r)| := by
let A := covariance (U k true j) (U l true r) μ
let B0 := covariance (U k true j) (U l false r) μ
let C := covariance (U k false j) (U l true r) μ
let D := covariance (U k false j) (U l false r) μ
have hAB : |A - B0| ≤ |A| + |B0| := by
simpa [sub_eq_add_neg] using abs_add_le A (-B0)
have hABC : |A - B0 - C| ≤ (|A| + |B0|) + |C| := by
have h0 : |A - B0 - C| ≤ |A - B0| + |C| := by
simpa [sub_eq_add_neg] using abs_add_le (A - B0) (-C)
exact h0.trans (add_le_add hAB le_rfl)
have htri : |A - B0 - C + D| ≤ ((|A| + |B0|) + |C|) + |D| :=
(abs_add_le _ _).trans (add_le_add hABC le_rfl)
have hsumEq :
(∑ a : Bool, ∑ b : Bool,
|centeredCrossMoment μ (U k a j) (U l b r)|) =
|A| + |B0| + (|C| + |D|) := by
simp [A, B0, C, D, centeredCrossMoment,
covariance_eq_sub (hU k true j) (hU l true r),
covariance_eq_sub (hU k true j) (hU l false r),
covariance_eq_sub (hU k false j) (hU l true r),
covariance_eq_sub (hU k false j) (hU l false r)]
rw [hsumEq]
dsimp [A, B0, C, D] at htri ⊢
convert htri using 1 <;> ring
rw [← mul_assoc]
calc
|c j| * |c r| *
|covariance (U k true j) (U l true r) μ -
covariance (U k true j) (U l false r) μ -
covariance (U k false j) (U l true r) μ +
covariance (U k false j) (U l false r) μ| ≤
|c j| * |c r| *
(∑ a : Bool, ∑ b : Bool,
|centeredCrossMoment μ (U k a j) (U l b r)|) := by
gcongr
_ = ∑ a : Bool, ∑ b : Bool,
|c j| * |c r| *
|centeredCrossMoment μ (U k a j) (U l b r)| := by
simp only [Finset.mul_sum]
_ = _ := by
calc
_ = ∑ j ∈ range (K - 1), ∑ a : Bool,
∑ r ∈ range (K - 1), ∑ b : Bool,
|shiftedCoefficient K j / B ^ (j + 1)| *
|shiftedCoefficient K r / B ^ (r + 1)| *
|centeredCrossMoment μ (U k a j) (U l b r)| := by
apply Finset.sum_congr rfl
intro j hj
rw [Finset.sum_comm]
_ = ∑ a : Bool, ∑ j ∈ range (K - 1),
∑ r ∈ range (K - 1), ∑ b : Bool,
|shiftedCoefficient K j / B ^ (j + 1)| *
|shiftedCoefficient K r / B ^ (r + 1)| *
|centeredCrossMoment μ (U k a j) (U l b r)| := by
rw [Finset.sum_comm]
_ = ∑ a : Bool, ∑ j ∈ range (K - 1), ∑ b : Bool,
∑ r ∈ range (K - 1),
|shiftedCoefficient K j / B ^ (j + 1)| *
|shiftedCoefficient K r / B ^ (r + 1)| *
|centeredCrossMoment μ (U k a j) (U l b r)| := by
apply Finset.sum_congr rfl
intro a ha
apply Finset.sum_congr rfl
intro j hj
rw [Finset.sum_comm]
_ = _ := by
apply Finset.sum_congr rfl
intro a ha
rw [Finset.sum_comm]
Under only the conditional second-moment envelope, a deterministic collection of signed one-mark ordered factorial statistics has the covariance bound used by the heavy/light estimator.
Formal statement
conclusion
∀ epsilon : ℝ,
0 < epsilon → epsilon < 1 / 2 → ∃ C_epsilon : ℝ, 0 < C_epsilon ∧ ∀ d m K : ℕ, ∀ M sigma B : ℝ, ∀ P : ModelClass d epsilon M sigma, ∀ S : Finset (Fin d), 2
≤ K → 0 < B → (∀ k ∈ S, P.law.cellMass k ≤ B / 4) → 4 * (K + 2) ^ 2 ≤ m → (4 : ℝ) * (K + 2) / m ≤ 3 * B / 4 → variance (allBlockMarkedPolynomialSum M B K S) (productLaw m P.law) ≤ C_epsilon / m
+ C_epsilon * 6 ^ (2 * K) * ((d : ℝ) * B ^ 2 + (d : ℝ) ^ 2 * K ^ 2 * B ^ 2 / m)
Proof (Lean source)
lemma linear_mark_factorial_covariance :
∀ epsilon : ℝ, 0 < epsilon → epsilon < 1 / 2 →
∃ C_epsilon : ℝ, 0 < C_epsilon ∧
∀ d m K : ℕ, ∀ M sigma B : ℝ,
∀ P : ModelClass d epsilon M sigma, ∀ S : Finset (Fin d),
2 ≤ K → 0 < B →
(∀ k ∈ S, P.law.cellMass k ≤ B / 4) →
4 * (K + 2) ^ 2 ≤ m →
(4 : ℝ) * (K + 2) / m ≤ 3 * B / 4 →
variance (allBlockMarkedPolynomialSum M B K S) (productLaw m P.law) ≤
C_epsilon / m + C_epsilon * 6 ^ (2 * K) *
((d : ℝ) * B ^ 2 + (d : ℝ) ^ 2 * K ^ 2 * B ^ 2 / m) := by
intro epsilon hepsilon hepsilon_half
refine ⟨64, by norm_num, ?_⟩
intro d m K M sigma B P S hK hB hlight hm hshift
let μ := Measure.infinitePi fun _ : ℕ => P.law.observedLaw
let F := fun (k : Fin d) (ω : ℕ → Obs d) =>
allBlockLightPolynomialTerm M B K (fun i : Fin m => ω i) k
have hmem (k : Fin d) : MemLp (F k) 2 μ := memLp_allBlockLightPolynomialTerm P k
have hsame (k : Fin d) (hk : k ∈ S) :
|covariance (F k) (F k) μ| ≤ 16 * (B * (6 : ℝ) ^ K) ^ 2 := by
exact (covariance_allBlockLightPolynomialTerm_abs_le P k k).trans
(weighted_centeredCrossMoment_same_cell_sum_le P k hK hB
(hlight k hk) hm hshift)
have hdiff (k l : Fin d) (hk : k ∈ S) (hl : l ∈ S) (hkl : k ≠ l) :
|covariance (F k) (F l) μ| ≤
32 * (K : ℝ) ^ 2 / m * (B * (6 : ℝ) ^ K) ^ 2 := by
exact (covariance_allBlockLightPolynomialTerm_abs_le P k l).trans
(weighted_centeredCrossMoment_cell_ne_sum_le P k l hkl hK hB
(hlight k hk) (hlight l hl) hm)
change variance (allBlockMarkedPolynomialSum M B K S)
(Measure.pi fun _ : Fin m => P.law.observedLaw) ≤ _
rw [variance_allBlockMarkedPolynomialSum_eq_infinitePi P.law.observedLaw M B K S]
change variance (fun ω => ∑ k ∈ S, F k ω) μ ≤ _
rw [variance_fun_sum' (fun k hk => hmem k)]
have hcard : (S.card : ℝ) ≤ d := by
have hcNat : S.card ≤ d := by simpa using S.card_le_univ
exact_mod_cast hcNat
have hcard0 : (0 : ℝ) ≤ S.card := by positivity
have hd0 : (0 : ℝ) ≤ d := by positivity
have hmpos : (0 : ℝ) < m := by
exact_mod_cast (lt_of_lt_of_le (by positivity : 0 < 4 * (K + 2) ^ 2) hm)
have hR0 : 0 ≤ (B * (6 : ℝ) ^ K) ^ 2 := sq_nonneg _
calc
(∑ k ∈ S, ∑ l ∈ S, covariance (F k) (F l) μ) ≤
∑ k ∈ S, ∑ l ∈ S, |covariance (F k) (F l) μ| := by
apply Finset.sum_le_sum
intro k hk
apply Finset.sum_le_sum
intro l hl
exact le_abs_self _
_ = ∑ k ∈ S, (|covariance (F k) (F k) μ| +
∑ l ∈ S.erase k, |covariance (F k) (F l) μ|) := by
apply Finset.sum_congr rfl
intro k hk
rw [← Finset.add_sum_erase _ _ hk]
_ ≤ ∑ _k ∈ S, (16 * (B * (6 : ℝ) ^ K) ^ 2 +
(d : ℝ) * (32 * (K : ℝ) ^ 2 / m *
(B * (6 : ℝ) ^ K) ^ 2)) := by
apply Finset.sum_le_sum
intro k hk
apply add_le_add (hsame k hk)
calc
(∑ l ∈ S.erase k, |covariance (F k) (F l) μ|) ≤
∑ _l ∈ S.erase k,
32 * (K : ℝ) ^ 2 / m * (B * (6 : ℝ) ^ K) ^ 2 := by
exact Finset.sum_le_sum fun l hl =>
hdiff k l hk (mem_of_mem_erase hl)
(Finset.ne_of_mem_erase hl).symm
_ = ((S.erase k).card : ℝ) *
(32 * (K : ℝ) ^ 2 / m * (B * (6 : ℝ) ^ K) ^ 2) := by
simp
_ ≤ (d : ℝ) *
(32 * (K : ℝ) ^ 2 / m * (B * (6 : ℝ) ^ K) ^ 2) := by
gcongr
exact_mod_cast (calc (S.erase k).card ≤ S.card := Finset.card_erase_le
_ ≤ d := by simpa using S.card_le_univ)
_ ≤ 16 * (d : ℝ) * (B * (6 : ℝ) ^ K) ^ 2 +
32 * (d : ℝ) ^ 2 * (K : ℝ) ^ 2 / m *
(B * (6 : ℝ) ^ K) ^ 2 := by
simp only [Finset.sum_const]
push_cast
have hinside0 : 0 ≤ 16 * (B * (6 : ℝ) ^ K) ^ 2 +
(d : ℝ) * (32 * (K : ℝ) ^ 2 / m *
(B * (6 : ℝ) ^ K) ^ 2) := by positivity
rw [nsmul_eq_mul]
push_cast
calc
(S.card : ℝ) * (16 * (B * 6 ^ K) ^ 2 +
(d : ℝ) * (32 * (K : ℝ) ^ 2 / m * (B * 6 ^ K) ^ 2)) ≤
(d : ℝ) * (16 * (B * 6 ^ K) ^ 2 +
(d : ℝ) * (32 * (K : ℝ) ^ 2 / m *
(B * 6 ^ K) ^ 2)) :=
mul_le_mul_of_nonneg_right hcard hinside0
_ = _ := by ring
_ ≤ 64 / m + 64 * 6 ^ (2 * K) *
((d : ℝ) * B ^ 2 + (d : ℝ) ^ 2 * K ^ 2 * B ^ 2 / m) := by
have hpow : ((6 : ℝ) ^ K) ^ 2 = 6 ^ (2 * K) := by
rw [← pow_mul]
congr 1
omega
rw [mul_pow, hpow]
have hA0 : 0 ≤ (d : ℝ) * B ^ 2 := by positivity
have hC0 : 0 ≤ (d : ℝ) ^ 2 * (K : ℝ) ^ 2 * B ^ 2 / m := by
positivity
have hsix0 : 0 ≤ (6 : ℝ) ^ (2 * K) := by positivity
have ht1 : 0 ≤ (d : ℝ) * B ^ 2 * 6 ^ (K * 2) := by positivity
have ht2 : 0 ≤ (d : ℝ) ^ 2 * B ^ 2 * (K : ℝ) ^ 2 *
(m : ℝ)⁻¹ * 6 ^ (K * 2) := by positivity
have hminv : 0 ≤ (m : ℝ)⁻¹ := by positivity
ring_nf
nlinarith
Helpers.FactorialCovarianceCoefficients 6 declarations
The signed coefficient in the real-outcome estimator is the same shifted Chebyshev coefficient used by the binary factorial construction.
Formal statement
hypotheses
K j :
ℕ
conclusion
shiftedCoefficient K j = CausalSmith.Stat.DiscreteAteMinimaxLoggap.gCoefficient K j
Proof (Lean source)
lemma shiftedCoefficient_eq_gCoefficient (K j : ℕ) :
shiftedCoefficient K j =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.gCoefficient K j := by
simp [shiftedCoefficient,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.gCoefficient]
Absolute shifted-coefficient series evaluated at a nonnegative intensity.
Definition (Lean source)
noncomputable def shiftedCoefficientEnvelope (K : ℕ) (x : ℝ) : ℝ :=
∑ j ∈ range (K - 1), |shiftedCoefficient K j| * x ^ j
theorem shiftedCoefficientEnvelope_eq_gpos
The marked-statistic coefficient envelope is the previously audited shifted-Chebyshev absolute series.
Formal statement
hypotheses
K :
ℕ
x :
ℝ
conclusion
shiftedCoefficientEnvelope K x = CausalSmith.Stat.DiscreteAteMinimaxLoggap.gpos K x
Proof (Lean source)
lemma shiftedCoefficientEnvelope_eq_gpos (K : ℕ) (x : ℝ) :
shiftedCoefficientEnvelope K x =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.gpos K x := by
unfold shiftedCoefficientEnvelope
CausalSmith.Stat.DiscreteAteMinimaxLoggap.gpos
simp_rw [shiftedCoefficient_eq_gCoefficient]
theorem shiftedCoefficientEnvelope_le
If the polynomial or elbow parameter satisfies its stated bound and the scalar satisfies the stated range condition, for nonnegative intensity, the absolute coefficient sum has the paper's 6^K max(1,x^(K-2)) envelope.
Formal statement
hypotheses
K :
ℕ
hK :
0 < K
x :
ℝ
hx :
0 ≤ x
conclusion
shiftedCoefficientEnvelope K x ≤ (6 : ℝ) ^ K * max 1 (x ^ (K - 2))
Proof (Lean source)
lemma shiftedCoefficientEnvelope_le {K : ℕ} (hK : 0 < K) {x : ℝ}
(hx : 0 ≤ x) :
shiftedCoefficientEnvelope K x ≤
(6 : ℝ) ^ K * max 1 (x ^ (K - 2)) := by
rw [shiftedCoefficientEnvelope_eq_gpos]
exact CausalSmith.Stat.DiscreteAteMinimaxLoggap.gpos_bound hK hx
theorem shiftedCoefficientEnvelope_le_six_pow
If the polynomial or elbow parameter satisfies its stated bound and the scalar is nonnegative and the scalar is at most one, on the unit intensity range, the absolute coefficient sum is at most 6^K.
Formal statement
hypotheses
K :
ℕ
hK :
0 < K
x :
ℝ
hx0 :
0 ≤ x
hx1 :
x ≤ 1
conclusion
shiftedCoefficientEnvelope K x ≤ (6 : ℝ) ^ K
Proof (Lean source)
lemma shiftedCoefficientEnvelope_le_six_pow {K : ℕ} (hK : 0 < K) {x : ℝ}
(hx0 : 0 ≤ x) (hx1 : x ≤ 1) :
shiftedCoefficientEnvelope K x ≤ (6 : ℝ) ^ K := by
refine (shiftedCoefficientEnvelope_le hK hx0).trans ?_
rw [max_eq_left]
· simp
· exact pow_le_one₀ hx0 hx1
theorem shiftedCoefficient_weighted_cellMass_sum_le Lemma shiftedCoefficient_weighted_cellMass_sum_le in the paper ↗
If the polynomial or elbow parameter satisfies its stated bound and the outcome bound is positive and the cell probability is nonnegative and the scaled cell probability satisfies the budget bound, on a light cell, the coefficient-weighted factorial mean series is at most B * 6^K. This is the one-dimensional factor in the cross-cell sum.
Formal statement
hypotheses
K :
ℕ
hK :
0 < K
B p :
ℝ
hB :
0 < B
hp0 :
0 ≤ p
hpB :
p ≤ B / 4
conclusion
∑ j ∈ range (K - 1), |shiftedCoefficient K j / B ^ (j + 1)| * p ^ (j + 2)
≤ B * (6 : ℝ) ^ K
Proof (Lean source)
lemma shiftedCoefficient_weighted_cellMass_sum_le
{K : ℕ} (hK : 0 < K) {B p : ℝ} (hB : 0 < B)
(hp0 : 0 ≤ p) (hpB : p ≤ B / 4) :
∑ j ∈ range (K - 1),
|shiftedCoefficient K j / B ^ (j + 1)| * p ^ (j + 2) ≤
B * (6 : ℝ) ^ K := by
have hB0 : 0 ≤ B := hB.le
have hp_le_B : p ≤ B := by linarith
have hratio0 : 0 ≤ p / B := div_nonneg hp0 hB0
have hratio1 : p / B ≤ 1 := (div_le_one hB).2 hp_le_B
have hrewrite :
(∑ j ∈ range (K - 1),
|shiftedCoefficient K j / B ^ (j + 1)| * p ^ (j + 2)) =
(p ^ 2 / B) * shiftedCoefficientEnvelope K (p / B) := by
unfold shiftedCoefficientEnvelope
rw [Finset.mul_sum]
apply Finset.sum_congr rfl
intro j hj
rw [abs_div, abs_pow, abs_of_pos hB]
have hBne : B ≠ 0 := ne_of_gt hB
rw [div_pow]
field_simp
ring
rw [hrewrite]
have henv := shiftedCoefficientEnvelope_le_six_pow hK hratio0 hratio1
have hp2 : p ^ 2 / B ≤ B := by
apply (div_le_iff₀ hB).2
nlinarith
exact (mul_le_mul_of_nonneg_left henv (div_nonneg (sq_nonneg p) hB0)).trans
(mul_le_mul_of_nonneg_right hp2 (by positivity))
Helpers.FactorialCovarianceExpansion 1 declarations
theorem centeredCrossMoment_allBlockOrderedMarkedFactorial
If the first factorial order is admissible and the second factorial order is admissible and the stated condition on the source size or matching order holds, the centered product of two paper-local all-block marked factorials is exactly the generic mixed-order partial-matching expansion.
Formal statement
hypotheses
d m K :
ℕ
epsilon M sigma :
ℝ
P :
ModelClass d epsilon M sigma
k l :
Fin d
a b :
j r :
ℕ
hj :
j + 2 ≤ K
hr :
r + 2 ≤ K
hm :
4 * (K + 2) ^ 2 ≤ m
conclusion
centeredCrossMoment (Measure.infinitePi fun _ : ℕ => P.law.observedLaw) (fun ω : ℕ → Obs d => allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) k a j) (fun ω : ℕ → Obs d => allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) l b r)
= (matchingNormalization m (empty (j + 2) (r + 2)) - 1) * orderedProductMean P.law.observedLaw (markedFactorialCoordinate M k a j) * orderedProductMean P.law.observedLaw (markedFactorialCoordinate M l b r)
+ ∑ h ∈ (range (min (j + 2) (r + 2) + 1)).filter (fun h => 0 < h), ∑ N ∈ partialMatchingsOfSize (j + 2) (r + 2) h, matchingNormalization m N * mergedProductMoment P.law.observedLaw (markedFactorialCoordinate M k a j) (markedFactorialCoordinate M l b r) N
Proof (Lean source)
lemma centeredCrossMoment_allBlockOrderedMarkedFactorial
{d m K : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (k l : Fin d) (a b : Bool)
(j r : ℕ) (hj : j + 2 ≤ K) (hr : r + 2 ≤ K)
(hm : 4 * (K + 2) ^ 2 ≤ m) :
centeredCrossMoment
(Measure.infinitePi fun _ : ℕ => P.law.observedLaw)
(fun ω : ℕ → Obs d =>
allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) k a j)
(fun ω : ℕ → Obs d =>
allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) l b r) =
(matchingNormalization m
(empty (j + 2) (r + 2)) - 1) *
orderedProductMean P.law.observedLaw
(markedFactorialCoordinate M k a j) *
orderedProductMean P.law.observedLaw
(markedFactorialCoordinate M l b r) +
∑ h ∈ (range (min (j + 2) (r + 2) + 1)).filter
(fun h => 0 < h),
∑ N ∈ partialMatchingsOfSize (j + 2) (r + 2) h,
matchingNormalization m N *
mergedProductMoment P.law.observedLaw
(markedFactorialCoordinate M k a j)
(markedFactorialCoordinate M l b r) N := by
let S0 := Causalean.Stat.iidSample_infinitePi P.law.observedLaw
have hKm : K ≤ m := by
calc
K ≤ K + 2 := by omega
_ ≤ (K + 2) ^ 2 := Nat.le_pow (by omega)
_ ≤ 4 * (K + 2) ^ 2 := by omega
_ ≤ m := hm
have hjm : j + 2 ≤ m := hj.trans hKm
have hrm : r + 2 ≤ m := hr.trans hKm
have hkstat :
(fun ω : ℕ → Obs d =>
allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) k a j) =
normalizedOrderedProductStatistic S0
(markedFactorialCoordinate M k a j) m := by
funext ω
exact allBlockOrderedMarkedFactorial_eq_normalizedOrderedProductStatistic
(m := m) S0 M k a j ω
have hlstat :
(fun ω : ℕ → Obs d =>
allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) l b r) =
normalizedOrderedProductStatistic S0
(markedFactorialCoordinate M l b r) m := by
funext ω
exact allBlockOrderedMarkedFactorial_eq_normalizedOrderedProductStatistic
(m := m) S0 M l b r ω
rw [hkstat, hlstat]
exact centeredCrossMoment_normalizedOrderedProductStatistic
S0 hjm hrm
(markedFactorialCoordinate M k a j)
(markedFactorialCoordinate M l b r)
(measurable_orderedProductKernel_markedFactorialCoordinate M k a j)
(measurable_orderedProductKernel_markedFactorialCoordinate M l b r)
(integrable_orderedProductKernel_markedFactorialCoordinate P k a j)
(integrable_orderedProductKernel_markedFactorialCoordinate P l b r)
(fun N => measurable_mergedProductKernel_markedFactorialCoordinate
M k l a b j r N)
(fun N => integrable_mergedProductKernel_markedFactorialCoordinate
P k l a b j r N)
Helpers.FactorialCovarianceMoments 8 declarations Paper-local moment bounds for marked factorial coordinates
Paper-local moment bounds for marked factorial coordinates
theorem observed_normalized_outcome_sq_arm_cell_integral_le
The squared normalized mark restricted to one arm-cell has mass-weighted second moment at most 5/4 times the cell mass.
Formal statement
hypotheses
conclusion
∫ o : Obs d, {z : Obs d | z.x = k ∧ z.a = a}.indicator (fun z => (z.y / M) ^ 2) o ∂P.law.observedLaw
≤ (5 / 4 : ℝ) * P.law.cellMass k
Proof (Lean source)
lemma observed_normalized_outcome_sq_arm_cell_integral_le
{d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (a : Bool) (k : Fin d) :
∫ o : Obs d, {z : Obs d | z.x = k ∧ z.a = a}.indicator
(fun z => (z.y / M) ^ 2) o ∂P.law.observedLaw ≤
(5 / 4 : ℝ) * P.law.cellMass k := by
have htuple : Measurable (fun o : Obs d => (o.x, o.a, o.y)) :=
measurable_iff_comap_le.mpr le_rfl
have hy : Measurable (fun o : Obs d => o.y) :=
measurable_snd.comp (measurable_snd.comp htuple)
have hxa : MeasurableSet {o : Obs d | o.x = k ∧ o.a = a} := by
have hx : Measurable (fun o : Obs d => o.x) := measurable_fst.comp htuple
have ha : Measurable (fun o : Obs d => o.a) :=
measurable_fst.comp (measurable_snd.comp htuple)
simpa only [preimage, Set.mem_singleton_iff, Prod.mk.injEq] using
(measurableSet_singleton (k, a)).preimage (prodMk hx ha)
by_cases hk : P.law.cellMass k = 0
· have hnull := observed_arm_cell_measure_eq_zero_of_cellMass_eq_zero
P.law a k hk
rw [integral_indicator hxa]
have hr : P.law.observedLaw.restrict {o : Obs d | o.x = k ∧ o.a = a} = 0 :=
Measure.restrict_eq_zero.mpr hnull
rw [hr, integral_zero_measure, hk, mul_zero]
· have hkpos : 0 < P.law.cellMass k :=
lt_of_le_of_ne (P.law.cellMass_range k).1 (Ne.symm hk)
have hc := normalized_outcome_second_moment_le_five_fourths P a k hkpos
have hprop : 0 ≤ if a then P.law.propensity k else 1 - P.law.propensity k := by
split <;> simp_all [(P.law.propensity_range k).1,
(P.law.propensity_range k).2]
have hprop1 : (if a then P.law.propensity k else 1 - P.law.propensity k) ≤ 1 := by
split <;> simp_all [(P.law.propensity_range k).1,
(P.law.propensity_range k).2]
let c := P.law.cellMass k *
(if a then P.law.propensity k else 1 - P.law.propensity k)
have hc0 : 0 ≤ c := mul_nonneg (P.law.cellMass_range k).1 hprop
rw [integral_indicator hxa]
have hmap := observed_arm_cell_outcome_measure P.law a k
have hmeasf : AEStronglyMeasurable (fun y : ℝ => (y / M) ^ 2)
(Measure.map (fun o : Obs d => o.y)
(P.law.observedLaw.restrict {o : Obs d | o.x = k ∧ o.a = a})) :=
((measurable_id.div measurable_const).pow_const 2).aestronglyMeasurable
rw [← integral_map hy.aemeasurable hmeasf, hmap, integral_smul_measure]
change toReal (ofReal c) *
(∫ y, (y / M) ^ 2 ∂P.law.outcomeLaw a k) ≤ _
rw [ENNReal.toReal_ofReal hc0]
calc
c * (∫ y, (y / M) ^ 2 ∂P.law.outcomeLaw a k) ≤ c * (5 / 4 : ℝ) :=
mul_le_mul_of_nonneg_left hc.2 hc0
_ ≤ (5 / 4 : ℝ) * P.law.cellMass k := by
dsimp [c]
nlinarith [P.law.cellMass_range k |>.1]
theorem integral_markedFactorialCoordinate_zero_abs_le
The unique marked coordinate has mean bounded by one half of its cell mass; this is the normalized conditional-mean audit.
Formal statement
hypotheses
conclusion
|∫ o : Obs d, markedFactorialCoordinate M k a j 0 o ∂P.law.observedLaw|
≤ (1 / 2 : ℝ) * P.law.cellMass k
Proof (Lean source)
lemma integral_markedFactorialCoordinate_zero_abs_le
{d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (a : Bool) (k : Fin d) (j : ℕ) :
|∫ o : Obs d, markedFactorialCoordinate M k a j 0 o
∂P.law.observedLaw| ≤ (1 / 2 : ℝ) * P.law.cellMass k := by
have htuple : Measurable (fun o : Obs d => (o.x, o.a, o.y)) :=
measurable_iff_comap_le.mpr le_rfl
have hy : Measurable (fun o : Obs d => o.y) :=
measurable_snd.comp (measurable_snd.comp htuple)
have hxa : MeasurableSet {o : Obs d | o.x = k ∧ o.a = a} := by
have hx : Measurable (fun o : Obs d => o.x) := measurable_fst.comp htuple
have ha : Measurable (fun o : Obs d => o.a) :=
measurable_fst.comp (measurable_snd.comp htuple)
simpa only [preimage, Set.mem_singleton_iff, Prod.mk.injEq] using
(measurableSet_singleton (k, a)).preimage (prodMk hx ha)
have hcoord : (fun o : Obs d => markedFactorialCoordinate M k a j 0 o) =
{o : Obs d | o.x = k ∧ o.a = a}.indicator (fun o => o.y / M) := by
funext o
by_cases ho : o.x = k ∧ o.a = a <;>
simp [markedFactorialCoordinate, indicator, ho]
rw [hcoord, integral_indicator hxa]
by_cases hk : P.law.cellMass k = 0
· have hnull := observed_arm_cell_measure_eq_zero_of_cellMass_eq_zero
P.law a k hk
have hr : P.law.observedLaw.restrict {o : Obs d | o.x = k ∧ o.a = a} = 0 :=
Measure.restrict_eq_zero.mpr hnull
rw [hr, integral_zero_measure, abs_zero, hk, mul_zero]
· have hkpos : 0 < P.law.cellMass k :=
lt_of_le_of_ne (P.law.cellMass_range k).1 (Ne.symm hk)
have hc := normalized_outcome_mean_abs_le_half P a k hkpos
have hprop : 0 ≤ if a then P.law.propensity k else 1 - P.law.propensity k := by
split <;> simp_all [(P.law.propensity_range k).1,
(P.law.propensity_range k).2]
have hprop1 : (if a then P.law.propensity k else 1 - P.law.propensity k) ≤ 1 := by
split <;> simp_all [(P.law.propensity_range k).1,
(P.law.propensity_range k).2]
let c := P.law.cellMass k *
(if a then P.law.propensity k else 1 - P.law.propensity k)
have hc0 : 0 ≤ c := mul_nonneg (P.law.cellMass_range k).1 hprop
have hmap := observed_arm_cell_outcome_measure P.law a k
have hmeasf : AEStronglyMeasurable (fun y : ℝ => y / M)
(Measure.map (fun o : Obs d => o.y)
(P.law.observedLaw.restrict {o : Obs d | o.x = k ∧ o.a = a})) :=
(measurable_id.div measurable_const).aestronglyMeasurable
rw [← integral_map hy.aemeasurable hmeasf, hmap, integral_smul_measure]
change |toReal (ofReal c) *
(∫ y, y / M ∂P.law.outcomeLaw a k)| ≤ _
rw [ENNReal.toReal_ofReal hc0, abs_mul, abs_of_nonneg hc0]
calc
c * |∫ y, y / M ∂P.law.outcomeLaw a k| ≤ c * (1 / 2 : ℝ) :=
mul_le_mul_of_nonneg_left hc.2 hc0
_ ≤ (1 / 2 : ℝ) * P.law.cellMass k := by
dsimp [c]
nlinarith [P.law.cellMass_range k |>.1]
theorem integral_markedFactorialCoordinate_abs_le_cellMass
Every coordinate mean is bounded by the cell mass. The marked coordinate uses the preceding mean audit; selector-only coordinates are event masses.
Formal statement
hypotheses
conclusion
|∫ o : Obs d, markedFactorialCoordinate M k a j q o ∂P.law.observedLaw| ≤ P.law.cellMass k
Proof (Lean source)
lemma integral_markedFactorialCoordinate_abs_le_cellMass
{d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (a : Bool) (k : Fin d)
(j : ℕ) (q : Fin (j + 2)) :
|∫ o : Obs d, markedFactorialCoordinate M k a j q o
∂P.law.observedLaw| ≤ P.law.cellMass k := by
by_cases hq0 : q.val = 0
· have hq : q = 0 := Fin.ext hq0
subst q
exact (integral_markedFactorialCoordinate_zero_abs_le P a k j).trans (by
have hp := P.law.cellMass_range k |>.1
nlinarith)
· have htuple : Measurable (fun o : Obs d => (o.x, o.a, o.y)) :=
measurable_iff_comap_le.mpr le_rfl
have hx : Measurable (fun o : Obs d => o.x) := measurable_fst.comp htuple
have ha : Measurable (fun o : Obs d => o.a) :=
measurable_fst.comp (measurable_snd.comp htuple)
have hxset : MeasurableSet {o : Obs d | o.x = k} :=
(measurableSet_singleton k).preimage hx
by_cases hq1 : q.val = 1
· have heq : (fun o : Obs d => markedFactorialCoordinate M k a j q o) =
{o : Obs d | o.x = k}.indicator (fun _ => (1 : ℝ)) := by
funext o
simp [markedFactorialCoordinate, hq1, indicator]
rw [heq]
have hi : (∫ o : Obs d, {o : Obs d | o.x = k}.indicator
(fun _ => (1 : ℝ)) o ∂P.law.observedLaw) =
(P.law.observedLaw {o : Obs d | o.x = k}).toReal := by
rw [integral_indicator hxset]
simp [Measure.real_def]
rw [hi, abs_of_nonneg ENNReal.toReal_nonneg]
exact (P.law.cellMass_eq k).ge
· have hxa : MeasurableSet {o : Obs d | o.x = k ∧ o.a = a} := by
simpa only [preimage, Set.mem_singleton_iff, Prod.mk.injEq] using
(measurableSet_singleton (k, a)).preimage (prodMk hx ha)
have heq : (fun o : Obs d => markedFactorialCoordinate M k a j q o) =
{o : Obs d | o.x = k ∧ o.a = a}.indicator (fun _ => (1 : ℝ)) := by
funext o
simp [markedFactorialCoordinate, hq0, hq1, indicator]
rw [heq]
have hi : (∫ o : Obs d, {o : Obs d | o.x = k ∧ o.a = a}.indicator
(fun _ => (1 : ℝ)) o ∂P.law.observedLaw) =
(P.law.observedLaw {o : Obs d | o.x = k ∧ o.a = a}).toReal := by
rw [integral_indicator hxa]
simp [Measure.real_def]
rw [hi, abs_of_nonneg ENNReal.toReal_nonneg, P.law.cellMass_eq]
exact ENNReal.toReal_mono (measure_ne_top _ _)
(measure_mono (by intro o (ho : o.x = k ∧ o.a = a); exact ho.1))
theorem integral_markedFactorialCoordinate_sq_le
Every coordinate has second moment at most 5/4 times its cell mass. The marked coordinate uses the outcome envelope, while every other coordinate is an idempotent selector.
Formal statement
hypotheses
conclusion
(∫ o : Obs d, (markedFactorialCoordinate M k a j q o) ^ 2 ∂P.law.observedLaw)
≤ (5 / 4 : ℝ) * P.law.cellMass k
Proof (Lean source)
lemma integral_markedFactorialCoordinate_sq_le
{d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (a : Bool) (k : Fin d)
(j : ℕ) (q : Fin (j + 2)) :
(∫ o : Obs d, (markedFactorialCoordinate M k a j q o) ^ 2
∂P.law.observedLaw) ≤ (5 / 4 : ℝ) * P.law.cellMass k := by
by_cases hq0 : q.val = 0
· have hq : q = 0 := Fin.ext hq0
subst q
simpa [markedFactorialCoordinate, indicator, mul_pow] using
observed_normalized_outcome_sq_arm_cell_integral_le P a k
· have hsq :
(fun o : Obs d => (markedFactorialCoordinate M k a j q o) ^ 2) =
markedFactorialCoordinate M k a j q := by
funext o
simp only [markedFactorialCoordinate, hq0, if_false]
split <;> split <;> norm_num
rw [hsq]
calc
(∫ o : Obs d, markedFactorialCoordinate M k a j q o
∂P.law.observedLaw) ≤
|∫ o : Obs d, markedFactorialCoordinate M k a j q o
∂P.law.observedLaw| := le_abs_self _
_ ≤ P.law.cellMass k :=
integral_markedFactorialCoordinate_abs_le_cellMass P a k j q
_ ≤ (5 / 4 : ℝ) * P.law.cellMass k := by
have hp := P.law.cellMass_range k |>.1
nlinarith
theorem integral_markedFactorialCoordinate_mul_abs_le
When two coordinate factors are assigned to the same observation in a partial matching, their product moment is still bounded by 5/4 times the cell mass. This uniformly covers a collision of the two real marks.
Formal statement
hypotheses
d :
ℕ
epsilon M sigma :
ℝ
P :
ModelClass d epsilon M sigma
k :
Fin d
a b :
j r :
ℕ
q :
Fin (j + 2)
s :
Fin (r + 2)
conclusion
|∫ o : Obs d, markedFactorialCoordinate M k a j q o * markedFactorialCoordinate M k b r s o ∂P.law.observedLaw|
≤ (5 / 4 : ℝ) * P.law.cellMass k
Proof (Lean source)
lemma integral_markedFactorialCoordinate_mul_abs_le
{d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (k : Fin d) (a b : Bool)
(j r : ℕ) (q : Fin (j + 2)) (s : Fin (r + 2)) :
|∫ o : Obs d,
markedFactorialCoordinate M k a j q o *
markedFactorialCoordinate M k b r s o
∂P.law.observedLaw| ≤ (5 / 4 : ℝ) * P.law.cellMass k := by
let f : Obs d → ℝ := markedFactorialCoordinate M k a j q
let g : Obs d → ℝ := markedFactorialCoordinate M k b r s
have hf2 : MemLp f 2 P.law.observedLaw :=
memLp_two_markedFactorialCoordinate P k a j q
have hg2 : MemLp g 2 P.law.observedLaw :=
memLp_two_markedFactorialCoordinate P k b r s
have hfg : Integrable (fun o => f o * g o) P.law.observedLaw := by
exact (hg2.mul' hf2 : MemLp (fun o => f o * g o) 1
P.law.observedLaw).integrable (by norm_num)
calc
|∫ o, f o * g o ∂P.law.observedLaw| ≤
∫ o, |f o * g o| ∂P.law.observedLaw :=
abs_integral_le_integral_abs
_ ≤ ∫ o, ((f o) ^ 2 + (g o) ^ 2) / 2
∂P.law.observedLaw := by
apply integral_mono_ae hfg.abs
· exact (hf2.integrable_sq.add hg2.integrable_sq).div_const 2
· filter_upwards with o
rw [abs_mul]
have h := two_mul_le_add_sq (|f o|) (|g o|)
rw [sq_abs, sq_abs] at h
linarith
_ = ((∫ o, (f o) ^ 2 ∂P.law.observedLaw) +
∫ o, (g o) ^ 2 ∂P.law.observedLaw) / 2 := by
rw [integral_div]
congr 1
rw [integral_add hf2.integrable_sq hg2.integrable_sq]
_ ≤ (5 / 4 : ℝ) * P.law.cellMass k := by
have hf := integral_markedFactorialCoordinate_sq_le P a k j q
have hg := integral_markedFactorialCoordinate_sq_le P b k r s
dsimp [f, g] at hf hg ⊢
linarith
def mergedMarkedCoordinateFactor
The factors assigned to one merged observation by a partial matching. Each fiber contains at most one coordinate from either ordered kernel.
Definition (Lean source)
noncomputable def mergedMarkedCoordinateFactor
{d : ℕ} (M : ℝ) (k : Fin d) (a b : Bool) (j r : ℕ)
(N : PartialMatching (j + 2) (r + 2))
(t : N.MergedIndex) (o : Obs d) : ℝ :=
(∏ q ∈ (Finset.univ : Finset (Fin (j + 2))).filter
(fun q => N.leftInjection q = t),
markedFactorialCoordinate M k a j q o) *
∏ s ∈ (Finset.univ : Finset (Fin (r + 2))).filter
(fun s => N.rightInjection s = t),
markedFactorialCoordinate M k b r s o
theorem mergedProductKernel_markedFactorialCoordinate_eq_prod_fibers
The merged marked kernel factors over its genuinely distinct observation indices, grouping the possible left/right coordinate collision in one fiber.
Formal statement
hypotheses
conclusion
= ∏ t : N.MergedIndex, mergedMarkedCoordinateFactor M k a b j r N t (z t)
Proof (Lean source)
lemma mergedProductKernel_markedFactorialCoordinate_eq_prod_fibers
{d : ℕ} (M : ℝ) (k : Fin d) (a b : Bool) (j r : ℕ)
(N : PartialMatching (j + 2) (r + 2))
(z : N.MergedIndex → Obs d) :
mergedProductKernel
(markedFactorialCoordinate M k a j)
(markedFactorialCoordinate M k b r) N z =
∏ t : N.MergedIndex,
mergedMarkedCoordinateFactor M k a b j r N t (z t) := by
classical
unfold mergedProductKernel mergedMarkedCoordinateFactor
rw [← Finset.prod_fiberwise (Finset.univ : Finset (Fin (j + 2)))
N.leftInjection
(fun q => markedFactorialCoordinate M k a j q (z (N.leftInjection q)))]
rw [← Finset.prod_fiberwise (Finset.univ : Finset (Fin (r + 2)))
N.rightInjection
(fun s => markedFactorialCoordinate M k b r s (z (N.rightInjection s)))]
rw [← Finset.prod_mul_distrib]
apply Finset.prod_congr rfl
intro t ht
congr 1
· apply Finset.prod_congr rfl
intro q hq
rw [(Finset.mem_filter.mp hq).2]
· apply Finset.prod_congr rfl
intro s hs
rw [(Finset.mem_filter.mp hs).2]
theorem orderedProductMean_markedFactorialCoordinate_abs_le
Independence across the ordered kernel coordinates bounds its population mean by the corresponding power of the cell mass.
Formal statement
hypotheses
conclusion
|orderedProductMean P.law.observedLaw (markedFactorialCoordinate M k a j)|
≤ P.law.cellMass k ^ (j + 2)
Proof (Lean source)
lemma orderedProductMean_markedFactorialCoordinate_abs_le
{d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (a : Bool) (k : Fin d) (j : ℕ) :
|orderedProductMean P.law.observedLaw
(markedFactorialCoordinate M k a j)| ≤ P.law.cellMass k ^ (j + 2) := by
unfold orderedProductMean orderedProductKernel
rw [MeasureTheory.integral_fintype_prod_eq_prod]
rw [← Real.norm_eq_abs, norm_prod univ]
simp only [Real.norm_eq_abs]
calc
(∏ q : Fin (j + 2),
|∫ x, markedFactorialCoordinate M k a j q x ∂P.law.observedLaw|) ≤
∏ _q : Fin (j + 2), P.law.cellMass k := by
apply Finset.prod_le_prod
· intro q _
exact abs_nonneg _
· intro q _
exact integral_markedFactorialCoordinate_abs_le_cellMass P a k j q
_ = P.law.cellMass k ^ (j + 2) := by simp
Helpers.FactorialCovarianceSameCell 5 declarations
theorem mergedMarkedCoordinateFactor_integral_abs_le
the absolute integral of a merged marked-coordinate factor satisfies the stated cell-probability bound.
Formal statement
hypotheses
d :
ℕ
epsilon M sigma :
ℝ
P :
ModelClass d epsilon M sigma
k :
Fin d
a b :
j r :
ℕ
N :
PartialMatching (j + 2) (r + 2)
t :
N.MergedIndex
conclusion
|∫ o : Obs d, mergedMarkedCoordinateFactor M k a b j r N t o ∂P.law.observedLaw|
≤ (5 / 4 : ℝ) * P.law.cellMass k
Proof (Lean source)
lemma mergedMarkedCoordinateFactor_integral_abs_le {d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (k : Fin d) (a b : Bool)
(j r : ℕ) (N : PartialMatching (j + 2) (r + 2))
(t : N.MergedIndex) :
|∫ o : Obs d, mergedMarkedCoordinateFactor M k a b j r N t o
∂P.law.observedLaw| ≤ (5 / 4 : ℝ) * P.law.cellMass k := by
classical
cases t with
| inl q =>
unfold mergedMarkedCoordinateFactor
simp only [leftInjection, Sum.inl.injEq,
Finset.filter_eq']
by_cases hq : q ∈ N.left
· let s : Fin (r + 2) := (N.equiv ⟨q, hq⟩).1
have hs : s ∈ N.right := (N.equiv ⟨q, hq⟩).2
have hfilter : (Finset.univ : Finset (Fin (r + 2))).filter
(fun x => N.rightInjection x = inl q) = {s} := by
ext x
simp only [mem_filter, Finset.mem_univ, true_and,
mem_singleton]
constructor
· intro hx
by_cases hxr : x ∈ N.right
· simp only [rightInjection, hxr,
dite_true] at hx
have he : N.equiv.symm ⟨x, hxr⟩ = ⟨q, hq⟩ := by
apply Subtype.ext
exact Sum.inl.inj hx
have he' := congrArg N.equiv he
simpa [s] using congrArg val he'
· simp [rightInjection, hxr] at hx
· intro hx
subst x
simp [rightInjection, hs, s]
rw [hfilter]
simpa using integral_markedFactorialCoordinate_mul_abs_le P k a b j r q s
· have hfilter : (Finset.univ : Finset (Fin (r + 2))).filter
(fun x => N.rightInjection x = inl q) = ∅ := by
ext x
simp only [mem_filter, Finset.mem_univ, true_and]
constructor
· intro hx
by_cases hxr : x ∈ N.right
· simp only [rightInjection, hxr,
dite_true] at hx
have hval : (N.equiv.symm ⟨x, hxr⟩).1 = q := Sum.inl.inj hx
have hf : False := hq (by
simpa [hval] using (N.equiv.symm ⟨x, hxr⟩).2)
exact hf.elim
· simp [rightInjection, hxr] at hx
· intro hx
have hf : False := by simpa using hx
exact hf.elim
rw [hfilter]
simpa using (integral_markedFactorialCoordinate_abs_le_cellMass P a k j q).trans
(by have hp := P.law.cellMass_range k |>.1; nlinarith)
| inr s =>
unfold mergedMarkedCoordinateFactor
simp only [leftInjection, Sum.inl.injEq,
Finset.filter_false, Finset.prod_empty, one_mul]
have hfilter : (Finset.univ : Finset (Fin (r + 2))).filter
(fun x => N.rightInjection x = inr s) = {s.1} := by
ext x
simp only [mem_filter, Finset.mem_univ, true_and,
mem_singleton]
constructor
· intro hx
by_cases hxr : x ∈ N.right
· simp [rightInjection, hxr] at hx
· simp only [rightInjection, hxr,
dite_false] at hx
apply Fin.ext
exact congrArg (fun z => z.1.1) (Sum.inr.inj hx)
· intro hx
subst x
simp [rightInjection, s.2]
rw [hfilter]
simpa using (integral_markedFactorialCoordinate_abs_le_cellMass
P b k r s.1).trans
(by have hp := P.law.cellMass_range k |>.1; nlinarith)
theorem mergedProductMoment_markedFactorialCoordinate_same_cell_abs_le
If the sampling budget satisfies the stated lower bound, the same-cell merged product moment of marked factorial coordinates satisfies the stated bound.
Formal statement
hypotheses
d :
ℕ
epsilon M sigma :
ℝ
P :
ModelClass d epsilon M sigma
k :
Fin d
a b :
j r h :
ℕ
N :
PartialMatching (j + 2) (r + 2)
hN :
N.size = h
conclusion
|mergedProductMoment P.law.observedLaw (markedFactorialCoordinate M k a j) (markedFactorialCoordinate M k b r) N|
≤ ((5 / 4 : ℝ) * P.law.cellMass k) ^ ((j + 2) + (r + 2) - h)
Proof (Lean source)
lemma mergedProductMoment_markedFactorialCoordinate_same_cell_abs_le {d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (k : Fin d) (a b : Bool)
(j r h : ℕ) (N : PartialMatching (j + 2) (r + 2))
(hN : N.size = h) :
|mergedProductMoment P.law.observedLaw
(markedFactorialCoordinate M k a j)
(markedFactorialCoordinate M k b r) N| ≤
((5 / 4 : ℝ) * P.law.cellMass k) ^ ((j + 2) + (r + 2) - h) := by
classical
unfold mergedProductMoment
simp_rw [mergedProductKernel_markedFactorialCoordinate_eq_prod_fibers]
rw [MeasureTheory.integral_fintype_prod_eq_prod, ← Real.norm_eq_abs,
norm_prod]
simp only [Real.norm_eq_abs]
calc
(∏ t : N.MergedIndex,
|∫ o : Obs d, mergedMarkedCoordinateFactor M k a b j r N t o
∂P.law.observedLaw|) ≤
∏ _t : N.MergedIndex, (5 / 4 : ℝ) * P.law.cellMass k := by
apply Finset.prod_le_prod
· intro t _
exact abs_nonneg _
· intro t _
exact mergedMarkedCoordinateFactor_integral_abs_le P k a b j r N t
_ = ((5 / 4 : ℝ) * P.law.cellMass k) ^ ((j + 2) + (r + 2) - h) := by
rw [Finset.prod_const, Finset.card_univ,
mergedIndex_card, hN]
If the first factorial order is admissible and the second factorial order is admissible and the stated condition on the source size or matching order holds, the centered cross-moment of two marked factorial statistics from the same cell satisfies the stated bound.
Formal statement
hypotheses
d m K :
ℕ
epsilon M sigma :
ℝ
P :
ModelClass d epsilon M sigma
k :
Fin d
a b :
j r :
ℕ
hj :
j + 2 ≤ K
hr :
r + 2 ≤ K
hm :
4 * (K + 2) ^ 2 ≤ m
conclusion
|centeredCrossMoment (Measure.infinitePi fun _ : ℕ => P.law.observedLaw) (fun ω : ℕ → Obs d => allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) k a j) (fun ω : ℕ → Obs d => allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) k b r)|
≤ (2 * ((K + 2 : ℕ) : ℝ) ^ 2 / m) * P.law.cellMass k ^ (j + 2) * P.law.cellMass k ^ (r + 2)
+ exp 1 * ((5 / 4 : ℝ) * P.law.cellMass k) ^ (r + 2) * ((5 / 4 : ℝ) * P.law.cellMass k + (r + 2 : ℕ) / m) ^ (j + 2)
Proof (Lean source)
lemma centeredCrossMoment_allBlockOrderedMarkedFactorial_same_cell_abs_le {d m K : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (k : Fin d) (a b : Bool)
(j r : ℕ) (hj : j + 2 ≤ K) (hr : r + 2 ≤ K)
(hm : 4 * (K + 2) ^ 2 ≤ m) :
|centeredCrossMoment
(Measure.infinitePi fun _ : ℕ => P.law.observedLaw)
(fun ω : ℕ → Obs d =>
allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) k a j)
(fun ω : ℕ → Obs d =>
allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) k b r)| ≤
(2 * ((K + 2 : ℕ) : ℝ) ^ 2 / m) *
P.law.cellMass k ^ (j + 2) * P.law.cellMass k ^ (r + 2) +
exp 1 * ((5 / 4 : ℝ) * P.law.cellMass k) ^ (r + 2) *
((5 / 4 : ℝ) * P.law.cellMass k + (r + 2 : ℕ) / m) ^ (j + 2) := by
classical
rw [centeredCrossMoment_allBlockOrderedMarkedFactorial P k k a b j r hj hr hm]
have hp : 0 ≤ (5 / 4 : ℝ) * P.law.cellMass k := by
exact mul_nonneg (by norm_num) (P.law.cellMass_range k).1
have hpos :
|∑ h ∈ (range (min (j + 2) (r + 2) + 1)).filter (fun h => 0 < h),
∑ N ∈ partialMatchingsOfSize (j + 2) (r + 2) h,
matchingNormalization m N *
mergedProductMoment P.law.observedLaw
(markedFactorialCoordinate M k a j)
(markedFactorialCoordinate M k b r) N| ≤
exp 1 * ((5 / 4 : ℝ) * P.law.cellMass k) ^ (r + 2) *
((5 / 4 : ℝ) * P.law.cellMass k + (r + 2 : ℕ) / m) ^ (j + 2) := by
calc
_ ≤ ∑ h ∈ (range (min (j + 2) (r + 2) + 1)).filter
(fun h => 0 < h),
(∑ N ∈ partialMatchingsOfSize (j + 2) (r + 2) h,
matchingNormalization m N) *
((5 / 4 : ℝ) * P.law.cellMass k) ^ ((j + 2) + (r + 2) - h) := by
calc
_ ≤ ∑ h ∈ (range (min (j + 2) (r + 2) + 1)).filter
(fun h => 0 < h),
|∑ N ∈ partialMatchingsOfSize (j + 2) (r + 2) h,
matchingNormalization m N *
mergedProductMoment P.law.observedLaw
(markedFactorialCoordinate M k a j)
(markedFactorialCoordinate M k b r) N| :=
Finset.abs_sum_le_sum_abs _ _
_ ≤ _ := by
apply Finset.sum_le_sum
intro h hh
calc
_ ≤ ∑ N ∈ partialMatchingsOfSize (j + 2) (r + 2) h,
|matchingNormalization m N *
mergedProductMoment P.law.observedLaw
(markedFactorialCoordinate M k a j)
(markedFactorialCoordinate M k b r) N| :=
Finset.abs_sum_le_sum_abs _ _
_ ≤
∑ N ∈ partialMatchingsOfSize (j + 2) (r + 2) h,
matchingNormalization m N *
((5 / 4 : ℝ) * P.law.cellMass k) ^ ((j + 2) + (r + 2) - h) := by
apply Finset.sum_le_sum
intro N hN
rw [abs_mul, abs_of_nonneg (by
unfold matchingNormalization
positivity)]
exact mul_le_mul_of_nonneg_left
(mergedProductMoment_markedFactorialCoordinate_same_cell_abs_le P k a b j r h N (by simpa using hN))
(by unfold matchingNormalization; positivity)
_ = _ := by rw [Finset.sum_mul]
_ ≤ _ := positiveMatchingNormalization_weighted_sum_le
(R := K + 2) (hj.trans (by omega)) (hr.trans (by omega)) hm hp
have hempty :
|(matchingNormalization m
(empty (j + 2) (r + 2)) - 1) *
orderedProductMean P.law.observedLaw
(markedFactorialCoordinate M k a j) *
orderedProductMean P.law.observedLaw
(markedFactorialCoordinate M k b r)| ≤
(2 * ((K + 2 : ℕ) : ℝ) ^ 2 / m) *
P.law.cellMass k ^ (j + 2) * P.law.cellMass k ^ (r + 2) := by
rw [abs_mul, abs_mul]
have hn := marked_emptyMatchingNormalization_sub_one_le hj hr hm
have hjm := orderedProductMean_markedFactorialCoordinate_abs_le P a k j
have hrm := orderedProductMean_markedFactorialCoordinate_abs_le P b k r
have hmpos : (0 : ℝ) < m := by
exact_mod_cast (lt_of_lt_of_le (by positivity : 0 < 4 * (K + 2) ^ 2) hm)
gcongr
exact mul_nonneg (by positivity) (pow_nonneg (P.law.cellMass_range k).1 _)
exact (abs_add_le _ _).trans (add_le_add hempty hpos)
theorem shiftedCoefficient_weighted_sum_le_of_le Lemma shiftedCoefficient_weighted_sum_le_of_le in the paper ↗
If the polynomial or elbow parameter satisfies its stated bound and the outcome bound is positive and the cell probability is nonnegative and the scaled cell probability satisfies the budget bound, a pointwise coefficient bound implies the corresponding weighted-sum bound.
Formal statement
hypotheses
K :
ℕ
hK :
0 < K
B p :
ℝ
hB :
0 < B
hp0 :
0 ≤ p
hpB :
p ≤ B
conclusion
∑ j ∈ range (K - 1), |shiftedCoefficient K j / B ^ (j + 1)| * p ^ (j + 2)
≤ B * (6 : ℝ) ^ K
Proof (Lean source)
lemma shiftedCoefficient_weighted_sum_le_of_le {K : ℕ} (hK : 0 < K) {B p : ℝ} (hB : 0 < B)
(hp0 : 0 ≤ p) (hpB : p ≤ B) :
∑ j ∈ range (K - 1),
|shiftedCoefficient K j / B ^ (j + 1)| * p ^ (j + 2) ≤
B * (6 : ℝ) ^ K := by
have hB0 : 0 ≤ B := hB.le
have hratio0 : 0 ≤ p / B := div_nonneg hp0 hB0
have hratio1 : p / B ≤ 1 := (div_le_one hB).2 hpB
have hrewrite :
(∑ j ∈ range (K - 1),
|shiftedCoefficient K j / B ^ (j + 1)| * p ^ (j + 2)) =
(p ^ 2 / B) * shiftedCoefficientEnvelope K (p / B) := by
unfold shiftedCoefficientEnvelope
rw [Finset.mul_sum]
apply Finset.sum_congr rfl
intro j hj
rw [abs_div, abs_pow, abs_of_pos hB]
have hBne : B ≠ 0 := ne_of_gt hB
rw [div_pow]
field_simp
ring
rw [hrewrite]
have henv := shiftedCoefficientEnvelope_le_six_pow hK hratio0 hratio1
have hp2 : p ^ 2 / B ≤ B := by
apply (div_le_iff₀ hB).2
nlinarith
exact (mul_le_mul_of_nonneg_left henv
(div_nonneg (sq_nonneg p) hB0)).trans
(mul_le_mul_of_nonneg_right hp2 (by positivity))
theorem weighted_centeredCrossMoment_same_cell_sum_le
If the polynomial or elbow parameter satisfies its stated bound and the outcome bound is positive and the stated condition on the cell holds and the stated condition on the source size or matching order holds and the stated shift condition holds, the weighted sum of same-cell centered cross-moments satisfies the stated bound.
Formal statement
hypotheses
d m K :
ℕ
epsilon M sigma B :
ℝ
P :
ModelClass d epsilon M sigma
k :
Fin d
hK :
2 ≤ K
hB :
0 < B
hk :
P.law.cellMass k ≤ B / 4
hm :
4 * (K + 2) ^ 2 ≤ m
hshift :
(4 : ℝ) * (K + 2) / m ≤ 3 * B / 4
conclusion
∑ a : Bool, ∑ b : Bool, ∑ j ∈ range (K - 1), ∑ r ∈ range (K - 1), |shiftedCoefficient K j / B ^ (j + 1)| * |shiftedCoefficient K r / B ^ (r + 1)| * |centeredCrossMoment (Measure.infinitePi fun _ : ℕ => P.law.observedLaw) (fun ω : ℕ → Obs d => allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) k a j) (fun ω : ℕ → Obs d => allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) k b r)|
≤ 16 * (B * (6 : ℝ) ^ K) ^ 2
Proof (Lean source)
lemma weighted_centeredCrossMoment_same_cell_sum_le {d m K : ℕ} {epsilon M sigma B : ℝ}
(P : ModelClass d epsilon M sigma) (k : Fin d)
(hK : 2 ≤ K) (hB : 0 < B) (hk : P.law.cellMass k ≤ B / 4)
(hm : 4 * (K + 2) ^ 2 ≤ m)
(hshift : (4 : ℝ) * (K + 2) / m ≤ 3 * B / 4) :
∑ a : Bool, ∑ b : Bool,
∑ j ∈ range (K - 1), ∑ r ∈ range (K - 1),
|shiftedCoefficient K j / B ^ (j + 1)| *
|shiftedCoefficient K r / B ^ (r + 1)| *
|centeredCrossMoment
(Measure.infinitePi fun _ : ℕ => P.law.observedLaw)
(fun ω : ℕ → Obs d =>
allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) k a j)
(fun ω : ℕ → Obs d =>
allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) k b r)| ≤
16 * (B * (6 : ℝ) ^ K) ^ 2 := by
classical
let p : ℝ := P.law.cellMass k
let p' : ℝ := (5 / 4 : ℝ) * p
let c : ℕ → ℝ := fun j => |shiftedCoefficient K j / B ^ (j + 1)|
have hmpos : (0 : ℝ) < m := by
exact_mod_cast (lt_of_lt_of_le (by positivity : 0 < 4 * (K + 2) ^ 2) hm)
have hp0 : 0 ≤ p := P.law.cellMass_range k |>.1
have hp'0 : 0 ≤ p' := by dsimp [p']; positivity
have hp'B : p' ≤ B := by dsimp [p', p]; nlinarith
have hvm (r : ℕ) (hr : r ∈ range (K - 1)) :
p' + (r + 2 : ℕ) / m ≤ B := by
have hrK : r + 2 ≤ K := by have := Finset.mem_range.mp hr; omega
have hcast : ((r + 2 : ℕ) : ℝ) ≤ K + 2 := by exact_mod_cast (hrK.trans (by omega))
have hdiv : ((r + 2 : ℕ) : ℝ) / m ≤ (K + 2 : ℝ) / m := by
exact (div_le_div_iff_of_pos_right hmpos).2 hcast
have hs : 4 * ((K + 2 : ℝ) / m) ≤ 3 * B / 4 := by
convert hshift using 1 <;> ring
have hKm : (K + 2 : ℝ) / m ≤ 3 * B / 16 := by nlinarith
dsimp [p', p]
nlinarith
have hq : 2 * ((K + 2 : ℕ) : ℝ) ^ 2 / m ≤ 1 / 2 := by
apply (div_le_iff₀ hmpos).2
have hmR : (4 : ℝ) * ((K + 2 : ℕ) : ℝ) ^ 2 ≤ m := by exact_mod_cast hm
nlinarith
have hterm (a b : Bool) (j r : ℕ)
(hj : j ∈ range (K - 1)) (hr : r ∈ range (K - 1)) :
c j * c r *
|centeredCrossMoment
(Measure.infinitePi fun _ : ℕ => P.law.observedLaw)
(fun ω : ℕ → Obs d =>
allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) k a j)
(fun ω : ℕ → Obs d =>
allBlockOrderedMarkedFactorial M (fun i : Fin m => ω i) k b r)| ≤
c j * c r * ((1 / 2 : ℝ) * p ^ (j + 2) * p ^ (r + 2) +
exp 1 * p' ^ (r + 2) *
(p' + (r + 2 : ℕ) / m) ^ (j + 2)) := by
have hjK : j + 2 ≤ K := by have := Finset.mem_range.mp hj; omega
have hrK : r + 2 ≤ K := by have := Finset.mem_range.mp hr; omega
have hc := centeredCrossMoment_allBlockOrderedMarkedFactorial_same_cell_abs_le P k a b j r hjK hrK hm
dsimp [c, p, p']
refine mul_le_mul_of_nonneg_left (hc.trans ?_)
(mul_nonneg (abs_nonneg _) (abs_nonneg _))
exact add_le_add (by gcongr) le_rfl
calc
_ ≤ ∑ _a : Bool, ∑ _b : Bool,
∑ j ∈ range (K - 1), ∑ r ∈ range (K - 1),
c j * c r * ((1 / 2 : ℝ) * p ^ (j + 2) * p ^ (r + 2) +
exp 1 * p' ^ (r + 2) *
(p' + (r + 2 : ℕ) / m) ^ (j + 2)) := by
apply Finset.sum_le_sum; intro a ha
apply Finset.sum_le_sum; intro b hb
apply Finset.sum_le_sum; intro j hj
apply Finset.sum_le_sum; intro r hr
exact hterm a b j r hj hr
_ ≤ 16 * (B * (6 : ℝ) ^ K) ^ 2 := by
have hpSum : ∑ j ∈ range (K - 1), c j * p ^ (j + 2) ≤
B * (6 : ℝ) ^ K := by
exact shiftedCoefficient_weighted_sum_le_of_le (show 0 < K by omega) hB hp0 (by dsimp [p]; linarith)
have hp'Sum : ∑ r ∈ range (K - 1), c r * p' ^ (r + 2) ≤
B * (6 : ℝ) ^ K := shiftedCoefficient_weighted_sum_le_of_le (show 0 < K by omega) hB hp'0 hp'B
have hxSum (r : ℕ) (hr : r ∈ range (K - 1)) :
∑ j ∈ range (K - 1),
c j * (p' + (r + 2 : ℕ) / m) ^ (j + 2) ≤
B * (6 : ℝ) ^ K :=
shiftedCoefficient_weighted_sum_le_of_le (show 0 < K by omega) hB (by positivity) (hvm r hr)
have he : exp 1 ≤ 3 := by
exact Real.exp_one_lt_d9.le.trans (by norm_num)
have hR0 : 0 ≤ B * (6 : ℝ) ^ K := by positivity
have hA0 : 0 ≤ ∑ j ∈ range (K - 1), c j * p ^ (j + 2) :=
sum_nonneg fun j _ => mul_nonneg (abs_nonneg _) (pow_nonneg hp0 _)
have hfirst :
(1 / 2 : ℝ) *
(∑ j ∈ range (K - 1), c j * p ^ (j + 2)) *
(∑ r ∈ range (K - 1), c r * p ^ (r + 2)) ≤
(1 / 2 : ℝ) * (B * 6 ^ K) ^ 2 := by
calc
_ ≤ (1 / 2 : ℝ) * (B * 6 ^ K) *
(∑ r ∈ range (K - 1), c r * p ^ (r + 2)) := by
exact mul_le_mul_of_nonneg_right
(mul_le_mul_of_nonneg_left hpSum (by norm_num)) hA0
_ ≤ (1 / 2 : ℝ) * (B * 6 ^ K) * (B * 6 ^ K) := by
exact mul_le_mul_of_nonneg_left hpSum
(mul_nonneg (by norm_num) hR0)
_ = _ := by ring
have hsecond :
∑ r ∈ range (K - 1),
c r * (exp 1 * p' ^ (r + 2) *
(∑ j ∈ range (K - 1),
c j * (p' + (r + 2 : ℕ) / m) ^ (j + 2))) ≤
3 * (B * 6 ^ K) ^ 2 := by
calc
_ ≤ ∑ r ∈ range (K - 1),
c r * (3 * p' ^ (r + 2) * (B * 6 ^ K)) := by
apply Finset.sum_le_sum
intro r hr
have hc0 : 0 ≤ c r := abs_nonneg _
have hpw0 : 0 ≤ p' ^ (r + 2) := pow_nonneg hp'0 _
have hx0 : 0 ≤ ∑ j ∈ range (K - 1),
c j * (p' + (r + 2 : ℕ) / m) ^ (j + 2) :=
sum_nonneg fun j _ => mul_nonneg (abs_nonneg _) (by positivity)
exact mul_le_mul_of_nonneg_left
(mul_le_mul
(mul_le_mul_of_nonneg_right he hpw0)
(hxSum r hr) hx0 (mul_nonneg (by positivity) hpw0)) hc0
_ = 3 * (B * 6 ^ K) *
(∑ r ∈ range (K - 1), c r * p' ^ (r + 2)) := by
rw [Finset.mul_sum]
apply Finset.sum_congr rfl
intro r hr
ring
_ ≤ 3 * (B * 6 ^ K) ^ 2 := by
rw [pow_two]
calc
_ ≤ (3 * (B * 6 ^ K)) * (B * 6 ^ K) :=
mul_le_mul_of_nonneg_left hp'Sum
(show (0 : ℝ) ≤ 3 * (B * 6 ^ K) by positivity)
_ = _ := by ring
have hdegree :
(∑ j ∈ range (K - 1), ∑ r ∈ range (K - 1),
c j * c r * ((1 / 2 : ℝ) * p ^ (j + 2) * p ^ (r + 2) +
exp 1 * p' ^ (r + 2) *
(p' + (r + 2 : ℕ) / m) ^ (j + 2))) =
(1 / 2 : ℝ) *
(∑ j ∈ range (K - 1), c j * p ^ (j + 2)) *
(∑ r ∈ range (K - 1), c r * p ^ (r + 2)) +
∑ r ∈ range (K - 1),
c r * (exp 1 * p' ^ (r + 2) *
(∑ j ∈ range (K - 1),
c j * (p' + (r + 2 : ℕ) / m) ^ (j + 2))) := by
simp_rw [mul_add, Finset.sum_add_distrib]
congr 1
· calc
(∑ j ∈ range (K - 1), ∑ r ∈ range (K - 1),
c j * c r * ((1 / 2 : ℝ) * p ^ (j + 2) * p ^ (r + 2))) =
∑ j ∈ range (K - 1),
((1 / 2 : ℝ) * c j * p ^ (j + 2)) *
(∑ r ∈ range (K - 1), c r * p ^ (r + 2)) := by
apply Finset.sum_congr rfl
intro j hj
rw [Finset.mul_sum]
apply Finset.sum_congr rfl
intro r hr
ring
_ = _ := by
rw [← Finset.sum_mul]
congr 1
rw [Finset.mul_sum]
apply Finset.sum_congr rfl
intro j hj
ring
· rw [Finset.sum_comm]
apply Finset.sum_congr rfl
intro r hr
rw [Finset.mul_sum]
rw [Finset.mul_sum]
apply Finset.sum_congr rfl
intro j hj
ring
have hrewrite :
(∑ _a : Bool, ∑ _b : Bool,
∑ j ∈ range (K - 1), ∑ r ∈ range (K - 1),
c j * c r * ((1 / 2 : ℝ) * p ^ (j + 2) * p ^ (r + 2) +
exp 1 * p' ^ (r + 2) *
(p' + (r + 2 : ℕ) / m) ^ (j + 2))) =
4 * ((1 / 2 : ℝ) *
(∑ j ∈ range (K - 1), c j * p ^ (j + 2)) *
(∑ r ∈ range (K - 1), c r * p ^ (r + 2)) +
∑ r ∈ range (K - 1),
c r * (exp 1 * p' ^ (r + 2) *
(∑ j ∈ range (K - 1),
c j * (p' + (r + 2 : ℕ) / m) ^ (j + 2)))) := by
simp only [hdegree, Finset.sum_const]
rw [show Finset.univ.card = 2 by decide]
norm_num
ring
rw [hrewrite]
nlinarith
Helpers.FactorialCovarianceSummation 2 declarations
theorem matchingNormalization_sum_le_asymmetric Lemma matchingNormalization_sum_le_asymmetric in the paper ↗
If the first order satisfies its stated bound and the second order satisfies its stated bound and the stated condition on the source size or matching order holds, the size-h normalization sum retains one binomial coefficient instead of replacing both by powers. This is the form that sums to a shifted intensity.
Formal statement
hypotheses
m u v R h :
ℕ
hu :
u ≤ R
hv :
v ≤ R
hm :
4 * R ^ 2 ≤ m
conclusion
∑ N ∈ partialMatchingsOfSize u v h, matchingNormalization m N
≤ (Nat.choose u h : ℝ) * (v : ℝ) ^ h * (exp 1 / (m : ℝ) ^ h)
Proof (Lean source)
lemma matchingNormalization_sum_le_asymmetric {m u v R h : ℕ}
(hu : u ≤ R) (hv : v ≤ R) (hm : 4 * R ^ 2 ≤ m) :
∑ N ∈ partialMatchingsOfSize u v h,
matchingNormalization m N ≤
(Nat.choose u h : ℝ) * (v : ℝ) ^ h *
(exp 1 / (m : ℝ) ^ h) := by
classical
have hnorm : ∀ N ∈ partialMatchingsOfSize u v h,
matchingNormalization m N ≤
exp 1 / (m : ℝ) ^ h := by
intro N hN
exact Causalean.Stat.matchingNormalization_le hu hv hm hN
have hcard :
((partialMatchingsOfSize u v h).card : ℝ) ≤
(Nat.choose u h : ℝ) * (v : ℝ) ^ h := by
rw [card_partialMatchingsOfSize]
push_cast
have hhfac : (0 : ℝ) < h.factorial := by positivity
have hvchoose : (Nat.choose v h : ℝ) ≤
(v : ℝ) ^ h / h.factorial := Nat.choose_le_pow_div h v
have hnonneg : (0 : ℝ) ≤ Nat.choose u h := by positivity
calc
(Nat.choose u h : ℝ) * Nat.choose v h * h.factorial ≤
(Nat.choose u h : ℝ) *
((v : ℝ) ^ h / h.factorial) * h.factorial := by
gcongr
_ = (Nat.choose u h : ℝ) * (v : ℝ) ^ h := by
field_simp
calc
∑ N ∈ partialMatchingsOfSize u v h,
matchingNormalization m N ≤
∑ _N ∈ partialMatchingsOfSize u v h,
(exp 1 / (m : ℝ) ^ h) := by
exact Finset.sum_le_sum fun N hN => hnorm N hN
_ = ((partialMatchingsOfSize u v h).card : ℝ) *
(exp 1 / (m : ℝ) ^ h) := by simp [mul_comm]
_ ≤ ((Nat.choose u h : ℝ) * (v : ℝ) ^ h) *
(exp 1 / (m : ℝ) ^ h) := by
gcongr
_ = (Nat.choose u h : ℝ) * (v : ℝ) ^ h *
(exp 1 / (m : ℝ) ^ h) := rfl
theorem positiveMatchingNormalization_weighted_sum_le Lemma positiveMatchingNormalization_weighted_sum_le in the paper ↗
If the first order satisfies its stated bound and the second order satisfies its stated bound and the stated condition on the source size or matching order holds and the probability lies in the stated range, after weighting a size-h overlap by the remaining cell-mass power, all positive overlap sizes are bounded by the binomially shifted intensity p + v / m.
Formal statement
hypotheses
m u v R :
ℕ
hu :
u ≤ R
hv :
v ≤ R
hm :
4 * R ^ 2 ≤ m
p :
ℝ
hp :
0 ≤ p
conclusion
∑ h ∈ (range (min u v + 1)).filter (fun h => 0 < h), (∑ N ∈ partialMatchingsOfSize u v h, matchingNormalization m N) * p ^ (u + v - h)
≤ exp 1 * p ^ v * (p + (v : ℝ) / m) ^ u
Proof (Lean source)
lemma positiveMatchingNormalization_weighted_sum_le
{m u v R : ℕ} (hu : u ≤ R) (hv : v ≤ R)
(hm : 4 * R ^ 2 ≤ m) {p : ℝ} (hp : 0 ≤ p) :
∑ h ∈ (range (min u v + 1)).filter (fun h => 0 < h),
(∑ N ∈ partialMatchingsOfSize u v h,
matchingNormalization m N) * p ^ (u + v - h) ≤
exp 1 * p ^ v * (p + (v : ℝ) / m) ^ u := by
classical
by_cases hR : R = 0
· subst R
have hu0 : u = 0 := Nat.eq_zero_of_le_zero hu
have hv0 : v = 0 := Nat.eq_zero_of_le_zero hv
subst u
subst v
have hempty : (range (min 0 0 + 1)).filter (fun h => 0 < h) = ∅ := by
ext h
simp
rw [hempty]
simp
positivity
have hmpos : (0 : ℝ) < m := by
have hRpos : 0 < R := Nat.pos_of_ne_zero hR
exact_mod_cast (lt_of_lt_of_le (by positivity : 0 < 4 * R ^ 2) hm)
calc
∑ h ∈ (range (min u v + 1)).filter (fun h => 0 < h),
(∑ N ∈ partialMatchingsOfSize u v h,
matchingNormalization m N) * p ^ (u + v - h) ≤
∑ h ∈ (range (min u v + 1)).filter (fun h => 0 < h),
((Nat.choose u h : ℝ) * (v : ℝ) ^ h *
(exp 1 / (m : ℝ) ^ h)) * p ^ (u + v - h) := by
apply Finset.sum_le_sum
intro h hh
exact mul_le_mul_of_nonneg_right
(matchingNormalization_sum_le_asymmetric hu hv hm)
(pow_nonneg hp _)
_ = ∑ h ∈ (range (min u v + 1)).filter (fun h => 0 < h),
exp 1 * p ^ v *
((Nat.choose u h : ℝ) * p ^ (u - h) *
((v : ℝ) / m) ^ h) := by
apply Finset.sum_congr rfl
intro h hh
simp only [mem_filter, Finset.mem_range] at hh
have hhmin : h ≤ min u v := by omega
have hhv : h ≤ v := hhmin.trans (min_le_right _ _)
rw [show u + v - h = (u - h) + v by omega, pow_add]
rw [div_pow]
field_simp
_ ≤ ∑ h ∈ range (u + 1),
exp 1 * p ^ v *
((Nat.choose u h : ℝ) * p ^ (u - h) *
((v : ℝ) / m) ^ h) := by
apply Finset.sum_le_sum_of_subset_of_nonneg
· intro h hh
simp only [mem_filter, Finset.mem_range] at hh
have hhmin : h ≤ min u v := by omega
exact Finset.mem_range.mpr
(Nat.lt_succ_of_le (hhmin.trans (min_le_left _ _)))
· intro h hh _
positivity
_ = exp 1 * p ^ v * (p + (v : ℝ) / m) ^ u := by
rw [← Finset.mul_sum]
congr 1
rw [add_comm p]
simpa [mul_assoc, mul_left_comm, mul_comm] using
(add_pow ((v : ℝ) / m) p u).symm
Helpers.LowerTransfer 25 declarations
def binaryFullChannel
Apply the same Bernoulli contraction to both potential responses, then scale the contracted bits. The formula is a law-level Markov pushforward and is hypothesis-independent: only the common success function is used.
Definition (Lean source)
noncomputable def binaryFullChannel {d e : ℕ}
(pad : Fin d → Fin e) (success scaled : Bool → ℝ)
(R : Measure (BinaryFullObs d)) : Measure (FullObs e) :=
∑ z : BinaryFullObs d, ∑ c0 : Bool, ∑ c1 : Bool,
(R {z} *
ofReal (if c0 then success z.b0 else 1 - success z.b0) *
ofReal (if c1 then success z.b1 else 1 - success z.b1)) •
Measure.dirac
(⟨pad z.x, z.a, scaled c0, scaled c1,
if z.a then scaled c1 else scaled c0⟩ : FullObs e)
Concrete data carried by the two padded, embedded least-favorable families.
Definition (Lean source)
parameters
n d :
ℕ
epsilon M sigma :
ℝ
fields
lambda :
ℝ
radialCap :
ℕ
exactCap :
ℕ
radialSource :
Set (CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n radialCap epsilon)
exactSource :
Set (BinaryExactLaw n exactCap epsilon)
radialIndex_injective :
Injective radialIndex
exactIndex_injective :
Injective exactIndex
radialEmbedding :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n radialCap epsilon → RealLaw d
exactEmbedding :
BinaryExactLaw n exactCap epsilon → RealLaw d
radialCoupling :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n radialCap epsilon → Measure (BinaryFullObs radialCap)
exactCoupling :
radial_mass_preserved :
∀ P ∈ radialSource,
∀ k,
(radialEmbedding P).cellMass (radialIndex k)
= CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P.1 k
exact_mass_preserved :
∀ P ∈ exactSource,
∀ k,
(exactEmbedding P).cellMass (exactIndex k)
= CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P.1 k
radial_zero_extension :
∀ P ∈ radialSource,
∀ k : Fin d, (∀ r, radialIndex r ≠ k) → (radialEmbedding P).cellMass k = 0
exact_zero_extension :
∀ P ∈ exactSource,
∀ k : Fin d, (∀ r, exactIndex r ≠ k) → (exactEmbedding P).cellMass k = 0
channelSuccess :
Bool → ℝ
scaledOutcome :
Bool → ℝ
radial_channel_law :
∀ P ∈ radialSource,
(radialEmbedding P).fullLaw
= binaryFullChannel radialIndex channelSuccess scaledOutcome (radialCoupling P)
exact_affine_law :
∀ P ∈ exactSource,
(exactEmbedding P).fullLaw
= Measure.map (BinaryFullObs.affine M exactIndex) (exactCoupling P)
The selected control-zero source family retains a genuine fixed-sample minimax lower bound; in particular, it cannot be an arbitrary nonempty set.
Definition (Lean source)
def RadialSourceHard {n d : ℕ} {epsilon M sigma : ℝ}
(H : LeastFavorableHandle n d epsilon M sigma) : Prop :=
∃ c : ℝ, 0 < c ∧
∀ est : {f : (Fin n →
CausalSmith.Stat.DiscreteAteMinimaxLoggap.Obs H.radialCap) → ℝ //
Measurable f},
∃ P ∈ H.radialSource,
c * CausalSmith.Stat.DiscreteAteMinimaxLoggap.minimaxRate n H.radialCap ≤
CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n)
est.1
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1)
The selected uniform exactly-homogeneous source family retains the cited fixed-sample lower bound.
Definition (Lean source)
def ExactSourceHard {n d : ℕ} {epsilon M sigma : ℝ}
(H : LeastFavorableHandle n d epsilon M sigma) : Prop :=
∃ c : ℝ, 0 < c ∧
∀ est : {f : (Fin n →
CausalSmith.Stat.DiscreteAteMinimaxLoggap.Obs H.exactCap) → ℝ //
Measurable f},
∃ P ∈ H.exactSource,
c * (1 / (n : ℝ) + (H.exactCap : ℝ) / (n : ℝ) ^ 2) ≤
CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n)
est.1
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1)
The chosen full-data couplings have the prescribed radial source margins.
Definition (Lean source)
def RadialCouplingCertificate {n d : ℕ} {epsilon M sigma : ℝ}
(H : LeastFavorableHandle n d epsilon M sigma) : Prop :=
∀ P ∈ H.radialSource, BinaryFullCoupling P.1 (H.radialCoupling P)
The chosen full-data couplings have the prescribed exact source margins.
Definition (Lean source)
def ExactCouplingCertificate {n d : ℕ} {epsilon M sigma : ℝ}
(H : LeastFavorableHandle n d epsilon M sigma) : Prop :=
∀ P ∈ H.exactSource, BinaryFullCoupling P.1 (H.exactCoupling P)
def leastFavorableAt
Pointwise realization of the two padded source families for fixed model parameters.
Definition (Lean source)
def leastFavorableAt (n d : ℕ) (epsilon M sigma bRad bExact : ℝ)
(H : LeastFavorableHandle n d epsilon M sigma) : Prop :=
0 < bRad ∧ 0 < bExact ∧
H.lambda = sigma / 2 ∧
H.radialCap = min d (max 1 (floor (bRad * n * logEN n))) ∧
H.exactCap = min d (max 1 (floor (bExact * (n : ℝ) ^ 2))) ∧
RadialSourceHard H ∧ ExactSourceHard H ∧
RadialCouplingCertificate H ∧ ExactCouplingCertificate H ∧
(∀ b, H.channelSuccess b =
1 / 2 + H.lambda * ((if b then 1 else 0) - 1 / 2)) ∧
(∀ b, H.scaledOutcome b = M * ((if b then 1 else 0) - 1 / 2))
A global least-favorable family. The two cap constants are selected from epsilon before every sample size, alphabet size, scale, and radius; hence they cannot vary with model parameters.
Definition (Lean source)
def leastFavorableFamily (epsilon : ℝ) : Prop :=
∃ bRad bExact : ℝ,
0 < bRad ∧ 0 < bExact ∧
∀ n d : ℕ, ∀ M sigma : ℝ,
0 < n → 0 < d → OutcomeScale M → -- @realizes M(range [1,infinity))
0 ≤ sigma → sigma ≤ 2 →
∃ H : LeastFavorableHandle n d epsilon M sigma,
leastFavorableAt n d epsilon M sigma bRad bExact H
Every radial embedding belongs to the radius-indexed ambient class. This is deliberately downstream of the construction handle.
Definition (Lean source)
def RadialEmbeddingMembership {n d : ℕ} {epsilon M sigma : ℝ}
(H : LeastFavorableHandle n d epsilon M sigma) : Prop :=
∀ P ∈ H.radialSource,
∃ Q : ModelClass d epsilon M sigma, Q.law = H.radialEmbedding P
Every exact-homogeneity embedding belongs to the radius-zero ambient class. This is deliberately downstream of the construction handle.
Definition (Lean source)
def ExactEmbeddingMembership {n d : ℕ} {epsilon M sigma : ℝ}
(H : LeastFavorableHandle n d epsilon M sigma) : Prop :=
∀ P ∈ H.exactSource,
∃ Q : ModelClass d epsilon M 0, Q.law = H.exactEmbedding P
Every embedded realization obeys the stated heterogeneity radius.
Definition (Lean source)
def RadiusCertificate {n d : ℕ} {epsilon M sigma : ℝ}
(H : LeastFavorableHandle n d epsilon M sigma) : Prop :=
∀ P ∈ H.radialSource, ∀ k : Fin d,
0 < (H.radialEmbedding P).cellMass k →
|cellDeviation (H.radialEmbedding P) k| ≤ sigma * M
The channel and affine embedding scale the source targets by their declared factors.
Definition (Lean source)
def TargetSeparationCertificate {n d : ℕ} {epsilon M sigma : ℝ}
(H : LeastFavorableHandle n d epsilon M sigma) : Prop :=
(∀ P0 ∈ H.radialSource, ∀ P1 ∈ H.radialSource,
rawAteFormula (H.radialEmbedding P1) -
rawAteFormula (H.radialEmbedding P0) =
M * H.lambda *
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P1.1 -
CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P0.1)) ∧
(∀ P0 ∈ H.exactSource, ∀ P1 ∈ H.exactSource,
rawAteFormula (H.exactEmbedding P1) -
rawAteFormula (H.exactEmbedding P0) =
M *
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P1.1 -
CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P0.1))
Product-sample total variation cannot increase under the radial channel.
Definition (Lean source)
def DataProcessingCertificate {n d : ℕ} {epsilon M sigma : ℝ}
(H : LeastFavorableHandle n d epsilon M sigma) : Prop :=
∀ P0 ∈ H.radialSource, ∀ P1 ∈ H.radialSource,
tvDist
(productLaw n (H.radialEmbedding P0))
(productLaw n (H.radialEmbedding P1)) ≤
tvDist
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P0.1 n)
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P1.1 n)
A source set, independently of the two-family construction handle, retains the fixed-sample one-arm lower bound.
Definition (Lean source)
def RadialSourceHardData {n cap : ℕ} {epsilon : ℝ}
(source : Set
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n cap epsilon)) : Prop :=
∃ c : ℝ, 0 < c ∧
∀ est : {f : (Fin n →
CausalSmith.Stat.DiscreteAteMinimaxLoggap.Obs cap) → ℝ //
Measurable f},
∃ P ∈ source,
c * CausalSmith.Stat.DiscreteAteMinimaxLoggap.minimaxRate n cap ≤
CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n)
est.1
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1)
The common data asserted for one realization of the radial Bernoulli channel. Factoring it out keeps the construction and its lower-transfer certificate tied to the same source, padding, embedding, and coupling.
Definition (Lean source)
def RadialChannelData (n d : ℕ) (epsilon M sigma bRad : ℝ)
(cap : ℕ)
(source : Set
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n cap epsilon))
(pad : Fin cap → Fin d)
(embedding :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n cap epsilon →
RealLaw d)
(coupling :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n cap epsilon →
Measure (BinaryFullObs cap)) : Prop :=
cap = min d (max 1 (floor (bRad * n * logEN n))) ∧
Injective pad ∧
RadialSourceHardData source ∧
(∀ P ∈ source, ∀ k,
(embedding P).cellMass (pad k) =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P.1 k) ∧
(∀ P ∈ source, ∀ k : Fin d,
(∀ r, pad r ≠ k) → (embedding P).cellMass k = 0) ∧
(∀ P ∈ source, BinaryFullCoupling P.1 (coupling P)) ∧
(∀ P ∈ source,
(embedding P).fullLaw =
binaryFullChannel pad
(fun b => 1 / 2 + sigma / 2 * ((if b then 1 else 0) - 1 / 2))
(fun b => M * ((if b then 1 else 0) - 1 / 2))
(coupling P)) ∧
(∀ P ∈ source,
∃ Q : ModelClass d epsilon M sigma, Q.law = embedding P) ∧
(∀ P ∈ source, ∀ k : Fin d,
0 < (embedding P).cellMass k →
|cellDeviation (embedding P) k| ≤ sigma * M) ∧
(∀ P0 ∈ source, ∀ P1 ∈ source,
tvDist
(productLaw n (embedding P0))
(productLaw n (embedding P1)) ≤
tvDist
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P0.1 n)
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P1.1 n))
The radial family and its Bernoulli channel, including capping, ambient membership, realization-wise radius, and product-sample data processing.
Definition (Lean source)
def RadialChannelConstruction (n d : ℕ) (epsilon M sigma bRad : ℝ) : Prop :=
∃ cap : ℕ,
∃ source : Set
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n cap epsilon),
∃ pad : Fin cap → Fin d,
∃ embedding :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n cap epsilon →
RealLaw d,
∃ coupling :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n cap epsilon →
Measure (BinaryFullObs cap),
RadialChannelData n d epsilon M sigma bRad cap source pad embedding coupling
def RadialTargetRiskTransferCertificate Definition aux_RadialTargetRiskTransferCertificate in the paper ↗
The same realized radial channel carries both its exact target scaling and an estimator-wise lower-risk witness on the channel image.
Definition (Lean source)
def RadialTargetRiskTransferCertificate (n d : ℕ)
(epsilon M sigma c bRad : ℝ) : Prop :=
∃ cap : ℕ,
∃ source : Set
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n cap epsilon),
∃ pad : Fin cap → Fin d,
∃ embedding :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n cap epsilon →
RealLaw d,
∃ coupling :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n cap epsilon →
Measure (BinaryFullObs cap),
RadialChannelData n d epsilon M sigma bRad cap source pad embedding coupling ∧
(∀ P0 ∈ source, ∀ P1 ∈ source,
rawAteFormula (embedding P1) - rawAteFormula (embedding P0) =
M * (sigma / 2) *
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P1.1 -
CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P0.1)) ∧
∀ est : Estimator n d M,
∃ P ∈ source,
c * M ^ 2 * sigma ^ 2 * min 1 (polynomialComponent n d) ≤
mse (embedding P) est.1
theorem radialChannelConstruction_of_handle Lemma radialChannelConstruction_of_handle in the paper ↗
If the specified least-favorable family is available and the transported family belongs to the target model class and the radial source lower bound holds and the channel satisfies data processing, project the radial construction asserted by the two-family internal handle to the radial-only public construction statement.
Formal statement
hypotheses
n d :
ℕ
epsilon M sigma bRad bExact :
ℝ
H :
LeastFavorableHandle n d epsilon M sigma
hfamily :
leastFavorableAt n d epsilon M sigma bRad bExact H
hmembership :
hradius :
hprocessing :
conclusion
RadialChannelConstruction n d epsilon M sigma bRad
Proof (Lean source)
lemma radialChannelConstruction_of_handle {n d : ℕ} {epsilon M sigma bRad bExact : ℝ}
(H : LeastFavorableHandle n d epsilon M sigma)
(hfamily : leastFavorableAt n d epsilon M sigma bRad bExact H)
(hmembership : RadialEmbeddingMembership H)
(hradius : RadiusCertificate H)
(hprocessing : DataProcessingCertificate H) :
RadialChannelConstruction n d epsilon M sigma bRad := by
rcases hfamily with
⟨_hbRad, _hbExact, hlambda, hradialCap, _hexactCap,
hradialHard, _hexactHard, hcoupling, _hexactCoupling,
hsuccess, hscaled⟩
refine ⟨H.radialCap, H.radialSource, H.radialIndex, H.radialEmbedding,
H.radialCoupling, hradialCap, H.radialIndex_injective, ?_,
H.radial_mass_preserved, H.radial_zero_extension, hcoupling, ?_,
hmembership, hradius, hprocessing⟩
· exact hradialHard
· intro P hP
have hsuccess' : H.channelSuccess =
fun b => 1 / 2 + sigma / 2 * ((if b then 1 else 0) - 1 / 2) := by
funext b
rw [hsuccess b, hlambda]
have hscaled' : H.scaledOutcome =
fun b => M * ((if b then 1 else 0) - 1 / 2) := by
funext b
exact hscaled b
rw [H.radial_channel_law P hP, hsuccess', hscaled']
After the finite-sample cutoffs, both capped alphabets lie in the respective source theorem ranges.
Definition (Lean source)
def SourceCutoffCertificate {n d : ℕ} {epsilon M sigma : ℝ}
(bRad bExact : ℝ) (NRad NExact : ℕ)
(H : LeastFavorableHandle n d epsilon M sigma) : Prop :=
0 < H.radialCap ∧ H.radialCap ≤ d ∧
0 < H.exactCap ∧ H.exactCap ≤ d ∧
(NRad ≤ n →
(H.radialCap : ℝ) ≤ bRad * (n : ℝ) * log n) ∧
(NExact ≤ n →
(H.exactCap : ℝ) ≤ bExact * (n : ℝ) ^ 2)
Each admissible ambient estimator is defeated by a member of each selected source family at the corresponding transferred lower-bound scale.
Definition (Lean source)
def RiskTransferCertificate {n d : ℕ} {epsilon M sigma : ℝ}
(c : ℝ) (H : LeastFavorableHandle n d epsilon M sigma) : Prop :=
∀ est : Estimator n d M,
(∃ P ∈ H.radialSource,
c * M ^ 2 * sigma ^ 2 * min 1 (polynomialComponent n d) ≤
mse (H.radialEmbedding P) est.1) ∧
(∃ P ∈ H.exactSource,
c * M ^ 2 *
(1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)) ≤
mse (H.exactEmbedding P) est.1)
theorem radialTargetRiskTransferCertificate_of_handle
If the specified least-favorable family is available and the transported family belongs to the target model class and the radial source lower bound holds and the channel satisfies data processing and the target separation certificate holds and the risk-transfer certificate holds, package one realized least-favorable handle as a radial certificate whose target scaling and estimator-wise risk transfer use those same witnesses.
Formal statement
hypotheses
n d :
ℕ
epsilon M sigma c bRad bExact :
ℝ
H :
LeastFavorableHandle n d epsilon M sigma
hfamily :
leastFavorableAt n d epsilon M sigma bRad bExact H
hmembership :
hradius :
hprocessing :
htarget :
htransfer :
conclusion
RadialTargetRiskTransferCertificate n d epsilon M sigma c bRad
Proof (Lean source)
lemma radialTargetRiskTransferCertificate_of_handle {n d : ℕ}
{epsilon M sigma c bRad bExact : ℝ}
(H : LeastFavorableHandle n d epsilon M sigma)
(hfamily : leastFavorableAt n d epsilon M sigma bRad bExact H)
(hmembership : RadialEmbeddingMembership H)
(hradius : RadiusCertificate H)
(hprocessing : DataProcessingCertificate H)
(htarget : TargetSeparationCertificate H)
(htransfer : RiskTransferCertificate c H) :
RadialTargetRiskTransferCertificate n d epsilon M sigma c bRad := by
rcases hfamily with
⟨_hbRad, _hbExact, hlambda, hradialCap, _hexactCap,
hradialHard, _hexactHard, hcoupling, _hexactCoupling,
hsuccess, hscaled⟩
refine ⟨H.radialCap, H.radialSource, H.radialIndex, H.radialEmbedding,
H.radialCoupling, ?_, ?_, ?_⟩
· refine ⟨hradialCap, H.radialIndex_injective, hradialHard,
H.radial_mass_preserved, H.radial_zero_extension, hcoupling, ?_,
hmembership, hradius, hprocessing⟩
intro P hP
have hsuccess' : H.channelSuccess =
fun b => 1 / 2 + sigma / 2 * ((if b then 1 else 0) - 1 / 2) := by
funext b
rw [hsuccess b, hlambda]
have hscaled' : H.scaledOutcome =
fun b => M * ((if b then 1 else 0) - 1 / 2) := by
funext b
exact hscaled b
rw [H.radial_channel_law P hP, hsuccess', hscaled']
· intro P0 hP0 P1 hP1
simpa [hlambda] using htarget.1 P0 hP0 P1 hP1
· intro est
exact (htransfer est).1
theorem radiusCertificate_of_radialEmbeddingMembership
If the transported family belongs to the target model class, ambient membership already contains the realization-wise homogeneity bound, so it immediately supplies the handle's radius certificate.
Formal statement
hypotheses
conclusion
Proof (Lean source)
lemma radiusCertificate_of_radialEmbeddingMembership {n d : ℕ}
{epsilon M sigma : ℝ}
(H : LeastFavorableHandle n d epsilon M sigma)
(hmembership : RadialEmbeddingMembership H) :
RadiusCertificate H := by
intro P hP k hk
obtain ⟨Q, hQ⟩ := hmembership P hP
have hhom := Q.homogeneity
rw [hQ] at hhom
exact hhom k hk
A one-cell two-point experiment gives the uniform parametric minimax term.
Formal statement
conclusion
∀ epsilon : ℝ,
0 < epsilon → epsilon < 1 / 2 → ∃ c_epsilon : ℝ, 0 < c_epsilon ∧ ∀ n d : ℕ, ∀ M sigma : ℝ, 0 < n → 0 < d → 1
≤ M → 0 ≤ sigma → sigma ≤ 2 → c_epsilon * M ^ 2 / n ≤ minimaxRisk n d epsilon M sigma
Proof (Lean source)
lemma parametric_lower :
∀ epsilon : ℝ, 0 < epsilon → epsilon < 1 / 2 →
∃ c_epsilon : ℝ, 0 < c_epsilon ∧
∀ n d : ℕ, ∀ M sigma : ℝ,
0 < n → 0 < d → 1 ≤ M → 0 ≤ sigma → sigma ≤ 2 →
c_epsilon * M ^ 2 / n ≤ minimaxRisk n d epsilon M sigma := by
exact parametric_lower_core
The cited exact-homogeneity binary converse transfers through the affine outcome embedding and capped-alphabet restriction.
Formal statement
conclusion
∀ epsilon : ℝ,
0 < epsilon → epsilon < 1 / 2 → ∃ c_epsilon : ℝ, 0 < c_epsilon ∧ ∀ n d : ℕ, ∀ M sigma : ℝ, 0 < n → 0 < d → 1
≤ M → 0 ≤ sigma → sigma ≤ 2 → c_epsilon * M ^ 2 * (1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)) ≤ minimaxRisk n d epsilon M sigma
Proof (Lean source)
lemma scaled_binary_exact_lower_transfer_all_d :
∀ epsilon : ℝ, 0 < epsilon → epsilon < 1 / 2 →
∃ c_epsilon : ℝ, 0 < c_epsilon ∧
∀ n d : ℕ, ∀ M sigma : ℝ,
0 < n → 0 < d → 1 ≤ M → 0 ≤ sigma → sigma ≤ 2 →
c_epsilon * M ^ 2 *
(1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)) ≤
minimaxRisk n d epsilon M sigma := by
intro epsilon hepsilon hepsilon_half
obtain ⟨a, b, N, ha, hb, hexact⟩ :=
zengBinaryExactHomogeneityLower epsilon ⟨hepsilon, hepsilon_half⟩
obtain ⟨cp, hcp, hparam⟩ := parametric_lower epsilon hepsilon hepsilon_half
let b0 : ℝ := min b (1 / 2)
have hb0 : 0 < b0 := lt_min hb (by norm_num)
have hb0b : b0 ≤ b := min_le_left _ _
have hb0half : b0 ≤ 1 / 2 := min_le_right _ _
let K : ℝ := 1 + (N : ℝ) + 1 / b0
have hK : 0 < K := by dsimp [K]; positivity
let c : ℝ := min (a * b0 / 4) (cp / K)
have hc : 0 < c := lt_min (by positivity) (by positivity)
refine ⟨c, hc, ?_⟩
intro n d M sigma hn hd hM hsigma hsigma_two
have hnR : 0 < (n : ℝ) := by exact_mod_cast hn
have hnOne : (1 : ℝ) ≤ n := by exact_mod_cast (Nat.one_le_iff_ne_zero.mpr hn.ne')
let x : ℝ := b0 * (n : ℝ) ^ 2
by_cases hlarge : N ≤ n ∧ 1 ≤ x
· let m : ℕ := min d (Nat.floor x)
have hfloor : 0 < Nat.floor x := Nat.floor_pos.mpr hlarge.2
have hm : 0 < m := (Nat.lt_min).2 ⟨hd, hfloor⟩
have hmd : m ≤ d := min_le_left _ _
have hmx : (m : ℝ) ≤ x := calc
(m : ℝ) ≤ Nat.floor x := by exact_mod_cast min_le_right d (Nat.floor x)
_ ≤ x := Nat.floor_le (le_trans (by norm_num) hlarge.2)
have hmb : (m : ℝ) ≤ b * (n : ℝ) ^ 2 := by
dsimp [x] at hmx
exact hmx.trans (mul_le_mul_of_nonneg_right hb0b (sq_nonneg (n : ℝ)))
have hbinary := hexact n m hm hlarge.1 hmb
let sourceRate : ℝ := 1 / (n : ℝ) + (m : ℝ) / (n : ℝ) ^ 2
let L : ℝ := a / 2 * sourceRate
have hsourceRate : 0 < sourceRate := by dsimp [sourceRate]; positivity
have hLlt : L < binaryExactMinimaxRisk n m epsilon := by
have hhalf : L < a * sourceRate := by dsimp [L]; nlinarith [ha, hsourceRate]
exact hhalf.trans_le (by simpa [sourceRate] using hbinary)
have hhard := binaryExactMinimaxRisk_hard_family_of_lt
(n := n) (d := m) hn hm hepsilon hepsilon_half hLlt
have htransfer := minimaxRisk_ge_of_padded_exact_hard_family
hmd hm hepsilon hepsilon_half hM hsigma hsigma_two hhard
have hfloorHalf : x / 2 ≤ (Nat.floor x : ℝ) :=
half_le_natFloor_of_one_le hlarge.2
have hb0le : b0 / 2 ≤ 1 := by linarith
have htarget : b0 / 2 *
(1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)) ≤ sourceRate := by
by_cases hdcap : d ≤ Nat.floor x
· have hmEq : m = d := min_eq_left hdcap
have hdRatio : (d : ℝ) / (n : ℝ) ^ 2 ≤ 1 := by
rw [div_le_one (sq_pos_of_pos hnR)]
calc
(d : ℝ) ≤ Nat.floor x := by exact_mod_cast hdcap
_ ≤ x := Nat.floor_le (le_trans (by norm_num) hlarge.2)
_ = b0 * (n : ℝ) ^ 2 := rfl
_ ≤ 1 * (n : ℝ) ^ 2 := by gcongr; linarith
_ = (n : ℝ) ^ 2 := one_mul _
rw [min_eq_right hdRatio]
dsimp [sourceRate]
rw [hmEq]
have hrate0 : 0 ≤ 1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2 := by positivity
nlinarith
· have hmEq : m = Nat.floor x := min_eq_right (Nat.le_of_not_ge hdcap)
have hmLower : b0 / 2 ≤ (m : ℝ) / (n : ℝ) ^ 2 := by
rw [hmEq]
apply (le_div_iff₀ (sq_pos_of_pos hnR)).2
dsimp [x] at hfloorHalf
nlinarith [hfloorHalf]
have hmin : min 1 ((d : ℝ) / (n : ℝ) ^ 2) ≤ 1 := min_le_left _ _
have hone : b0 / 2 * (1 / (n : ℝ)) ≤ 1 / (n : ℝ) := by
simpa using mul_le_mul_of_nonneg_right hb0le (one_div_nonneg.mpr hnR.le)
dsimp [sourceRate]
nlinarith
have hcHigh : c ≤ a * b0 / 4 := min_le_left _ _
have hrateNonneg : 0 ≤
1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2) := by positivity
have hcoef : c *
(1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)) ≤
a / 2 * sourceRate := calc
c * _ ≤ (a * b0 / 4) * _ :=
mul_le_mul_of_nonneg_right hcHigh hrateNonneg
_ = a / 2 * (b0 / 2 *
(1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2))) := by ring
_ ≤ a / 2 * sourceRate :=
mul_le_mul_of_nonneg_left htarget (by positivity)
calc
c * M ^ 2 * (1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2))
≤ M ^ 2 * (a / 2 * sourceRate) := by
calc
_ = M ^ 2 * (c *
(1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2))) := by ring
_ ≤ M ^ 2 * (a / 2 * sourceRate) := by gcongr
_ = M ^ 2 * L := by rfl
_ ≤ minimaxRisk n d epsilon M sigma := htransfer
· have hfallback :
1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2) ≤ K / (n : ℝ) := by
have hmin : min 1 ((d : ℝ) / (n : ℝ) ^ 2) ≤ 1 := min_le_left _ _
rcases not_and_or.mp hlarge with hN | hx
· have hnN : n < N := Nat.lt_of_not_ge hN
have hnNreal : (n : ℝ) ≤ N := by exact_mod_cast (le_of_lt hnN)
have hone : 1 ≤ (N : ℝ) / (n : ℝ) := by
apply (le_div_iff₀ hnR).2
simpa using hnNreal
have hbterm : 0 ≤ 1 / b0 / (n : ℝ) := by positivity
calc
1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)
≤ 1 / (n : ℝ) + 1 := by
simpa [add_comm] using add_le_add_right hmin (1 / (n : ℝ))
_ ≤ 1 / (n : ℝ) + (N : ℝ) / (n : ℝ) + 1 / b0 / (n : ℝ) := by
linarith
_ = K / (n : ℝ) := by dsimp [K]; ring
· have hxlt : x < 1 := lt_of_not_ge hx
have hbn : b0 * (n : ℝ) ≤ 1 := by
dsimp [x] at hxlt
nlinarith [hnOne, mul_pos hb0 hnR]
have hone : 1 ≤ (1 / b0) / (n : ℝ) := by
rw [div_div]
apply (le_div_iff₀ (mul_pos hb0 hnR)).2
simpa [mul_comm] using hbn
have hNterm : 0 ≤ (N : ℝ) / (n : ℝ) := by positivity
calc
1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)
≤ 1 / (n : ℝ) + 1 := by
simpa [add_comm] using add_le_add_right hmin (1 / (n : ℝ))
_ ≤ 1 / (n : ℝ) + (N : ℝ) / (n : ℝ) + 1 / b0 / (n : ℝ) := by
linarith
_ = K / (n : ℝ) := by dsimp [K]; ring
have hcFallback : c ≤ cp / K := min_le_right _ _
have hscaled : c *
(1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)) ≤ cp / (n : ℝ) := by
have hrate0 : 0 ≤ 1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2) := by
positivity
have h1 := mul_le_mul_of_nonneg_left hfallback (le_of_lt hc)
have h2 := mul_le_mul_of_nonneg_right hcFallback hrate0
calc
c * _ ≤ (cp / K) * _ := h2
_ ≤ (cp / K) * (K / (n : ℝ)) := by gcongr
_ = cp / (n : ℝ) := by field_simp [hK.ne', hnR.ne']
have hp := hparam n d M sigma hn hd hM hsigma hsigma_two
calc
c * M ^ 2 * (1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2))
≤ cp * M ^ 2 / n := by
have hM2 : 0 ≤ M ^ 2 := sq_nonneg M
calc
_ = M ^ 2 * (c * (1 / (n : ℝ) +
min 1 ((d : ℝ) / (n : ℝ) ^ 2))) := by ring
_ ≤ M ^ 2 * (cp / (n : ℝ)) := by gcongr
_ = cp * M ^ 2 / n := by ring
_ ≤ minimaxRisk n d epsilon M sigma := hp
theorem scaled_binary_exact_lower_transfer
Restricted-range corollary of the exact-homogeneity transfer.
Formal statement
conclusion
∀ epsilon : ℝ,
0 < epsilon → epsilon < 1 / 2 → ∃ a_epsilon b_epsilon : ℝ, ∃ N_epsilon : ℕ, 0 < a_epsilon ∧ 0 < b_epsilon ∧ ∀ n d : ℕ, ∀ M sigma : ℝ, N_epsilon
≤ n → 0 < d → (d : ℝ) ≤ b_epsilon * (n : ℝ) ^ 2 → 1 ≤ M → 0 ≤ sigma → sigma ≤ 2 → a_epsilon * M ^ 2 * (1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2) ≤ minimaxRisk n d epsilon M sigma
Proof (Lean source)
lemma scaled_binary_exact_lower_transfer :
∀ epsilon : ℝ, 0 < epsilon → epsilon < 1 / 2 →
∃ a_epsilon b_epsilon : ℝ, ∃ N_epsilon : ℕ,
0 < a_epsilon ∧ 0 < b_epsilon ∧
∀ n d : ℕ, ∀ M sigma : ℝ,
N_epsilon ≤ n → 0 < d → (d : ℝ) ≤ b_epsilon * (n : ℝ) ^ 2 →
1 ≤ M → 0 ≤ sigma → sigma ≤ 2 →
a_epsilon * M ^ 2 * (1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2) ≤
minimaxRisk n d epsilon M sigma := by
intro epsilon hepsilon hepsilon_half
obtain ⟨c_epsilon, hc, hall⟩ :=
scaled_binary_exact_lower_transfer_all_d epsilon hepsilon hepsilon_half
refine ⟨c_epsilon, 1, 1, hc, zero_lt_one, ?_⟩
intro n d M sigma hn hd hd_range hM hsigma hsigma_two
have hn_pos : 0 < n := Nat.zero_lt_of_lt hn
have hn_real : 0 < (n : ℝ) := by exact_mod_cast hn_pos
have hratio : (d : ℝ) / (n : ℝ) ^ 2 ≤ 1 := by
rw [div_le_one (sq_pos_of_pos hn_real)]
simpa using hd_range
simpa [min_eq_right hratio] using
hall n d M sigma hn_pos hd hM hsigma hsigma_two
Helpers.OccupancyDischarge 1 declarations This file combines independent-Poisson usable-occupancy bounds with monotone half-intensity de-Poissonization and transports the result back to the original fixed-size real-outcome sample.
Discharge of the usable-occupancy citation
This file combines independent-Poisson usable-occupancy bounds with monotone half-intensity de-Poissonization and transports the result back to the original fixed-size real-outcome sample.
The cited usable-occupancy reciprocal interface follows from the formal Poissonization and de-Poissonization argument.
Formal statement
Proof (Lean source)
theorem zengUsableOccupancyReciprocal (epsilon : ℝ) :
ZengUsableOccupancyReciprocal epsilon := by
let c := poissonUsableLaplaceConstant epsilon
let valid : Prop := 0 < epsilon ∧ epsilon < 1 / 2
let b : ℝ := if valid then c / 8 else 1
let B : ℝ := if valid then max (64 / c) 2 else 1
refine ⟨b, B, ?_, ?_, ?_⟩
· by_cases hv : valid
· simp only [b, if_pos hv]
exact div_pos (poissonUsableLaplaceConstant_pos hv.1
(lt_trans hv.2 (by norm_num))) (by norm_num)
· simp [b, hv]
· by_cases hv : valid
· simp only [B, if_pos hv]
exact lt_of_lt_of_le (by norm_num) (le_max_right _ _)
· simp [B, hv]
intro n d M sigma P
have hv : valid := ⟨P.epsilon_pos, P.epsilon_lt_half⟩
have hc : 0 < c := poissonUsableLaplaceConstant_pos hv.1
(lt_trans hv.2 (by norm_num))
have hb : b = c / 8 := by simp [b, hv]
have hB : B = max (64 / c) 2 := by simp [B, hv]
by_cases hd : d = 0
· subst d
have hmass := occupancy_sum_cellMass_eq_one P.law
simp at hmass
by_cases hn : n = 0
· subst n
constructor
· have hle : realMass (productLaw 0 P.law)
{s | usableTotal s = 0} ≤ 1 := by
unfold realMass
calc
_ ≤ ((productLaw 0 P.law) univ).toReal :=
ENNReal.toReal_mono (measure_ne_top _ _) (measure_mono (Set.subset_univ _))
_ = 1 := by rw [measure_univ]; simp
simpa [hb] using hle.trans (by norm_num : (1 : ℝ) ≤ 2)
· have hzero : (fun s : Fin 0 → Obs d ↦
if 0 < usableTotal s then (1 : ℝ) / usableTotal s else 0) = 0 := by
funext s
simp [usableTotal, usableCell, armCount]
have hzint : (∫ s, (if 0 < usableTotal s then (1 : ℝ) / usableTotal s else 0)
∂productLaw 0 P.law) = 0 := by
apply integral_eq_zero_of_ae
filter_upwards with s
exact congrFun hzero s
rw [hzint]
positivity
have hnpos : 0 < n := Nat.pos_of_ne_zero hn
have hdpos : 0 < d := Nat.pos_of_ne_zero hd
let Q := observedMarkLaw P.law
let lam : ℝ≥0 := (n : ℝ≥0) / 2
let A : ℝ := c * (lam : ℝ) ^ 2 / max (lam : ℝ) (d : ℝ)
let rate : ℝ := (n : ℝ) ^ 2 / (max n d : ℕ)
have hlam : 0 < lam := by
dsimp [lam]
positivity
have hA : 0 < A := by
dsimp [A]
have hmax : 0 < max (lam : ℝ) (d : ℝ) := by positivity
positivity
have hrate : 0 < rate := by
dsimp [rate]
positivity
have hA_rate : c / 4 * rate ≤ A := by
have hden : max (lam : ℝ) (d : ℝ) ≤ max (n : ℝ) (d : ℝ) := by
apply max_le_max_right
dsimp [lam]
norm_num
have hmaxpos : 0 < max (lam : ℝ) (d : ℝ) := by positivity
have hmaxNpos : 0 < max (n : ℝ) (d : ℝ) := by positivity
dsimp [A, rate, lam]
rw [Nat.cast_max]
have heq : c / 4 * ((n : ℝ) ^ 2 / max (n : ℝ) (d : ℝ)) =
(c * ((n : ℝ) / 2) ^ 2) / max (n : ℝ) (d : ℝ) := by ring
rw [heq]
exact div_le_div_of_nonneg_left (by positivity) hmaxpos hden
have hmarked := markedPoisson_lintegral_bounds hdpos hv.1 hv.2
P.law P.overlap lam hlam
let hzero : ℕ → ℝ≥0∞ := fun r ↦ if r = 0 then 1 else 0
let hpen : ℕ → ℝ≥0∞ := fun r ↦ if r = 0 then 1 else (r : ℝ≥0∞)⁻¹
have hdepo0 := augmented_half_depoissonization Q n augmentedZero
measurable_augmentedZero (fun z _ _ hmn ↦ augmentedZero_antitone z hmn)
have hdepoP := augmented_half_depoissonization Q n augmentedPenalty
measurable_augmentedPenalty (fun z _ _ hmn ↦ augmentedPenalty_antitone z hmn)
have hfix0 : (∫⁻ z, streamUsableZero n z ∂iidStreamLaw Q) ≤
2 * ofReal (exp (-A)) := by
have hleft := lintegral_augmentedPrefix_eq (n := n) Q hzero
have hright := lintegral_poissonStream_eq_markedUsable Q lam hzero
dsimp [augmentedZero, hzero] at hdepo0 hleft hright
rw [hleft, hright] at hdepo0
have hm := mul_le_mul_of_nonneg_left hdepo0 (by positivity : (0 : ℝ≥0∞) ≤ 2)
simp only [div_eq_mul_inv, one_mul] at hm
rw [← mul_assoc,
ENNReal.mul_inv_cancel (by norm_num : (2 : ℝ≥0∞) ≠ 0)
(by norm_num : (2 : ℝ≥0∞) ≠ ∞), one_mul] at hm
simpa [streamUsableZero, A, c] using
hm.trans (mul_le_mul_of_nonneg_left hmarked.1 (by positivity))
have hfixP : (∫⁻ z, streamUsablePenalty n z ∂iidStreamLaw Q) ≤
2 * ofReal (2 / A + exp (-A / 2)) := by
have hleft := lintegral_augmentedPrefix_eq (n := n) Q hpen
have hright := lintegral_poissonStream_eq_markedUsable Q lam hpen
dsimp [augmentedPenalty, hpen] at hdepoP hleft hright
rw [hleft, hright] at hdepoP
have hm := mul_le_mul_of_nonneg_left hdepoP (by positivity : (0 : ℝ≥0∞) ≤ 2)
simp only [div_eq_mul_inv, one_mul] at hm
rw [← mul_assoc,
ENNReal.mul_inv_cancel (by norm_num : (2 : ℝ≥0∞) ≠ 0)
(by norm_num : (2 : ℝ≥0∞) ≠ ∞), one_mul] at hm
simpa [streamUsablePenalty, A, c] using
hm.trans (mul_le_mul_of_nonneg_left hmarked.2 (by positivity))
have hprod0 := lintegral_usableZero_productLaw_eq_stream (n := n) P.law
have hprodP := lintegral_usablePenalty_productLaw_eq_stream (n := n) P.law
constructor
· have hset : MeasurableSet {s : Fin n → Obs d | usableTotal s = 0} :=
(measurableSet_singleton 0).preimage measurable_usableTotal
have hmassLin : (productLaw n P.law) {s | usableTotal s = 0} =
∫⁻ s, (if usableTotal s = 0 then (1 : ℝ≥0∞) else 0)
∂productLaw n P.law := by
rw [← lintegral_indicator_one hset]
apply lintegral_congr
intro s
by_cases hs : usableTotal s = 0 <;> simp [hs]
have hENN : (productLaw n P.law) {s | usableTotal s = 0} ≤
2 * ofReal (exp (-A)) := by
rw [hmassLin, hprod0]
exact hfix0
unfold realMass
have hreal := ENNReal.toReal_mono (by finiteness) hENN
rw [ENNReal.toReal_mul, ENNReal.toReal_ofReal (Real.exp_pos _).le] at hreal
norm_num at hreal
rw [hb]
refine hreal.trans ?_
gcongr
have hratio : 0 ≤ (n : ℝ) ^ 2 / (max n d : ℕ) := by positivity
have hweak : c / 8 * ((n : ℝ) ^ 2 / (max n d : ℕ)) ≤ A := by
apply le_trans _ hA_rate
dsimp [rate]
nlinarith [hc, hratio]
calc
-A ≤ -(c / 8 * ((n : ℝ) ^ 2 / (max n d : ℕ))) := neg_le_neg hweak
_ = -(c / 8) * (n : ℝ) ^ 2 / (max n d : ℕ) := by ring
· let recip : (Fin n → Obs d) → ℝ := fun s ↦
if 0 < usableTotal s then 1 / usableTotal s else 0
have hrecipInt : Integrable recip (productLaw n P.law) := by
refine (integrable_const (1 : ℝ)).mono
((measurable_of_countable (fun r : ℕ ↦
if 0 < r then (1 : ℝ) / r else 0)).comp measurable_usableTotal).aestronglyMeasurable ?_
filter_upwards with s
simp only [recip]
by_cases hs : 0 < usableTotal s
· rw [if_pos hs, Real.norm_eq_abs, abs_of_nonneg (by positivity), norm_one]
exact (div_le_one (by exact_mod_cast hs)).2 (by exact_mod_cast hs)
· simp [hs]
have hrecENN : ofReal (∫ s, recip s ∂productLaw n P.law) ≤
∫⁻ s, (if usableTotal s = 0 then (1 : ℝ≥0∞)
else ((usableTotal s : ℕ) : ℝ≥0∞)⁻¹) ∂productLaw n P.law := by
rw [ofReal_integral_eq_lintegral_ofReal hrecipInt]
· apply lintegral_mono
intro s
by_cases hs : usableTotal s = 0
· simp [recip, hs]
· simp [recip, hs, Nat.pos_of_ne_zero hs, ENNReal.ofReal_inv_of_pos,
Nat.cast_pos.mpr (Nat.pos_of_ne_zero hs)]
· filter_upwards with s
simp [recip]
split <;> positivity
have hrecBound : ofReal (∫ s, recip s ∂productLaw n P.law) ≤
2 * ofReal (2 / A + exp (-A / 2)) := by
exact hrecENN.trans (hprodP.le.trans hfixP)
have hreal := ENNReal.toReal_mono (by finiteness) hrecBound
rw [ENNReal.toReal_ofReal (integral_nonneg (fun s ↦ by
simp [recip]; split <;> positivity)), ENNReal.toReal_mul,
ENNReal.toReal_ofReal (by positivity : 0 ≤ 2 / A + exp (-A / 2))] at hreal
norm_num at hreal
change (∫ s, recip s ∂productLaw n P.law) ≤ _
rw [hB, hb]
refine hreal.trans ?_
rw [show -(c / 8) * (n : ℝ) ^ 2 / (max n d : ℕ) =
-(c / 8) * rate by dsimp [rate]; ring]
have hinv : 4 / A ≤ (64 / c) * ((max n d : ℕ) / (n : ℝ) ^ 2) := by
rw [show ((max n d : ℕ) / (n : ℝ) ^ 2) = 1 / rate by
dsimp [rate]
field_simp]
rw [show 64 / c * (1 / rate) = 64 / (c * rate) by field_simp]
apply (div_le_div_iff₀ hA (mul_pos hc hrate)).2
nlinarith [hA_rate]
have hexp : 2 * exp (-A / 2) ≤
2 * exp (-(c / 8) * rate) := by
gcongr
nlinarith
have hcoef1 : 64 / c ≤ max (64 / c) 2 := le_max_left _ _
have hcoef2 : (2 : ℝ) ≤ max (64 / c) 2 := le_max_right _ _
dsimp [recip] at *
calc
2 * (2 / A + exp (-A / 2)) =
4 / A + 2 * exp (-A / 2) := by ring
_ ≤ 64 / c * ((max n d : ℕ) / (n : ℝ) ^ 2) +
2 * exp (-(c / 8) * rate) := add_le_add hinv hexp
_ ≤ max (64 / c) 2 * ((max n d : ℕ) / (n : ℝ) ^ 2) +
max (64 / c) 2 * exp (-(c / 8) * rate) :=
add_le_add
(mul_le_mul_of_nonneg_right hcoef1 (by positivity))
(mul_le_mul_of_nonneg_right hcoef2
(Real.exp_pos (-(c / 8) * rate)).le)
_ = max (64 / c) 2 *
((max n d : ℕ) / (n : ℝ) ^ 2 + exp (-(c / 8) * rate)) := by ring
Helpers.OccupancyTransport 33 declarations This file forgets real outcomes while retaining the finite cell-and-arm marks that determine usable occupancy.
Occupancy transport and monotone de-Poissonization
This file forgets real outcomes while retaining the finite cell-and-arm marks that determine usable occupancy. It also supplies deterministic prefix monotonicity and a paper-independent transfer from an i.i.d. stream stopped at an independent Poisson count to a fixed prefix.
def observedMark
The finite cell-and-arm mark obtained by forgetting an observation's outcome.
Definition (Lean source)
theorem measurable_observedMark
Forgetting the outcome is measurable.
Formal statement
Proof (Lean source)
lemma measurable_observedMark {d : ℕ} : Measurable (observedMark : Obs d → Fin d × Bool) := by
have htuple : Measurable (fun o : Obs d => (o.x, o.a, o.y)) := by
rw [measurable_iff_comap_le]
rfl
exact prodMk htuple.fst htuple.snd.fst
def observedMarkLaw
The one-observation law of the finite cell-and-arm mark.
Definition (Lean source)
noncomputable def observedMarkLaw {d : ℕ} (P : RealLaw d) : Measure (Fin d × Bool) :=
Measure.map observedMark P.observedLaw
instance instIsProbabilityMeasureProdFinBoolObservedMarkLaw
The finite product law of observed binary marks is a probability measure.
Definition (Lean source)
instance {d : ℕ} (P : RealLaw d) : IsProbabilityMeasure (observedMarkLaw P) :=
Measure.isProbabilityMeasure_map measurable_observedMark.aemeasurable
theorem realMass_observedMarkLaw_singleton
The factorization field of a real law gives the exact mass of every cell-and-arm atom after outcomes are forgotten.
Formal statement
hypotheses
conclusion
realMass (observedMarkLaw P) {(k, a)}
= P.cellMass k * (if a then P.propensity k else 1 - P.propensity k)
Proof (Lean source)
lemma realMass_observedMarkLaw_singleton {d : ℕ} (P : RealLaw d)
(k : Fin d) (a : Bool) :
realMass (observedMarkLaw P) {(k, a)} =
P.cellMass k * (if a then P.propensity k else 1 - P.propensity k) := by
unfold observedMarkLaw realMass
rw [Measure.map_apply measurable_observedMark
(measurableSet_singleton (k, a))]
have hfactor := P.arm_outcome_factorization a k Set.univ MeasurableSet.univ
let _ : IsProbabilityMeasure (P.outcomeLaw a k) := P.outcome_isProbability a k
have hout : realMass (P.outcomeLaw a k) Set.univ = 1 := by
simp [realMass]
rw [hout, mul_one] at hfactor
rw [show observedMark ⁻¹' {(k, a)} =
{o : Obs d | o.x = k ∧ o.a = a ∧ o.y ∈ Set.univ} by
ext o
simp [observedMark]]
simpa only [realMass, Set.mem_univ, and_true] using hfactor.symm
theorem productLaw_map_observedMarks
Mapping every coordinate of a real-outcome product sample to its finite cell-and-arm mark gives the product of the corresponding mark law.
Formal statement
hypotheses
n d :
ℕ
P :
RealLaw d
conclusion
Measure.map (fun sample : Fin n → Obs d => fun i => observedMark (sample i)) (productLaw n P)
= Measure.pi (fun _ : Fin n => observedMarkLaw P)
Proof (Lean source)
lemma productLaw_map_observedMarks {n d : ℕ} (P : RealLaw d) :
Measure.map (fun sample : Fin n → Obs d => fun i => observedMark (sample i))
(productLaw n P) =
Measure.pi (fun _ : Fin n => observedMarkLaw P) := by
unfold productLaw observedMarkLaw
exact Measure.pi_map_pi (fun _ : Fin n => measurable_observedMark.aemeasurable)
def streamArmCount
Number of occurrences of one arm and cell in the first n marks of a stream.
def streamCellCount
Total number of marks in a cell in the first n stream positions.
Definition (Lean source)
def streamCellCount {d : ℕ} (z : ℕ → Fin d × Bool)
(n : ℕ) (k : Fin d) : ℕ :=
streamArmCount z n false k + streamArmCount z n true k
def streamUsableTotal
Usable occupancy in the first n positions of a cell-and-arm stream.
Definition (Lean source)
def streamUsableTotal {d : ℕ} (z : ℕ → Fin d × Bool) (n : ℕ) : ℕ :=
∑ k : Fin d,
if 0 < streamArmCount z n false k ∧ 0 < streamArmCount z n true k then
streamCellCount z n k
else 0
theorem streamArmCount_mono
If the shorter prefix is contained in the longer prefix, arm counts cannot decrease when a stream prefix is enlarged.
Formal statement
hypotheses
conclusion
streamArmCount z m a k ≤ streamArmCount z n a k
Proof (Lean source)
lemma streamArmCount_mono {d : ℕ} (z : ℕ → Fin d × Bool)
{m n : ℕ} (hmn : m ≤ n) (a : Bool) (k : Fin d) :
streamArmCount z m a k ≤ streamArmCount z n a k := by
apply Finset.card_le_card
intro i hi
simp only [mem_filter, Finset.mem_range] at hi ⊢
exact ⟨lt_of_lt_of_le hi.1 hmn, hi.2⟩
theorem streamUsableTotal_mono
If the shorter prefix is contained in the longer prefix, usable occupancy cannot decrease when a stream prefix is enlarged.
Formal statement
hypotheses
conclusion
streamUsableTotal z m ≤ streamUsableTotal z n
Proof (Lean source)
lemma streamUsableTotal_mono {d : ℕ} (z : ℕ → Fin d × Bool)
{m n : ℕ} (hmn : m ≤ n) :
streamUsableTotal z m ≤ streamUsableTotal z n := by
unfold streamUsableTotal
apply Finset.sum_le_sum
intro k _
by_cases hm : 0 < streamArmCount z m false k ∧ 0 < streamArmCount z m true k
· have hn : 0 < streamArmCount z n false k ∧ 0 < streamArmCount z n true k :=
⟨lt_of_lt_of_le hm.1 (streamArmCount_mono z hmn false k),
lt_of_lt_of_le hm.2 (streamArmCount_mono z hmn true k)⟩
rw [if_pos hm, if_pos hn]
unfold streamCellCount
exact Nat.add_le_add (streamArmCount_mono z hmn false k)
(streamArmCount_mono z hmn true k)
· rw [if_neg hm]
exact zero_le _
theorem iidStream_antitone_depoissonization
If the specified object and the specified function or embedding and the stream functional is measurable and the stream functional decreases with sample size, a reusable monotone de-Poissonization inequality. If a nonnegative stream statistic decreases with the prefix length, then its fixed-n expectation, multiplied by the probability that the independent Poisson count does not exceed n, is bounded by the statistic evaluated at that random count.
Formal statement
hypotheses
X :
Type*
P :
lam :
ℝ≥0
n :
ℕ
g :
ℕ → (ℕ → X) → ℝ≥0∞
hg :
Measurable (fun z : ℕ × (ℕ → X) => g z.1 z.2)
hanti :
∀ z m n
if
m ≤ n
then
g n z ≤ g m z
conclusion
(poissonMeasure lam) (Iic n) * (∫⁻ z, g n z ∂iidStreamLaw P)
≤ ∫⁻ z, g z.1 z.2 ∂poissonIIDStreamLaw P lam
Proof (Lean source)
lemma iidStream_antitone_depoissonization
{X : Type*} [MeasurableSpace X]
(P : Measure X) [IsProbabilityMeasure P]
(lam : ℝ≥0) (n : ℕ) (g : ℕ → (ℕ → X) → ℝ≥0∞)
(hg : Measurable (fun z : ℕ × (ℕ → X) => g z.1 z.2))
(hanti : ∀ z m n, m ≤ n → g n z ≤ g m z) :
(poissonMeasure lam) (Iic n) *
(∫⁻ z, g n z ∂iidStreamLaw P) ≤
∫⁻ z, g z.1 z.2 ∂poissonIIDStreamLaw P lam := by
let A : ℝ≥0∞ := ∫⁻ z, g n z ∂iidStreamLaw P
have hiter : Measurable (fun m : ℕ => ∫⁻ z, g m z ∂iidStreamLaw P) :=
hg.lintegral_prod_right'
calc
(poissonMeasure lam) (Iic n) * A =
∫⁻ _m in Iic n, A ∂poissonMeasure lam := by
rw [setLIntegral_const]
simp [mul_comm]
_ ≤ ∫⁻ m in Iic n, ∫⁻ z, g m z ∂iidStreamLaw P ∂poissonMeasure lam := by
apply setLIntegral_mono hiter
intro m hm
apply lintegral_mono
intro z
exact hanti z m n hm
_ ≤ ∫⁻ m, ∫⁻ z, g m z ∂iidStreamLaw P ∂poissonMeasure lam :=
setLIntegral_le_lintegral _ _
_ = ∫⁻ z, g z.1 z.2 ∂poissonIIDStreamLaw P lam := by
unfold poissonIIDStreamLaw
exact (lintegral_prod _ hg.aemeasurable).symm
theorem poisson_lintegral_natCast
If the specified object, the first moment of a Poisson count is its intensity, in lintegral form.
Formal statement
hypotheses
r :
ℝ≥0
conclusion
∫⁻ n : ℕ, (n : ℝ≥0∞) ∂poissonMeasure r = (r : ℝ≥0∞)
Proof (Lean source)
lemma poisson_lintegral_natCast (r : ℝ≥0) :
∫⁻ n : ℕ, (n : ℝ≥0∞) ∂poissonMeasure r = (r : ℝ≥0∞) := by
rw [lintegral_countable']
simp_rw [poissonMeasure_singleton_eq_poissonPMF]
rw [tsum_eq_zero_add' ENNReal.summable]
simp only [cast_zero, zero_mul, zero_add]
simp_rw [occupancy_poissonPMF_succ_mul]
rw [ENNReal.tsum_mul_left, PMF.tsum_coe, mul_one]
theorem poisson_half_intensity_Iic
A Poisson count with mean n/2 is at most n with probability at least one half.
Formal statement
Proof (Lean source)
lemma poisson_half_intensity_Iic (n : ℕ) :
(1 / 2 : ℝ≥0∞) ≤
(poissonMeasure ((n : ℝ≥0) / 2)) (Iic n) := by
let r : ℝ≥0 := (n : ℝ≥0) / 2
let tail : Set ℕ := {m | n + 1 ≤ m}
have hmarkov := meas_ge_le_lintegral_div
(μ := poissonMeasure r) (f := fun m : ℕ => (m : ℝ≥0∞))
(measurable_of_countable _).aemeasurable
(ε := ((n + 1 : ℕ) : ℝ≥0∞)) (by simp) (by simp)
rw [poisson_lintegral_natCast] at hmarkov
have htail_set : {m : ℕ | ((n + 1 : ℕ) : ℝ≥0∞) ≤ (m : ℝ≥0∞)} = tail := by
ext m
simp only [tail]
norm_cast
rw [htail_set] at hmarkov
have hratio : (r : ℝ≥0∞) / ((n + 1 : ℕ) : ℝ≥0∞) ≤ 1 / 2 := by
rw [ENNReal.div_le_iff_le_mul (inl (by simp)) (inl (by simp))]
change (((n : ℝ≥0) / 2 : ℝ≥0) : ℝ≥0∞) ≤
(1 / 2 : ℝ≥0∞) * (n + 1 : ℕ)
rw [ENNReal.coe_div (by norm_num)]
change (n : ℝ≥0∞) / 2 ≤ 1 / 2 * (n + 1 : ℕ)
rw [div_eq_mul_inv, one_div, mul_comm (n : ℝ≥0∞)]
gcongr
exact_mod_cast Nat.le_succ n
have htail : (poissonMeasure r) tail ≤ 1 / 2 := hmarkov.trans hratio
have hcompl : tail = (Iic n)ᶜ := by
ext m
simp [tail]
have htail_meas : MeasurableSet tail := by
rw [hcompl]
exact measurableSet_Iic.compl
have hIic : (poissonMeasure r) (Iic n) = 1 - (poissonMeasure r) tail := by
rw [← compl_compl (Iic n), ← hcompl,
measure_compl htail_meas (measure_ne_top _ _), measure_univ]
rw [hIic]
apply ENNReal.le_sub_of_add_le_right
(ne_top_of_le_ne_top ENNReal.one_ne_top (htail.trans (by norm_num)))
calc
(1 / 2 : ℝ≥0∞) + (poissonMeasure r) tail ≤ 1 / 2 + 1 / 2 :=
add_le_add_right htail _
_ = 1 := by simpa only [one_div] using ENNReal.inv_two_add_inv_two
def streamUsableZero
The zero-occupancy indicator on a stream prefix.
Definition (Lean source)
def streamUsableZero {d : ℕ} (n : ℕ) (z : ℕ → Fin d × Bool) : ℝ≥0∞ :=
if streamUsableTotal z n = 0 then 1 else 0
def streamUsablePenalty
The combined zero-event and reciprocal penalty used for monotone de-Poissonization.
Definition (Lean source)
noncomputable def streamUsablePenalty {d : ℕ} (n : ℕ)
(z : ℕ → Fin d × Bool) : ℝ≥0∞ :=
if streamUsableTotal z n = 0 then 1
else ((streamUsableTotal z n : ℕ) : ℝ≥0∞)⁻¹
def finiteMarkUsableTotal
Usable occupancy computed directly on a finite tuple of cell-and-arm marks.
Definition (Lean source)
theorem streamUsableTotal_eq_finiteMarkUsableTotal
Range-based stream occupancy equals direct occupancy of the finite prefix.
Formal statement
hypotheses
conclusion
Proof (Lean source)
lemma streamUsableTotal_eq_finiteMarkUsableTotal {d : ℕ}
(z : ℕ → Fin d × Bool) (n : ℕ) :
streamUsableTotal z n = finiteMarkUsableTotal (fun i : Fin n => z i) := by
classical
unfold streamUsableTotal streamCellCount streamArmCount finiteMarkUsableTotal
apply Finset.sum_congr rfl
intro k _
have hcount (a : Bool) :
((range n).filter fun i => (z i).1 = k ∧ (z i).2 = a).card =
(Finset.univ.filter fun i : Fin n => (z i).1 = k ∧ (z i).2 = a).card := by
rw [← Finset.image_fin_univ, Finset.filter_image]
exact Finset.card_image_of_injective _ Fin.val_injective
rw [hcount false, hcount true]
theorem measurable_streamUsableTotal_fixed
Fixed-prefix usable occupancy is measurable as a function of the stream.
Formal statement
Proof (Lean source)
lemma measurable_streamUsableTotal_fixed {d : ℕ} (n : ℕ) :
Measurable (fun z : ℕ → Fin d × Bool => streamUsableTotal z n) := by
have hprefix : Measurable (fun z : ℕ → Fin d × Bool => fun i : Fin n => z i) :=
measurable_pi_lambda _ fun i => measurable_pi_apply (i : ℕ)
rw [show (fun z : ℕ → Fin d × Bool => streamUsableTotal z n) =
finiteMarkUsableTotal ∘ (fun z => fun i : Fin n => z i) by
funext z
exact streamUsableTotal_eq_finiteMarkUsableTotal z n]
exact (measurable_of_countable finiteMarkUsableTotal).comp hprefix
theorem streamUsableZero_antitone
If the shorter prefix is contained in the longer prefix, the zero-occupancy indicator decreases along stream prefixes.
Formal statement
hypotheses
conclusion
streamUsableZero n z ≤ streamUsableZero m z
Proof (Lean source)
lemma streamUsableZero_antitone {d : ℕ} (z : ℕ → Fin d × Bool)
{m n : ℕ} (hmn : m ≤ n) :
streamUsableZero n z ≤ streamUsableZero m z := by
unfold streamUsableZero
by_cases hm : streamUsableTotal z m = 0
· rw [if_pos hm]
split <;> simp
· have hn : streamUsableTotal z n ≠ 0 := by
exact Nat.ne_zero_of_lt (lt_of_lt_of_le (Nat.pos_of_ne_zero hm)
(streamUsableTotal_mono z hmn))
simp [hm, hn]
theorem streamUsablePenalty_antitone
If the shorter prefix is contained in the longer prefix, the combined zero-event/reciprocal penalty decreases along stream prefixes.
Formal statement
hypotheses
conclusion
streamUsablePenalty n z ≤ streamUsablePenalty m z
Proof (Lean source)
lemma streamUsablePenalty_antitone {d : ℕ} (z : ℕ → Fin d × Bool)
{m n : ℕ} (hmn : m ≤ n) :
streamUsablePenalty n z ≤ streamUsablePenalty m z := by
unfold streamUsablePenalty
by_cases hm : streamUsableTotal z m = 0
· rw [if_pos hm]
by_cases hn : streamUsableTotal z n = 0
· simp [hn]
· rw [if_neg hn]
apply (ENNReal.inv_le_one).2
exact_mod_cast Nat.one_le_iff_ne_zero.2 hn
· have hn : streamUsableTotal z n ≠ 0 := by
exact Nat.ne_zero_of_lt (lt_of_lt_of_le (Nat.pos_of_ne_zero hm)
(streamUsableTotal_mono z hmn))
rw [if_neg hm, if_neg hn]
exact ENNReal.inv_le_inv.mpr (by exact_mod_cast streamUsableTotal_mono z hmn)
def armCellPartition
The identity partition of the finite cell-and-arm mark space.
Definition (Lean source)
def armCellPartition (d : ℕ) :
FiniteMeasurablePartition (Fin d × Bool) (Fin d × Bool) where
cell := id
measurable_cell := measurable_id
def markedArmCellCounts
Counts of every arm-cell atom in a finite marked sample.
Definition (Lean source)
noncomputable def markedArmCellCounts {d : ℕ}
(s : FiniteSample ((Fin d × Bool) × ℝ)) : Fin d × Bool → ℕ :=
fun j => ((armCellPartition d).restrictCell j s).count
theorem measurable_markedArmCellCounts
The arm-cell count vector is measurable.
Formal statement
hypotheses
d :
ℕ
conclusion
Proof (Lean source)
lemma measurable_markedArmCellCounts {d : ℕ} :
Measurable (markedArmCellCounts :
FiniteSample ((Fin d × Bool) × ℝ) → Fin d × Bool → ℕ) := by
unfold markedArmCellCounts
exact measurable_pi_lambda _ fun j =>
measurable_finiteSample_count.comp ((armCellPartition d).measurable_restrictCell j)
theorem map_markedArmCellCounts_finiteMarkedPoissonSampleLaw
If the specified object, under the marked Poisson construction, all arm-cell counts are jointly independent Poisson variables with their atom-specific intensities.
Formal statement
hypotheses
P :
lam :
ℝ≥0
conclusion
Measure.map markedArmCellCounts (finiteMarkedPoissonSampleLaw P (Measure.dirac 0) lam)
Proof (Lean source)
lemma map_markedArmCellCounts_finiteMarkedPoissonSampleLaw
(P : Measure (Fin d × Bool)) [IsProbabilityMeasure P]
(lam : ℝ≥0) :
Measure.map markedArmCellCounts
(finiteMarkedPoissonSampleLaw P (Measure.dirac 0) lam) =
Measure.pi (fun j : Fin d × Bool =>
poissonMeasure (lam * (P {j}).toNNReal)) := by
let p := armCellPartition d
let R : Measure ℝ := Measure.dirac 0
let mu := finiteMarkedPoissonSampleLaw P R lam
have hjoint := FiniteMeasurablePartition.map_restrictPartition_finiteMarkedPoissonSampleLaw
p P R lam
let countFamily : (Fin d × Bool → FiniteSample ((Fin d × Bool) × ℝ)) →
(Fin d × Bool → ℕ) := fun q j => (q j).count
have hcountFamily : Measurable countFamily :=
measurable_pi_lambda _ fun j => measurable_finiteSample_count.comp (measurable_pi_apply j)
calc
Measure.map markedArmCellCounts mu =
Measure.map countFamily (Measure.map p.restrictPartition mu) := by
rw [Measure.map_map hcountFamily p.measurable_restrictPartition]
rfl
_ = Measure.map countFamily
(Measure.pi (fun j : Fin d × Bool =>
finiteMarkedPoissonSampleLaw (p.cellObservationLaw P j) R
(lam * p.cellMass P j))) := by rw [hjoint]
_ = Measure.pi (fun j : Fin d × Bool =>
Measure.map FiniteSample.count
(finiteMarkedPoissonSampleLaw (p.cellObservationLaw P j) R
(lam * p.cellMass P j))) := by
exact Measure.pi_map_pi (fun _ => measurable_finiteSample_count.aemeasurable)
_ = Measure.pi (fun j : Fin d × Bool =>
poissonMeasure (lam * (P {j}).toNNReal)) := by
congr with j
rw [finiteMarkedPoissonSampleLaw_map_count]
rfl
def regroupArmCellCounts
Regroup an arm-cell count vector into false/true counts within each cell.
def usableTotalOfRegroupedCounts
Usable occupancy computed from a regrouped pair of arm counts.
Definition (Lean source)
def usableTotalOfRegroupedCounts {d : ℕ} (c : Fin d → ℕ × ℕ) : ℕ :=
∑ k : Fin d, if 0 < (c k).1 ∧ 0 < (c k).2 then (c k).1 + (c k).2 else 0
theorem map_regroupedArmCellCounts_finiteMarkedPoissonSampleLaw
If the specified object, the regrouped arm-count vector is the pushforward of the independent atom-count product law.
Formal statement
hypotheses
P :
lam :
ℝ≥0
conclusion
Measure.map (regroupArmCellCounts ∘ markedArmCellCounts) (finiteMarkedPoissonSampleLaw P (Measure.dirac 0) lam)
= Measure.map regroupArmCellCounts (Measure.pi (fun j : Fin d × Bool => poissonMeasure (lam * (P {j}).toNNReal)))
Proof (Lean source)
lemma map_regroupedArmCellCounts_finiteMarkedPoissonSampleLaw
(P : Measure (Fin d × Bool)) [IsProbabilityMeasure P]
(lam : ℝ≥0) :
Measure.map (regroupArmCellCounts ∘ markedArmCellCounts)
(finiteMarkedPoissonSampleLaw P (Measure.dirac 0) lam) =
Measure.map regroupArmCellCounts
(Measure.pi (fun j : Fin d × Bool =>
poissonMeasure (lam * (P {j}).toNNReal))) := by
rw [← Measure.map_map (measurable_of_countable regroupArmCellCounts)
measurable_markedArmCellCounts,
map_markedArmCellCounts_finiteMarkedPoissonSampleLaw]
theorem usableTotalOfRegroupedCounts_eq
Regrouping the atom counts preserves the stream usable-total formula.
Formal statement
hypotheses
conclusion
= streamUsableTotal z n
Proof (Lean source)
lemma usableTotalOfRegroupedCounts_eq {d : ℕ} (z : ℕ → Fin d × Bool) (n : ℕ) :
usableTotalOfRegroupedCounts
(regroupArmCellCounts (fun j => streamArmCount z n j.2 j.1)) =
streamUsableTotal z n := by
rfl
def markedUsableTotal
The marked finite-sample usable total is exactly the deterministic regrouping of its arm-cell count vector.
Definition (Lean source)
noncomputable def markedUsableTotal {d : ℕ}
(s : FiniteSample ((Fin d × Bool) × ℝ)) : ℕ :=
usableTotalOfRegroupedCounts (regroupArmCellCounts (markedArmCellCounts s))
theorem markedArmCellCounts_streamToFiniteSample
Restricting a retained marked prefix to an arm-cell and counting it agrees with the corresponding range count in the underlying stream.
Formal statement
hypotheses
conclusion
= streamArmCount (fun i => (q.2 i).1) q.1 j.2 j.1
Proof (Lean source)
lemma markedArmCellCounts_streamToFiniteSample {d : ℕ}
(q : ℕ × (ℕ → (Fin d × Bool) × ℝ)) (j : Fin d × Bool) :
markedArmCellCounts (streamToFiniteSample q) j =
streamArmCount (fun i => (q.2 i).1) q.1 j.2 j.1 := by
classical
unfold markedArmCellCounts FiniteMeasurablePartition.restrictCell
FiniteMeasurablePartition.cellIndices armCellPartition streamToFiniteSample
streamArmCount FiniteSample.count FiniteSample.points
dsimp only
rw [← Finset.image_fin_univ, Finset.filter_image]
rw [Finset.card_image_of_injective _ Fin.val_injective]
congr 1
ext i
simp [Prod.ext_iff]
theorem markedUsableTotal_streamToFiniteSample
Marked-sample usable occupancy is exactly usable occupancy of the retained random prefix after the auxiliary real marks are forgotten.
Formal statement
hypotheses
conclusion
Proof (Lean source)
lemma markedUsableTotal_streamToFiniteSample {d : ℕ}
(q : ℕ × (ℕ → (Fin d × Bool) × ℝ)) :
markedUsableTotal (streamToFiniteSample q) =
streamUsableTotal (fun i => (q.2 i).1) q.1 := by
unfold markedUsableTotal
rw [← usableTotalOfRegroupedCounts_eq (fun i => (q.2 i).1) q.1]
congr 2
funext j
exact markedArmCellCounts_streamToFiniteSample q j
theorem finiteMarkedPoissonSampleLaw_eq_randomPrefixLaw
If the specified object, the marked Poisson law is the random-prefix iid-stream law on observation-mark pairs.
Formal statement
hypotheses
P :
lam :
ℝ≥0
conclusion
finiteMarkedPoissonSampleLaw P (Measure.dirac 0) lam
= Measure.map streamToFiniteSample (poissonIIDStreamLaw (P.prod (Measure.dirac 0)) lam)
Proof (Lean source)
lemma finiteMarkedPoissonSampleLaw_eq_randomPrefixLaw
(P : Measure (Fin d × Bool)) [IsProbabilityMeasure P] (lam : ℝ≥0) :
finiteMarkedPoissonSampleLaw P (Measure.dirac 0) lam =
Measure.map streamToFiniteSample
(poissonIIDStreamLaw (P.prod (Measure.dirac 0)) lam) := by
rfl
Helpers.OccupancyUpper 13 declarations
def collisionDesignCenter
The conditional-on-design center of the unclipped occupancy estimator, totalized by zero when no cell contains both treatment arms.
Definition (Lean source)
noncomputable def collisionDesignCenter {n d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (sample : Fin n → Obs d) : ℝ :=
if 0 < usableTotal sample then
(∑ k : Fin d, if usableCell sample k then
(cellCount sample k : ℝ) * cellEffect P.law k else 0) /
usableTotal sample
else 0
theorem collisionDesignCenter_sub_ate_eq
If the cell is usable, on the usable event, subtracting the ATE from the conditional design center is exactly the occupancy-weighted average of the cell deviations.
Formal statement
hypotheses
n d :
ℕ
epsilon M sigma :
ℝ
P :
ModelClass d epsilon M sigma
sample :
Fin n → Obs d
husable :
0 < usableTotal sample
conclusion
collisionDesignCenter P sample - rawAteFormula P.law
= (∑ k : Fin d, if usableCell sample k then (cellCount sample k : ℝ) * cellDeviation P.law k else 0) / usableTotal sample
Proof (Lean source)
lemma collisionDesignCenter_sub_ate_eq {n d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (sample : Fin n → Obs d)
(husable : 0 < usableTotal sample) :
collisionDesignCenter P sample - rawAteFormula P.law =
(∑ k : Fin d, if usableCell sample k then
(cellCount sample k : ℝ) * cellDeviation P.law k else 0) /
usableTotal sample := by
have hsumNat :
(∑ k : Fin d, if usableCell sample k then cellCount sample k else 0) =
usableTotal sample := rfl
have hsumReal :
(∑ k : Fin d, if usableCell sample k then
(cellCount sample k : ℝ) else 0) = (usableTotal sample : ℝ) := by
exact_mod_cast hsumNat
have hden : (usableTotal sample : ℝ) ≠ 0 := by
exact_mod_cast (Nat.ne_of_gt husable)
let w : Fin d → ℝ := fun k =>
if usableCell sample k then (cellCount sample k : ℝ) else 0
have hw_sum : ∑ k : Fin d, w k = (usableTotal sample : ℝ) := by
simpa [w] using hsumReal
have hcenter :
(∑ k : Fin d, if usableCell sample k then
(cellCount sample k : ℝ) * cellEffect P.law k else 0) =
∑ k : Fin d, w k * cellEffect P.law k := by
apply Finset.sum_congr rfl
intro k hk
by_cases hu : usableCell sample k <;> simp [w, hu]
have hdeviation :
(∑ k : Fin d, if usableCell sample k then
(cellCount sample k : ℝ) * cellDeviation P.law k else 0) =
∑ k : Fin d, w k * cellDeviation P.law k := by
apply Finset.sum_congr rfl
intro k hk
by_cases hu : usableCell sample k <;> simp [w, hu]
unfold collisionDesignCenter
rw [if_pos husable]
rw [hcenter, hdeviation]
simp_rw [cellDeviation, mul_sub]
rw [Finset.sum_sub_distrib, ← Finset.sum_mul, hw_sum]
field_simp
theorem collisionDesignCenter_bias_abs_le
If the cell is usable and the stated support-size bound holds, if every empirically usable cell is in the population support, approximate homogeneity bounds the conditional design bias by sigma * M.
Formal statement
hypotheses
n d :
ℕ
epsilon M sigma :
ℝ
P :
ModelClass d epsilon M sigma
sample :
Fin n → Obs d
husable :
0 < usableTotal sample
hsupport :
∀ k
if
usableCell sample k
then
0 < P.law.cellMass k
conclusion
|collisionDesignCenter P sample - rawAteFormula P.law| ≤ sigma * M
Proof (Lean source)
lemma collisionDesignCenter_bias_abs_le {n d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (sample : Fin n → Obs d)
(husable : 0 < usableTotal sample)
(hsupport : ∀ k, usableCell sample k → 0 < P.law.cellMass k) :
|collisionDesignCenter P sample - rawAteFormula P.law| ≤ sigma * M := by
rw [collisionDesignCenter_sub_ate_eq P sample husable, abs_div]
have hden : (0 : ℝ) < usableTotal sample := by exact_mod_cast husable
have hnum :
|∑ k : Fin d, if usableCell sample k then
(cellCount sample k : ℝ) * cellDeviation P.law k else 0| ≤
(usableTotal sample : ℝ) * (sigma * M) := by
calc
_ ≤ ∑ k : Fin d, |if usableCell sample k then
(cellCount sample k : ℝ) * cellDeviation P.law k else 0| :=
Finset.abs_sum_le_sum_abs _ _
_ ≤ ∑ k : Fin d, if usableCell sample k then
(cellCount sample k : ℝ) * (sigma * M) else 0 := by
apply Finset.sum_le_sum
intro k hk
by_cases hu : usableCell sample k
· simp only [if_pos hu, abs_mul,
abs_of_nonneg (show (0 : ℝ) ≤ cellCount sample k by positivity)]
exact mul_le_mul_of_nonneg_left
(P.homogeneity k (hsupport k hu)) (by positivity)
· simp [hu]
_ = (usableTotal sample : ℝ) * (sigma * M) := by
have hrewrite :
(∑ k : Fin d, if usableCell sample k then
(cellCount sample k : ℝ) * (sigma * M) else 0) =
(∑ k : Fin d, if usableCell sample k then
(cellCount sample k : ℝ) else 0) * (sigma * M) := by
rw [Finset.sum_mul]
apply Finset.sum_congr rfl
intro k hk
by_cases hu : usableCell sample k <;> simp [hu]
rw [hrewrite]
congr 1
exact_mod_cast
(show (∑ k : Fin d, if usableCell sample k then
cellCount sample k else 0) = usableTotal sample from rfl)
rw [abs_of_pos hden]
exact (div_le_iff₀ hden).2 (by simpa [mul_comm] using hnum)
theorem modelClass_rawAte_abs_le
Mean normalization bounds the ATE of every radius-indexed model by the outcome scale.
Formal statement
hypotheses
conclusion
|rawAteFormula P.law| ≤ M
Proof (Lean source)
lemma modelClass_rawAte_abs_le {d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) : |rawAteFormula P.law| ≤ M := by
let Q : UnrestrictedClass d epsilon M := {
law := P.law
epsilon_pos := P.epsilon_pos
epsilon_lt_half := P.epsilon_lt_half
M_ge_one := P.M_ge_one
consistency := P.consistency
exchangeability := P.exchangeability
overlap := P.overlap
mean_normalization := P.mean_normalization
second_moment := P.second_moment }
exact (scale_sanity.1 Q).2.1
theorem collisionDesignCenter_bias_sq_le
If the stated support-size bound holds, the conditional-design squared bias is bounded by the homogeneity radius, plus the indicator of the zero-usable-occupancy fallback.
Formal statement
hypotheses
n d :
ℕ
epsilon M sigma :
ℝ
P :
ModelClass d epsilon M sigma
sample :
Fin n → Obs d
hsupport :
∀ k
if
usableCell sample k
then
0 < P.law.cellMass k
conclusion
(collisionDesignCenter P sample - rawAteFormula P.law) ^ 2
≤ M ^ 2 * (sigma ^ 2 + if usableTotal sample = 0 then 1 else 0)
Proof (Lean source)
lemma collisionDesignCenter_bias_sq_le {n d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (sample : Fin n → Obs d)
(hsupport : ∀ k, usableCell sample k → 0 < P.law.cellMass k) :
(collisionDesignCenter P sample - rawAteFormula P.law) ^ 2 ≤
M ^ 2 * (sigma ^ 2 + if usableTotal sample = 0 then 1 else 0) := by
have hM : 0 ≤ M := le_trans zero_le_one P.M_ge_one
by_cases husable : 0 < usableTotal sample
· have hne : usableTotal sample ≠ 0 := Nat.ne_of_gt husable
have hbias := collisionDesignCenter_bias_abs_le P sample husable hsupport
rw [if_neg hne]
simp only [add_zero]
calc
(collisionDesignCenter P sample - rawAteFormula P.law) ^ 2 =
|collisionDesignCenter P sample - rawAteFormula P.law| ^ 2 :=
(sq_abs _).symm
_ ≤ (sigma * M) ^ 2 :=
(sq_le_sq₀ (abs_nonneg _) (mul_nonneg P.sigma_nonneg hM)).2 hbias
_ = M ^ 2 * sigma ^ 2 := by ring
· have hzero : usableTotal sample = 0 := Nat.eq_zero_of_not_pos husable
have hate := modelClass_rawAte_abs_le P
rw [if_pos hzero]
simp only [collisionDesignCenter, hzero, lt_self_iff_false, ↓reduceIte,
zero_sub] at hate ⊢
calc
(-rawAteFormula P.law) ^ 2 = |rawAteFormula P.law| ^ 2 := by
rw [sq_abs]
ring
_ ≤ M ^ 2 := (sq_le_sq₀ (abs_nonneg _) hM).2 hate
_ ≤ M ^ 2 * (sigma ^ 2 + 1) := by
nlinarith [sq_nonneg M, sq_nonneg sigma,
mul_nonneg (sq_nonneg M) (sq_nonneg sigma)]
theorem usableCell_supported_ae
Empirically usable cells have positive population mass almost surely under the product experiment.
Formal statement
hypotheses
n d :
ℕ
P :
RealLaw d
conclusion
∀ᵐ sample ∂productLaw n P, ∀ k, usableCell sample k → 0 < P.cellMass k
Proof (Lean source)
-- @node: usableCell_supported_ae
lemma usableCell_supported_ae {n d : ℕ} (P : RealLaw d) :
∀ᵐ sample ∂productLaw n P, ∀ k, usableCell sample k → 0 < P.cellMass k := by
have hcoord (i : Fin n) (k : Fin d) :
∀ᵐ sample ∂productLaw n P, (sample i).x = k → 0 < P.cellMass k := by
by_cases hk : 0 < P.cellMass k
· exact Filter.Eventually.of_forall (fun _ _ => hk)
· have hkzero : P.cellMass k = 0 :=
le_antisymm (not_lt.mp hk) (P.cellMass_range k).1
let S : Set (Obs d) := {o | o.x = k}
have hSzero : P.observedLaw S = 0 := by
have htoreal : (P.observedLaw S).toReal = 0 := by
change realMass P.observedLaw {o | o.x = k} = 0
rw [← P.cellMass_eq k, hkzero]
rcases (ENNReal.toReal_eq_zero_iff _).mp htoreal with hzero | htop
· exact hzero
· exact (measure_ne_top P.observedLaw S htop).elim
have hpre : (productLaw n P) (Function.eval i ⁻¹' S) = 0 := by
simpa [productLaw] using
(Measure.pi_eval_preimage_null
(μ := fun _ : Fin n => P.observedLaw) (i := i) hSzero)
have hae : ∀ᵐ sample ∂productLaw n P, sample i ∉ S :=
measure_eq_zero_iff_ae_notMem.mp hpre
filter_upwards [hae] with sample hs hxi
exact (hs hxi).elim
filter_upwards [Filter.eventually_all.2 (fun i =>
Filter.eventually_all.2 (fun k => hcoord i k))] with sample hs
intro k hu
have hcounts : 0 < armCount sample false k ∧ 0 < armCount sample true k := by
simpa [usableCell] using hu
obtain ⟨i, hi⟩ := Finset.card_pos.mp hcounts.1
have hix : (sample i).x = k ∧ (sample i).a = false := by
simpa [armCount] using hi
exact hs i k hix.1
theorem measurable_collisionDesignCenter
The conditional design center is measurable because it factors through the finite cell-and-treatment design.
Formal statement
hypotheses
conclusion
Measurable (collisionDesignCenter P : (Fin n → Obs d) → ℝ)
Proof (Lean source)
-- @node: measurable_collisionDesignCenter
lemma measurable_collisionDesignCenter {n d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) :
Measurable (collisionDesignCenter P : (Fin n → Obs d) → ℝ) := by
let design : (Fin n → Obs d) → (Fin n → Fin d × Bool) :=
fun s i => ((s i).x, (s i).a)
let f : (Fin n → Fin d × Bool) → ℝ := fun z =>
collisionDesignCenter P (fun i => ⟨(z i).1, (z i).2, 0⟩)
have hdesign : Measurable design := by
have hobs : Measurable (fun o : Obs d => (o.x, o.a, o.y)) :=
measurable_iff_comap_le.mpr le_rfl
have hx : Measurable (fun o : Obs d => o.x) := measurable_fst.comp hobs
have ha : Measurable (fun o : Obs d => o.a) :=
measurable_fst.comp (measurable_snd.comp hobs)
fun_prop
rw [show (collisionDesignCenter P : (Fin n → Obs d) → ℝ) = f ∘ design by
funext s
rfl]
exact (measurable_of_finite f).comp hdesign
theorem measurable_usableTotal
The usable total is a measurable finite-design statistic.
Formal statement
Proof (Lean source)
lemma measurable_usableTotal {n d : ℕ} :
Measurable (usableTotal : (Fin n → Obs d) → ℕ) := by
let design : (Fin n → Obs d) → (Fin n → Fin d × Bool) :=
fun s i => ((s i).x, (s i).a)
let f : (Fin n → Fin d × Bool) → ℕ := fun z =>
usableTotal (fun i => ⟨(z i).1, (z i).2, 0⟩)
have hdesign : Measurable design := by
have hobs : Measurable (fun o : Obs d => (o.x, o.a, o.y)) :=
measurable_iff_comap_le.mpr le_rfl
have hx : Measurable (fun o : Obs d => o.x) := measurable_fst.comp hobs
have ha : Measurable (fun o : Obs d => o.a) :=
measurable_fst.comp (measurable_snd.comp hobs)
fun_prop
rw [show (usableTotal : (Fin n → Obs d) → ℕ) = f ∘ design by
funext s
rfl]
exact (measurable_of_finite f).comp hdesign
theorem integral_collisionDesignCenter_bias_sq_le
Integrating the conditional-design squared bias introduces exactly the zero-usable-occupancy probability and no unsupported-cell contribution.
Formal statement
hypotheses
conclusion
(∫ sample, (collisionDesignCenter P sample - rawAteFormula P.law) ^ 2 ∂productLaw n P.law)
≤ M ^ 2 * (sigma ^ 2 + realMass (productLaw n P.law) {sample | usableTotal sample = 0})
Proof (Lean source)
-- @node: integral_collisionDesignCenter_bias_sq_le
lemma integral_collisionDesignCenter_bias_sq_le {n d : ℕ}
{epsilon M sigma : ℝ} (P : ModelClass d epsilon M sigma) :
(∫ sample, (collisionDesignCenter P sample - rawAteFormula P.law) ^ 2
∂productLaw n P.law) ≤
M ^ 2 * (sigma ^ 2 +
realMass (productLaw n P.law) {sample | usableTotal sample = 0}) := by
let μ := productLaw n P.law
let bad : Set (Fin n → Obs d) := {sample | usableTotal sample = 0}
let f : (Fin n → Obs d) → ℝ := fun sample =>
(collisionDesignCenter P sample - rawAteFormula P.law) ^ 2
let g : (Fin n → Obs d) → ℝ := fun sample =>
M ^ 2 * (sigma ^ 2 + if usableTotal sample = 0 then 1 else 0)
have hbad : MeasurableSet bad := by
exact (measurableSet_singleton 0).preimage measurable_usableTotal
have hfmeas : Measurable f := by
exact ((measurable_collisionDesignCenter P).sub measurable_const).pow_const 2
have hgmeas : Measurable g := by
apply measurable_const.mul
apply measurable_const.add
exact Measurable.ite
((measurableSet_singleton 0).preimage measurable_usableTotal)
measurable_const measurable_const
have hgint : Integrable g μ := by
refine (integrable_const (M ^ 2 * (sigma ^ 2 + 1) : ℝ)).mono
hgmeas.aestronglyMeasurable ?_
filter_upwards with sample
dsimp [g]
by_cases hs : usableTotal sample = 0
· rw [if_pos hs]
· rw [if_neg hs, add_zero]
rw [abs_of_nonneg (by positivity), abs_of_nonneg (by positivity)]
nlinarith [sq_nonneg M]
have hfle : f ≤ᵐ[μ] g := by
filter_upwards [usableCell_supported_ae P.law] with sample hs
exact collisionDesignCenter_bias_sq_le P sample hs
have hfint : Integrable f μ := by
refine hgint.mono hfmeas.aestronglyMeasurable ?_
filter_upwards [hfle] with sample hle
have hfnonneg : 0 ≤ f sample := by dsimp [f]; positivity
have hgnonneg : 0 ≤ g sample := hfnonneg.trans hle
simpa [Real.norm_eq_abs, abs_of_nonneg hfnonneg, abs_of_nonneg hgnonneg]
have hint := integral_mono_ae hfint hgint hfle
change (∫ sample, f sample ∂μ) ≤ _
calc
(∫ sample, f sample ∂μ) ≤ ∫ sample, g sample ∂μ := hint
_ = M ^ 2 * (sigma ^ 2 + realMass μ bad) := by
have hindicator : (fun sample : Fin n → Obs d =>
if usableTotal sample = 0 then (1 : ℝ) else 0) = bad.indicator 1 := by
funext sample
by_cases hs : usableTotal sample = 0 <;> simp [bad, hs]
rw [show g = fun sample =>
M ^ 2 * sigma ^ 2 + M ^ 2 *
(if usableTotal sample = 0 then 1 else 0) by
funext sample
dsimp [g]
ring]
rw [integral_add]
· rw [integral_const, integral_const_mul, hindicator,
integral_indicator_one hbad]
simp [μ, realMass]
change M ^ 2 * sigma ^ 2 + M ^ 2 * (μ bad).toReal =
M ^ 2 * (sigma ^ 2 + (μ bad).toReal)
ring
· exact integrable_const _
· have hi :=
((integrable_const (μ := μ) (1 : ℝ)).indicator hbad).const_mul (M ^ 2)
simpa [bad, indicator, mul_ite] using hi
_ = _ := by rfl
If the sample is nonempty, the de-Poissonization occupancy scale is bounded by the advertised parametric-plus-alphabet rate.
Formal statement
hypotheses
n d :
ℕ
hn :
0 < n
conclusion
((max n d : ℕ) : ℝ) / (n : ℝ) ^ 2 ≤ 1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2
Proof (Lean source)
lemma occupancy_max_rate_le {n d : ℕ} (hn : 0 < n) :
((max n d : ℕ) : ℝ) / (n : ℝ) ^ 2 ≤
1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2 := by
have hnR : (0 : ℝ) < n := by exact_mod_cast hn
rw [Nat.cast_max]
by_cases hnd : n ≤ d
· rw [max_eq_right (by exact_mod_cast hnd)]
have hone : 0 ≤ 1 / (n : ℝ) := by positivity
linarith
· have hdn : d ≤ n := Nat.le_of_not_ge hnd
rw [max_eq_left (by exact_mod_cast hdn)]
have hne : (n : ℝ) ≠ 0 := hnR.ne'
have hrewrite : (n : ℝ) / (n : ℝ) ^ 2 = 1 / (n : ℝ) := by
field_simp
rw [hrewrite]
have hdnonneg : 0 ≤ (d : ℝ) / (n : ℝ) ^ 2 := by positivity
linarith
theorem exp_neg_div_absorbed_by_linear_rate Lemma exp_neg_div_absorbed_by_linear_rate in the paper ↗
If the radial cap satisfies its stated bound and the scalar satisfies the stated range condition, an inverse-scale exponential tail is absorbed by a constant multiple of the scale, uniformly over every positive scale.
Formal statement
Proof (Lean source)
lemma exp_neg_div_absorbed_by_linear_rate {b x : ℝ} (hb : 0 < b) (hx : 0 < x) :
exp (-b / x) ≤ (1 / b) * x := by
let y := b / x
have hy : 0 < y := div_pos hb hx
have hy_exp : y ≤ exp y :=
le_trans (by linarith : y ≤ y + 1) (Real.add_one_le_exp y)
have hinv : (exp y)⁻¹ ≤ y⁻¹ :=
(inv_le_inv₀ (Real.exp_pos y) hy).2 hy_exp
have harg : -b / x = -y := by
dsimp [y]
ring
rw [harg, Real.exp_neg]
calc
(exp y)⁻¹ ≤ y⁻¹ := hinv
_ = (1 / b) * x := by
dsimp [y]
field_simp
theorem observed_arm_cell_centered_integral_eq_zero
Within one supported arm and cell, the observed centered outcome has zero integral. The statement is totalized over zero-mass cells, where the arm-cell event is null.
Formal statement
hypotheses
conclusion
(∫ o in {o : Obs d | o.x = k ∧ o.a = a}, (o.y - P.law.outcomeMean a k) ∂P.law.observedLaw)
= 0
Proof (Lean source)
lemma observed_arm_cell_centered_integral_eq_zero {d : ℕ}
{epsilon M sigma : ℝ} (P : ModelClass d epsilon M sigma)
(a : Bool) (k : Fin d) :
(∫ o in {o : Obs d | o.x = k ∧ o.a = a},
(o.y - P.law.outcomeMean a k) ∂P.law.observedLaw) = 0 := by
let E : Set (Obs d) := {o | o.x = k ∧ o.a = a}
have htuple : Measurable (fun o : Obs d => (o.x, o.a, o.y)) :=
measurable_iff_comap_le.mpr le_rfl
have hy : Measurable (fun o : Obs d => o.y) :=
measurable_snd.comp (measurable_snd.comp htuple)
by_cases hk : P.law.cellMass k = 0
· have hnull := observed_arm_cell_measure_eq_zero_of_cellMass_eq_zero
P.law a k hk
exact MeasureTheory.setIntegral_measure_zero _ hnull
· have hkpos : 0 < P.law.cellMass k :=
lt_of_le_of_ne (P.law.cellMass_range k).1 (Ne.symm hk)
let _ : IsProbabilityMeasure (P.law.outcomeLaw a k) :=
P.law.outcome_isProbability a k
have hc := (P.second_moment a k hkpos).1
have hcLp : MemLp (fun y : ℝ => y - P.law.outcomeMean a k) 2
(P.law.outcomeLaw a k) :=
(memLp_two_iff_integrable_sq
((measurable_id.sub measurable_const).aestronglyMeasurable)).2 hc
have hcInt : Integrable (fun y : ℝ => y - P.law.outcomeMean a k)
(P.law.outcomeLaw a k) := hcLp.integrable (by norm_num)
have hyInt : Integrable (fun y : ℝ => y) (P.law.outcomeLaw a k) := by
refine (hcInt.add (integrable_const (P.law.outcomeMean a k))).congr ?_
filter_upwards with y
simp
have hmap := observed_arm_cell_outcome_measure P.law a k
have hcenter : (∫ y, y - P.law.outcomeMean a k
∂P.law.outcomeLaw a k) = 0 := by
rw [integral_sub hyInt
(integrable_const (P.law.outcomeMean a k)), integral_const,
← P.law.outcomeMean_eq]
simp
calc
(∫ o in {o : Obs d | o.x = k ∧ o.a = a},
(o.y - P.law.outcomeMean a k) ∂P.law.observedLaw) =
∫ y, y - P.law.outcomeMean a k
∂Measure.map (fun o : Obs d => o.y)
(P.law.observedLaw.restrict E) := by
convert (integral_map hy.aemeasurable
(measurable_id.sub measurable_const).aestronglyMeasurable).symm using 1 <;>
rfl
_ = ∫ y, y - P.law.outcomeMean a k ∂(
ofReal (P.law.cellMass k *
(if a then P.law.propensity k else 1 - P.law.propensity k)) •
P.law.outcomeLaw a k) := by rw [hmap]
_ = 0 := by rw [integral_smul_measure, hcenter]; simp
theorem observed_arm_cell_centered_sq_integral_le
The observed centered second moment on an arm-cell event is bounded by its arm-cell probability times M², including the null-cell boundary.
Formal statement
hypotheses
conclusion
(∫ o in {o : Obs d | o.x = k ∧ o.a = a}, (o.y - P.law.outcomeMean a k) ^ 2 ∂P.law.observedLaw)
≤ P.law.cellMass k * (if a then P.law.propensity k else 1 - P.law.propensity k) * M ^ 2
Proof (Lean source)
lemma observed_arm_cell_centered_sq_integral_le {d : ℕ}
{epsilon M sigma : ℝ} (P : ModelClass d epsilon M sigma)
(a : Bool) (k : Fin d) :
(∫ o in {o : Obs d | o.x = k ∧ o.a = a},
(o.y - P.law.outcomeMean a k) ^ 2 ∂P.law.observedLaw) ≤
P.law.cellMass k *
(if a then P.law.propensity k else 1 - P.law.propensity k) * M ^ 2 := by
let E : Set (Obs d) := {o | o.x = k ∧ o.a = a}
have htuple : Measurable (fun o : Obs d => (o.x, o.a, o.y)) :=
measurable_iff_comap_le.mpr le_rfl
have hy : Measurable (fun o : Obs d => o.y) :=
measurable_snd.comp (measurable_snd.comp htuple)
have hq : 0 ≤ if a then P.law.propensity k else 1 - P.law.propensity k := by
split
· exact (P.law.propensity_range k).1
· linarith [(P.law.propensity_range k).2]
by_cases hk : P.law.cellMass k = 0
· have hnull := observed_arm_cell_measure_eq_zero_of_cellMass_eq_zero
P.law a k hk
rw [MeasureTheory.setIntegral_measure_zero _ hnull, hk, zero_mul]
positivity
· have hkpos : 0 < P.law.cellMass k :=
lt_of_le_of_ne (P.law.cellMass_range k).1 (Ne.symm hk)
have hc := P.second_moment a k hkpos
have hmap := observed_arm_cell_outcome_measure P.law a k
have hscale : 0 ≤ P.law.cellMass k *
(if a then P.law.propensity k else 1 - P.law.propensity k) :=
mul_nonneg hkpos.le hq
calc
(∫ o in {o : Obs d | o.x = k ∧ o.a = a},
(o.y - P.law.outcomeMean a k) ^ 2 ∂P.law.observedLaw) =
∫ y, (y - P.law.outcomeMean a k) ^ 2
∂Measure.map (fun o : Obs d => o.y)
(P.law.observedLaw.restrict E) := by
convert (integral_map hy.aemeasurable
((measurable_id.sub measurable_const).pow_const 2).aestronglyMeasurable).symm using 1 <;>
rfl
_ = ∫ y, (y - P.law.outcomeMean a k) ^ 2 ∂(
ofReal (P.law.cellMass k *
(if a then P.law.propensity k else 1 - P.law.propensity k)) •
P.law.outcomeLaw a k) := by rw [hmap]
_ = (P.law.cellMass k *
(if a then P.law.propensity k else 1 - P.law.propensity k)) *
(∫ y, (y - P.law.outcomeMean a k) ^ 2
∂P.law.outcomeLaw a k) := by
rw [integral_smul_measure]
rw [ENNReal.toReal_ofReal hscale]
rfl
_ ≤ P.law.cellMass k *
(if a then P.law.propensity k else 1 - P.law.propensity k) * M ^ 2 :=
mul_le_mul_of_nonneg_left hc.2 hscale
Helpers.OccupancyUpperAssembly 11 declarations
theorem test_groupArm
the test sample's arm-cell count agrees with the generic grouped count.
Formal statement
hypotheses
conclusion
groupArmCount (fun o : Obs d => o.x) (fun o => o.a) sample a k = armCount sample a k
Proof (Lean source)
lemma test_groupArm {n d : ℕ} (sample : Fin n → Obs d) (a : Bool) (k : Fin d) :
groupArmCount (fun o : Obs d => o.x) (fun o => o.a) sample a k =
armCount sample a k := by
rfl
theorem test_group
the sum of arm-specific group counts equals the total count for the cell.
Formal statement
hypotheses
conclusion
groupCount (fun o : Obs d => o.x) (fun o => o.a) sample k = cellCount sample k
Proof (Lean source)
lemma test_group {n d : ℕ} (sample : Fin n → Obs d) (k : Fin d) :
groupCount (fun o : Obs d => o.x) (fun o => o.a) sample k =
cellCount sample k := by
rfl
theorem test_group_total
summing the test sample's cell counts gives the sample size.
Formal statement
hypotheses
n d :
ℕ
sample :
Fin n → Obs d
conclusion
usableGroupTotal (fun o : Obs d => o.x) (fun o => o.a) sample = usableTotal sample
Proof (Lean source)
lemma test_group_total {n d : ℕ} (sample : Fin n → Obs d) :
usableGroupTotal (fun o : Obs d => o.x) (fun o => o.a) sample =
usableTotal sample := by
unfold usableGroupTotal usableTotal
apply Finset.sum_congr rfl
intro k _
rw [show usableGroup (fun o : Obs d => o.x) (fun o => o.a) sample k ↔
usableCell sample k by
simp only [usableGroup, usableCell, test_groupArm, decide_eq_true_eq]]
simp only [test_group]
def testUnclipped
This is the untruncated occupancy estimator before final clipping.
Definition (Lean source)
noncomputable def testUnclipped {n d : ℕ} (sample : Fin n → Obs d) : ℝ :=
if 0 < usableTotal sample then
(∑ k : Fin d, if usableCell sample k then
(cellCount sample k : ℝ) *
(empiricalArmMean sample true k - empiricalArmMean sample false k)
else 0) / usableTotal sample
else 0
theorem test_armResidualMean
If the stated count condition holds, the arm-specific residual contribution has mean zero.
Formal statement
hypotheses
conclusion
armResidualMean (fun o : Obs d => o.x) (fun o => o.a) (fun o => o.y) P.outcomeMean sample a k
= empiricalArmMean sample a k - P.outcomeMean a k
Proof (Lean source)
lemma test_armResidualMean {n d : ℕ} (P : RealLaw d)
(sample : Fin n → Obs d) (a : Bool) (k : Fin d)
(hcount : 0 < armCount sample a k) :
armResidualMean (fun o : Obs d => o.x) (fun o => o.a) (fun o => o.y)
P.outcomeMean sample a k =
empiricalArmMean sample a k - P.outcomeMean a k := by
rw [armResidualMean, if_pos]
· rw [empiricalArmMean, if_pos hcount]
unfold armResidualSum supportedArmGroupResidual armGroupResidual armGroupEvent armSum
simp only [Set.indicator_apply]
have hcountR : (armCount sample a k : ℝ) ≠ 0 := by exact_mod_cast hcount.ne'
simp only [test_groupArm]
have hsum : (∑ i : Fin n, if sample i ∈
{omega : Obs d | omega.x = k ∧ omega.a = a}
then (sample i).y - P.outcomeMean a k else 0) =
(∑ i : Fin n, if (sample i).x = k ∧ (sample i).a = a
then (sample i).y else 0) -
(armCount sample a k : ℝ) * P.outcomeMean a k := by
unfold armCount
simp only [Set.mem_ofPred_eq]
rw [← Finset.sum_filter, ← Finset.sum_filter]
rw [Finset.sum_sub_distrib, Finset.sum_const, nsmul_eq_mul]
rw [hsum]
field_simp
· simpa only [test_groupArm] using hcount
theorem testUnclipped_sub_center_eq
the untruncated occupancy estimator minus its design center equals the sum of its arm-specific residual contributions.
Formal statement
hypotheses
conclusion
testUnclipped sample - collisionDesignCenter P sample
= occupancyWeightedResidual P.law.observedLaw (fun o : Obs d => o.x) (fun o => o.a) (fun o => o.y) P.law.outcomeMean sample
Proof (Lean source)
lemma testUnclipped_sub_center_eq {n d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (sample : Fin n → Obs d) :
testUnclipped sample - collisionDesignCenter P sample =
occupancyWeightedResidual P.law.observedLaw
(fun o : Obs d => o.x) (fun o => o.a) (fun o => o.y)
P.law.outcomeMean sample := by
unfold testUnclipped collisionDesignCenter occupancyWeightedResidual
rw [test_group_total]
by_cases ht : 0 < usableTotal sample
· simp only [if_pos ht]
rw [div_eq_inv_mul, div_eq_inv_mul]
rw [← mul_sub]
congr 1
rw [← Finset.sum_sub_distrib]
apply Finset.sum_congr rfl
intro k _
have hu : usableGroup (fun o : Obs d => o.x) (fun o => o.a) sample k ↔
usableCell sample k := by
simp only [usableGroup, usableCell, test_groupArm, decide_eq_true_eq]
rw [hu]
by_cases huk : usableCell sample k
· rw [if_pos huk, if_pos huk, test_group]
have hcounts : 0 < armCount sample false k ∧ 0 < armCount sample true k := by
simpa [usableCell] using huk
rw [test_armResidualMean P.law sample true k hcounts.2,
test_armResidualMean P.law sample false k hcounts.1]
simp only [if_pos huk]
unfold cellEffect
ring
· simp [huk]
· simp only [if_neg ht]
ring
theorem test_arm_mass
the integral of an arm indicator equals its treatment-arm probability.
Formal statement
hypotheses
conclusion
realMass P.observedLaw (armGroupEvent (fun o : Obs d => o.x) (fun o => o.a) a k)
= P.cellMass k * (if a then P.propensity k else 1 - P.propensity k)
Proof (Lean source)
lemma test_arm_mass {d : ℕ} (P : RealLaw d) (a : Bool) (k : Fin d) :
realMass P.observedLaw (armGroupEvent (fun o : Obs d => o.x)
(fun o => o.a) a k) =
P.cellMass k * (if a then P.propensity k else 1 - P.propensity k) := by
let _ : IsProbabilityMeasure (P.outcomeLaw a k) := P.outcome_isProbability a k
have h := P.arm_outcome_factorization a k univ MeasurableSet.univ
simp [realMass] at h
simpa [armGroupEvent, realMass] using h.symm
theorem test_memLp
the untruncated occupancy estimator has a finite second moment.
Formal statement
hypotheses
conclusion
MemLp (supportedArmGroupResidual (fun o : Obs d => o.x) (fun o => o.a) (fun o => o.y) P.law.outcomeMean a k) 2 P.law.observedLaw
Proof (Lean source)
lemma test_memLp {d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (a : Bool) (k : Fin d) :
MemLp (supportedArmGroupResidual (fun o : Obs d => o.x) (fun o => o.a)
(fun o => o.y) P.law.outcomeMean a k) 2 P.law.observedLaw := by
have hM : M ≠ 0 := ne_of_gt (lt_of_lt_of_le (by norm_num) P.M_ge_one)
have hy : MemLp (fun o : Obs d => o.y) 2 P.law.observedLaw := by
have h := (memLp_two_observed_normalized_outcome P).const_mul M
convert h using 1
funext o
field_simp
have hr : MemLp (fun o : Obs d => o.y - P.law.outcomeMean a k) 2
P.law.observedLaw := hy.sub (memLp_const _)
have htuple : Measurable (fun o : Obs d => (o.x, o.a, o.y)) :=
measurable_iff_comap_le.mpr le_rfl
have hx : Measurable (fun o : Obs d => o.x) := measurable_fst.comp htuple
have ha : Measurable (fun o : Obs d => o.a) :=
measurable_fst.comp (measurable_snd.comp htuple)
have hevent := measurableSet_armGroupEvent (fun o : Obs d => o.x)
(fun o => o.a) hx ha a k
change MemLp ((armGroupEvent (fun o : Obs d => o.x) (fun o => o.a) a k).indicator
(fun o => o.y - P.law.outcomeMean a k)) 2 P.law.observedLaw
exact hr.indicator hevent
theorem test_center_memLp
the centered occupancy estimator has a finite second moment.
Formal statement
hypotheses
conclusion
MemLp (fun sample : Fin n → Obs d => collisionDesignCenter P sample - rawAteFormula P.law) 2 (productLaw n P.law)
Proof (Lean source)
lemma test_center_memLp {n d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) :
MemLp (fun sample : Fin n → Obs d =>
collisionDesignCenter P sample - rawAteFormula P.law) 2
(productLaw n P.law) := by
have hm : Measurable (fun sample : Fin n → Obs d =>
collisionDesignCenter P sample - rawAteFormula P.law) :=
(measurable_collisionDesignCenter P).sub measurable_const
apply MemLp.of_bound hm.aestronglyMeasurable
(M * sqrt (sigma ^ 2 + 1))
filter_upwards [usableCell_supported_ae P.law] with sample hs
have hsq := collisionDesignCenter_bias_sq_le P sample hs
rw [Real.norm_eq_abs]
have hnonneg : 0 ≤ sigma ^ 2 + 1 := by positivity
have hsqrt : 0 ≤ sqrt (sigma ^ 2 + 1) := Real.sqrt_nonneg _
have hM : 0 ≤ M := le_trans zero_le_one P.M_ge_one
have hboundnonneg : 0 ≤ M * sqrt (sigma ^ 2 + 1) := mul_nonneg hM hsqrt
apply (sq_le_sq₀ (abs_nonneg _) hboundnonneg).mp
rw [sq_abs, mul_pow, Real.sq_sqrt hnonneg]
have hfactor : sigma ^ 2 + (if usableTotal sample = 0 then 1 else 0) ≤
sigma ^ 2 + 1 := by
split <;> simp
exact hsq.trans (mul_le_mul_of_nonneg_left hfactor (sq_nonneg M))
for every alphabet size, the collision estimator achieves the stated continuous-outcome risk upper bound.
Formal statement
conclusion
∀ epsilon : ℝ,
0 < epsilon → epsilon < 1 / 2 → ∃ C_epsilon : ℝ, 0 < C_epsilon ∧ ∀ n d : ℕ, ∀ M sigma : ℝ, 0 < n → 0 < d → ∀ P : ModelClass d epsilon M sigma, mse P.law (collisionEstimator (n := n) (d := d) M)
≤ C_epsilon * M ^ 2 * (1 / (n : ℝ) + sigma ^ 2 + (d : ℝ) / (n : ℝ) ^ 2)
Proof (Lean source)
lemma continuous_occupancy_collision_upper_all_d :
∀ epsilon : ℝ, 0 < epsilon → epsilon < 1 / 2 →
∃ C_epsilon : ℝ, 0 < C_epsilon ∧
∀ n d : ℕ, ∀ M sigma : ℝ,
0 < n → 0 < d →
∀ P : ModelClass d epsilon M sigma,
mse P.law (collisionEstimator (n := n) (d := d) M) ≤
C_epsilon * M ^ 2 *
(1 / (n : ℝ) + sigma ^ 2 + (d : ℝ) / (n : ℝ) ^ 2) := by
intro epsilon hepsilon hepsilon_half
rcases zengUsableOccupancyReciprocal epsilon with
⟨b, B, hb, hB, hoccupancy⟩
let K : ℝ := 16 / (epsilon ^ 2 * (1 - epsilon))
let C : ℝ := 2 * K * B + (2 * K * B + 4) / b + 2
have hK : 0 < K := by
dsimp [K]
have : 0 < 1 - epsilon := by linarith
positivity
have hC : 0 < C := by
dsimp [C]
positivity
refine ⟨C, hC, ?_⟩
intro n d M sigma hn hd P
have hM : 0 ≤ M := le_trans zero_le_one P.M_ge_one
have htuple : Measurable (fun o : Obs d => (o.x, o.a, o.y)) :=
measurable_iff_comap_le.mpr le_rfl
have hx : Measurable (fun o : Obs d => o.x) := measurable_fst.comp htuple
have ha : Measurable (fun o : Obs d => o.a) :=
measurable_fst.comp (measurable_snd.comp htuple)
have hy : Measurable (fun o : Obs d => o.y) :=
measurable_snd.comp (measurable_snd.comp htuple)
have hmem : ∀ a k, MemLp
(supportedArmGroupResidual (fun o : Obs d => o.x) (fun o => o.a)
(fun o => o.y) P.law.outcomeMean a k) 2 P.law.observedLaw :=
fun a k => test_memLp P a k
have hcenter : ∀ a k,
∫ o in armGroupEvent (fun o : Obs d => o.x) (fun o => o.a) a k,
armGroupResidual (fun o : Obs d => o.y) P.law.outcomeMean a k o
∂P.law.observedLaw = 0 := by
intro a k
simpa [armGroupEvent, armGroupResidual] using
observed_arm_cell_centered_integral_eq_zero P a k
have hsq : ∀ a k,
∫ o in armGroupEvent (fun o : Obs d => o.x) (fun o => o.a) a k,
(armGroupResidual (fun o : Obs d => o.y) P.law.outcomeMean a k o) ^ 2
∂P.law.observedLaw ≤
(P.law.observedLaw
(armGroupEvent (fun o : Obs d => o.x) (fun o => o.a) a k)).toReal * M ^ 2 := by
intro a k
rw [show (P.law.observedLaw
(armGroupEvent (fun o : Obs d => o.x) (fun o => o.a) a k)).toReal =
P.law.cellMass k * (if a then P.law.propensity k else 1 - P.law.propensity k) by
simpa [realMass] using test_arm_mass P.law a k]
simpa [armGroupEvent, armGroupResidual] using
observed_arm_cell_centered_sq_integral_le P a k
have hoverlap : ∀ k,
0 < (P.law.observedLaw
(groupEvent (fun o : Obs d => o.x) k)).toReal → ∀ a,
epsilon * (P.law.observedLaw
(groupEvent (fun o : Obs d => o.x) k)).toReal ≤
(P.law.observedLaw
(armGroupEvent (fun o : Obs d => o.x) (fun o => o.a) a k)).toReal := by
intro k hk a
have hg : (P.law.observedLaw
(groupEvent (fun o : Obs d => o.x) k)).toReal = P.law.cellMass k := by
simpa [groupEvent, realMass] using (P.law.cellMass_eq k).symm
rw [hg, show (P.law.observedLaw
(armGroupEvent (fun o : Obs d => o.x) (fun o => o.a) a k)).toReal =
P.law.cellMass k * (if a then P.law.propensity k else 1 - P.law.propensity k) by
simpa [realMass] using test_arm_mass P.law a k]
have hov := P.overlap k (by simpa [hg] using hk)
cases a <;> simp only [Bool.false_eq_true, ↓reduceIte] <;>
nlinarith [P.law.cellMass_range k]
have hnoise0 := integral_occupancyWeightedResidual_sq_le_reciprocal
(n := n) P.law.observedLaw (fun o : Obs d => o.x) (fun o => o.a)
(fun o => o.y) P.law.outcomeMean M epsilon hx ha hy hmem hcenter hsq
hepsilon hepsilon_half.le hoverlap
have hnoise :
(∫ s, (testUnclipped s - collisionDesignCenter P s) ^ 2 ∂productLaw n P.law) ≤
K * M ^ 2 *
(∫ s, (if 0 < usableTotal s then (1 : ℝ) / usableTotal s else 0)
∂productLaw n P.law) := by
rw [show productLaw n P.law = Measure.pi (fun _ : Fin n => P.law.observedLaw) from rfl]
simpa only [test_group_total, one_div, K, testUnclipped_sub_center_eq] using hnoise0
have hbias := integral_collisionDesignCenter_bias_sq_le (n := n) P
have hocc := hoccupancy n d M sigma P
let x : ℝ := (max n d : ℕ) / (n : ℝ) ^ 2
have hxpos : 0 < x := by dsimp [x]; positivity
have hexpeq : -b * (n : ℝ) ^ 2 / (max n d : ℕ) = -b / x := by
dsimp [x]
field_simp
have hexp : exp (-b * (n : ℝ) ^ 2 / (max n d : ℕ)) ≤ (1 / b) * x := by
rw [hexpeq]
exact exp_neg_div_absorbed_by_linear_rate hb hxpos
have hxrate := occupancy_max_rate_le (d := d) hn
have hnoise' :
(∫ s, (testUnclipped s - collisionDesignCenter P s) ^ 2 ∂productLaw n P.law) ≤
K * M ^ 2 * B * (x + exp (-b * (n : ℝ) ^ 2 / (max n d : ℕ))) := by
have hocc2 : (∫ s, (if 0 < usableTotal s then (1 : ℝ) / usableTotal s else 0)
∂productLaw n P.law) ≤
B * (x + exp (-b * (n : ℝ) ^ 2 / (max n d : ℕ))) := by
simpa [x] using hocc.2
calc
_ ≤ K * M ^ 2 * (B * (x + exp (-b * (n : ℝ) ^ 2 / (max n d : ℕ)))) :=
hnoise.trans (mul_le_mul_of_nonneg_left hocc2
(mul_nonneg hK.le (sq_nonneg M)))
_ = _ := by ring
have hbias' :
(∫ s, (collisionDesignCenter P s - rawAteFormula P.law) ^ 2
∂productLaw n P.law) ≤
M ^ 2 * (sigma ^ 2 + 2 * exp (-b * (n : ℝ) ^ 2 / (max n d : ℕ))) := by
exact hbias.trans (mul_le_mul_of_nonneg_left
(add_le_add (le_refl (sigma ^ 2)) hocc.1) (sq_nonneg M))
have hresLp := occupancyWeightedResidual_memLp_two
(n := n) P.law.observedLaw (fun o : Obs d => o.x) (fun o => o.a)
(fun o => o.y) P.law.outcomeMean hx ha hy hmem
have hnoiseLp : MemLp (fun s : Fin n → Obs d =>
testUnclipped s - collisionDesignCenter P s) 2 (productLaw n P.law) := by
rw [show productLaw n P.law = Measure.pi (fun _ : Fin n => P.law.observedLaw) from rfl]
convert hresLp using 1
funext s
exact testUnclipped_sub_center_eq P s
have hrawLp : MemLp (fun s : Fin n → Obs d =>
testUnclipped s - rawAteFormula P.law) 2 (productLaw n P.law) := by
convert hnoiseLp.add (test_center_memLp (n := n) P) using 1
funext s
change testUnclipped s - rawAteFormula P.law =
(testUnclipped s - collisionDesignCenter P s) +
(collisionDesignCenter P s - rawAteFormula P.law)
ring
have hrawSq := hrawLp.integrable_sq
have hclipMeas := (collisionEstimator_admissible (n := n) (d := d) hM).1
have hclipLp : MemLp (fun s : Fin n → Obs d =>
collisionEstimator M s - rawAteFormula P.law) 2 (productLaw n P.law) := by
apply hrawLp.of_le (hclipMeas.sub measurable_const).aestronglyMeasurable
filter_upwards with s
rw [Real.norm_eq_abs, Real.norm_eq_abs]
have ht : rawAteFormula P.law ∈ Icc (-M) M := by
have habs := modelClass_rawAte_abs_le P
exact (abs_le.mp habs)
simpa [collisionEstimator, clip, testUnclipped,
clipIcc] using
abs_clipIcc_sub_le ht (testUnclipped s)
have hclipSq := hclipLp.integrable_sq
have hclipRaw : mse P.law (collisionEstimator (n := n) (d := d) M) ≤
∫ s, (testUnclipped s - rawAteFormula P.law) ^ 2 ∂productLaw n P.law := by
unfold mse
apply integral_mono hclipSq hrawSq
intro s
have ht : rawAteFormula P.law ∈ Icc (-M) M := by
have habs := modelClass_rawAte_abs_le P
exact (abs_le.mp habs)
simpa [collisionEstimator, clip, testUnclipped,
clipIcc] using
clipIcc_sub_sq_le ht (testUnclipped s)
calc
mse P.law (collisionEstimator M) ≤
∫ s, (testUnclipped s - rawAteFormula P.law) ^ 2 ∂productLaw n P.law := hclipRaw
_ ≤ 2 * (∫ s, (testUnclipped s - collisionDesignCenter P s) ^ 2 ∂productLaw n P.law) +
2 * (∫ s, (collisionDesignCenter P s - rawAteFormula P.law) ^ 2 ∂productLaw n P.law) := by
rw [← integral_const_mul, ← integral_const_mul, ← integral_add]
· apply integral_mono hrawSq
((hnoiseLp.integrable_sq.const_mul 2).add
((test_center_memLp (n := n) P).integrable_sq.const_mul 2))
intro s
change (testUnclipped s - rawAteFormula P.law) ^ 2 ≤
2 * (testUnclipped s - collisionDesignCenter P s) ^ 2 +
2 * (collisionDesignCenter P s - rawAteFormula P.law) ^ 2
nlinarith [sq_nonneg (testUnclipped s - collisionDesignCenter P s -
(collisionDesignCenter P s - rawAteFormula P.law))]
· exact hnoiseLp.integrable_sq.const_mul 2
· exact (test_center_memLp (n := n) P).integrable_sq.const_mul 2
_ ≤ 2 * (K * M ^ 2 * B * (x + exp (-b * (n : ℝ) ^ 2 / (max n d : ℕ)))) +
2 * (M ^ 2 * (sigma ^ 2 + 2 * exp (-b * (n : ℝ) ^ 2 / (max n d : ℕ)))) :=
add_le_add (mul_le_mul_of_nonneg_left hnoise' (by norm_num))
(mul_le_mul_of_nonneg_left hbias' (by norm_num))
_ ≤ C * M ^ 2 * (1 / (n : ℝ) + sigma ^ 2 + (d : ℝ) / (n : ℝ) ^ 2) := by
have hsig : 0 ≤ sigma ^ 2 := sq_nonneg sigma
have hrate : 0 ≤ 1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2 := by positivity
have hKB : 0 ≤ K * B := (mul_pos hK hB).le
have hMb : 0 ≤ (2 * K * B + 4) / b := by positivity
have hcoef : 0 ≤ 2 * K * B + 4 := by positivity
have hexpCoef := mul_le_mul_of_nonneg_left hexp hcoef
have hD : 0 ≤ 2 * K * B + (2 * K * B + 4) / b := by positivity
have hxrate' : x ≤ 1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2 := by
simpa [x] using hxrate
have hxD := mul_le_mul_of_nonneg_left hxrate' hD
have hinter :
2 * K * B * (x + exp (-b * (n : ℝ) ^ 2 / (max n d : ℕ))) +
2 * (sigma ^ 2 + 2 * exp (-b * (n : ℝ) ^ 2 / (max n d : ℕ))) ≤
C * (1 / (n : ℝ) + sigma ^ 2 + (d : ℝ) / (n : ℝ) ^ 2) := by
calc
_ = 2 * K * B * x + (2 * K * B + 4) *
exp (-b * (n : ℝ) ^ 2 / (max n d : ℕ)) + 2 * sigma ^ 2 := by ring
_ ≤ 2 * K * B * x + (2 * K * B + 4) * ((1 / b) * x) +
2 * sigma ^ 2 := by linarith
_ = (2 * K * B + (2 * K * B + 4) / b) * x + 2 * sigma ^ 2 := by ring
_ ≤ (2 * K * B + (2 * K * B + 4) / b) *
(1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2) + 2 * sigma ^ 2 := by
linarith
_ ≤ C * (1 / (n : ℝ) + sigma ^ 2 + (d : ℝ) / (n : ℝ) ^ 2) := by
dsimp [C]
nlinarith [mul_nonneg hD hrate, mul_nonneg (by norm_num : (0 : ℝ) ≤ 2) hsig]
have := mul_le_mul_of_nonneg_left hinter (sq_nonneg M)
nlinarith
theorem continuous_occupancy_collision_upper
Restricted-range form of the occupancy estimator upper bound.
Formal statement
conclusion
∀ epsilon : ℝ,
0 < epsilon → epsilon < 1 / 2 → ∃ C_epsilon c_epsilon : ℝ, 0 < C_epsilon ∧ 0 < c_epsilon ∧ ∀ n d : ℕ, ∀ M sigma : ℝ, 0 < n → 0 < d → (d : ℝ)
≤ c_epsilon * (n : ℝ) ^ 2 / logEN n → ∀ P : ModelClass d epsilon M sigma, mse P.law (collisionEstimator (n := n) (d := d) M) ≤ C_epsilon * M ^ 2 * (1 / (n : ℝ) + sigma ^ 2 + (d : ℝ) / (n : ℝ) ^ 2)
Proof (Lean source)
lemma continuous_occupancy_collision_upper :
∀ epsilon : ℝ, 0 < epsilon → epsilon < 1 / 2 →
∃ C_epsilon c_epsilon : ℝ, 0 < C_epsilon ∧ 0 < c_epsilon ∧
∀ n d : ℕ, ∀ M sigma : ℝ,
0 < n → 0 < d →
(d : ℝ) ≤ c_epsilon * (n : ℝ) ^ 2 / logEN n →
∀ P : ModelClass d epsilon M sigma,
mse P.law (collisionEstimator (n := n) (d := d) M) ≤
C_epsilon * M ^ 2 *
(1 / (n : ℝ) + sigma ^ 2 + (d : ℝ) / (n : ℝ) ^ 2) := by
intro epsilon hepsilon hepsilon_half
obtain ⟨C_epsilon, hC, hbound⟩ :=
continuous_occupancy_collision_upper_all_d epsilon hepsilon hepsilon_half
refine ⟨C_epsilon, 1, hC, zero_lt_one, ?_⟩
intro n d M sigma hn hd _ P
exact hbound n d M sigma hn hd P
Helpers.OneArmLowerDischarge 1 declarations This module exposes the proved binary one-arm lower bound in the local heterogeneity-frontier namespace.
One-arm lower-bound discharge
This module exposes the proved binary one-arm lower bound in the local heterogeneity-frontier namespace.
The proved binary one-arm minimax result discharges the local cited interface.
Formal statement
Proof (Lean source)
theorem zengOneArmMinimaxLower (epsilon : ℝ) :
ZengOneArmMinimaxLower epsilon := by
exact CausalSmith.Stat.DiscreteAteMinimaxLoggap.zengOneArmMinimaxLower epsilon
Helpers.ParametricLower 42 declarations
def testFullAtom
This is the full-data atom used in the parametric two-point lower-bound experiment.
Definition (Lean source)
noncomputable def testFullAtom {d : ℕ} (k : Fin d) (a : Bool) (y : ℝ) : FullObs d :=
⟨k, a, 0, y, if a then y else 0⟩
theorem measurable_testFullAtom
the parametric full-data atom is measurable as a function of its outcome.
Formal statement
Proof (Lean source)
lemma measurable_testFullAtom {d : ℕ} (k : Fin d) (a : Bool) :
Measurable (testFullAtom k a) := by
cases a
· rw [measurable_comap_iff]
change Measurable (fun y : ℝ => (k, false, 0, y, 0))
fun_prop
· rw [measurable_comap_iff]
change Measurable (fun y : ℝ => (k, true, 0, y, y))
fun_prop
theorem measurable_fullObserved
the projection from full data to observed data is measurable.
Formal statement
Proof (Lean source)
lemma measurable_fullObserved {d : ℕ} : Measurable (FullObs.observed : FullObs d → Obs d) := by
have htuple : Measurable (fun z : FullObs d => (z.x, z.a, z.y0, z.y1, z.y)) := by
rw [measurable_iff_comap_le]
rfl
rw [measurable_comap_iff]
change Measurable (fun z : FullObs d => (z.x, z.a, z.y))
have hx : Measurable (fun z : FullObs d => z.x) := measurable_fst.comp htuple
have ha : Measurable (fun z : FullObs d => z.a) :=
measurable_fst.comp (measurable_snd.comp htuple)
have hy : Measurable (fun z : FullObs d => z.y) :=
measurable_snd.comp (measurable_snd.comp (measurable_snd.comp
(measurable_snd.comp htuple)))
exact Measurable.prod hx (Measurable.prod ha hy)
theorem measurable_full_x
the full-data covariate coordinate is measurable.
Formal statement
Proof (Lean source)
lemma measurable_full_x {d : ℕ} : Measurable (fun z : FullObs d => z.x) := by
have h : Measurable (fun z : FullObs d => (z.x, z.a, z.y0, z.y1, z.y)) := by
rw [measurable_iff_comap_le]; rfl
exact measurable_fst.comp h
theorem measurable_full_a
the full-data treatment coordinate is measurable.
Formal statement
Proof (Lean source)
lemma measurable_full_a {d : ℕ} : Measurable (fun z : FullObs d => z.a) := by
have h : Measurable (fun z : FullObs d => (z.x, z.a, z.y0, z.y1, z.y)) := by
rw [measurable_iff_comap_le]; rfl
exact measurable_fst.comp (measurable_snd.comp h)
theorem measurable_full_y0
the control potential-outcome coordinate is measurable.
Formal statement
Proof (Lean source)
lemma measurable_full_y0 {d : ℕ} : Measurable (fun z : FullObs d => z.y0) := by
have h : Measurable (fun z : FullObs d => (z.x, z.a, z.y0, z.y1, z.y)) := by
rw [measurable_iff_comap_le]; rfl
exact measurable_fst.comp (measurable_snd.comp (measurable_snd.comp h))
theorem measurable_full_y1
the treated potential-outcome coordinate is measurable.
Formal statement
Proof (Lean source)
lemma measurable_full_y1 {d : ℕ} : Measurable (fun z : FullObs d => z.y1) := by
have h : Measurable (fun z : FullObs d => (z.x, z.a, z.y0, z.y1, z.y)) := by
rw [measurable_iff_comap_le]; rfl
exact measurable_fst.comp (measurable_snd.comp (measurable_snd.comp
(measurable_snd.comp h)))
theorem measurable_full_y
the observed-outcome coordinate on full data is measurable.
Formal statement
Proof (Lean source)
lemma measurable_full_y {d : ℕ} : Measurable (fun z : FullObs d => z.y) := by
have h : Measurable (fun z : FullObs d => (z.x, z.a, z.y0, z.y1, z.y)) := by
rw [measurable_iff_comap_le]; rfl
exact measurable_snd.comp (measurable_snd.comp (measurable_snd.comp
(measurable_snd.comp h)))
def testFullLaw
This is the full-data law of the parametric two-point experiment.
Definition (Lean source)
noncomputable def testFullLaw {d : ℕ} (k : Fin d) (B u : ℝ) : Measure (FullObs d) :=
(2 : ENNReal)⁻¹ • Measure.map (testFullAtom k false)
(twoPointMean B u) +
(2 : ENNReal)⁻¹ • Measure.map (testFullAtom k true)
(twoPointMean B u)
theorem testFullLaw_prob
If the outcome bound is positive and the local mean lies within the outcome bound, the test full law is a probability measure.
Formal statement
hypotheses
conclusion
IsProbabilityMeasure (testFullLaw k B u)
Proof (Lean source)
lemma testFullLaw_prob {d : ℕ} (k : Fin d) {B u : ℝ}
(hB : 0 < B) (hu : |u| ≤ B) : IsProbabilityMeasure (testFullLaw k B u) := by
letI : IsProbabilityMeasure (twoPointMean B u) :=
twoPointMean_isProbabilityMeasure hB hu
rw [isProbabilityMeasure_iff, testFullLaw, Measure.add_apply,
Measure.smul_apply, Measure.smul_apply]
rw [Measure.map_apply (measurable_testFullAtom k false) MeasurableSet.univ,
Measure.map_apply (measurable_testFullAtom k true) MeasurableSet.univ]
simp only [Set.preimage_univ, measure_univ, smul_eq_mul, mul_one]
rw [← two_mul]
exact ENNReal.mul_inv_cancel (by norm_num) (by norm_num)
def testObsAtom
This is the observed-data atom associated with the parametric full-data atom.
Definition (Lean source)
theorem measurable_testObsAtom
the parametric observed-data atom is measurable as a function of its outcome.
Formal statement
Proof (Lean source)
lemma measurable_testObsAtom {d : ℕ} (k : Fin d) (a : Bool) :
Measurable (testObsAtom k a) := by
rw [measurable_comap_iff]
change Measurable (fun y : ℝ => (k, a, y))
fun_prop
theorem measurable_obs_x
the observed covariate coordinate is measurable.
Formal statement
Proof (Lean source)
lemma measurable_obs_x {d : ℕ} : Measurable (fun o : Obs d => o.x) := by
exact measurable_fst.comp (measurable_iff_comap_le.mpr le_rfl)
theorem measurable_obs_a
the observed treatment coordinate is measurable.
Formal statement
Proof (Lean source)
lemma measurable_obs_a {d : ℕ} : Measurable (fun o : Obs d => o.a) := by
exact measurable_fst.comp (measurable_snd.comp (measurable_iff_comap_le.mpr le_rfl))
theorem measurable_obs_y
the observed outcome coordinate is measurable.
Formal statement
Proof (Lean source)
lemma measurable_obs_y {d : ℕ} : Measurable (fun o : Obs d => o.y) := by
exact measurable_snd.comp (measurable_snd.comp (measurable_iff_comap_le.mpr le_rfl))
def testObservedLaw
This is the observed-data margin of the parametric two-point experiment.
Definition (Lean source)
noncomputable def testObservedLaw {d : ℕ} (k : Fin d) (B u : ℝ) : Measure (Obs d) :=
(2 : ENNReal)⁻¹ • Measure.dirac (testObsAtom k false 0) +
(2 : ENNReal)⁻¹ • Measure.map (testObsAtom k true)
(twoPointMean B u)
theorem testObservedLaw_prob
If the outcome bound is positive and the local mean lies within the outcome bound, the test observed law is a probability measure.
Formal statement
hypotheses
conclusion
IsProbabilityMeasure (testObservedLaw k B u)
Proof (Lean source)
lemma testObservedLaw_prob {d : ℕ} (k : Fin d) {B u : ℝ}
(hB : 0 < B) (hu : |u| ≤ B) : IsProbabilityMeasure (testObservedLaw k B u) := by
letI : IsProbabilityMeasure (twoPointMean B u) :=
twoPointMean_isProbabilityMeasure hB hu
rw [isProbabilityMeasure_iff, testObservedLaw, Measure.add_apply,
Measure.smul_apply, Measure.smul_apply,
Measure.map_apply (measurable_testObsAtom k true) MeasurableSet.univ]
simp only [Measure.dirac_apply, Set.indicator_of_mem, Set.mem_univ, Pi.one_apply,
Set.preimage_univ, measure_univ, smul_eq_mul, mul_one]
rw [← two_mul]
exact ENNReal.mul_inv_cancel (by norm_num) (by norm_num)
theorem test_observed_margin
If the outcome bound is positive and the local mean lies within the outcome bound, the observed margin of the parametric full-data law equals the constructed observed-data law.
Formal statement
hypotheses
conclusion
Measure.map FullObs.observed (testFullLaw k B u) = testObservedLaw k B u
Proof (Lean source)
lemma test_observed_margin {d : ℕ} (k : Fin d) {B u : ℝ}
(hB : 0 < B) (hu : |u| ≤ B) :
Measure.map FullObs.observed (testFullLaw k B u) = testObservedLaw k B u := by
letI : IsProbabilityMeasure (twoPointMean B u) :=
twoPointMean_isProbabilityMeasure hB hu
rw [testFullLaw, testObservedLaw, Measure.map_add]
· rw [Measure.map_smul, Measure.map_smul, Measure.map_map,
Measure.map_map]
· congr 2
· rw [show FullObs.observed ∘ testFullAtom k false =
fun _ : ℝ => testObsAtom k false 0 by funext y; rfl,
Measure.map_const, measure_univ, one_smul]
all_goals first | exact measurable_fullObserved | fun_prop
· exact measurable_fullObserved
This is the conditional outcome distribution in the parametric test model.
Definition (Lean source)
noncomputable def testOutcomeLaw (B u : ℝ) (a : Bool) : Measure ℝ :=
if a then twoPointMean B u else Measure.dirac 0
theorem testOutcomeLaw_prob
If the outcome bound is positive and the local mean lies within the outcome bound, the test outcome law is a probability measure.
Formal statement
hypotheses
conclusion
IsProbabilityMeasure (testOutcomeLaw B u a)
Proof (Lean source)
lemma testOutcomeLaw_prob {B u : ℝ} (hB : 0 < B) (hu : |u| ≤ B) (a : Bool) :
IsProbabilityMeasure (testOutcomeLaw B u a) := by
cases a
· change IsProbabilityMeasure (Measure.dirac 0)
infer_instance
· simpa [testOutcomeLaw] using
twoPointMean_isProbabilityMeasure hB hu
theorem test_cellMass_eq
If the outcome bound is positive and the local mean lies within the outcome bound, the test law assigns unit cell probability to the designated cell and zero to every other cell.
Formal statement
hypotheses
conclusion
realMass (testObservedLaw k B u) {o | o.x = l} = if l = k then 1 else 0
Proof (Lean source)
lemma test_cellMass_eq {d : ℕ} (k l : Fin d) {B u : ℝ}
(hB : 0 < B) (hu : |u| ≤ B) :
realMass (testObservedLaw k B u) {o | o.x = l} = if l = k then 1 else 0 := by
letI : IsProbabilityMeasure (twoPointMean B u) :=
Causalean.Mathlib.Probability.twoPointMean_isProbabilityMeasure hB hu
have hs : MeasurableSet {o : Obs d | o.x = l} := by
have ht : Measurable (fun o : Obs d => (o.x, o.a, o.y)) := by
rw [measurable_iff_comap_le]
rfl
exact (measurableSet_singleton l).preimage (measurable_fst.comp ht)
rw [realMass, testObservedLaw, Measure.add_apply,
Measure.smul_apply, Measure.smul_apply,
Measure.map_apply (measurable_testObsAtom k true) hs]
by_cases hl : l = k
· subst l
simp only [Measure.dirac_apply' _ hs, Set.mem_setOf_eq, testObsAtom,
Set.preimage_setOf_eq, true_iff, if_true]
simp [ENNReal.toReal_add, ENNReal.toReal_mul]
rw [← two_mul]
norm_num
· have hkl : k ≠ l := Ne.symm hl
simp [Measure.dirac_apply' _ hs, testObsAtom, hl, hkl]
theorem test_outcomeMean
If the outcome bound is positive and the local mean lies within the outcome bound, the test outcome mean is the local mean in the treated arm and zero in the control arm.
Formal statement
hypotheses
conclusion
∫ y, y ∂testOutcomeLaw B u a = if a then u else 0
Proof (Lean source)
lemma test_outcomeMean {B u : ℝ} (hB : 0 < B) (hu : |u| ≤ B) (a : Bool) :
∫ y, y ∂testOutcomeLaw B u a = if a then u else 0 := by
cases a
· simp [testOutcomeLaw]
· simpa [testOutcomeLaw] using
twoPointMean_mean hB hu
theorem testOutcome_second
If the outcome bound is positive and the local mean lies within the outcome bound, the conditional outcome law has an integrable squared centered outcome with second moment at most the squared outcome bound.
Formal statement
hypotheses
conclusion
Integrable (fun y => (y - if a then u else 0) ^ 2) (testOutcomeLaw B u a) ∧
∫ y, (y - if a then u else 0) ^ 2 ∂testOutcomeLaw B u a ≤ B ^ 2
Proof (Lean source)
lemma testOutcome_second {B u : ℝ} (hB : 0 < B) (hu : |u| ≤ B) (a : Bool) :
Integrable (fun y => (y - if a then u else 0) ^ 2) (testOutcomeLaw B u a) ∧
∫ y, (y - if a then u else 0) ^ 2 ∂testOutcomeLaw B u a ≤ B ^ 2 := by
cases a
· constructor
· simpa [testOutcomeLaw] using
(integrable_dirac (f := fun y : ℝ => (y - 0) ^ 2) (a := 0) (by simp [enorm]))
· simp [testOutcomeLaw]
positivity
· have hint : Integrable (fun y : ℝ => (y - u) ^ 2)
(twoPointMean B u) := by
unfold twoPointMean
apply Integrable.add_measure
· exact Integrable.smul_measure
(integrable_dirac (f := fun y : ℝ => (y - u) ^ 2) (a := B)
(by simp [enorm])) (by simp)
· exact Integrable.smul_measure
(integrable_dirac (f := fun y : ℝ => (y - u) ^ 2) (a := -B)
(by simp [enorm])) (by simp)
refine ⟨by simpa [testOutcomeLaw] using hint, ?_⟩
rw [show testOutcomeLaw B u true =
twoPointMean B u by rfl,
twoPointMean_integral hB hu]
have hcalc : ((1 + u / B) / 2) * (B - u) ^ 2 +
((1 - u / B) / 2) * (-B - u) ^ 2 = B ^ 2 - u ^ 2 := by
field_simp [hB.ne']
ring
simp only [Bool.true_eq, if_true]
rw [hcalc]
nlinarith [sq_nonneg u]
def testRealLaw
This assembles the real-outcome law for the parametric two-point experiment.
Definition (Lean source)
noncomputable def testRealLaw {d : ℕ} (k : Fin d) (B u : ℝ)
(hB : 0 < B) (hu : |u| ≤ B) : RealLaw d where
observedLaw := testObservedLaw k B u
observed_isProbability := testObservedLaw_prob k hB hu
fullLaw := testFullLaw k B u
full_isProbability := testFullLaw_prob k hB hu
cellMass := fun l => if l = k then 1 else 0
propensity := fun _ => 1 / 2
outcomeLaw := fun a _ => testOutcomeLaw B u a
outcome_isProbability := fun a _ => testOutcomeLaw_prob hB hu a
outcomeMean := fun a _ => if a then u else 0
observed_margin := test_observed_margin k hB hu
cellMass_eq := fun l => (test_cellMass_eq k l hB hu).symm
cellMass_range := by
intro l
split <;> simp_all
propensity_range := by norm_num
arm_outcome_factorization := by
intro a l s hs
have hset : MeasurableSet {o : Obs d | o.x = l ∧ o.a = a ∧ o.y ∈ s} :=
((measurableSet_singleton l).preimage measurable_obs_x).inter
(((measurableSet_singleton a).preimage measurable_obs_a).inter
(hs.preimage measurable_obs_y))
unfold realMass
rw [testObservedLaw, Measure.add_apply,
Measure.smul_apply, Measure.smul_apply,
Measure.map_apply (measurable_testObsAtom k true) hset]
by_cases hl : l = k
· subst l
cases a
· by_cases h0 : (0 : ℝ) ∈ s <;>
simp [testOutcomeLaw, testObsAtom, Measure.dirac_apply' _ hset,
ENNReal.toReal_add, ENNReal.toReal_mul, h0] <;> norm_num
· simp [testOutcomeLaw, testObsAtom, Measure.dirac_apply' _ hset,
ENNReal.toReal_add, ENNReal.toReal_mul]
· have hkl : k ≠ l := Ne.symm hl
cases a <;>
simp [testOutcomeLaw, testObsAtom, Measure.dirac_apply' _ hset, hl, hkl]
outcomeMean_eq := by
intro a l
exact (test_outcomeMean hB hu a).symm
theorem testRealLaw_rawAte
If the outcome bound is positive and the local mean lies within the outcome bound, the raw average-treatment-effect formula of the parametric test law equals its local mean parameter.
Formal statement
hypotheses
conclusion
rawAteFormula (testRealLaw k B u hB hu) = u
Proof (Lean source)
lemma testRealLaw_rawAte {d : ℕ} (k : Fin d) {B u : ℝ}
(hB : 0 < B) (hu : |u| ≤ B) : rawAteFormula (testRealLaw k B u hB hu) = u := by
classical
simp [rawAteFormula, cellEffect, testRealLaw]
theorem testRealLaw_consistency
If the outcome bound is positive and the local mean lies within the outcome bound, the test real law satisfies consistency.
Formal statement
hypotheses
conclusion
Consistency (testRealLaw k B u hB hu)
Proof (Lean source)
lemma testRealLaw_consistency {d : ℕ} (k : Fin d) {B u : ℝ}
(hB : 0 < B) (hu : |u| ≤ B) : Consistency (testRealLaw k B u hB hu) := by
unfold Consistency
change testFullLaw k B u {z | z.y ≠ if z.a then z.y1 else z.y0} = 0
have hsel : Measurable (fun z : FullObs d => if z.a then z.y1 else z.y0) := by
apply Measurable.ite
· exact (measurableSet_singleton true).preimage measurable_full_a
· exact measurable_full_y1
· exact measurable_full_y0
have hbad : MeasurableSet {z : FullObs d | z.y ≠ if z.a then z.y1 else z.y0} :=
(measurableSet_eq_fun measurable_full_y hsel).compl
rw [testFullLaw, Measure.add_apply, Measure.smul_apply, Measure.smul_apply,
Measure.map_apply (measurable_testFullAtom k false) hbad,
Measure.map_apply (measurable_testFullAtom k true) hbad]
simp [testFullAtom]
theorem testFullLaw_joint
If the control-potential event is measurable and the treated-potential event is measurable, the joint covariate-treatment-potential-outcome probability under the test law has the stated two-point formula.
Formal statement
hypotheses
conclusion
testFullLaw k B u {z | z.x = l ∧ z.a = a ∧ z.y0 ∈ s0 ∧ z.y1 ∈ s1} = if l = k ∧
0 ∈ s0 then (2 : ENNReal)⁻¹ * twoPointMean B u s1 else 0
Proof (Lean source)
lemma testFullLaw_joint {d : ℕ} (k l : Fin d) {B u : ℝ} (a : Bool)
{s0 s1 : Set ℝ} (hs0 : MeasurableSet s0) (hs1 : MeasurableSet s1) :
testFullLaw k B u {z | z.x = l ∧ z.a = a ∧ z.y0 ∈ s0 ∧ z.y1 ∈ s1} =
if l = k ∧ 0 ∈ s0 then
(2 : ENNReal)⁻¹ * twoPointMean B u s1
else 0 := by
have hset : MeasurableSet
{z : FullObs d | z.x = l ∧ z.a = a ∧ z.y0 ∈ s0 ∧ z.y1 ∈ s1} :=
((measurableSet_singleton l).preimage measurable_full_x).inter
(((measurableSet_singleton a).preimage measurable_full_a).inter
((hs0.preimage measurable_full_y0).inter (hs1.preimage measurable_full_y1)))
rw [testFullLaw, Measure.add_apply, Measure.smul_apply, Measure.smul_apply,
Measure.map_apply (measurable_testFullAtom k false) hset,
Measure.map_apply (measurable_testFullAtom k true) hset]
by_cases hl : l = k
· subst l
by_cases h0 : (0 : ℝ) ∈ s0 <;> cases a <;>
simp [testFullAtom, h0]
· have hkl : k ≠ l := Ne.symm hl
cases a <;> simp [testFullAtom, hl, hkl]
theorem testFullLaw_x
If the outcome bound is positive and the local mean lies within the outcome bound, the covariate marginal of the parametric full-data law is concentrated on the designated cell.
Formal statement
hypotheses
conclusion
testFullLaw k B u {z | z.x = l} = if l = k then 1 else 0
Proof (Lean source)
lemma testFullLaw_x {d : ℕ} (k l : Fin d) {B u : ℝ}
(hB : 0 < B) (hu : |u| ≤ B) :
testFullLaw k B u {z | z.x = l} = if l = k then 1 else 0 := by
letI : IsProbabilityMeasure (twoPointMean B u) :=
Causalean.Mathlib.Probability.twoPointMean_isProbabilityMeasure hB hu
have hset : MeasurableSet {z : FullObs d | z.x = l} :=
(measurableSet_singleton l).preimage measurable_full_x
rw [testFullLaw, Measure.add_apply, Measure.smul_apply, Measure.smul_apply,
Measure.map_apply (measurable_testFullAtom k false) hset,
Measure.map_apply (measurable_testFullAtom k true) hset]
by_cases hl : l = k
· subst l
simp only [testFullAtom, Set.preimage_setOf_eq, true_iff, measure_univ, if_true,
smul_eq_mul, mul_one]
rw [show {a : ℝ | True} = univ by ext; simp, measure_univ]
simp only [mul_one]
rw [← two_mul]
exact ENNReal.mul_inv_cancel (a := (2 : ENNReal)) (by norm_num) (by norm_num)
· have hkl : k ≠ l := Ne.symm hl
simp [testFullAtom, hl, hkl]
theorem testFullLaw_arm
If the outcome bound is positive and the local mean lies within the outcome bound, the treatment-arm marginal of the parametric full-data law is one half.
Formal statement
hypotheses
conclusion
testFullLaw k B u {z | z.x = l ∧ z.a = a} = if l = k then (2 : ENNReal)⁻¹ else 0
Proof (Lean source)
lemma testFullLaw_arm {d : ℕ} (k l : Fin d) {B u : ℝ} (a : Bool)
(hB : 0 < B) (hu : |u| ≤ B) :
testFullLaw k B u {z | z.x = l ∧ z.a = a} =
if l = k then (2 : ENNReal)⁻¹ else 0 := by
letI : IsProbabilityMeasure (twoPointMean B u) :=
Causalean.Mathlib.Probability.twoPointMean_isProbabilityMeasure hB hu
have hset : MeasurableSet {z : FullObs d | z.x = l ∧ z.a = a} :=
((measurableSet_singleton l).preimage measurable_full_x).inter
((measurableSet_singleton a).preimage measurable_full_a)
rw [testFullLaw, Measure.add_apply, Measure.smul_apply, Measure.smul_apply,
Measure.map_apply (measurable_testFullAtom k false) hset,
Measure.map_apply (measurable_testFullAtom k true) hset]
by_cases hl : l = k
· subst l
cases a <;> simp [testFullAtom]
· have hkl : k ≠ l := Ne.symm hl
cases a <;> simp [testFullAtom, hl, hkl]
theorem testFullLaw_potential
If the control-potential event is measurable and the treated-potential event is measurable, the potential-outcome marginal of the parametric full-data law has the stated formula.
Formal statement
hypotheses
conclusion
testFullLaw k B u {z | z.x = l ∧ z.y0 ∈ s0 ∧ z.y1 ∈ s1} = if l = k ∧
0 ∈ s0 then twoPointMean B u s1 else 0
Proof (Lean source)
lemma testFullLaw_potential {d : ℕ} (k l : Fin d) {B u : ℝ}
{s0 s1 : Set ℝ} (hs0 : MeasurableSet s0) (hs1 : MeasurableSet s1) :
testFullLaw k B u {z | z.x = l ∧ z.y0 ∈ s0 ∧ z.y1 ∈ s1} =
if l = k ∧ 0 ∈ s0 then
twoPointMean B u s1 else 0 := by
have hset : MeasurableSet {z : FullObs d | z.x = l ∧ z.y0 ∈ s0 ∧ z.y1 ∈ s1} :=
((measurableSet_singleton l).preimage measurable_full_x).inter
((hs0.preimage measurable_full_y0).inter (hs1.preimage measurable_full_y1))
rw [testFullLaw, Measure.add_apply, Measure.smul_apply, Measure.smul_apply,
Measure.map_apply (measurable_testFullAtom k false) hset,
Measure.map_apply (measurable_testFullAtom k true) hset]
by_cases hl : l = k
· subst l
by_cases h0 : (0 : ℝ) ∈ s0
· simp only [testFullAtom, Set.preimage_setOf_eq, true_and, h0, if_true,
smul_eq_mul]
rw [← add_mul, ← two_mul]
rw [ENNReal.mul_inv_cancel (by norm_num) (by norm_num), one_mul]
rfl
· simp [testFullAtom, h0]
· have hkl : k ≠ l := Ne.symm hl
simp [testFullAtom, hl, hkl]
theorem testRealLaw_exchangeability
If the outcome bound is positive and the local mean lies within the outcome bound, the parametric test law satisfies conditional exchangeability.
Formal statement
hypotheses
conclusion
ConditionalExchangeability (testRealLaw k B u hB hu)
Proof (Lean source)
lemma testRealLaw_exchangeability {d : ℕ} (k : Fin d) {B u : ℝ}
(hB : 0 < B) (hu : |u| ≤ B) :
ConditionalExchangeability (testRealLaw k B u hB hu) := by
unfold ConditionalExchangeability
intro l a s0 s1 hs0 hs1
change testFullLaw k B u
{z | z.x = l ∧ z.a = a ∧ z.y0 ∈ s0 ∧ z.y1 ∈ s1} *
testFullLaw k B u {z | z.x = l} =
testFullLaw k B u {z | z.x = l ∧ z.a = a} *
testFullLaw k B u {z | z.x = l ∧ z.y0 ∈ s0 ∧ z.y1 ∈ s1}
rw [testFullLaw_joint k l a hs0 hs1, testFullLaw_x k l hB hu,
testFullLaw_arm k l a hB hu, testFullLaw_potential k l hs0 hs1]
by_cases hl : l = k <;> by_cases h0 : (0 : ℝ) ∈ s0 <;>
simp [hl, h0] <;> ring
def testModelClass
This packages the parametric test law as a member of the heterogeneous model class.
Definition (Lean source)
noncomputable def testModelClass {d : ℕ} (k : Fin d)
(epsilon M sigma u : ℝ) (he0 : 0 < epsilon) (he1 : epsilon < 1 / 2)
(hM : 1 ≤ M) (hs0 : 0 ≤ sigma) (hs2 : sigma ≤ 2) (hu : |u| ≤ M / 2) :
ModelClass d epsilon M sigma := by
have hB : 0 < M / 2 := by positivity
let P := testRealLaw k (M / 2) u hB hu
refine {
law := P
epsilon_pos := he0
epsilon_lt_half := he1
M_ge_one := hM
sigma_nonneg := hs0
sigma_le_two := hs2
consistency := testRealLaw_consistency k hB hu
exchangeability := testRealLaw_exchangeability k hB hu
overlap := ?_
mean_normalization := ?_
second_moment := ?_
homogeneity := ?_ }
· intro l hl
have hlk : l = k := by
by_contra hne
simp [P, testRealLaw, hne] at hl
subst l
simp [P, testRealLaw]
constructor <;> linarith
· intro a l hl
have hlk : l = k := by
by_contra hne
simp [P, testRealLaw, hne] at hl
subst l
cases a
· simp [P, testRealLaw]
positivity
· simpa [P, testRealLaw] using hu
· intro a l hl
have hlk : l = k := by
by_contra hne
simp [P, testRealLaw, hne] at hl
subst l
have hsec := testOutcome_second hB hu a
refine ⟨by simpa [P, testRealLaw] using hsec.1, ?_⟩
calc
∫ y, (y - P.outcomeMean a k) ^ 2 ∂P.outcomeLaw a k ≤ (M / 2) ^ 2 := by
simpa [P, testRealLaw] using hsec.2
_ ≤ M ^ 2 := by nlinarith
· intro l hl
have hlk : l = k := by
by_contra hne
simp [P, testRealLaw, hne] at hl
subst l
have hate := testRealLaw_rawAte k hB hu
rw [show P = testRealLaw k (M / 2) u hB hu by rfl]
unfold cellDeviation cellEffect
rw [hate]
simp only [testRealLaw, Bool.true_eq, if_true, Bool.false_eq_true, if_false,
sub_zero, sub_self, abs_zero]
exact mul_nonneg hs0 (le_trans zero_le_one hM)
def testScaleBinaryObs
This rescales a binary test observation into the real-outcome observation space.
Definition (Lean source)
noncomputable def testScaleBinaryObs {d : ℕ} (k : Fin d) (M : ℝ)
(z : CausalSmith.Stat.DiscreteAteMinimaxLoggap.Obs 1) : Obs d :=
⟨k, z.2.1, if z.2.1 then M * ((if z.2.2 then 1 else 0) - 1 / 2) else 0⟩
theorem measurable_testScaleBinaryObs
the affine map from binary test observations to real observations is measurable.
Formal statement
Proof (Lean source)
lemma measurable_testScaleBinaryObs {d : ℕ} (k : Fin d) (M : ℝ) :
Measurable (testScaleBinaryObs k M) := measurable_of_finite _
theorem test_map_endpoint_null
If the outcome scale satisfies its stated bound and the second order satisfies its stated bound, at the null endpoint, affine rescaling maps the test observed law to the binary null law.
Formal statement
hypotheses
conclusion
Measure.map (testScaleBinaryObs k M) (CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw (CausalSmith.Stat.DiscreteAteMinimaxLoggap.endpointParametricLaw hv))
= testObservedLaw k (M / 2) (M * (g - 1 / 2))
Proof (Lean source)
lemma test_map_endpoint_null {d : ℕ} (k : Fin d) {M g : ℝ}
(hM : 0 < M)
(hv : ValidDGP
(mC (C := Fin 1) (1 / 2))
(gNull (C := Fin 1) g g)) :
Measure.map (testScaleBinaryObs k M)
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.endpointParametricLaw hv)) =
testObservedLaw k (M / 2) (M * (g - 1 / 2)) := by
ext s hs
rw [Measure.map_apply (measurable_testScaleBinaryObs k M) hs]
change (obsPMF hv).toMeasure
((testScaleBinaryObs k M) ⁻¹' s) = _
rw [PMF.toMeasure_apply _ (hs.preimage (measurable_testScaleBinaryObs k M)), tsum_fintype]
rw [testObservedLaw, twoPointMean,
Measure.add_apply, Measure.smul_apply, Measure.smul_apply]
rw [Measure.map_apply (measurable_testObsAtom k true) hs]
rw [Measure.add_apply, Measure.smul_apply, Measure.smul_apply]
simp [Fintype.sum_prod_type, obsPMF,
obsReal,
mC,
gNull,
testScaleBinaryObs, testObsAtom, PMF.ofFintype_apply,
Measure.dirac_apply' _ hs, indicator]
have hg := hv.g_mem true (0 : Fin 1)
have hgg : g ∈ Icc (0 : ℝ) 1 := by
simpa [gNull] using hg
have hplus : ofReal ((1 + M * (g - 1 / 2) / (M / 2)) / 2) =
ofReal g := by
congr 1
field_simp [hM.ne']
ring
have hminus : ofReal ((1 - M * (g - 1 / 2) / (M / 2)) / 2) =
ofReal (1 - g) := by
congr 1
field_simp [hM.ne']
ring
norm_num at hplus hminus ⊢
rw [hplus, hminus]
have hcontrol : ofReal ((1 - (1 / 2 : ℝ)) * g) +
ofReal ((1 - (1 / 2 : ℝ)) * (1 - g)) = (2 : ENNReal)⁻¹ := by
rw [ENNReal.ofReal_mul (by norm_num : 0 ≤ (1 - (1 / 2 : ℝ))),
ENNReal.ofReal_mul (by norm_num : 0 ≤ (1 - (1 / 2 : ℝ))), ← mul_add]
have hsum : ofReal g + ofReal (1 - g) = 1 := by
rw [← ENNReal.ofReal_add hgg.1 (sub_nonneg.mpr hgg.2)]
norm_num
rw [hsum]
rw [show (1 - (1 / 2 : ℝ)) = 1 / 2 by ring]
rw [ENNReal.ofReal_div_of_pos (by norm_num : (0 : ℝ) < 2)]
norm_num
have hcontrol' : ofReal (g * (1 / 2)) +
ofReal (1 / 2 + g * (-1 / 2)) = (2 : ENNReal)⁻¹ := by
convert hcontrol using 1 <;> ring
have hsum' : ofReal g + ofReal (1 - g) = 1 := by
rw [← ENNReal.ofReal_add hgg.1 (sub_nonneg.mpr hgg.2)]
norm_num
have hhalf : (2 : ENNReal)⁻¹ * ofReal g +
(2 : ENNReal)⁻¹ * ofReal (1 - g) = (2 : ENNReal)⁻¹ := by
rw [← mul_add, hsum']
simp
have heqPlus : M * (2 : ℝ)⁻¹ = M * (1 / 2 : ℝ) := by norm_num
have heqMinus : -(M * (2 : ℝ)⁻¹) = M * (-1 / 2 : ℝ) := by ring
split <;> split <;> split <;>
simp_all [indicator, hcontrol', hhalf, heqPlus, heqMinus,
div_eq_mul_inv, smul_eq_mul] <;>
abel
theorem test_map_endpoint_pert
If the outcome scale satisfies its stated bound and the second order satisfies its stated bound, at the perturbed endpoint, affine rescaling maps the test observed law to the binary alternative law.
Formal statement
hypotheses
conclusion
Measure.map (testScaleBinaryObs k M) (CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw (CausalSmith.Stat.DiscreteAteMinimaxLoggap.endpointParametricPertLaw hv))
= testObservedLaw k (M / 2) (M * (g + delta - 1 / 2))
Proof (Lean source)
lemma test_map_endpoint_pert {d : ℕ} (k : Fin d) {M g delta : ℝ}
(hM : 0 < M)
(hv : ValidDGP
(mC (C := Fin 1) (1 / 2))
(gPert (C := Fin 1) g g delta)) :
Measure.map (testScaleBinaryObs k M)
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.endpointParametricPertLaw hv)) =
testObservedLaw k (M / 2) (M * (g + delta - 1 / 2)) := by
ext s hs
rw [Measure.map_apply (measurable_testScaleBinaryObs k M) hs]
change (obsPMF hv).toMeasure
((testScaleBinaryObs k M) ⁻¹' s) = _
rw [PMF.toMeasure_apply _ (hs.preimage (measurable_testScaleBinaryObs k M)), tsum_fintype]
rw [testObservedLaw, twoPointMean,
Measure.add_apply, Measure.smul_apply, Measure.smul_apply]
rw [Measure.map_apply (measurable_testObsAtom k true) hs]
rw [Measure.add_apply, Measure.smul_apply, Measure.smul_apply]
simp [Fintype.sum_prod_type, obsPMF,
obsReal,
mC,
gPert,
testScaleBinaryObs, testObsAtom, PMF.ofFintype_apply,
Measure.dirac_apply' _ hs, indicator]
have hgc : g ∈ Icc (0 : ℝ) 1 := by
simpa [gPert] using
hv.g_mem false (0 : Fin 1)
have hgt : g + delta ∈ Icc (0 : ℝ) 1 := by
simpa [gPert] using
hv.g_mem true (0 : Fin 1)
have hplus : ofReal
((1 + M * (g + delta - 1 / 2) / (M / 2)) / 2) =
ofReal (g + delta) := by
congr 1
field_simp [hM.ne']
ring
have hminus : ofReal
((1 - M * (g + delta - 1 / 2) / (M / 2)) / 2) =
ofReal (1 - (g + delta)) := by
congr 1
field_simp [hM.ne']
ring
norm_num at hplus hminus ⊢
rw [hplus, hminus]
have hsumC : ofReal g + ofReal (1 - g) = 1 := by
rw [← ENNReal.ofReal_add hgc.1 (sub_nonneg.mpr hgc.2)]
norm_num
have hsumT : ofReal (g + delta) +
ofReal (1 - (g + delta)) = 1 := by
rw [← ENNReal.ofReal_add hgt.1 (sub_nonneg.mpr hgt.2)]
norm_num
have hcontrol : ofReal ((1 - (1 / 2 : ℝ)) * g) +
ofReal ((1 - (1 / 2 : ℝ)) * (1 - g)) = (2 : ENNReal)⁻¹ := by
rw [ENNReal.ofReal_mul (by norm_num : 0 ≤ (1 - (1 / 2 : ℝ))),
ENNReal.ofReal_mul (by norm_num : 0 ≤ (1 - (1 / 2 : ℝ))), ← mul_add,
hsumC, mul_one, show (1 - (1 / 2 : ℝ)) = 1 / 2 by ring,
ENNReal.ofReal_div_of_pos (by norm_num : (0 : ℝ) < 2)]
norm_num
have hhalfT : (2 : ENNReal)⁻¹ * ofReal (g + delta) +
(2 : ENNReal)⁻¹ * ofReal (1 - (g + delta)) =
(2 : ENNReal)⁻¹ := by
rw [← mul_add, hsumT]
simp
have hhalfC : (2 : ENNReal)⁻¹ * ofReal g +
(2 : ENNReal)⁻¹ * ofReal (1 - g) = (2 : ENNReal)⁻¹ := by
rw [← mul_add, hsumC]
simp
have heqPlus : M * (2 : ℝ)⁻¹ = M * (1 / 2 : ℝ) := by norm_num
have heqMinus : -(M * (2 : ℝ)⁻¹) = M * (-1 / 2 : ℝ) := by ring
split <;> split <;> split <;>
simp_all [indicator, hcontrol, hhalfT, hhalfC, heqPlus, heqMinus,
div_eq_mul_inv, smul_eq_mul] <;>
abel
theorem test_product_map_null
If the outcome scale satisfies its stated bound and the second order satisfies its stated bound and the first order satisfies its stated bound, coordinatewise affine rescaling maps the null test-sample law to the binary null product law.
Formal statement
hypotheses
conclusion
productLaw n (testRealLaw k (M / 2) (M * (g - 1 / 2)) (by positivity) hu)
= Measure.map (fun sample i => testScaleBinaryObs k M (sample i)) (productLaw hv n)
Proof (Lean source)
lemma test_product_map_null {n d : ℕ} (k : Fin d) {M g : ℝ}
(hM : 0 < M)
(hv : ValidDGP
(mC (C := Fin 1) (1 / 2))
(gNull (C := Fin 1) g g))
(hu : |M * (g - 1 / 2)| ≤ M / 2) :
productLaw n (testRealLaw k (M / 2) (M * (g - 1 / 2)) (by positivity) hu) =
Measure.map (fun sample i => testScaleBinaryObs k M (sample i))
(productLaw hv n) := by
unfold productLaw productLaw
change Measure.pi (fun _ : Fin n => testObservedLaw k (M / 2) (M * (g - 1 / 2))) = _
rw [← test_map_endpoint_null k hM hv]
exact (Measure.pi_map_pi
(μ := fun _ : Fin n =>
CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.endpointParametricLaw hv))
(f := fun _ : Fin n => testScaleBinaryObs k M)
(fun _ => (measurable_testScaleBinaryObs k M).aemeasurable)).symm
theorem test_product_map_pert
If the outcome scale satisfies its stated bound and the second order satisfies its stated bound and the first order satisfies its stated bound, coordinatewise affine rescaling maps the perturbed test-sample law to the binary alternative product law.
Formal statement
hypotheses
conclusion
productLaw n (testRealLaw k (M / 2) (M * (g + delta - 1 / 2)) (by positivity) hu)
= Measure.map (fun sample i => testScaleBinaryObs k M (sample i)) (productLaw hv n)
Proof (Lean source)
lemma test_product_map_pert {n d : ℕ} (k : Fin d) {M g delta : ℝ}
(hM : 0 < M)
(hv : ValidDGP
(mC (C := Fin 1) (1 / 2))
(gPert (C := Fin 1) g g delta))
(hu : |M * (g + delta - 1 / 2)| ≤ M / 2) :
productLaw n
(testRealLaw k (M / 2) (M * (g + delta - 1 / 2)) (by positivity) hu) =
Measure.map (fun sample i => testScaleBinaryObs k M (sample i))
(productLaw hv n) := by
unfold productLaw productLaw
change Measure.pi (fun _ : Fin n =>
testObservedLaw k (M / 2) (M * (g + delta - 1 / 2))) = _
rw [← test_map_endpoint_pert k hM hv]
exact (Measure.pi_map_pi
(μ := fun _ : Fin n =>
CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.endpointParametricPertLaw hv))
(f := fun _ : Fin n => testScaleBinaryObs k M)
(fun _ => (measurable_testScaleBinaryObs k M).aemeasurable)).symm
theorem test_model_mse_le
risk on the parametric test model is bounded by the ambient minimax risk.
Formal statement
hypotheses
conclusion
mse P.law est.1 ≤ (2 * M) ^ 2
Proof (Lean source)
lemma test_model_mse_le {n d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (est : Estimator n d M) :
mse P.law est.1 ≤ (2 * M) ^ 2 := by
let U : UnrestrictedClass d epsilon M :=
{ law := P.law
epsilon_pos := P.epsilon_pos
epsilon_lt_half := P.epsilon_lt_half
M_ge_one := P.M_ge_one
consistency := P.consistency
exchangeability := P.exchangeability
overlap := P.overlap
mean_normalization := P.mean_normalization
second_moment := P.second_moment }
have htau : |rawAteFormula P.law| ≤ M := by
simpa [U] using ((scale_sanity (d := d) (epsilon := epsilon) (M := M)).1 U).2.1
have hM0 : 0 ≤ M := le_trans zero_le_one P.M_ge_one
have hpoint (x : Fin n → Obs d) :
(est.1 x - rawAteFormula P.law) ^ 2 ≤ (2 * M) ^ 2 := by
have hest : |est.1 x| ≤ M := (abs_le).2 (est.2.2 x)
have habs : |est.1 x - rawAteFormula P.law| ≤ 2 * M := by
calc
|est.1 x - rawAteFormula P.law| ≤ |est.1 x| + |rawAteFormula P.law| :=
abs_sub _ _
_ ≤ M + M := add_le_add hest htau
_ = 2 * M := by ring
exact (sq_le_sq).2 (by simpa [abs_mul, abs_of_nonneg hM0] using habs)
have hint : Integrable (fun x => (est.1 x - rawAteFormula P.law) ^ 2)
(productLaw n P.law) := by
apply Integrable.of_bound (C := (2 * M) ^ 2)
((est.2.1.sub measurable_const).pow_const 2).aestronglyMeasurable
filter_upwards with x
rw [Real.norm_eq_abs, abs_of_nonneg (sq_nonneg _)]
exact hpoint x
unfold mse
calc
∫ x, (est.1 x - rawAteFormula P.law) ^ 2 ∂productLaw n P.law ≤
∫ _x, (2 * M) ^ 2 ∂productLaw n P.law := by
exact integral_mono hint (integrable_const _) hpoint
_ = (2 * M) ^ 2 := by simp
theorem test_model_sq_integrable
finite ambient risk makes the squared error on the parametric test model integrable.
Formal statement
hypotheses
conclusion
Integrable (fun x => (est.1 x - rawAteFormula P.law) ^ 2) (productLaw n P.law)
Proof (Lean source)
lemma test_model_sq_integrable {n d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (est : Estimator n d M) :
Integrable (fun x => (est.1 x - rawAteFormula P.law) ^ 2) (productLaw n P.law) := by
let U : UnrestrictedClass d epsilon M :=
{ law := P.law
epsilon_pos := P.epsilon_pos
epsilon_lt_half := P.epsilon_lt_half
M_ge_one := P.M_ge_one
consistency := P.consistency
exchangeability := P.exchangeability
overlap := P.overlap
mean_normalization := P.mean_normalization
second_moment := P.second_moment }
have htau : |rawAteFormula P.law| ≤ M := by
simpa [U] using ((scale_sanity (d := d) (epsilon := epsilon) (M := M)).1 U).2.1
have hM0 : 0 ≤ M := le_trans zero_le_one P.M_ge_one
apply Integrable.of_bound (C := (2 * M) ^ 2)
((est.2.1.sub measurable_const).pow_const 2).aestronglyMeasurable
filter_upwards with x
rw [Real.norm_eq_abs, abs_of_nonneg (sq_nonneg _)]
have hest : |est.1 x| ≤ M := (abs_le).2 (est.2.2 x)
have habs : |est.1 x - rawAteFormula P.law| ≤ 2 * M := by
calc
|est.1 x - rawAteFormula P.law| ≤ |est.1 x| + |rawAteFormula P.law| :=
abs_sub _ _
_ ≤ M + M := add_le_add hest htau
_ = 2 * M := by ring
exact (sq_le_sq).2 (by simpa [abs_mul, abs_of_nonneg hM0] using habs)
theorem test_minimax_two_point
If the outcome scale satisfies its stated bound and the bad-event probability is bounded as stated and the stated tau0 condition holds and the stated tau1 condition holds and the stated tv condition holds, the two parametric test laws give the stated two-point minimax lower bound.
Formal statement
hypotheses
n d :
ℕ
epsilon M sigma delta :
ℝ
P0 P1 :
ModelClass d epsilon M sigma
hM :
1 ≤ M
hdelta :
0 ≤ delta
htau0 :
rawAteFormula P0.law = 0
htau1 :
rawAteFormula P1.law = delta
htv :
tvDist (productLaw n P0.law) (productLaw n P1.law) ≤ 1 / 2
conclusion
delta ^ 2 / 16 ≤ minimaxRisk n d epsilon M sigma
Proof (Lean source)
lemma test_minimax_two_point {n d : ℕ} {epsilon M sigma delta : ℝ}
(P0 P1 : ModelClass d epsilon M sigma) (hM : 1 ≤ M)
(hdelta : 0 ≤ delta)
(htau0 : rawAteFormula P0.law = 0)
(htau1 : rawAteFormula P1.law = delta)
(htv : tvDist (productLaw n P0.law) (productLaw n P1.law) ≤ 1 / 2) :
delta ^ 2 / 16 ≤ minimaxRisk n d epsilon M sigma := by
let est0 : Estimator n d M := ⟨fun _ ↦ 0, measurable_const, fun _ ↦ by simp; linarith⟩
letI : Nonempty (Estimator n d M) := ⟨est0⟩
unfold minimaxRisk
apply le_ciInf
intro est
have hprob := two_point_lower_bound_of_tvDist_le
(P₀ := productLaw n P0.law) (P₁ := productLaw n P1.law) est.2.1
(θ₀ := rawAteFormula P0.law) (θ₁ := rawAteFormula P1.law)
(s := delta / 2) (c := (1 / 2 : ℝ))
(by rw [htau0, htau1, zero_sub, abs_neg, abs_of_nonneg hdelta]; linarith) htv
have hmse (P : ModelClass d epsilon M sigma) :
(delta / 2) ^ 2 * (productLaw n P.law).real
{x | delta / 2 ≤ |est.1 x - rawAteFormula P.law|} ≤ mse P.law est.1 := by
unfold mse
have hset : {x | delta / 2 ≤ |est.1 x - rawAteFormula P.law|} =
{x | (delta / 2) ^ 2 ≤ (est.1 x - rawAteFormula P.law) ^ 2} := by
ext x
simp only [Set.mem_setOf_eq]
constructor <;> intro h <;>
nlinarith [hdelta, abs_nonneg (est.1 x - rawAteFormula P.law),
sq_abs (est.1 x - rawAteFormula P.law)]
rw [hset]
exact mul_meas_ge_le_integral_of_nonneg
(Filter.Eventually.of_forall fun x ↦ sq_nonneg (est.1 x - rawAteFormula P.law))
(test_model_sq_integrable P est) ((delta / 2) ^ 2)
have htwo : delta ^ 2 / 16 ≤ max (mse P0.law est.1) (mse P1.law est.1) := by
have hscale : 0 ≤ (delta / 2) ^ 2 := sq_nonneg _
have hp := mul_le_mul_of_nonneg_left hprob hscale
norm_num at hp
rw [mul_max_of_nonneg _ _ hscale] at hp
calc
delta ^ 2 / 16 = (delta / 2) ^ 2 * (1 / 4) := by ring
_ ≤ max
((delta / 2) ^ 2 * (productLaw n P0.law).real
{x | delta / 2 ≤ |est.1 x - rawAteFormula P0.law|})
((delta / 2) ^ 2 * (productLaw n P1.law).real
{x | delta / 2 ≤ |est.1 x - rawAteFormula P1.law|}) := hp
_ ≤ _ := max_le_max (hmse P0) (hmse P1)
have hb : BddAbove (range (fun P : ModelClass d epsilon M sigma ↦
mse P.law est.1)) := by
refine ⟨(2 * M) ^ 2, ?_⟩
rintro _ ⟨P, rfl⟩
exact test_model_mse_le P est
exact htwo.trans (max_le (le_ciSup hb P0) (le_ciSup hb P1))
theorem parametric_lower_core
the parametric two-point experiment yields the stated inverse-sample-size minimax lower bound.
Formal statement
conclusion
∀ epsilon : ℝ,
0 < epsilon → epsilon < 1 / 2 → ∃ c_epsilon : ℝ, 0 < c_epsilon ∧ ∀ n d : ℕ, ∀ M sigma : ℝ, 0 < n → 0 < d → 1
≤ M → 0 ≤ sigma → sigma ≤ 2 → c_epsilon * M ^ 2 / n ≤ minimaxRisk n d epsilon M sigma
Proof (Lean source)
lemma parametric_lower_core :
∀ epsilon : ℝ, 0 < epsilon → epsilon < 1 / 2 →
∃ c_epsilon : ℝ, 0 < c_epsilon ∧
∀ n d : ℕ, ∀ M sigma : ℝ,
0 < n → 0 < d → 1 ≤ M → 0 ≤ sigma → sigma ≤ 2 →
c_epsilon * M ^ 2 / n ≤ minimaxRisk n d epsilon M sigma := by
intro epsilon he0 hehalf
refine ⟨1 / 100, by norm_num, ?_⟩
intro n d M sigma hn hd hM hs0 hs2
letI : Nonempty (Fin d) := Fin.pos_iff_nonempty.mp hd
let k : Fin d := Classical.arbitrary (Fin d)
let delta : ℝ := (2 / 5) / Real.sqrt n
have hnR : (0 : ℝ) < n := by exact_mod_cast hn
have hsqrt : 0 < Real.sqrt (n : ℝ) := Real.sqrt_pos.2 hnR
have hdelta0 : 0 ≤ delta := by dsimp [delta]; positivity
have hdeltasq : delta ^ 2 = 4 / (25 * (n : ℝ)) := by
dsimp [delta]
rw [div_pow, Real.sq_sqrt hnR.le]
ring
have hsqrt_one : 1 ≤ Real.sqrt (n : ℝ) := by
rw [← Real.sqrt_one]
exact Real.sqrt_le_sqrt (by exact_mod_cast hn)
have hdeltaU : delta ≤ 2 / 5 := by
dsimp [delta]
exact div_le_self (by norm_num) hsqrt_one
have hM0 : 0 < M := lt_of_lt_of_le zero_lt_one hM
let u0 : ℝ := M * ((1 / 2 : ℝ) - 1 / 2)
let u1 : ℝ := M * ((1 / 2 : ℝ) + delta - 1 / 2)
have hu0 : |u0| ≤ M / 2 := by dsimp [u0]; simp; positivity
have hu1 : |u1| ≤ M / 2 := by
have hdhalf : delta ≤ 1 / 2 := hdeltaU.trans (by norm_num)
rw [show u1 = M * delta by dsimp [u1]; ring, abs_mul,
abs_of_nonneg hdelta0, abs_of_nonneg hM0.le]
nlinarith
let P0 : ModelClass d epsilon M sigma :=
testModelClass k epsilon M sigma u0 he0 hehalf hM hs0 hs2 hu0
let P1 : ModelClass d epsilon M sigma :=
testModelClass k epsilon M sigma u1 he0 hehalf hM hs0 hs2 hu1
let hv0 := validDGP_null
(C := Fin 1) (m₀ := (1 / 2 : ℝ)) (g₀ := (1 / 2 : ℝ)) (g₁ := (1 / 2 : ℝ))
(by norm_num) (by norm_num) (by norm_num) (by norm_num) (by norm_num) (by norm_num)
let hv1 := validDGP_pert
(C := Fin 1) (m₀ := (1 / 2 : ℝ)) (g₀ := (1 / 2 : ℝ)) (g₁ := (1 / 2 : ℝ))
(δ := delta) (by norm_num) (by norm_num) (by norm_num) (by norm_num)
(by norm_num) hdelta0 (by linarith)
have hreg : (n : ℝ) *
((1 / 2 : ℝ) * delta ^ 2 / ((1 / 2 : ℝ) * (1 - 1 / 2))) ≤ log 2 := by
rw [hdeltasq]
have hlog : (8 / 25 : ℝ) ≤ log 2 :=
le_trans (by norm_num) (le_of_lt Real.log_two_gt_d9)
convert hlog using 1 <;> field_simp <;> ring
have htvSource : tvDist
(productLaw hv0 n)
(productLaw hv1 n) ≤ 1 / 2 :=
tvDist_productLaw_le_half
hv0 hv1 (by norm_num) (by norm_num) (by norm_num) (by norm_num)
(by norm_num) (by norm_num) hreg
have hmap0 : productLaw n P0.law =
Measure.map (fun sample i => testScaleBinaryObs k M (sample i))
(productLaw hv0 n) := by
simpa [P0, u0, testModelClass] using
(test_product_map_null (n := n) k hM0 hv0 hu0)
have hmap1 : productLaw n P1.law =
Measure.map (fun sample i => testScaleBinaryObs k M (sample i))
(productLaw hv1 n) := by
simpa [P1, u1, testModelClass] using
(test_product_map_pert (n := n) k hM0 hv1 hu1)
have htv : tvDist (productLaw n P0.law) (productLaw n P1.law) ≤ 1 / 2 := by
rw [hmap0, hmap1]
exact (CausalSmith.Stat.DiscreteAteMinimaxLoggap.tvDist_map_le
(productLaw hv0 n)
(productLaw hv1 n)
(fun sample i => testScaleBinaryObs k M (sample i)) (by fun_prop)).trans htvSource
have htau0 : rawAteFormula P0.law = 0 := by
simpa [P0, u0, testModelClass] using
(testRealLaw_rawAte k (show 0 < M / 2 by positivity) hu0)
have htau1 : rawAteFormula P1.law = M * delta := by
have hraw := testRealLaw_rawAte k (show 0 < M / 2 by positivity) hu1
simpa [P1, u1, testModelClass] using hraw
have hlow := test_minimax_two_point P0 P1 hM
(mul_nonneg hM0.le hdelta0) htau0 htau1 htv
rw [show (M * delta) ^ 2 = M ^ 2 * delta ^ 2 by ring, hdeltasq] at hlow
convert hlow using 1 <;> field_simp <;> ring
Helpers.PolynomialUpper.AggregateBridge 30 declarations
def markedTupleEquiv
This bijection decomposes a marked tuple into its distinguished coordinate and the remaining coordinates.
Definition (Lean source)
noncomputable def markedTupleEquiv (j : ℕ) (J : Type*) [DecidableEq J] :
(Fin (j + 2) ↪ J) ≃ Σ i : J, Fin (j + 1) ↪ {q : J // q ≠ i} where
toFun e := ⟨e 0, {
toFun := fun q => ⟨e q.succ, fun h =>
Fin.succ_ne_zero q (e.injective h)⟩
inj' := fun q r h => by
apply Fin.ext
apply Nat.succ.inj
exact congrArg val (e.injective (congrArg val h)) }⟩
invFun p := {
toFun := Fin.cons p.1 (fun q => (p.2 q).val)
inj' := by
rw [Fin.cons_injective_iff]
refine ⟨?_, ?_⟩
· rintro ⟨q, hq⟩
exact (p.2 q).property hq
· exact fun q r h => p.2.injective (Subtype.ext h) }
left_inv e := by
ext q
refine Fin.cases ?_ (fun q => ?_) q
· rfl
· rfl
right_inv p := by
rcases p with ⟨i, e⟩
rfl
theorem markedTupleEquiv_symm_zero
If the specified marked tuple, the inverse marked-tuple decomposition recovers the distinguished coordinate at position zero.
Formal statement
hypotheses
conclusion
((markedTupleEquiv j J).symm p) 0 = p.1
Proof (Lean source)
@[simp] lemma markedTupleEquiv_symm_zero (j : ℕ) (J : Type*) [DecidableEq J]
(p : Σ i : J, Fin (j + 1) ↪ {q : J // q ≠ i}) :
((markedTupleEquiv j J).symm p) 0 = p.1 := rfl
theorem markedTupleEquiv_symm_succ
If the specified marked tuple, the inverse marked-tuple decomposition recovers each tail coordinate at its successor position.
Formal statement
hypotheses
conclusion
((markedTupleEquiv j J).symm p) q.succ = (p.2 q).val
Proof (Lean source)
@[simp] lemma markedTupleEquiv_symm_succ (j : ℕ) (J : Type*) [DecidableEq J]
(p : Σ i : J, Fin (j + 1) ↪ {q : J // q ≠ i})
(q : Fin (j + 1)) :
((markedTupleEquiv j J).symm p) q.succ = (p.2 q).val := rfl
theorem weightedMatchingCount_remove_zero
removing the distinguished coordinate rewrites the weighted matching count in terms of its tail pattern.
Formal statement
hypotheses
Proof (Lean source)
lemma weightedMatchingCount_remove_zero {j : ℕ} {A J : Type*}
[Fintype A] [DecidableEq A] [Fintype J] [DecidableEq J]
(g : Fin (j + 2) → A) (z : J → A) (w : J → ℝ) :
(∑ e : Fin (j + 2) ↪ J,
if ∀ q, z (e q) = g q then w (e 0) else 0) =
∑ i : J, w i * (if z i = g 0 then
matchingCount (fun q : Fin (j + 1) => g q.succ)
(fun q : {q : J // q ≠ i} => z q) else 0) := by
classical
rw [Fintype.sum_equiv (markedTupleEquiv j J)
(fun e : Fin (j + 2) ↪ J =>
if ∀ q, z (e q) = g q then w (e 0) else 0)
(fun p => if ∀ q, z (((markedTupleEquiv j J).symm p) q) = g q then
w (((markedTupleEquiv j J).symm p) 0) else 0)
(fun e => by rw [Equiv.symm_apply_apply])]
rw [Fintype.sum_sigma]
apply Finset.sum_congr rfl
intro i _
simp only [Fin.forall_fin_succ, markedTupleEquiv_symm_zero,
markedTupleEquiv_symm_succ]
simp only [matchingCount]
by_cases h0 : z i = g 0
· simp only [h0, true_and, if_pos]
rw [Finset.mul_sum]
apply Finset.sum_congr rfl
intro f _
by_cases ht : ∀ q : Fin (j + 1), z (f q) = g q.succ
· have ht' : z (f 0) = g (succ 0) ∧
∀ q : Fin j, z (f q.succ) = g q.succ.succ := by
simpa only [Fin.forall_fin_succ] using ht
simp [ht, ht']
· have ht' : ¬ (z (f 0) = g (succ 0) ∧
∀ q : Fin j, z (f q.succ) = g q.succ.succ) := by
simpa only [Fin.forall_fin_succ] using ht
have hsplit : ¬ (z (f 0) = g 1 ∧
∀ q : Fin j, z (f q.succ) = g q.succ.succ) := by
intro hs
apply ht'
exact ⟨by simpa using hs.1, hs.2⟩
simp [hsplit, ht]
· simp [h0]
theorem complementFiber_card
If the stated i condition holds, the complement fiber of a finite Boolean pattern has cardinality equal to the total size minus the selected fiber size.
Formal statement
hypotheses
Proof (Lean source)
lemma complementFiber_card {A J : Type*} [Fintype J] [DecidableEq J]
[DecidableEq A] (z : J → A) (i : J) (a c : A) (hi : z i = a) :
card {q : {q : J // q ≠ i} // z q = c} =
if c = a then card {q : J // z q = c} - 1
else card {q : J // z q = c} := by
classical
let e : {q : {q : J // q ≠ i} // z q = c} ≃
{q : {q : J // z q = c} // q.1 ≠ i} :=
{ toFun := fun q => ⟨⟨q.1.1, q.2⟩, q.1.2⟩
invFun := fun q => ⟨⟨q.1.1, q.2⟩, q.1.2⟩
left_inv := fun q => by cases q; rfl
right_inv := fun q => by cases q; rfl }
rw [Fintype.card_congr e]
by_cases hca : c = a
· subst c
simp only [if_true, Fintype.card_subtype]
let ii : {q : J // z q = a} := ⟨i, hi⟩
rw [show (Finset.univ.filter fun q : {q : J // z q = a} => q.1 ≠ i) =
Finset.univ.erase ii by
ext q
simp [ii, Subtype.ext_iff]]
rw [Finset.card_erase_of_mem (Finset.mem_univ ii), Finset.card_univ]
rw [← Fintype.card_subtype]
· simp only [if_neg hca, Fintype.card_subtype]
rw [show (Finset.univ.filter fun q : {q : J // z q = c} => q.1 ≠ i) =
univ by
ext q
simp only [mem_filter, Finset.mem_univ, iff_true]
refine ⟨trivial, ?_⟩
intro hqi
apply hca
subst i
exact q.2.symm.trans hi]
rw [Finset.card_univ, ← Fintype.card_subtype]
def boolTailPattern
This Boolean tail pattern records whether each remaining coordinate matches a designated arm.
Definition (Lean source)
def boolMarkedPattern
This Boolean marked pattern adds the distinguished arm to the tail pattern.
Definition (Lean source)
def boolMarkedPattern (j : ℕ) (a b : Bool) : Fin (j + 2) → Bool :=
Fin.cons a (boolTailPattern j a b)
theorem boolTailPattern_same
every coordinate of the same-arm Boolean tail pattern equals the designated arm.
Formal statement
Proof (Lean source)
lemma boolTailPattern_same (j : ℕ) (a : Bool) :
boolTailPattern j a a = fun _ => a := by
funext q
refine Fin.cases rfl (fun _ => rfl) q
theorem matchingCount_boolTail_same
If the stated i condition holds, the same-arm Boolean tail pattern has the full matching count.
Formal statement
hypotheses
conclusion
matchingCount (boolTailPattern j a a) (fun q : {q : J // q ≠ i} => z q)
= ((card {q : J // z q = a} - 1).descFactorial (j + 1) : ℝ)
Proof (Lean source)
lemma matchingCount_boolTail_same {J : Type*} [Fintype J] [DecidableEq J]
(z : J → Bool) (i : J) (a : Bool) (j : ℕ) (hi : z i = a) :
matchingCount (boolTailPattern j a a)
(fun q : {q : J // q ≠ i} => z q) =
((card {q : J // z q = a} - 1).descFactorial (j + 1) : ℝ) := by
classical
rw [boolTailPattern_same]
rw [matchingCount_eq_prod]
simp_rw [complementFiber_card z i a _ hi]
cases a <;>
simp [PatternFiber, Fintype.card_subtype]
theorem card_boolTailPattern_other_fiber_self
for the opposite-arm tail pattern, the fiber over the designated arm has cardinality zero.
Formal statement
Proof (Lean source)
lemma card_boolTailPattern_other_fiber_self (j : ℕ) (a : Bool) :
card (PatternFiber (boolTailPattern j a (!a)) a) = j := by
classical
let e : PatternFiber (boolTailPattern j a (!a)) a ≃
{q : Fin (j + 1) // q ≠ 0} :=
Equiv.subtypeEquivProp (by
funext q
apply propext
refine Fin.cases ?_ (fun q => ?_) q <;>
cases a <;> simp [boolTailPattern])
rw [Fintype.card_congr e, ← Fintype.card_congr (finSuccAboveEquiv 0)]
exact Fintype.card_fin j
theorem card_boolTailPattern_other_fiber_not
for the opposite-arm tail pattern, the fiber over the other arm has full tail cardinality.
Formal statement
Proof (Lean source)
lemma card_boolTailPattern_other_fiber_not (j : ℕ) (a : Bool) :
card (PatternFiber (boolTailPattern j a (!a)) (!a)) = 1 := by
classical
let e : PatternFiber (boolTailPattern j a (!a)) (!a) ≃
{q : Fin (j + 1) // q = 0} :=
Equiv.subtypeEquivProp (by
funext q
apply propext
refine Fin.cases ?_ (fun q => ?_) q <;>
cases a <;> simp [boolTailPattern])
rw [Fintype.card_congr e]
exact Fintype.card_subtype_eq 0
theorem matchingCount_boolTail_other
If the stated i condition holds, the opposite-arm Boolean tail pattern has the stated complementary matching count.
Formal statement
hypotheses
conclusion
matchingCount (boolTailPattern j a (!a)) (fun q : {q : J // q ≠ i} => z q)
Proof (Lean source)
lemma matchingCount_boolTail_other {J : Type*} [Fintype J] [DecidableEq J]
(z : J → Bool) (i : J) (a : Bool) (j : ℕ) (hi : z i = a) :
matchingCount (boolTailPattern j a (!a))
(fun q : {q : J // q ≠ i} => z q) =
((card {q : J // z q = a} - 1).descFactorial j : ℝ) *
card {q : J // z q = !a} := by
classical
rw [matchingCount_eq_prod]
simp_rw [complementFiber_card z i a _ hi]
have hpattern : ∀ c : Bool,
card (PatternFiber (boolTailPattern j a (!a)) c) =
if c = a then j else 1 := by
intro c
by_cases hca : c = a
· subst c
simpa using card_boolTailPattern_other_fiber_self j a
· have hcnot : c = !a := by
cases a <;> cases c <;> simp_all
subst c
simpa using card_boolTailPattern_other_fiber_not j a
simp_rw [hpattern]
cases a <;>
simp [boolTailPattern, PatternFiber, Fintype.card_subtype,
Nat.descFactorial_one, mul_comm]
theorem weightedMatching_bool_same
the same-arm Boolean pattern has the stated weighted matching value.
Formal statement
hypotheses
conclusion
(∑ e : Fin (j + 2) ↪ J, if ∀ q, z (e q) = boolMarkedPattern j a a q then w (e 0) else 0)
= (∑ i : J, if z i = a then w i else 0) * ((card {i : J // z i = a} - 1).descFactorial (j + 1) : ℝ)
Proof (Lean source)
lemma weightedMatching_bool_same {J : Type*} [Fintype J] [DecidableEq J]
(z : J → Bool) (w : J → ℝ) (a : Bool) (j : ℕ) :
(∑ e : Fin (j + 2) ↪ J,
if ∀ q, z (e q) = boolMarkedPattern j a a q then w (e 0) else 0) =
(∑ i : J, if z i = a then w i else 0) *
((card {i : J // z i = a} - 1).descFactorial (j + 1) : ℝ) := by
classical
rw [weightedMatchingCount_remove_zero]
rw [Finset.sum_mul]
apply Finset.sum_congr rfl
intro i _
by_cases hi : z i = a
· simp only [boolMarkedPattern, Fin.cons_zero, hi, if_pos]
change w i * matchingCount (boolTailPattern j a a)
(fun q : {q : J // q ≠ i} => z q) = _
rw [matchingCount_boolTail_same z i a j hi]
· simp [boolMarkedPattern, hi]
theorem weightedMatching_bool_other
the opposite-arm Boolean pattern has the stated weighted matching value.
Formal statement
hypotheses
conclusion
(∑ e : Fin (j + 2) ↪ J, if ∀ q, z (e q) = boolMarkedPattern j a (!a) q then w (e 0) else 0)
Proof (Lean source)
lemma weightedMatching_bool_other {J : Type*} [Fintype J] [DecidableEq J]
(z : J → Bool) (w : J → ℝ) (a : Bool) (j : ℕ) :
(∑ e : Fin (j + 2) ↪ J,
if ∀ q, z (e q) = boolMarkedPattern j a (!a) q then w (e 0) else 0) =
(∑ i : J, if z i = a then w i else 0) *
(((card {i : J // z i = a} - 1).descFactorial j : ℝ) *
card {i : J // z i = !a}) := by
classical
rw [weightedMatchingCount_remove_zero]
rw [Finset.sum_mul]
apply Finset.sum_congr rfl
intro i _
by_cases hi : z i = a
· simp only [boolMarkedPattern, Fin.cons_zero, hi, if_pos]
change w i * matchingCount (boolTailPattern j a (!a))
(fun q : {q : J // q ≠ i} => z q) = _
rw [matchingCount_boolTail_other z i a j hi]
· simp [boolMarkedPattern, hi]
theorem bool_fiber_card_add
the two Boolean fibers of a marked pattern have cardinalities that sum to the pattern length.
Formal statement
Proof (Lean source)
lemma bool_fiber_card_add (J : Type*) [Fintype J] [DecidableEq J]
(z : J → Bool) (a : Bool) :
card {i : J // z i = a} + card {i : J // z i = !a} =
card J := by
classical
simp only [Fintype.card_subtype]
rw [← Finset.card_univ]
rw [← Finset.card_filter_add_card_filter_not (s := univ)
(fun i : J => z i = a)]
congr 2
ext i
cases a <;> cases z i <;> simp
theorem forall_boolMarkedPattern_iff
a property holds for every marked Boolean pattern exactly when it holds for the two possible marked arms and every tail pattern.
Formal statement
hypotheses
conclusion
(∀ q, z (e q) = boolMarkedPattern j a b q) ↔ z (e 0) = a ∧
z (e 1) = b ∧
∀ q : Fin j, z (e q.succ.succ) = a
Proof (Lean source)
lemma forall_boolMarkedPattern_iff {J : Type*} (z : J → Bool)
{j : ℕ} (e : Fin (j + 2) ↪ J) (a b : Bool) :
(∀ q, z (e q) = boolMarkedPattern j a b q) ↔
z (e 0) = a ∧ z (e 1) = b ∧
∀ q : Fin j, z (e q.succ.succ) = a := by
simp [boolMarkedPattern, boolTailPattern, Fin.forall_fin_succ]
theorem weightedMarkedUnrestricted
the unrestricted weighted marked sum reduces to the sum of the two Boolean arm cases.
Formal statement
hypotheses
Proof (Lean source)
lemma weightedMarkedUnrestricted {J : Type*} [Fintype J] [DecidableEq J]
(z : J → Bool) (w : J → ℝ) (a : Bool) (j : ℕ) :
(∑ e : Fin (j + 2) ↪ J,
if z (e 0) = a ∧ ∀ q : Fin j, z (e q.succ.succ) = a
then w (e 0) else 0) =
(∑ i : J, if z i = a then w i else 0) *
((card {i : J // z i = a} - 1).descFactorial j : ℝ) *
(card J - (j + 1) : ℕ) := by
classical
have hsplit : (∑ e : Fin (j + 2) ↪ J,
if z (e 0) = a ∧ ∀ q : Fin j, z (e q.succ.succ) = a
then w (e 0) else 0) =
(∑ e : Fin (j + 2) ↪ J,
if ∀ q, z (e q) = boolMarkedPattern j a a q then w (e 0) else 0) +
(∑ e : Fin (j + 2) ↪ J,
if ∀ q, z (e q) = boolMarkedPattern j a (!a) q then w (e 0) else 0) := by
rw [← Finset.sum_add_distrib]
apply Finset.sum_congr rfl
intro e _
simp only [forall_boolMarkedPattern_iff]
by_cases h0 : z (e 0) = a
· by_cases h1 : z (e 1) = a
· simp [h0, h1, boolMarkedPattern, boolTailPattern,
Fin.forall_fin_succ]
· have h1' : z (e 1) = !a := Bool.eq_not_iff.mpr h1
simp [h0, h1, h1', boolMarkedPattern, boolTailPattern,
Fin.forall_fin_succ]
· simp [h0, boolMarkedPattern, Fin.forall_fin_succ]
rw [hsplit, weightedMatching_bool_same, weightedMatching_bool_other]
let S : ℝ := ∑ i : J, if z i = a then w i else 0
let A : ℕ := card {i : J // z i = a}
let O : ℕ := card {i : J // z i = !a}
have hJO : A + O = card J := bool_fiber_card_add J z a
change S * ((A - 1).descFactorial (j + 1) : ℝ) +
S * (((A - 1).descFactorial j : ℝ) * O) =
S * ((A - 1).descFactorial j : ℝ) * (card J - (j + 1) : ℕ)
by_cases hA : A = 0
· have hS : S = 0 := by
unfold S A at *
rw [Fintype.card_eq_zero_iff] at hA
apply Finset.sum_eq_zero
intro i _
by_cases hi : z i = a
· rw [if_pos hi]
have hf : False := @isEmptyElim _ hA (fun _ => False) ⟨i, hi⟩
exact hf.elim
· simp [hi]
simp [hS]
· have hApos : 0 < A := Nat.pos_of_ne_zero hA
rw [Nat.descFactorial_succ]
by_cases hj : j ≤ A - 1
· have harith : (A - 1 - j) + O = A + O - (j + 1) := by omega
push_cast
rw [← hJO, ← harith]
rw [Nat.cast_add]
ring
· have hfact : (A - 1).descFactorial j = 0 :=
Nat.descFactorial_eq_zero_iff_lt.mpr (Nat.lt_of_not_ge hj)
simp [hfact]
def embeddingSubtypeRangeEquiv
An injective embedding identifies its domain with the subtype of points in its range.
Definition (Lean source)
noncomputable def embeddingSubtypeRangeEquiv {J : Type*} (r : ℕ)
(p : J → Prop) [DecidablePred p] :
(Fin r ↪ {i : J // p i}) ≃ {e : Fin r ↪ J // ∀ q, p (e q)} where
toFun e := ⟨e.trans (subtype _), fun q => (e q).2⟩
invFun e := {
toFun := fun q => ⟨e.1 q, e.2 q⟩
inj' := fun q s h => e.1.injective (congrArg val h) }
left_inv e := by ext q; rfl
right_inv e := by ext q; rfl
theorem sum_embedding_range_indicator
summing a range-indicator over the target of an embedding recovers the sum over the source.
Formal statement
hypotheses
Proof (Lean source)
lemma sum_embedding_range_indicator {J : Type*} [Fintype J] [DecidableEq J]
(r : ℕ) (p : J → Prop) [DecidablePred p]
(F : (Fin r ↪ J) → ℝ) :
(∑ e : Fin r ↪ J, if ∀ q, p (e q) then F e else 0) =
∑ e : Fin r ↪ {i : J // p i},
F (e.trans (subtype _)) := by
classical
rw [← Finset.sum_filter]
rw [Finset.sum_subtype (p := fun e : Fin r ↪ J => ∀ q, p (e q))
((Finset.univ : Finset (Fin r ↪ J)).filter fun e => ∀ q, p (e q))
(fun e => by simp) F]
exact Fintype.sum_equiv (embeddingSubtypeRangeEquiv r p).symm _ _
(fun _ => rfl)
theorem sum_function_injective_indicator
for an injective function, summing its range-indicator over the codomain recovers the domain sum.
Formal statement
hypotheses
I J :
F :
(I → J) → ℝ
conclusion
(∑ f : I → J, if Injective f then F f else 0) = ∑ e : I ↪ J, F e
Proof (Lean source)
lemma sum_function_injective_indicator {I J : Type*} [Fintype I] [Fintype J]
[DecidableEq I] [DecidableEq J] (F : (I → J) → ℝ) :
(∑ f : I → J, if Injective f then F f else 0) =
∑ e : I ↪ J, F e := by
classical
rw [← Finset.sum_filter]
rw [Finset.sum_subtype (p := Injective)
((Finset.univ : Finset (I → J)).filter Injective)
(fun f => by simp) F]
let e : {f : I → J // Injective f} ≃ (I ↪ J) :=
{ toFun := fun f => ⟨f.1, f.2⟩
invFun := fun f => ⟨f, f.injective⟩
left_inv := fun f => by cases f; rfl
right_inv := fun f => by cases f; rfl }
exact Fintype.sum_equiv e _ _ (fun _ => rfl)
theorem markedKernel_eq_ite
the marked kernel equals its explicit conditional product formula.
Formal statement
hypotheses
conclusion
((sample (e 0)).y / M) * (if (sample (e 0)).x = k ∧ (sample (e 0)).a = a then 1 else 0) * (if (sample (e 1)).x = k then 1 else 0) * (∏ q : Fin (j + 2), if 2 ≤ q.val then if (sample (e q)).x = k ∧ (sample (e q)).a = a then 1 else 0 else 1)
= if ((sample (e 0)).x = k ∧ (sample (e 0)).a = a) ∧
(sample (e 1)).x = k ∧
(∀ q : Fin (j + 2), 2 ≤ q.val → (sample (e q)).x = k ∧ (sample (e q)).a = a) then (sample (e 0)).y / M else 0
Proof (Lean source)
lemma markedKernel_eq_ite {m d j : ℕ} (M : ℝ) (sample : Fin m → Obs d)
(k : Fin d) (a : Bool) (e : Fin (j + 2) ↪ Fin m) :
((sample (e 0)).y / M) *
(if (sample (e 0)).x = k ∧ (sample (e 0)).a = a then 1 else 0) *
(if (sample (e 1)).x = k then 1 else 0) *
(∏ q : Fin (j + 2),
if 2 ≤ q.val then
if (sample (e q)).x = k ∧ (sample (e q)).a = a then 1 else 0
else 1) =
if ((sample (e 0)).x = k ∧ (sample (e 0)).a = a) ∧
(sample (e 1)).x = k ∧
(∀ q : Fin (j + 2), 2 ≤ q.val →
(sample (e q)).x = k ∧ (sample (e q)).a = a)
then (sample (e 0)).y / M else 0 := by
classical
simp_rw [show ∀ q : Fin (j + 2),
(if 2 ≤ q.val then
if (sample (e q)).x = k ∧ (sample (e q)).a = a then (1 : ℝ) else 0
else 1) =
if (2 ≤ q.val → (sample (e q)).x = k ∧ (sample (e q)).a = a)
then 1 else 0 by
intro q
by_cases hq : 2 ≤ q.val <;> simp [hq]]
rw [Fintype.prod_boole]
by_cases h0 : (sample (e 0)).x = k ∧ (sample (e 0)).a = a <;>
by_cases h1 : (sample (e 1)).x = k <;>
simp [h0, h1]
theorem forall_two_le_fin_iff
a property holds for every finite index at least two exactly when it holds for every index after removing the first two positions.
Formal statement
Proof (Lean source)
lemma forall_two_le_fin_iff {j : ℕ} {P : Fin (j + 2) → Prop} :
(∀ q : Fin (j + 2), 2 ≤ q.val → P q) ↔
∀ q : Fin j, P q.succ.succ := by
simp [Fin.forall_fin_succ]
theorem groupArmSubtype_card
the subtype of sample indices in a given arm and cell has cardinality equal to the corresponding arm-cell count.
Formal statement
Proof (Lean source)
lemma groupArmSubtype_card {m d : ℕ} (sample : Fin m → Obs d)
(k : Fin d) (a : Bool) :
card {i : {i : Fin m // (sample i).x = k} // (sample i).a = a} =
groupArmCount (fun o : Obs d => o.x) (fun o => o.a)
sample a k := by
classical
let e : {i : {i : Fin m // (sample i).x = k} // (sample i).a = a} ≃
{i : Fin m // (sample i).x = k ∧ (sample i).a = a} :=
{ toFun := fun i => ⟨i.1.1, i.1.2, i.2⟩
invFun := fun i => ⟨⟨i.1, i.2.1⟩, i.2.2⟩
left_inv := fun i => by cases i; rfl
right_inv := fun i => by cases i; rfl }
rw [Fintype.card_congr e]
unfold groupArmCount
exact Fintype.card_subtype _
theorem groupSubtype_card
the subtype of sample indices in a given cell has cardinality equal to the corresponding cell count.
Formal statement
hypotheses
conclusion
= groupCount (fun o : Obs d => o.x) (fun o => o.a) sample k
Proof (Lean source)
lemma groupSubtype_card {m d : ℕ} (sample : Fin m → Obs d) (k : Fin d) :
card {i : Fin m // (sample i).x = k} =
groupCount (fun o : Obs d => o.x) (fun o => o.a)
sample k := by
rw [← bool_fiber_card_add {i : Fin m // (sample i).x = k}
(fun i => (sample i).a) false]
rw [groupArmSubtype_card, groupArmSubtype_card]
rfl
theorem groupSubtype_markSum_div
the normalized mark sum over a cell subtype equals the corresponding grouped mark sum divided by the outcome scale.
Formal statement
hypotheses
conclusion
(∑ i : {i : Fin m // (sample i).x = k}, if (sample i).a = a then (sample i).y / M else 0)
= armMarkSum (fun o : Obs d => o.x) (fun o => o.a) (fun o => o.y) sample a k / M
Proof (Lean source)
lemma groupSubtype_markSum_div {m d : ℕ} (M : ℝ)
(sample : Fin m → Obs d) (k : Fin d) (a : Bool) :
(∑ i : {i : Fin m // (sample i).x = k},
if (sample i).a = a then (sample i).y / M else 0) =
armMarkSum
(fun o : Obs d => o.x) (fun o => o.a) (fun o => o.y) sample a k / M := by
classical
unfold armMarkSum
supportedArmMark
armCategoryEvent
armGroupEvent indicator
simp only [Set.mem_ofPred_eq]
rw [Finset.sum_div]
rw [← Finset.sum_filter]
rw [Finset.sum_subtype
(p := fun i : {i : Fin m // (sample i).x = k} => (sample i).a = a)
((Finset.univ : Finset {i : Fin m // (sample i).x = k}).filter
fun i => (sample i).a = a) (fun i => by simp)
(fun i => (sample i).y / M)]
let e : {i : {i : Fin m // (sample i).x = k} // (sample i).a = a} ≃
{i : Fin m // (sample i).x = k ∧ (sample i).a = a} :=
{ toFun := fun i => ⟨i.1.1, i.1.2, i.2⟩
invFun := fun i => ⟨⟨i.1, i.2.1⟩, i.2.2⟩
left_inv := fun i => by cases i; rfl
right_inv := fun i => by cases i; rfl }
rw [Fintype.sum_equiv e (fun i => (sample i.1.1).y / M)
(fun i => (sample i.1).y / M) (fun _ => rfl)]
rw [← Finset.sum_subtype
(p := fun i : Fin m => (sample i).x = k ∧ (sample i).a = a)
((Finset.univ : Finset (Fin m)).filter
fun i => (sample i).x = k ∧ (sample i).a = a) (fun i => by simp)
(fun i => (sample i).y / M)]
rw [Finset.sum_filter]
apply Finset.sum_congr rfl
intro i _
by_cases hi : (sample i).x = k ∧ (sample i).a = a <;> simp [hi]
theorem markedKernel_condition_iff
the marked-kernel matching condition is equivalent to the stated arm-and-cell coordinate conditions.
Formal statement
hypotheses
Proof (Lean source)
lemma markedKernel_condition_iff {m d j : ℕ} (sample : Fin m → Obs d)
(k : Fin d) (a : Bool) (e : Fin (j + 2) ↪ Fin m) :
((((sample (e 0)).x = k ∧ (sample (e 0)).a = a) ∧
(sample (e 1)).x = k ∧
(∀ q : Fin (j + 2), 2 ≤ q.val →
(sample (e q)).x = k ∧ (sample (e q)).a = a))) ↔
((∀ q : Fin (j + 2), (sample (e q)).x = k) ∧
(sample (e 0)).a = a ∧
∀ q : Fin j, (sample (e q.succ.succ)).a = a) := by
rw [forall_two_le_fin_iff]
simp only [Fin.forall_fin_succ]
aesop
theorem allBlockOrderedMarkedFactorial_eq_closed
the all-block ordered marked factorial statistic equals its closed-form expression in the arm mark sum and descending factorial counts.
Formal statement
hypotheses
conclusion
allBlockOrderedMarkedFactorial M sample k a j
= (armMarkSum (fun o : Obs d => o.x) (fun o => o.a) (fun o => o.y) sample a k / M) * ((groupArmCount (fun o : Obs d => o.x) (fun o => o.a) sample a k - 1).descFactorial j : ℝ) * (groupCount (fun o : Obs d => o.x) (fun o => o.a) sample k - (j + 1) : ℕ) / m.descFactorial (j + 2)
Proof (Lean source)
lemma allBlockOrderedMarkedFactorial_eq_closed {m d : ℕ} (M : ℝ)
(sample : Fin m → Obs d) (k : Fin d) (a : Bool) (j : ℕ) :
allBlockOrderedMarkedFactorial M sample k a j =
(armMarkSum
(fun o : Obs d => o.x) (fun o => o.a) (fun o => o.y) sample a k / M) *
((groupArmCount (fun o : Obs d => o.x) (fun o => o.a)
sample a k - 1).descFactorial j : ℝ) *
(groupCount (fun o : Obs d => o.x) (fun o => o.a)
sample k - (j + 1) : ℕ) /
m.descFactorial (j + 2) := by
classical
unfold allBlockOrderedMarkedFactorial
rw [sum_function_injective_indicator]
simp_rw [markedKernel_eq_ite]
simp_rw [markedKernel_condition_iff]
have hsplit : (∑ e : Fin (j + 2) ↪ Fin m,
if ((∀ q : Fin (j + 2), (sample (e q)).x = k) ∧
(sample (e 0)).a = a ∧
∀ q : Fin j, (sample (e q.succ.succ)).a = a)
then (sample (e 0)).y / M else 0) =
∑ e : Fin (j + 2) ↪ Fin m,
if ∀ q, (sample (e q)).x = k then
(if (sample (e 0)).a = a ∧
∀ q : Fin j, (sample (e q.succ.succ)).a = a
then (sample (e 0)).y / M else 0)
else 0 := by
apply Finset.sum_congr rfl
intro e _
by_cases hg : ∀ q, (sample (e q)).x = k <;> simp [hg]
rw [hsplit]
have hrange : (∑ e : Fin (j + 2) ↪ Fin m,
if ∀ q, (sample (e q)).x = k then
(if (sample (e 0)).a = a ∧
∀ q : Fin j, (sample (e q.succ.succ)).a = a
then (sample (e 0)).y / M else 0)
else 0) =
∑ e : Fin (j + 2) ↪ {i : Fin m // (sample i).x = k},
if (sample (e 0)).a = a ∧
∀ q : Fin j, (sample (e q.succ.succ)).a = a
then (sample (e 0)).y / M else 0 := by
simpa using sum_embedding_range_indicator (J := Fin m) (r := j + 2)
(fun i => (sample i).x = k)
(fun e => if (sample (e 0)).a = a ∧
∀ q : Fin j, (sample (e q.succ.succ)).a = a
then (sample (e 0)).y / M else 0)
rw [hrange]
have hweighted : (∑ e : Fin (j + 2) ↪ {i : Fin m // (sample i).x = k},
if (sample (e 0)).a = a ∧
∀ q : Fin j, (sample (e q.succ.succ)).a = a
then (sample (e 0)).y / M else 0) =
(∑ i : {i : Fin m // (sample i).x = k},
if (sample i).a = a then (sample i).y / M else 0) *
((card {i : {i : Fin m // (sample i).x = k} //
(sample i).a = a} - 1).descFactorial j : ℝ) *
(card {i : Fin m // (sample i).x = k} - (j + 1) : ℕ) := by
simpa using weightedMarkedUnrestricted
(J := {i : Fin m // (sample i).x = k})
(fun i => (sample i).a) (fun i => (sample i).y / M) a j
rw [hweighted]
rw [groupSubtype_markSum_div, groupArmSubtype_card, groupSubtype_card]
theorem orderedMarkedFactorial_rebuild_eq_allBlock
The concrete estimation-block factorial is exactly the all-block statistic on the canonically reindexed estimation fold.
Formal statement
hypotheses
n d :
ℕ
P :
RealLaw d
base :
Obs d
x :
(polynomialBalancedSplit P).foldB n → Obs d
M :
ℝ
k :
Fin d
a :
j :
ℕ
conclusion
orderedMarkedFactorial M (rebuildPolynomialEstimationSample P base x) k a j
= allBlockOrderedMarkedFactorial M (polynomialFoldBReindex P x) k a j
Proof (Lean source)
lemma orderedMarkedFactorial_rebuild_eq_allBlock {n d : ℕ} (P : RealLaw d)
(base : Obs d) (x : (polynomialBalancedSplit P).foldB n → Obs d)
(M : ℝ) (k : Fin d) (a : Bool) (j : ℕ) :
orderedMarkedFactorial M (rebuildPolynomialEstimationSample P base x) k a j =
allBlockOrderedMarkedFactorial M (polynomialFoldBReindex P x) k a j := by
unfold orderedMarkedFactorial estimationBlockSize
rw [estimationArmSum_rebuild_eq_reindex,
estimationArmCount_rebuild_eq_reindex,
estimationCellCount_rebuild_eq_reindex,
allBlockOrderedMarkedFactorial_eq_closed]
theorem lightPolynomialTerm_rebuild_eq_allBlock
Each concrete light-cell polynomial is the finite-product all-block polynomial on the canonically reindexed estimation fold.
Formal statement
hypotheses
n d :
ℕ
P :
RealLaw d
base :
Obs d
x :
(polynomialBalancedSplit P).foldB n → Obs d
M B :
ℝ
K :
ℕ
k :
Fin d
conclusion
lightPolynomialTerm M B K (rebuildPolynomialEstimationSample P base x) k
= allBlockLightPolynomialTerm M B K (polynomialFoldBReindex P x) k
Proof (Lean source)
lemma lightPolynomialTerm_rebuild_eq_allBlock {n d : ℕ} (P : RealLaw d)
(base : Obs d) (x : (polynomialBalancedSplit P).foldB n → Obs d)
(M B : ℝ) (K : ℕ) (k : Fin d) :
lightPolynomialTerm M B K (rebuildPolynomialEstimationSample P base x) k =
allBlockLightPolynomialTerm M B K (polynomialFoldBReindex P x) k := by
unfold lightPolynomialTerm allBlockLightPolynomialTerm
simp_rw [orderedMarkedFactorial_rebuild_eq_allBlock]
theorem lightPolynomialSum_rebuild_eq_allBlock
Summing over a deterministic light set preserves the exact fold identification.
Formal statement
hypotheses
conclusion
(∑ k ∈ S, lightPolynomialTerm M B K (rebuildPolynomialEstimationSample P base x) k)
= allBlockMarkedPolynomialSum M B K S (polynomialFoldBReindex P x)
Proof (Lean source)
lemma lightPolynomialSum_rebuild_eq_allBlock {n d : ℕ} (P : RealLaw d)
(base : Obs d) (x : (polynomialBalancedSplit P).foldB n → Obs d)
(M B : ℝ) (K : ℕ) (S : Finset (Fin d)) :
(∑ k ∈ S,
lightPolynomialTerm M B K (rebuildPolynomialEstimationSample P base x) k) =
allBlockMarkedPolynomialSum M B K S (polynomialFoldBReindex P x) := by
unfold allBlockMarkedPolynomialSum
apply Finset.sum_congr rfl
intro k _
exact lightPolynomialTerm_rebuild_eq_allBlock P base x M B K k
Helpers.PolynomialUpper.Assembly 2 declarations
The explicit heavy/light signed one-mark estimator is total and has the capped all-alphabet polynomial risk bound under conditional second moments.
Formal statement
conclusion
∀ epsilon : ℝ,
0 < epsilon → epsilon < 1 / 2 → ∃ C_epsilon : ℝ, ∃ handle : PolynomialHandle, 0 < C_epsilon ∧ PolynomialComplexityBound handle ∧ ∀ n d : ℕ, ∀ M sigma : ℝ, 0 < n → 0 < d → 1
≤ M → 0 ≤ sigma → sigma ≤ 2 → Measurable (polyEstimator handle (n := n) (d := d) M) ∧
(∀ s : Fin n → Obs d, polyEstimator handle (n := n) (d := d) M s ∈ Icc (-M) M) ∧
(∀ P : ModelClass d epsilon M sigma, mse P.law (polyEstimator handle (n := n) (d := d) M) ≤ C_epsilon * M ^ 2 * (1 / (n : ℝ) + min 1 (polynomialComponent n d)))
Proof (Lean source)
lemma continuous_ratio_polynomial_upper_all_d :
∀ epsilon : ℝ, 0 < epsilon → epsilon < 1 / 2 →
∃ C_epsilon : ℝ, ∃ handle : PolynomialHandle,
0 < C_epsilon ∧
PolynomialComplexityBound handle ∧
∀ n d : ℕ, ∀ M sigma : ℝ,
0 < n → 0 < d → 1 ≤ M → 0 ≤ sigma → sigma ≤ 2 →
Measurable
(polyEstimator handle (n := n) (d := d) M) ∧
(∀ s : Fin n → Obs d,
polyEstimator handle (n := n) (d := d) M s ∈ Icc (-M) M) ∧
(∀ P : ModelClass d epsilon M sigma,
mse P.law
(polyEstimator handle (n := n) (d := d) M) ≤
C_epsilon * M ^ 2 *
(1 / (n : ℝ) + min 1 (polynomialComponent n d))) := by
intro epsilon hepsilon hepsilon_half
obtain ⟨Cfix, hCfix, hfixed⟩ :=
polynomial_fixedBranch_uniform_bound epsilon hepsilon hepsilon_half
obtain ⟨Ndeg, hNdeg⟩ := polynomialDegree_eventually_two
obtain ⟨Nlog, hNlog⟩ := logEN_eventually_ge_240
let N : ℕ := max 8 (max Ndeg Nlog)
let rho : ℝ := 1
have hrho : 0 < rho := by simp [rho]
let handle : PolynomialHandle :=
⟨(N, rho), hrho, by
intro n d hn hd hactive
exact hNdeg n ((le_max_left Ndeg Nlog).trans
((le_max_right 8 (max Ndeg Nlog)).trans hactive.1))⟩
obtain ⟨Cfall, hCfall, hfallback⟩ :=
polyEstimator_uncalibrated_rate (N := N) hrho
let D : ℝ := 128 / epsilon + 96 + 2 / epsilon ^ 4 + 4 * Cfix +
6 * Cfix * 8192 ^ 2 +
2 * (32768 / (epsilon * polynomialAlpha0 ^ 2)) ^ 2
have hD : 0 < D := by
dsimp [D]
positivity
let C : ℝ := max Cfall (D + 1)
have hC : 0 < C := hCfall.trans_le (le_max_left _ _)
refine ⟨C, handle, hC, polynomialComplexityBound handle, ?_⟩
intro n d M sigma hn hd hM hsigma hsigma_two
have htotal := polyEstimator_total_and_clipped
(n := n) (d := d) (N := N) (M := M) (rho := rho)
(le_trans zero_le_one hM)
have htotal' :
Measurable (polyEstimator handle (n := n) (d := d) M) ∧
∀ s : Fin n → Obs d,
polyEstimator handle (n := n) (d := d) M s ∈ Icc (-M) M := by
change Measurable
(rawPolyEstimator (n := n) (d := d) handle.N handle.rho M) ∧
∀ s : Fin n → Obs d,
rawPolyEstimator handle.N handle.rho M s ∈ Icc (-M) M
simpa [handle, PolynomialHandle.N, PolynomialHandle.rho] using htotal
refine ⟨htotal'.1, htotal'.2, ?_⟩
· intro P
by_cases hcal : N ≤ n ∧ (d : ℝ) ≤ rho * n * logEN n
· have hn8 : 8 ≤ n := (le_max_left 8 (max Ndeg Nlog)).trans hcal.1
have hnNdeg : Ndeg ≤ n :=
(le_max_left Ndeg Nlog).trans
((le_max_right 8 (max Ndeg Nlog)).trans hcal.1)
have hnNlog : Nlog ≤ n :=
(le_max_right Ndeg Nlog).trans
((le_max_right 8 (max Ndeg Nlog)).trans hcal.1)
have hK : 2 ≤ polynomialDegree n := hNdeg n hnNdeg
have hL : 240 ≤ logEN n := hNlog n hnNlog
have hn2 : 2 ≤ n := by omega
have hlower : 0 < polynomialPilotLowerBand n := by
unfold polynomialPilotLowerBand
have hp : 0 < n / 2 := Nat.div_pos (by omega) (by norm_num)
have hlog : 0 < logEN n := by linarith
positivity
have hrate0 : 0 ≤ 1 / (n : ℝ) + polynomialComponent n d := by
unfold polynomialComponent
positivity
have hR : 0 ≤ D *
(1 / (n : ℝ) + polynomialComponent n d) :=
mul_nonneg hD.le hrate0
let base : Obs d := ⟨⟨0, hd⟩, false, 0⟩
have hbranchFixed : ∀ H : Finset (Fin d),
polynomialSelectorEligible P.law (polynomialPilotLowerBand n)
(polynomialPilotUpperBand n) H →
∫ z : Fin (n - n / 2) → Obs d,
(polynomialFixedBranchNormalizedError
(K := polynomialDegree n) P.law M
(4096 * logEN n / (n - n / 2 : ℕ)) H z) ^ 2
∂(productLaw (n - n / 2) P.law) ≤
D * (1 / (n : ℝ) + polynomialComponent n d) := by
intro H helig
apply (hfixed n d M sigma (polynomialPilotLowerBand n)
(polynomialPilotUpperBand n) P H helig hK hlower hn2
(polynomialPilotUpperBand_le_lightScale_quarter hn2)
(polynomial_fixedBranch_size_condition hn hL)
(polynomial_shift_condition hn)).trans
exact polynomial_uniformBranch_expression_le_rate hn8 hepsilon hCfix
hK hL
let delta : ℝ := 2 * (d : ℝ) * exp (-32 * logEN n)
have hdelta : 0 ≤ delta := by dsimp [delta]; positivity
have hpilotBad :
(Measure.infinitePi fun _ : ℕ => P.law.observedLaw).real
(polynomialPilotGood (n := n) P.law
(polynomialPilotLowerBand n) (polynomialPilotUpperBand n))ᶜ ≤
delta := by
exact polynomialPilotGood_compl_probability_calibrated_le P.law hn2
have hstream := polynomial_clipped_finiteSelector_risk P base hlower hR
hdelta hbranchFixed hpilotBad
have hdcal : (d : ℝ) ≤ (n : ℝ) * logEN n := by
simpa [rho] using hcal.2
have hbadAbs := polynomial_bad_event_absorption hn hL hdcal
have hstreamRate :
∫ omega : ℕ → Obs d,
(clip (-1) 1 (polynomialNormalizedSum M
(fun i : Fin n => omega i)) - rawAteFormula P.law / M) ^ 2
∂(Measure.infinitePi fun _ : ℕ => P.law.observedLaw) ≤
(D + 1) * (1 / (n : ℝ) + polynomialComponent n d) := by
apply hstream.trans
dsimp [delta]
have hinvRate : 1 / (n : ℝ) ≤
1 / (n : ℝ) + polynomialComponent n d := by
apply le_add_of_nonneg_right
unfold polynomialComponent
positivity
nlinarith
have hmse := mse_polyEstimator_le_of_clipped_stream P hcal hstreamRate
apply hmse.trans
have hDC : D + 1 ≤ C := le_max_right _ _
have hM2 : 0 ≤ M ^ 2 := sq_nonneg M
calc
M ^ 2 * ((D + 1) *
(1 / (n : ℝ) + polynomialComponent n d)) ≤
M ^ 2 * (C *
(1 / (n : ℝ) + polynomialComponent n d)) := by gcongr
_ = C * M ^ 2 *
(1 / (n : ℝ) + min 1 (polynomialComponent n d)) := by
have hpolyLe : polynomialComponent n d ≤ 1 := by
by_contra hnot
have hone : 1 < polynomialComponent n d := lt_of_not_ge hnot
have hdlarge : (n : ℝ) * logEN n < (d : ℝ) := by
unfold polynomialComponent at hone
have hnR : 0 < (n : ℝ) := by exact_mod_cast hn
have hlog : 0 < logEN n := by linarith
have hden : 0 < (n : ℝ) ^ 2 * logEN n ^ 2 := by positivity
rw [lt_div_iff₀ hden] at hone
nlinarith [sq_nonneg ((d : ℝ) - (n : ℝ) * logEN n)]
linarith
rw [min_eq_right hpolyLe]
ring
· have hfb := hfallback hn hd hcal P
exact hfb.trans (by
have hFC : Cfall ≤ C := le_max_left _ _
have hnonneg : 0 ≤ M ^ 2 *
(1 / (n : ℝ) + min 1 (polynomialComponent n d)) := by
have hpoly : 0 ≤ polynomialComponent n d := by
unfold polynomialComponent
positivity
positivity
nlinarith)
theorem continuous_ratio_polynomial_upper
Restricted-range form of the polynomial upper bound.
Formal statement
conclusion
∀ epsilon : ℝ,
0 < epsilon → epsilon < 1 / 2 → ∃ C_epsilon c_epsilon : ℝ, ∃ handle : PolynomialHandle, 0 < C_epsilon ∧ 0 < c_epsilon ∧ PolynomialComplexityBound handle ∧ ∀ n d : ℕ, ∀ M sigma : ℝ, 0 < n → 0 < d → 1
≤ M → 0 ≤ sigma → sigma ≤ 2 → (d : ℝ) ≤ c_epsilon * (n : ℝ) ^ 2 / logEN n → Measurable (polyEstimator handle (n := n) (d := d) M) ∧
(∀ s : Fin n → Obs d, polyEstimator handle (n := n) (d := d) M s ∈ Icc (-M) M) ∧
(∀ P : ModelClass d epsilon M sigma, mse P.law (polyEstimator handle (n := n) (d := d) M) ≤ C_epsilon * M ^ 2 * (1 / (n : ℝ) + min 1 (polynomialComponent n d)))
Proof (Lean source)
lemma continuous_ratio_polynomial_upper :
∀ epsilon : ℝ, 0 < epsilon → epsilon < 1 / 2 →
∃ C_epsilon c_epsilon : ℝ, ∃ handle : PolynomialHandle,
0 < C_epsilon ∧ 0 < c_epsilon ∧
PolynomialComplexityBound handle ∧
∀ n d : ℕ, ∀ M sigma : ℝ,
0 < n → 0 < d → 1 ≤ M → 0 ≤ sigma → sigma ≤ 2 →
(d : ℝ) ≤ c_epsilon * (n : ℝ) ^ 2 / logEN n →
Measurable
(polyEstimator handle (n := n) (d := d) M) ∧
(∀ s : Fin n → Obs d,
polyEstimator handle (n := n) (d := d) M s ∈ Icc (-M) M) ∧
(∀ P : ModelClass d epsilon M sigma,
mse P.law
(polyEstimator handle (n := n) (d := d) M) ≤
C_epsilon * M ^ 2 *
(1 / (n : ℝ) + min 1 (polynomialComponent n d))) := by
intro epsilon hepsilon hepsilon_half
obtain ⟨C_epsilon, handle, hC, hcomplexity, hbound⟩ :=
continuous_ratio_polynomial_upper_all_d epsilon hepsilon hepsilon_half
refine ⟨C_epsilon, 1, handle, hC, zero_lt_one, hcomplexity, ?_⟩
intro n d M sigma hn hd hM hsigma hsigma_two _
exact hbound n d M sigma hn hd hM hsigma hsigma_two
Helpers.PolynomialUpper.Bias 6 declarations
theorem shiftedCoefficient_sum_eq_gPolynomial Lemma shiftedCoefficient_sum_eq_gPolynomial in the paper ↗
The coefficients used by the real-outcome estimator are exactly the shifted-Chebyshev reciprocal polynomial from the binary construction.
Formal statement
hypotheses
K :
ℕ
x :
ℝ
conclusion
∑ j ∈ range (K - 1), shiftedCoefficient K j * x ^ j
= CausalSmith.Stat.DiscreteAteMinimaxLoggap.gPolynomial K x
Proof (Lean source)
lemma shiftedCoefficient_sum_eq_gPolynomial (K : ℕ) (x : ℝ) :
∑ j ∈ range (K - 1), shiftedCoefficient K j * x ^ j =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.gPolynomial K x := by
unfold CausalSmith.Stat.DiscreteAteMinimaxLoggap.gPolynomial
apply Finset.sum_congr rfl
intro j hj
rw [shiftedCoefficient_eq_gCoefficient]
If the polynomial or elbow parameter satisfies its stated bound and the outcome bound is positive and the overlap constant is positive and the probability lies in the stated range and the scaled probability satisfies the budget bound and the shifted probability satisfies the stated range and the shifted scaled probability satisfies the budget bound and the approximation argument lies in its stated range and the evaluation point lies in its stated range, on a genuinely light cell, the shifted-Chebyshev arm polynomial has the paper's deterministic B /(2 ε K²) bias bound.
Formal statement
hypotheses
K :
ℕ
hK :
0 < K
B epsilon p s nu z :
ℝ
hB :
0 < B
hepsilon :
0 < epsilon
hp :
0 ≤ p
_hpB :
p ≤ B
hsp :
epsilon * p ≤ s
hsB :
s ≤ B
hnu :
|nu| ≤ 1 / 2
hz :
z = s * nu
conclusion
|p * nu - p * z / B * (∑ j ∈ range (K - 1), shiftedCoefficient K j * (s / B) ^ j)|
≤ B / (2 * epsilon * (K : ℝ) ^ 2)
Proof (Lean source)
lemma lightArmPolynomial_bias_le {K : ℕ} (hK : 0 < K)
{B epsilon p s nu z : ℝ} (hB : 0 < B) (hepsilon : 0 < epsilon)
(hp : 0 ≤ p) (_hpB : p ≤ B) (hsp : epsilon * p ≤ s)
(hsB : s ≤ B) (hnu : |nu| ≤ 1 / 2) (hz : z = s * nu) :
|p * nu - p * z / B *
(∑ j ∈ range (K - 1), shiftedCoefficient K j * (s / B) ^ j)|
≤ B / (2 * epsilon * (K : ℝ) ^ 2) := by
have hs0 : 0 ≤ s := le_trans (mul_nonneg hepsilon.le hp) hsp
have hx : s / B ∈ Icc (0 : ℝ) 1 :=
⟨div_nonneg hs0 hB.le, (div_le_one hB).2 hsB⟩
have hcert :=
CausalSmith.Stat.DiscreteAteMinimaxLoggap.gPolynomial_certificate hK hx
rw [shiftedCoefficient_sum_eq_gPolynomial, hz]
have hKreal : 0 < (K : ℝ) := by exact_mod_cast hK
have hp_eps : p ≤ s / epsilon := by
apply (le_div_iff₀ hepsilon).2
simpa [mul_comm] using hsp
have hfactor : p * |nu| ≤ B / (2 * epsilon) * (s / B) := by
calc
p * |nu| ≤ (s / epsilon) * (1 / 2) :=
mul_le_mul hp_eps hnu (abs_nonneg nu) (div_nonneg hs0 hepsilon.le)
_ = B / (2 * epsilon) * (s / B) := by field_simp [hB.ne']
have hcoef0 : 0 ≤ B / (2 * epsilon) := by positivity
have hE0 : 0 ≤ |1 - (s / B) *
CausalSmith.Stat.DiscreteAteMinimaxLoggap.gPolynomial K (s / B)| :=
abs_nonneg _
have hKsq : 0 < (K : ℝ) ^ 2 := sq_pos_of_pos hKreal
have hcert' :
(s / B) * |1 - (s / B) *
CausalSmith.Stat.DiscreteAteMinimaxLoggap.gPolynomial K (s / B)| ≤
1 / (K : ℝ) ^ 2 := by
change _ ≤ ((K : ℝ) ^ 2)⁻¹ at hcert
rw [inv_eq_one_div] at hcert
exact hcert
have hid :
p * nu - p * (s * nu) / B *
CausalSmith.Stat.DiscreteAteMinimaxLoggap.gPolynomial K (s / B) =
p * nu * (1 - (s / B) *
CausalSmith.Stat.DiscreteAteMinimaxLoggap.gPolynomial K (s / B)) := by
field_simp [hB.ne']
rw [hid, abs_mul, abs_mul, abs_of_nonneg hp]
calc
p * |nu| *
|1 - (s / B) *
CausalSmith.Stat.DiscreteAteMinimaxLoggap.gPolynomial K (s / B)|
≤ (B / (2 * epsilon) * (s / B)) *
|1 - (s / B) *
CausalSmith.Stat.DiscreteAteMinimaxLoggap.gPolynomial K (s / B)| :=
mul_le_mul_of_nonneg_right hfactor hE0
_ = B / (2 * epsilon) * ((s / B) *
|1 - (s / B) *
CausalSmith.Stat.DiscreteAteMinimaxLoggap.gPolynomial K (s / B)|) := by
ring
_ ≤ B / (2 * epsilon) * (1 / (K : ℝ) ^ 2) :=
mul_le_mul_of_nonneg_left hcert' hcoef0
_ = B / (2 * epsilon * (K : ℝ) ^ 2) := by field_simp [hKsq.ne']
def polynomialLightCellPopulationTerm
Treated-minus-control population polynomial for one fixed light cell.
Definition (Lean source)
noncomputable def polynomialLightCellPopulationTerm {d : ℕ} (P : RealLaw d)
(M B : ℝ) (K : ℕ) (k : Fin d) : ℝ :=
let p := P.cellMass k
let s (a : Bool) := p * (if a then P.propensity k else 1 - P.propensity k)
let nu (a : Bool) := P.outcomeMean a k / M
let arm (a : Bool) := p * (s a * nu a) / B *
(∑ j ∈ range (K - 1),
shiftedCoefficient K j * (s a / B) ^ j)
arm true - arm false
theorem polynomialLightCellPopulationTerm_bias_le
If the polynomial or elbow parameter satisfies its stated bound and the outcome bound is positive and the cell is classified as light, on a genuinely light supported cell, the signed population polynomial approximates its normalized cell contribution with bias at most B /(ε K²).
Formal statement
hypotheses
d K :
ℕ
epsilon M sigma B :
ℝ
P :
ModelClass d epsilon M sigma
k :
Fin d
hK :
0 < K
hB :
0 < B
hlight :
P.law.cellMass k ≤ B
conclusion
|P.law.cellMass k * cellEffect P.law k / M
- polynomialLightCellPopulationTerm P.law M B K k|
≤ B / (epsilon * (K : ℝ) ^ 2)
Proof (Lean source)
lemma polynomialLightCellPopulationTerm_bias_le {d K : ℕ}
{epsilon M sigma B : ℝ} (P : ModelClass d epsilon M sigma)
(k : Fin d) (hK : 0 < K) (hB : 0 < B)
(hlight : P.law.cellMass k ≤ B) :
|P.law.cellMass k * cellEffect P.law k / M -
polynomialLightCellPopulationTerm P.law M B K k| ≤
B / (epsilon * (K : ℝ) ^ 2) := by
have hM : 0 < M := lt_of_lt_of_le zero_lt_one P.M_ge_one
by_cases hk : 0 < P.law.cellMass k
· let p := P.law.cellMass k
let q : Bool → ℝ := fun a =>
if a then P.law.propensity k else 1 - P.law.propensity k
let s : Bool → ℝ := fun a => p * q a
let nu : Bool → ℝ := fun a => P.law.outcomeMean a k / M
let arm : Bool → ℝ := fun a => p * (s a * nu a) / B *
(∑ j ∈ range (K - 1),
shiftedCoefficient K j * (s a / B) ^ j)
have hp0 : 0 ≤ p := (P.law.cellMass_range k).1
have hq_bounds (a : Bool) : q a ∈ Icc (epsilon : ℝ) 1 := by
have hov := P.overlap k hk
rcases P.law.propensity_range k with ⟨hprop0, hprop1⟩
cases a
· simp only [q, Bool.false_eq_true, ↓reduceIte]
constructor <;> linarith
· simp only [q, ↓reduceIte]
exact ⟨hov.1, hprop1⟩
have hsB (a : Bool) : s a ≤ B := by
calc
s a = p * q a := rfl
_ ≤ p * 1 := mul_le_mul_of_nonneg_left (hq_bounds a).2 hp0
_ ≤ B := by simpa using hlight
have hsp (a : Bool) : epsilon * p ≤ s a := by
simpa [s, mul_comm] using
mul_le_mul_of_nonneg_left (hq_bounds a).1 hp0
have hnu (a : Bool) : |nu a| ≤ 1 / 2 := by
dsimp [nu]
rw [abs_div, abs_of_pos hM]
apply (div_le_iff₀ hM).2
simpa [div_eq_mul_inv, mul_comm] using P.mean_normalization a k hk
have harm (a : Bool) :
|p * nu a - arm a| ≤ B / (2 * epsilon * (K : ℝ) ^ 2) := by
exact lightArmPolynomial_bias_le hK hB P.epsilon_pos hp0 hlight
(hsp a) (hsB a) (hnu a) rfl
have htarget :
P.law.cellMass k * cellEffect P.law k / M =
p * nu true - p * nu false := by
dsimp [p, nu, cellEffect]
field_simp [hM.ne']
have hpoly : polynomialLightCellPopulationTerm P.law M B K k =
arm true - arm false := by
rfl
rw [htarget, hpoly]
have hrearrange :
p * nu true - p * nu false - (arm true - arm false) =
(p * nu true - arm true) - (p * nu false - arm false) := by ring
rw [hrearrange]
calc
|(p * nu true - arm true) - (p * nu false - arm false)| ≤
|p * nu true - arm true| + |p * nu false - arm false| :=
abs_sub _ _
_ ≤ B / (2 * epsilon * (K : ℝ) ^ 2) +
B / (2 * epsilon * (K : ℝ) ^ 2) :=
add_le_add (harm true) (harm false)
_ = B / (epsilon * (K : ℝ) ^ 2) := by
ring
· have hp : P.law.cellMass k = 0 :=
le_antisymm (not_lt.mp hk) (P.law.cellMass_range k).1
have hright : 0 ≤ B / (epsilon * (K : ℝ) ^ 2) := by
exact div_nonneg hB.le
(mul_nonneg P.epsilon_pos.le (sq_nonneg (K : ℝ)))
simpa [polynomialLightCellPopulationTerm, hp] using hright
def polynomialFixedLightPopulationBias
Deterministic bias of the population polynomial over a fixed genuinely light set.
Definition (Lean source)
noncomputable def polynomialFixedLightPopulationBias {d : ℕ} (P : RealLaw d)
(M B : ℝ) (K : ℕ) (S : Finset (Fin d)) : ℝ :=
∑ k ∈ S, (P.cellMass k * cellEffect P k / M -
polynomialLightCellPopulationTerm P M B K k)
theorem polynomialFixedLightPopulationBias_abs_le
If the polynomial or elbow parameter satisfies its stated bound and the outcome bound is positive and the cell is classified as light, summing the cellwise Chebyshev approximation bound costs only the number of genuinely light cells.
Formal statement
hypotheses
d K :
ℕ
epsilon M sigma B :
ℝ
P :
ModelClass d epsilon M sigma
hK :
0 < K
hB :
0 < B
hlight :
∀ k ∈ S, P.law.cellMass k ≤ B
conclusion
|polynomialFixedLightPopulationBias P.law M B K S|
≤ (S.card : ℝ) * (B / (epsilon * (K : ℝ) ^ 2))
Proof (Lean source)
lemma polynomialFixedLightPopulationBias_abs_le {d K : ℕ}
{epsilon M sigma B : ℝ} (P : ModelClass d epsilon M sigma)
(S : Finset (Fin d)) (hK : 0 < K) (hB : 0 < B)
(hlight : ∀ k ∈ S, P.law.cellMass k ≤ B) :
|polynomialFixedLightPopulationBias P.law M B K S| ≤
(S.card : ℝ) * (B / (epsilon * (K : ℝ) ^ 2)) := by
unfold polynomialFixedLightPopulationBias
calc
|∑ k ∈ S, (P.law.cellMass k * cellEffect P.law k / M -
polynomialLightCellPopulationTerm P.law M B K k)| ≤
∑ k ∈ S, |P.law.cellMass k * cellEffect P.law k / M -
polynomialLightCellPopulationTerm P.law M B K k| :=
Finset.abs_sum_le_sum_abs _ _
_ ≤ ∑ _k ∈ S, B / (epsilon * (K : ℝ) ^ 2) := by
exact Finset.sum_le_sum fun k hk =>
polynomialLightCellPopulationTerm_bias_le P k hK hB (hlight k hk)
_ = (S.card : ℝ) * (B / (epsilon * (K : ℝ) ^ 2)) := by simp
Helpers.PolynomialUpper.Calibration 24 declarations
The explicit shifted-Chebyshev degree constant is strictly positive.
Formal statement
conclusion
0 < polynomialAlpha0
theorem polynomialAlpha0_le_one
the polynomial calibration constant is at most one.
Formal statement
conclusion
polynomialAlpha0 ≤ 1
Proof (Lean source)
lemma polynomialAlpha0_le_one : polynomialAlpha0 ≤ 1 := by
unfold polynomialAlpha0
exact min_le_left _ _
The declared degree constant fits the coefficient-growth budget used by the shifted-Chebyshev variance bound.
Formal statement
conclusion
polynomialAlpha0 ≤ 1 / (64 * log 6)
Proof (Lean source)
lemma polynomialAlpha0_le_log_six_budget :
polynomialAlpha0 ≤ 1 / (64 * log 6) := by
unfold polynomialAlpha0
exact (min_le_right _ _).trans (min_le_left _ _)
theorem polynomialAlpha0_le_covariance_budget Lemma polynomialAlpha0_le_covariance_budget in the paper ↗
The same degree constant fits the stronger covariance calibration budget.
Formal statement
conclusion
polynomialAlpha0 ≤ 1 / (512 * (8 * log (27 / 4)))
Proof (Lean source)
lemma polynomialAlpha0_le_covariance_budget :
polynomialAlpha0 ≤ 1 / (512 * (8 * log (27 / 4))) := by
unfold polynomialAlpha0
exact (min_le_right _ _).trans (min_le_right _ _)
theorem polynomialDegree_cast_le
Removing the floor can only increase the calibrated real-valued degree.
Formal statement
Proof (Lean source)
lemma polynomialDegree_cast_le (n : ℕ) :
(polynomialDegree n : ℝ) ≤ polynomialAlpha0 * logEN n := by
unfold polynomialDegree
apply Nat.floor_le
apply mul_nonneg polynomialAlpha0_pos.le
by_cases hn : n = 0
· subst n
simp [logEN]
· have hnR : 1 ≤ (n : ℝ) := by exact_mod_cast Nat.one_le_iff_ne_zero.mpr hn
unfold logEN
exact Real.log_nonneg (by
have : (1 : ℝ) ≤ exp 1 := Real.one_le_exp zero_le_one
nlinarith)
theorem logEN_nonneg_poly
the effective logarithmic sample size is nonnegative.
Formal statement
Proof (Lean source)
lemma logEN_nonneg_poly (n : ℕ) : 0 ≤ logEN n := by
by_cases hn : n = 0
· subst n
simp [logEN]
· have hnR : 1 ≤ (n : ℝ) := by exact_mod_cast Nat.one_le_iff_ne_zero.mpr hn
unfold logEN
exact Real.log_nonneg (by
have : (1 : ℝ) ≤ exp 1 := Real.one_le_exp zero_le_one
nlinarith)
theorem one_le_logEN
If the sample is nonempty, the effective logarithmic sample size is at least one.
Formal statement
Proof (Lean source)
lemma one_le_logEN {n : ℕ} (hn : 0 < n) : 1 ≤ logEN n := by
have hnR : 1 ≤ (n : ℝ) := by exact_mod_cast hn
rw [logEN, Real.log_mul (Real.exp_ne_zero 1)
(ne_of_gt (lt_of_lt_of_le zero_lt_one hnR)), Real.log_exp]
linarith [Real.log_nonneg hnR]
theorem polynomialDegree_cast_le_logEN
If the sample is nonempty, the polynomial degree is at most the effective logarithmic sample size.
Formal statement
Proof (Lean source)
lemma polynomialDegree_cast_le_logEN {n : ℕ} (hn : 0 < n) :
(polynomialDegree n : ℝ) ≤ logEN n := by
exact (polynomialDegree_cast_le n).trans
(mul_le_of_le_one_left (logEN_nonneg_poly n) polynomialAlpha0_le_one)
theorem polynomialDegree_cast_lower
If the polynomial or elbow parameter satisfies its stated bound, the polynomial degree satisfies its stated lower bound in terms of the effective logarithmic sample size.
Formal statement
hypotheses
n :
ℕ
hK :
2 ≤ polynomialDegree n
conclusion
polynomialAlpha0 * logEN n / 2 ≤ (polynomialDegree n : ℝ)
Proof (Lean source)
lemma polynomialDegree_cast_lower {n : ℕ} (hK : 2 ≤ polynomialDegree n) :
polynomialAlpha0 * logEN n / 2 ≤ (polynomialDegree n : ℝ) := by
have hfloor := Nat.lt_floor_add_one (polynomialAlpha0 * logEN n)
change polynomialAlpha0 * logEN n <
(polynomialDegree n : ℝ) + 1 at hfloor
have hKreal : (2 : ℝ) ≤ polynomialDegree n := by exact_mod_cast hK
nlinarith
theorem six_pow_two_degree_le_exp
The squared coefficient-growth factor is dominated by a small exponential power of the effective logarithmic sample size.
Formal statement
Proof (Lean source)
lemma six_pow_two_degree_le_exp (n : ℕ) :
(6 : ℝ) ^ (2 * polynomialDegree n) ≤
exp (logEN n / 32) := by
rw [Real.pow_le_iff_le_log (by norm_num) (Real.exp_pos _), Real.log_exp]
have hK := polynomialDegree_cast_le n
have ha := polynomialAlpha0_le_log_six_budget
have hlog6 : 0 < log 6 := Real.log_pos (by norm_num)
have hL := logEN_nonneg_poly n
have ha' : 2 * polynomialAlpha0 * log 6 ≤ 1 / 32 := by
calc
2 * polynomialAlpha0 * log 6 =
(2 * log 6) * polynomialAlpha0 := by ring
_ ≤ (2 * log 6) * (1 / (64 * log 6)) :=
mul_le_mul_of_nonneg_left ha (by positivity)
_ = 1 / 32 := by field_simp [hlog6.ne']; ring
norm_num only [Nat.cast_mul, Nat.cast_ofNat]
calc
2 * (polynomialDegree n : ℝ) * log 6 =
(2 * log 6) * (polynomialDegree n : ℝ) := by ring
_ ≤ (2 * log 6) * (polynomialAlpha0 * logEN n) :=
mul_le_mul_of_nonneg_left hK (by positivity)
_ = (2 * polynomialAlpha0 * log 6) * logEN n := by ring
_ ≤ (1 / 32) * logEN n := mul_le_mul_of_nonneg_right ha' hL
_ = logEN n / 32 := by ring
theorem six_pow_four_degree_le_exp
The fourth-power coefficient factor obeys the matching doubled exponent budget.
Formal statement
Proof (Lean source)
lemma six_pow_four_degree_le_exp (n : ℕ) :
(6 : ℝ) ^ (4 * polynomialDegree n) ≤
exp (logEN n / 16) := by
have h := six_pow_two_degree_le_exp n
have h0 : 0 ≤ (6 : ℝ) ^ (2 * polynomialDegree n) := by positivity
have he0 : 0 ≤ exp (logEN n / 32) := (Real.exp_pos _).le
calc
(6 : ℝ) ^ (4 * polynomialDegree n) =
((6 : ℝ) ^ (2 * polynomialDegree n)) ^ 2 := by
rw [show 4 * polynomialDegree n =
(2 * polynomialDegree n) * 2 by omega, pow_mul]
_ ≤ (exp (logEN n / 32)) ^ 2 :=
pow_le_pow_left₀ h0 h 2
_ = exp (logEN n / 16) := by
rw [← Real.exp_nat_mul]
congr 1
ring
theorem polynomial_estimationBlock_pos
If the sample is nonempty, the polynomial estimation block is nonempty.
Formal statement
hypotheses
n :
ℕ
hn :
0 < n
conclusion
0 < n - n / 2
Proof (Lean source)
lemma polynomial_estimationBlock_pos {n : ℕ} (hn : 0 < n) :
0 < n - n / 2 := by
omega
theorem polynomial_lightScale_pos
If the sample is nonempty, the polynomial light-cell scale is positive.
Formal statement
Proof (Lean source)
lemma polynomial_lightScale_pos {n : ℕ} (hn : 0 < n) :
0 < 4096 * logEN n / (n - n / 2 : ℕ) := by
have hm := polynomial_estimationBlock_pos hn
have hL := one_le_logEN hn
positivity
theorem polynomial_lightScale_le
If the sample is nonempty, the polynomial light-cell scale satisfies its stated upper bound.
Formal statement
Proof (Lean source)
lemma polynomial_lightScale_le {n : ℕ} (hn : 0 < n) :
4096 * logEN n / (n - n / 2 : ℕ) ≤
8192 * logEN n / (n : ℝ) := by
have hm : 0 < ((n - n / 2 : ℕ) : ℝ) := by
exact_mod_cast polynomial_estimationBlock_pos hn
have hnR : 0 < (n : ℝ) := by exact_mod_cast hn
have hL : 0 ≤ logEN n := logEN_nonneg_poly n
have htwice : n ≤ 2 * (n - n / 2) := by omega
have htwiceR : (n : ℝ) ≤ 2 * ((n - n / 2 : ℕ) : ℝ) := by
exact_mod_cast htwice
apply (div_le_div_iff₀ hm hnR).2
nlinarith
theorem polynomial_pilot_denominator_ge
If the sample size satisfies the stated lower bound and the overlap constant is positive, the pilot denominator satisfies the stated lower bound.
Formal statement
Proof (Lean source)
lemma polynomial_pilot_denominator_ge {n : ℕ} {epsilon : ℝ}
(hn : 8 ≤ n) (hepsilon : 0 < epsilon) :
4 * (n : ℝ) * epsilon ^ 2 * logEN n ≤
(((((n - n / 2 - 2 : ℕ) : ℝ) / 2 * epsilon) ^ 2) *
(128 * logEN n / (n / 2 : ℕ))) := by
have hnpos : 0 < n := by omega
have hnR : 0 < (n : ℝ) := by exact_mod_cast hnpos
have hpNat : 0 < n / 2 := Nat.div_pos (by omega) (by norm_num)
have hp : 0 < ((n / 2 : ℕ) : ℝ) := by exact_mod_cast hpNat
have hL : 0 < logEN n := zero_lt_one.trans_le (one_le_logEN hnpos)
have hblockNat : n ≤ 4 * (n - n / 2 - 2) := by omega
have hblock : (n : ℝ) / 8 ≤ ((n - n / 2 - 2 : ℕ) : ℝ) / 2 := by
have hc : (n : ℝ) ≤ 4 * ((n - n / 2 - 2 : ℕ) : ℝ) := by
exact_mod_cast hblockNat
linarith
have hlower : 256 * logEN n / (n : ℝ) ≤
128 * logEN n / (n / 2 : ℕ) := by
apply (div_le_div_iff₀ hnR hp).2
have hhalf : 2 * (n / 2) ≤ n := by omega
have hhalfR : 2 * ((n / 2 : ℕ) : ℝ) ≤ (n : ℝ) := by exact_mod_cast hhalf
nlinarith
have hsq : ((n : ℝ) / 8 * epsilon) ^ 2 ≤
(((n - n / 2 - 2 : ℕ) : ℝ) / 2 * epsilon) ^ 2 := by
have hmul := mul_le_mul_of_nonneg_right hblock hepsilon.le
nlinarith [sq_nonneg
((((n - n / 2 - 2 : ℕ) : ℝ) / 2 * epsilon) -
((n : ℝ) / 8 * epsilon))]
calc
4 * (n : ℝ) * epsilon ^ 2 * logEN n =
(((n : ℝ) / 8 * epsilon) ^ 2) *
(256 * logEN n / (n : ℝ)) := by field_simp [hnR.ne']; ring
_ ≤ ((((n - n / 2 - 2 : ℕ) : ℝ) / 2 * epsilon) ^ 2) *
(128 * logEN n / (n / 2 : ℕ)) :=
mul_le_mul hsq hlower (by positivity) (by positivity)
theorem polynomial_shift_condition
If the sample is nonempty, the calibrated polynomial shift satisfies the required approximation condition.
Formal statement
hypotheses
n :
ℕ
hn :
0 < n
conclusion
(4 : ℝ) * (polynomialDegree n + 2) / (n - n / 2 : ℕ)
≤ 3 * (4096 * logEN n / (n - n / 2 : ℕ)) / 4
Proof (Lean source)
lemma polynomial_shift_condition {n : ℕ} (hn : 0 < n) :
(4 : ℝ) * (polynomialDegree n + 2) / (n - n / 2 : ℕ) ≤
3 * (4096 * logEN n / (n - n / 2 : ℕ)) / 4 := by
have hm : 0 < ((n - n / 2 : ℕ) : ℝ) := by
exact_mod_cast polynomial_estimationBlock_pos hn
have hK := polynomialDegree_cast_le_logEN hn
have hL := one_le_logEN hn
rw [show 3 * (4096 * logEN n / ((n - n / 2 : ℕ) : ℝ)) / 4 =
(3072 * logEN n) / ((n - n / 2 : ℕ) : ℝ) by ring]
apply (div_le_div_iff_of_pos_right hm).2
push_cast
nlinarith
theorem polynomial_log_growth_four
If the sample is nonempty and the stated llarge condition holds, the fourth-power logarithmic growth term satisfies the stated sample-size bound.
Formal statement
hypotheses
conclusion
(6 : ℝ) ^ (4 * polynomialDegree n) * logEN n ^ 6 ≤ n
Proof (Lean source)
lemma polynomial_log_growth_four {n : ℕ} (hn : 0 < n)
(hLlarge : 240 ≤ logEN n) :
(6 : ℝ) ^ (4 * polynomialDegree n) * logEN n ^ 6 ≤ n := by
have hL : 0 < logEN n := by linarith
have hnR : 0 < (n : ℝ) := by exact_mod_cast hn
have hlogn : logEN n = 1 + log (n : ℝ) := by
rw [logEN, Real.log_mul (Real.exp_ne_zero 1) hnR.ne', Real.log_exp]
have hlog12 : log (12 : ℝ) ≤ 11 :=
(Real.log_le_sub_one_of_pos (by norm_num)).trans_eq (by norm_num)
have hlogL : log (logEN n) ≤ logEN n / 8 := by
have hsplit : log (logEN n) =
log 12 + log (logEN n / 12) := by
rw [← Real.log_mul (by norm_num : (12 : ℝ) ≠ 0) (by positivity)]
congr 1
field_simp
rw [hsplit]
have hsmall := Real.log_le_sub_one_of_pos (by positivity : 0 < logEN n / 12)
nlinarith
have hLpow : logEN n ^ 6 ≤ exp (3 * logEN n / 4) := by
rw [show logEN n ^ 6 = exp (6 * log (logEN n)) by
calc
logEN n ^ 6 = (exp (log (logEN n))) ^ 6 := by
rw [Real.exp_log hL]
_ = exp ((6 : ℕ) * log (logEN n)) := by
rw [Real.exp_nat_mul]
_ = _ := by norm_num]
exact Real.exp_le_exp.mpr (by nlinarith)
calc
(6 : ℝ) ^ (4 * polynomialDegree n) * logEN n ^ 6 ≤
exp (logEN n / 16) * exp (3 * logEN n / 4) := by
gcongr
exact six_pow_four_degree_le_exp n
_ = exp (13 * logEN n / 16) := by
rw [← Real.exp_add]
congr 1
ring
_ ≤ exp (logEN n - 1) := by
apply Real.exp_le_exp.mpr
nlinarith
_ = n := by
rw [hlogn]
simp [Real.exp_log hnR]
theorem polynomial_log_growth_two
If the sample is nonempty and the stated llarge condition holds, the squared logarithmic growth term satisfies the stated sample-size bound.
Formal statement
hypotheses
conclusion
(6 : ℝ) ^ (2 * polynomialDegree n) * logEN n ^ 6 ≤ n
Proof (Lean source)
lemma polynomial_log_growth_two {n : ℕ} (hn : 0 < n)
(hLlarge : 240 ≤ logEN n) :
(6 : ℝ) ^ (2 * polynomialDegree n) * logEN n ^ 6 ≤ n := by
have hpow : (6 : ℝ) ^ (2 * polynomialDegree n) ≤
(6 : ℝ) ^ (4 * polynomialDegree n) := by
exact pow_le_pow_right₀ (by norm_num) (by omega)
exact (mul_le_mul_of_nonneg_right hpow (by positivity)).trans
(polynomial_log_growth_four hn hLlarge)
theorem polynomial_fixedBranch_size_condition
If the sample is nonempty and the stated llarge condition holds, the calibrated fixed branch satisfies the required degree-versus-block-size condition.
Formal statement
hypotheses
conclusion
4 * (polynomialDegree n + 2) ^ 2 ≤ n - n / 2
Proof (Lean source)
lemma polynomial_fixedBranch_size_condition {n : ℕ} (hn : 0 < n)
(hLlarge : 240 ≤ logEN n) :
4 * (polynomialDegree n + 2) ^ 2 ≤ n - n / 2 := by
have hK := polynomialDegree_cast_le_logEN hn
have hgrowth := polynomial_log_growth_four hn hLlarge
have hsix : 1 ≤ (6 : ℝ) ^ (4 * polynomialDegree n) :=
one_le_pow₀ (by norm_num)
have hL : 0 ≤ logEN n := by linarith
have hL6 : logEN n ^ 6 ≤ (n : ℝ) := by
calc
logEN n ^ 6 = 1 * logEN n ^ 6 := by ring
_ ≤ (6 : ℝ) ^ (4 * polynomialDegree n) * logEN n ^ 6 := by gcongr
_ ≤ n := hgrowth
have hnum : (8 : ℝ) * (polynomialDegree n + 2) ^ 2 ≤ n := by
have hK2 : (polynomialDegree n : ℝ) + 2 ≤ 2 * logEN n := by
norm_num only [Nat.cast_add, Nat.cast_ofNat]
nlinarith
have hsq : ((polynomialDegree n : ℝ) + 2) ^ 2 ≤
(2 * logEN n) ^ 2 := by
have hp : 0 ≤
(2 * logEN n - ((polynomialDegree n : ℝ) + 2)) *
(2 * logEN n + ((polynomialDegree n : ℝ) + 2)) := by
positivity
nlinarith
have h32 : 32 * logEN n ^ 2 ≤ logEN n ^ 6 := by
have hp : (240 : ℝ) ^ 4 ≤ logEN n ^ 4 :=
pow_le_pow_left₀ (by norm_num) hLlarge 4
have hc : (32 : ℝ) ≤ logEN n ^ 4 := by norm_num at hp ⊢; linarith
calc
32 * logEN n ^ 2 ≤ logEN n ^ 4 * logEN n ^ 2 :=
mul_le_mul_of_nonneg_right hc (sq_nonneg _)
_ = logEN n ^ 6 := by ring
calc
(8 : ℝ) * (polynomialDegree n + 2) ^ 2 ≤
8 * (2 * logEN n) ^ 2 := by gcongr
_ = 32 * logEN n ^ 2 := by ring
_ ≤ logEN n ^ 6 := h32
_ ≤ n := hL6
have hhalf : (n : ℝ) / 2 ≤ ((n - n / 2 : ℕ) : ℝ) := by
have htwice : n ≤ 2 * (n - n / 2) := by omega
have htwiceR : (n : ℝ) ≤ 2 * ((n - n / 2 : ℕ) : ℝ) := by
exact_mod_cast htwice
linarith
exact_mod_cast (show (4 : ℝ) * (polynomialDegree n + 2) ^ 2 ≤
((n - n / 2 : ℕ) : ℝ) by nlinarith)
theorem logEN_eventually_ge_240
the effective logarithmic sample size is eventually at least 240.
Formal statement
conclusion
∃ N : ℕ, ∀ n : ℕ, N ≤ n → 240 ≤ logEN n
Proof (Lean source)
lemma logEN_eventually_ge_240 :
∃ N : ℕ, ∀ n : ℕ, N ≤ n → 240 ≤ logEN n := by
let N : ℕ := ceil (exp 239)
refine ⟨N, ?_⟩
intro n hn
have hceil : exp 239 ≤ N := Nat.le_ceil _
have hnreal : (N : ℝ) ≤ n := by exact_mod_cast hn
have hnpos : 0 < (n : ℝ) :=
lt_of_lt_of_le (Real.exp_pos _) (hceil.trans hnreal)
have hlog : (239 : ℝ) ≤ log n := by
rw [Real.le_log_iff_exp_le hnpos]
exact hceil.trans hnreal
rw [logEN, Real.log_mul (Real.exp_ne_zero 1) hnpos.ne', Real.log_exp]
linarith
theorem polynomial_bad_event_absorption
If the sample is nonempty and the stated llarge condition holds and the alphabet size satisfies the stated condition, the pilot bad-event term is absorbed by the target polynomial rate.
Formal statement
Proof (Lean source)
lemma polynomial_bad_event_absorption {n d : ℕ} (hn : 0 < n)
(hLlarge : 240 ≤ logEN n)
(hd : (d : ℝ) ≤ (n : ℝ) * logEN n) :
8 * (d : ℝ) * exp (-32 * logEN n) ≤ 1 / (n : ℝ) := by
have hnR : 0 < (n : ℝ) := by exact_mod_cast hn
have hL : 0 < logEN n := by linarith
have hnid : exp (logEN n - 1) = (n : ℝ) := by
rw [logEN, Real.log_mul (Real.exp_ne_zero 1) hnR.ne', Real.log_exp]
ring_nf
rw [Real.exp_log hnR]
have hLexp : logEN n ≤ exp (logEN n) := by
exact (show logEN n ≤ logEN n + 1 by linarith).trans
(Real.add_one_le_exp (logEN n))
have h8exp : (8 : ℝ) ≤ exp (logEN n) := by
calc
(8 : ℝ) ≤ 1 + logEN n := by linarith
_ ≤ exp (logEN n) := by
simpa [add_comm] using Real.add_one_le_exp (logEN n)
have hpolyexp : 8 * logEN n ≤ exp (2 * logEN n) := by
calc
8 * logEN n ≤ exp (logEN n) * exp (logEN n) :=
mul_le_mul h8exp hLexp hL.le (Real.exp_pos _).le
_ = exp (2 * logEN n) := by
rw [← Real.exp_add]
congr 1
ring
have hmajor : 8 * (n : ℝ) ^ 2 * logEN n ≤
exp (32 * logEN n) := by
rw [← hnid, ← Real.exp_nat_mul]
calc
8 * exp (2 * (logEN n - 1)) * logEN n =
(8 * logEN n) * exp (2 * (logEN n - 1)) := by ring
_ ≤ exp (2 * logEN n) * exp (2 * (logEN n - 1)) := by
gcongr
_ = exp (4 * logEN n - 2) := by
rw [← Real.exp_add]
congr 1
ring
_ ≤ exp (32 * logEN n) := by
apply Real.exp_le_exp.mpr
nlinarith
apply (le_div_iff₀ hnR).2
calc
8 * (d : ℝ) * exp (-32 * logEN n) * n ≤
8 * ((n : ℝ) * logEN n) * exp (-32 * logEN n) * n := by
gcongr
_ = (8 * (n : ℝ) ^ 2 * logEN n) * exp (-32 * logEN n) := by ring
_ ≤ exp (32 * logEN n) * exp (-32 * logEN n) := by
gcongr
_ = 1 := by rw [← Real.exp_add]; ring_nf; exact Real.exp_zero
Beyond an explicit finite cutoff, the calibrated polynomial degree is at least two.
Formal statement
conclusion
∃ N : ℕ, ∀ n : ℕ, N ≤ n → 2 ≤ polynomialDegree n
Proof (Lean source)
lemma polynomialDegree_eventually_two :
∃ N : ℕ, ∀ n : ℕ, N ≤ n → 2 ≤ polynomialDegree n := by
let N : ℕ := ceil (exp (2 / polynomialAlpha0))
refine ⟨N, ?_⟩
intro n hn
unfold polynomialDegree
apply Nat.le_floor
have hceil : exp (2 / polynomialAlpha0) ≤ N := Nat.le_ceil _
have hnreal : (N : ℝ) ≤ n := by exact_mod_cast hn
have hnpos : 0 < (n : ℝ) :=
lt_of_lt_of_le (Real.exp_pos _) (hceil.trans hnreal)
have hlog : 2 / polynomialAlpha0 ≤ log n := by
rw [Real.le_log_iff_exp_le hnpos]
exact hceil.trans hnreal
have hlogEN : log n ≤ logEN n := by
rw [logEN, Real.log_mul (Real.exp_ne_zero 1) hnpos.ne', Real.log_exp]
linarith
have hscaled := mul_le_mul_of_nonneg_left
(hlog.trans hlogEN) polynomialAlpha0_pos.le
have hcancel : polynomialAlpha0 * (2 / polynomialAlpha0) = 2 := by
field_simp [polynomialAlpha0_pos.ne']
rw [hcancel] at hscaled
exact hscaled
If the outcome scale satisfies its stated bound, for every cutoff choice, the concrete estimator is measurable, uses its declared zero fallback, and is clipped to the required range.
Formal statement
hypotheses
n d N :
ℕ
M rho :
ℝ
hM :
0 ≤ M
conclusion
Measurable (rawPolyEstimator (n := n) (d := d) N rho M) ∧
∀ s : Fin n → Obs d, rawPolyEstimator (n := n) (d := d) N rho M s ∈ Icc (-M) M
Proof (Lean source)
lemma polyEstimator_total_and_clipped {n d N : ℕ} {M rho : ℝ} (hM : 0 ≤ M) :
Measurable (rawPolyEstimator (n := n) (d := d) N rho M) ∧
∀ s : Fin n → Obs d,
rawPolyEstimator (n := n) (d := d) N rho M s ∈ Icc (-M) M := by
exact polyEstimator_admissible hM
One finite cutoff simultaneously supplies the degree side condition and, for every alphabet and scale, the total clipped estimator certificate used in the all-alphabet assembly.
Formal statement
conclusion
∃ N : ℕ,
∀ n d : ℕ,
∀ M rho : ℝ,
0 ≤ M → (N ≤ n ∧ (d : ℝ) ≤ rho * n * logEN n → 2 ≤ polynomialDegree n) ∧
Measurable (rawPolyEstimator (n := n) (d := d) N rho M) ∧
(∀ s : Fin n → Obs d, rawPolyEstimator (n := n) (d := d) N rho M s ∈ Icc (-M) M)
Proof (Lean source)
lemma polynomialCalibrationPackage :
∃ N : ℕ, ∀ n d : ℕ, ∀ M rho : ℝ,
0 ≤ M →
(N ≤ n ∧ (d : ℝ) ≤ rho * n * logEN n → 2 ≤ polynomialDegree n) ∧
Measurable (rawPolyEstimator (n := n) (d := d) N rho M) ∧
(∀ s : Fin n → Obs d,
rawPolyEstimator (n := n) (d := d) N rho M s ∈ Icc (-M) M) := by
obtain ⟨N, hN⟩ := polynomialDegree_eventually_two
refine ⟨N, ?_⟩
intro n d M rho hM
obtain ⟨hmeas, hrange⟩ :=
polyEstimator_total_and_clipped (n := n) (d := d) (N := N)
(M := M) (rho := rho) hM
exact ⟨fun hcalibrated => hN n hcalibrated.1, hmeas, hrange⟩
Helpers.PolynomialUpper.ClippingAssembly 4 declarations
def polynomialNormalizedSum
The normalized heavy/light sum before the estimator's final clipping.
Definition (Lean source)
noncomputable def polynomialNormalizedSum {n d : ℕ} (M : ℝ)
(sample : Fin n → Obs d) : ℝ :=
let K := polynomialDegree n
let B := 4096 * logEN n / (n - n / 2 : ℕ)
∑ k : Fin d,
if 256 * logEN n < pilotCount sample k then
heavyEmpiricalTerm M sample k
else lightPolynomialTerm M B K sample k
theorem polyEstimator_eq_scaled_clip_of_calibrated Lemma polyEstimator_eq_scaled_clip_of_calibrated in the paper ↗
If the indicated calibration branch applies, on the calibrated branch, the concrete estimator is exactly the outcome scale times the clipped normalized heavy/light sum.
Formal statement
hypotheses
conclusion
rawPolyEstimator (n := n) (d := d) N rho M
= fun sample => M * clip (-1) 1 (polynomialNormalizedSum M sample)
Proof (Lean source)
lemma polyEstimator_eq_scaled_clip_of_calibrated {n d N : ℕ} {M rho : ℝ}
(hbranch : N ≤ n ∧ (d : ℝ) ≤ rho * n * logEN n) :
rawPolyEstimator (n := n) (d := d) N rho M =
fun sample => M * clip (-1) 1 (polynomialNormalizedSum M sample) := by
funext sample
simp [rawPolyEstimator, hbranch, polynomialNormalizedSum]
theorem modelClass_normalized_ate_mem_Icc
The model's ATE divided by its positive outcome scale lies in the clipping interval.
Formal statement
hypotheses
conclusion
rawAteFormula P.law / M ∈ Icc (-1 : ℝ) 1
Proof (Lean source)
lemma modelClass_normalized_ate_mem_Icc {d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) :
rawAteFormula P.law / M ∈ Icc (-1 : ℝ) 1 := by
let Q : UnrestrictedClass d epsilon M :=
{ law := P.law
epsilon_pos := P.epsilon_pos
epsilon_lt_half := P.epsilon_lt_half
M_ge_one := P.M_ge_one
consistency := P.consistency
exchangeability := P.exchangeability
overlap := P.overlap
mean_normalization := P.mean_normalization
second_moment := P.second_moment }
have hraw : |rawAteFormula P.law| ≤ M := scale_sanity.1 Q |>.2.1
have hM : 0 < M := lt_of_lt_of_le zero_lt_one P.M_ge_one
constructor
· apply (le_div_iff₀ hM).2
simpa using (neg_le_of_abs_le hraw)
· rw [div_le_one hM]
exact le_of_abs_le hraw
theorem mse_polyEstimator_le_of_normalized_error Lemma mse_polyEstimator_le_of_normalized_error in the paper ↗
If the indicated calibration branch applies and the specified random quantity is integrable and the raw normalized-error bound holds, a normalized pre-clipping mean-square bound transfers to the scaled, clipped estimator whenever the normalized target lies in [-1,1].
Formal statement
hypotheses
n d N :
ℕ
epsilon M sigma rho R :
ℝ
P :
ModelClass d epsilon M sigma
hbranch :
N ≤ n ∧ (d : ℝ) ≤ rho * n * logEN n
hright :
Integrable (fun sample : Fin n → Obs d => M ^ 2 * (polynomialNormalizedSum M sample - rawAteFormula P.law / M) ^ 2) (productLaw n P.law)
hraw :
(∫ sample : Fin n → Obs d, (polynomialNormalizedSum M sample - rawAteFormula P.law / M) ^ 2 ∂productLaw n P.law)
≤ R
conclusion
mse P.law (rawPolyEstimator (n := n) (d := d) N rho M) ≤ M ^ 2 * R
Proof (Lean source)
lemma mse_polyEstimator_le_of_normalized_error {n d N : ℕ}
{epsilon M sigma rho R : ℝ} (P : ModelClass d epsilon M sigma)
(hbranch : N ≤ n ∧ (d : ℝ) ≤ rho * n * logEN n)
(hright : Integrable
(fun sample : Fin n → Obs d =>
M ^ 2 * (polynomialNormalizedSum M sample -
rawAteFormula P.law / M) ^ 2)
(productLaw n P.law))
(hraw : (∫ sample : Fin n → Obs d,
(polynomialNormalizedSum M sample - rawAteFormula P.law / M) ^ 2
∂productLaw n P.law) ≤ R) :
mse P.law (rawPolyEstimator (n := n) (d := d) N rho M) ≤ M ^ 2 * R := by
have hM : M ≠ 0 := ne_of_gt (lt_of_lt_of_le zero_lt_one P.M_ge_one)
have hpoint (sample : Fin n → Obs d) :
(rawPolyEstimator N rho M sample - rawAteFormula P.law) ^ 2 ≤
M ^ 2 * (polynomialNormalizedSum M sample -
rawAteFormula P.law / M) ^ 2 := by
rw [polyEstimator_eq_scaled_clip_of_calibrated hbranch]
have hclip := clip_normalized_sq_error_le
(polynomialNormalizedSum M sample) (rawAteFormula P.law / M)
(modelClass_normalized_ate_mem_Icc P)
have hid :
(M * clip (-1) 1 (polynomialNormalizedSum M sample) -
rawAteFormula P.law) ^ 2 =
M ^ 2 * (clip (-1) 1 (polynomialNormalizedSum M sample) -
rawAteFormula P.law / M) ^ 2 := by
field_simp [hM]
rw [hid]
exact mul_le_mul_of_nonneg_left hclip (sq_nonneg M)
have hleft : Integrable
(fun sample : Fin n → Obs d =>
(rawPolyEstimator N rho M sample - rawAteFormula P.law) ^ 2)
(productLaw n P.law) := by
have hadm := polyEstimator_admissible (n := n) (d := d) (N := N)
(rho := rho) (le_trans zero_le_one P.M_ge_one)
have htau : |rawAteFormula P.law| ≤ M := by
let Q : UnrestrictedClass d epsilon M :=
{ law := P.law
epsilon_pos := P.epsilon_pos
epsilon_lt_half := P.epsilon_lt_half
M_ge_one := P.M_ge_one
consistency := P.consistency
exchangeability := P.exchangeability
overlap := P.overlap
mean_normalization := P.mean_normalization
second_moment := P.second_moment }
exact scale_sanity.1 Q |>.2.1
have hM0 : 0 ≤ M := le_trans zero_le_one P.M_ge_one
apply Integrable.of_bound (C := (2 * M) ^ 2)
((hadm.1.sub measurable_const).pow_const 2).aestronglyMeasurable
filter_upwards with sample
rw [Real.norm_eq_abs, abs_of_nonneg (sq_nonneg _)]
have hest : |rawPolyEstimator N rho M sample| ≤ M :=
(abs_le).2 (hadm.2 sample)
have habs : |rawPolyEstimator N rho M sample - rawAteFormula P.law| ≤ 2 * M :=
(abs_sub _ _).trans (by linarith)
exact (sq_le_sq).2 (by
simpa [abs_mul, abs_of_nonneg hM0] using habs)
unfold mse
calc
(∫ sample, (rawPolyEstimator N rho M sample - rawAteFormula P.law) ^ 2
∂productLaw n P.law) ≤
∫ sample, M ^ 2 * (polynomialNormalizedSum M sample -
rawAteFormula P.law / M) ^ 2 ∂productLaw n P.law :=
integral_mono hleft hright hpoint
_ = M ^ 2 * (∫ sample,
(polynomialNormalizedSum M sample - rawAteFormula P.law / M) ^ 2
∂productLaw n P.law) := by rw [integral_const_mul]
_ ≤ M ^ 2 * R := mul_le_mul_of_nonneg_left hraw (sq_nonneg M)
Helpers.PolynomialUpper.Complexity 13 declarations
This straight-line program computes an empirical treatment-arm mean.
Definition (Lean source)
noncomputable def polynomialArmMeanCode {d : ℕ} (a : Bool) (k : Fin d) :
AggregatedArithmeticProgram d :=
.iteLt (.const 0) (.count a k) (.div (.outcomeSum a k) (.count a k)) (.const 0)
This straight-line program computes the heavy-cell contribution to the polynomial estimator.
Definition (Lean source)
noncomputable def polynomialHeavyCode {n d : ℕ} (M : ℝ) (k : Fin d) :
AggregatedArithmeticProgram d :=
.mul
(.div (.add (.count false k) (.count true k)) (.const (estimationBlockSize n)))
(.div (.sub (polynomialArmMeanCode true k) (polynomialArmMeanCode false k))
(.const M))
This straight-line program computes a marked polynomial term.
Definition (Lean source)
noncomputable def polynomialMarkedCode {n d : ℕ} (M : ℝ)
(k : Fin d) (a : Bool) (j : ℕ) : AggregatedArithmeticProgram d :=
.div
(.mul (.mul (.div (.outcomeSum a k) (.const M))
(.armDescFactorial a k 1 j)) (.cellSub k (j + 1)))
(.const ((estimationBlockSize n).descFactorial (j + 2)))
This straight-line program computes the light-cell polynomial contribution.
Definition (Lean source)
noncomputable def polynomialLightCode {n d : ℕ} (M B : ℝ) (K : ℕ)
(k : Fin d) : AggregatedArithmeticProgram d :=
.sumTerms (K - 1) fun j =>
.mul (.const (shiftedCoefficient K j.val / B ^ (j.val + 1)))
(.sub (polynomialMarkedCode (n := n) M k true j.val)
(polynomialMarkedCode (n := n) M k false j.val))
This straight-line program assembles the calibrated polynomial estimator.
Definition (Lean source)
noncomputable def calibratedPolynomialCode (n d : ℕ) (M : ℝ) :
AggregatedArithmeticProgram d :=
let K := polynomialDegree n
let B := 4096 * logEN n / (estimationBlockSize n : ℕ)
.mul (.const M)
(.maximum (.const (-1)) (.minimum (.const 1)
(.sumCells fun k =>
.iteLt (.const (256 * logEN n)) (.pilot k)
(polynomialHeavyCode (n := n) M k)
(polynomialLightCode (n := n) M B K k))))
theorem polynomialMarkedCode_eval
Each marked syntax node evaluates to the aggregate falling-factorial statistic.
Formal statement
hypotheses
conclusion
(polynomialMarkedCode (n := n) M k a j).eval (aggregatePolynomialSample sample)
= orderedMarkedFactorial M sample k a j
Proof (Lean source)
lemma polynomialMarkedCode_eval {n d : ℕ} (M : ℝ) (sample : Fin n → Obs d)
(k : Fin d) (a : Bool) (j : ℕ) :
(polynomialMarkedCode (n := n) M k a j).eval
(aggregatePolynomialSample sample) =
orderedMarkedFactorial M sample k a j := by
simp [polynomialMarkedCode, AggregatedArithmeticProgram.eval,
aggregatePolynomialSample, orderedMarkedFactorial, estimationCellCount,
estimationBlockSize]
theorem polynomialLightCode_eval
Each light-cell syntax subtree evaluates to the declared polynomial term.
Formal statement
hypotheses
conclusion
(polynomialLightCode (n := n) M B K k).eval (aggregatePolynomialSample sample)
= lightPolynomialTerm M B K sample k
Proof (Lean source)
lemma polynomialLightCode_eval {n d : ℕ} (M B : ℝ) (K : ℕ)
(sample : Fin n → Obs d) (k : Fin d) :
(polynomialLightCode (n := n) M B K k).eval
(aggregatePolynomialSample sample) =
lightPolynomialTerm M B K sample k := by
classical
unfold polynomialLightCode lightPolynomialTerm
simp only [AggregatedArithmeticProgram.eval]
simp_rw [polynomialMarkedCode_eval]
simpa using Fin.sum_univ_eq_sum_range
(fun j : ℕ => shiftedCoefficient K j / B ^ (j + 1) *
(orderedMarkedFactorial M sample k true j -
orderedMarkedFactorial M sample k false j)) (K - 1)
theorem calibratedPolynomialCode_computes
Evaluation of the explicit aggregate syntax is the calibrated estimator branch.
Formal statement
hypotheses
conclusion
(calibratedPolynomialCode n d M).eval (aggregatePolynomialSample sample)
= M * clip (-1) 1 (polynomialNormalizedSum M sample)
Proof (Lean source)
lemma calibratedPolynomialCode_computes {n d : ℕ} (M : ℝ)
(sample : Fin n → Obs d) :
(calibratedPolynomialCode n d M).eval (aggregatePolynomialSample sample) =
M * clip (-1) 1 (polynomialNormalizedSum M sample) := by
classical
simp only [calibratedPolynomialCode, AggregatedArithmeticProgram.eval,
polynomialLightCode_eval]
simp [polynomialHeavyCode, polynomialArmMeanCode, aggregatePolynomialSample,
AggregatedArithmeticProgram.eval, polynomialNormalizedSum,
heavyEmpiricalTerm, estimationArmMean,
estimationCellCount, estimationBlockSize, clip]
theorem calibratedPolynomialCode_operationCount_le
The explicit syntax fits the fixed quadratic-in-degree per-cell budget.
Formal statement
hypotheses
n d :
ℕ
M :
ℝ
conclusion
(calibratedPolynomialCode n d M).operationCount ≤ polynomialOperationBudget n d
Proof (Lean source)
lemma calibratedPolynomialCode_operationCount_le (n d : ℕ) (M : ℝ) :
(calibratedPolynomialCode n d M).operationCount ≤ polynomialOperationBudget n d := by
classical
let K := polynomialDegree n
have hterm (j : Fin (K - 1)) : 12 + 2 * j.val ≤ 14 * (K + 1) := by
omega
have hsum : (∑ j : Fin (K - 1), (12 + 2 * j.val)) ≤
(K - 1) * (14 * (K + 1)) := by
have h := Finset.sum_le_sum (s := univ) fun j _ => hterm j
simpa using h
dsimp [K] at hsum
have hsum' :
(∑ j : Fin (polynomialDegree n - 1),
(1 + (1 + (1 + (1 + (2 + j.val) + 1)) +
(1 + (1 + (2 + j.val) + 1))))) ≤
(polynomialDegree n - 1) * (14 * (polynomialDegree n + 1)) := by
calc
_ = ∑ j : Fin (polynomialDegree n - 1), (12 + 2 * j.val) := by
apply Finset.sum_congr rfl
intro j _
omega
_ ≤ _ := hsum
have hlight : polynomialDegree n - 1 +
(∑ j : Fin (polynomialDegree n - 1),
(1 + (1 + (1 + (1 + (2 + j.val) + 1)) +
(1 + (1 + (2 + j.val) + 1))))) ≤
16 * (polynomialDegree n + 1) ^ 2 := by
have hk : polynomialDegree n - 1 ≤ polynomialDegree n := Nat.sub_le _ _
have hprod : (polynomialDegree n - 1) *
(14 * (polynomialDegree n + 1)) ≤
polynomialDegree n * (14 * (polynomialDegree n + 1)) :=
Nat.mul_le_mul_right _ hk
have hcoarse := Nat.add_le_add hk (hsum'.trans hprod)
apply hcoarse.trans
nlinarith
have hsq : 1 ≤ (polynomialDegree n + 1) ^ 2 := by
nlinarith
have hmax : max 9 (polynomialDegree n - 1 +
(∑ j : Fin (polynomialDegree n - 1),
(1 + (1 + (1 + (1 + (2 + j.val) + 1)) +
(1 + (1 + (2 + j.val) + 1)))))) ≤
16 * (polynomialDegree n + 1) ^ 2 := by
exact max_le (by nlinarith [hsq]) hlight
simp [calibratedPolynomialCode, polynomialHeavyCode, polynomialArmMeanCode,
polynomialLightCode, polynomialMarkedCode,
AggregatedArithmeticProgram.operationCount, polynomialOperationBudget]
have hmul := Nat.mul_le_mul_left d (Nat.add_le_add_left hmax 1)
nlinarith [hmul, hsq]
theorem polynomialComplexityBound_of_calibrated Lemma polynomialComplexityBound_of_calibrated in the paper ↗
If the indicated calibration branch applies, the calibrated branch has a concrete aggregate program with the advertised budget.
Formal statement
hypotheses
conclusion
PolynomialExactComplexityCertificate n d handle M
Proof (Lean source)
lemma polynomialComplexityBound_of_calibrated {n d : ℕ}
{handle : PolynomialHandle} {M : ℝ}
(hbranch : handle.N ≤ n ∧ (d : ℝ) ≤ handle.rho * n * logEN n) :
PolynomialExactComplexityCertificate n d handle M := by
let program : AggregatedPolynomialProgram n d handle M :=
{ code := calibratedPolynomialCode n d M
computesEstimator := by
intro sample
rw [calibratedPolynomialCode_computes]
simp [polyEstimator, rawPolyEstimator, hbranch, polynomialNormalizedSum] }
exact ⟨program, calibratedPolynomialCode_operationCount_le n d M⟩
theorem polynomialComplexityBound_of_not_calibrated Lemma polynomialComplexityBound_of_not_calibrated in the paper ↗
If the indicated calibration branch applies, on either declared fallback branch, the constant-zero aggregate program computes the estimator and has zero arithmetic cost.
Formal statement
hypotheses
conclusion
PolynomialExactComplexityCertificate n d handle M
Proof (Lean source)
lemma polynomialComplexityBound_of_not_calibrated {n d : ℕ}
{handle : PolynomialHandle} {M : ℝ}
(hbranch : ¬ (handle.N ≤ n ∧
(d : ℝ) ≤ handle.rho * n * logEN n)) :
PolynomialExactComplexityCertificate n d handle M := by
let program : AggregatedPolynomialProgram n d handle M :=
{ code := .const 0
computesEstimator := by
intro sample
simp [AggregatedArithmeticProgram.eval, polyEstimator,
rawPolyEstimator, hbranch] }
refine ⟨program, ?_⟩
exact zero_le _
theorem polynomialComplexityBound_all_branches Lemma polynomialComplexityBound_all_branches in the paper ↗
If the calibration handle is available, every branch of the total polynomial estimator has an executable certificate.
Formal statement
hypotheses
conclusion
PolynomialExactComplexityCertificate n d handle M
Proof (Lean source)
lemma polynomialComplexityBound_all_branches (n d : ℕ)
(handle : PolynomialHandle) (M : ℝ) :
PolynomialExactComplexityCertificate n d handle M := by
by_cases hbranch : handle.N ≤ n ∧
(d : ℝ) ≤ handle.rho * n * logEN n
· exact polynomialComplexityBound_of_calibrated hbranch
· exact polynomialComplexityBound_of_not_calibrated hbranch
theorem polynomialComplexityBound
If the calibration handle is available, the fixed 128 proof-local certificate implies the public family-level O(d K²) arithmetic statement.
Formal statement
Proof (Lean source)
lemma polynomialComplexityBound (handle : PolynomialHandle) :
PolynomialComplexityBound handle := by
refine ⟨128, by norm_num, ?_⟩
intro n d M
obtain ⟨program, hprogram⟩ :=
polynomialComplexityBound_all_branches n d handle M
exact ⟨program, by simpa [polynomialOperationBudget] using hprogram⟩
Helpers.PolynomialUpper.Expectation 5 declarations
theorem integral_markedFactorialCoordinate_exact
The marked coordinate has mean p_k q_{ak} μ_{ak}/M, the cell-only coordinate has mean p_k, and every later arm-cell coordinate has mean p_k q_{ak}. This is the exact coordinate audit behind the marked-factorial expectation formula.
Formal statement
hypotheses
conclusion
(∫ o : Obs d, markedFactorialCoordinate M k a j q o ∂P.law.observedLaw)
= if q.val = 0 then P.law.cellMass k * (if a then P.law.propensity k else 1 - P.law.propensity k) * (P.law.outcomeMean a k / M) else if q.val = 1 then P.law.cellMass k else P.law.cellMass k * (if a then P.law.propensity k else 1 - P.law.propensity k)
Proof (Lean source)
lemma integral_markedFactorialCoordinate_exact
{d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (k : Fin d) (a : Bool) (j : ℕ)
(q : Fin (j + 2)) :
(∫ o : Obs d, markedFactorialCoordinate M k a j q o
∂P.law.observedLaw) =
if q.val = 0 then
P.law.cellMass k *
(if a then P.law.propensity k else 1 - P.law.propensity k) *
(P.law.outcomeMean a k / M)
else if q.val = 1 then P.law.cellMass k
else P.law.cellMass k *
(if a then P.law.propensity k else 1 - P.law.propensity k) := by
have htuple : Measurable (fun o : Obs d => (o.x, o.a, o.y)) :=
measurable_iff_comap_le.mpr le_rfl
have hx : Measurable (fun o : Obs d => o.x) := measurable_fst.comp htuple
have ha : Measurable (fun o : Obs d => o.a) :=
measurable_fst.comp (measurable_snd.comp htuple)
have hy : Measurable (fun o : Obs d => o.y) :=
measurable_snd.comp (measurable_snd.comp htuple)
have hxset : MeasurableSet {o : Obs d | o.x = k} :=
(measurableSet_singleton k).preimage hx
have hxa : MeasurableSet {o : Obs d | o.x = k ∧ o.a = a} := by
simpa only [preimage, Set.mem_singleton_iff, Prod.mk.injEq] using
(measurableSet_singleton (k, a)).preimage (prodMk hx ha)
let c : ℝ := P.law.cellMass k *
(if a then P.law.propensity k else 1 - P.law.propensity k)
have hc0 : 0 ≤ c := by
dsimp [c]
apply mul_nonneg (P.law.cellMass_range k).1
split <;> simp_all [(P.law.propensity_range k).1,
(P.law.propensity_range k).2]
by_cases hq0 : q.val = 0
· rw [if_pos hq0]
have hq : q = 0 := Fin.ext hq0
subst q
simp only [markedFactorialCoordinate, Fin.val_zero, ↓reduceIte]
have hmark :
(fun o : Obs d => o.y / M * if o.x = k ∧ o.a = a then 1 else 0) =
{o : Obs d | o.x = k ∧ o.a = a}.indicator (fun o => o.y / M) := by
funext o
by_cases ho : o.x = k ∧ o.a = a <;> simp [indicator, ho]
rw [hmark]
by_cases hk : P.law.cellMass k = 0
· have hnull := observed_arm_cell_measure_eq_zero_of_cellMass_eq_zero
P.law a k hk
rw [integral_indicator hxa]
have hr : P.law.observedLaw.restrict {o : Obs d | o.x = k ∧ o.a = a} = 0 :=
Measure.restrict_eq_zero.mpr hnull
simp [hr, hk]
· have hkpos : 0 < P.law.cellMass k :=
lt_of_le_of_ne (P.law.cellMass_range k).1 (Ne.symm hk)
have hint := (normalized_outcome_mean_abs_le_half P a k hkpos).1
rw [integral_indicator hxa]
have hmap := observed_arm_cell_outcome_measure P.law a k
have hmeas : AEStronglyMeasurable (fun y : ℝ => y / M)
(Measure.map (fun o : Obs d => o.y)
(P.law.observedLaw.restrict {o : Obs d | o.x = k ∧ o.a = a})) :=
(measurable_id.div measurable_const).aestronglyMeasurable
rw [← integral_map hy.aemeasurable hmeas, hmap, integral_smul_measure]
change toReal (ofReal c) *
(∫ y, y / M ∂P.law.outcomeLaw a k) = _
rw [ENNReal.toReal_ofReal hc0, integral_div, ← P.law.outcomeMean_eq]
· rw [if_neg hq0]
by_cases hq1 : q.val = 1
· rw [if_pos hq1]
have heq : (fun o : Obs d => markedFactorialCoordinate M k a j q o) =
{o : Obs d | o.x = k}.indicator (fun _ => (1 : ℝ)) := by
funext o
simp [markedFactorialCoordinate, hq1, indicator]
rw [heq, integral_indicator hxset]
rw [show (∫ _x : Obs d in {o : Obs d | o.x = k}, (1 : ℝ)
∂P.law.observedLaw) = realMass P.law.observedLaw {o | o.x = k} by
simp [realMass, Measure.real_def]]
exact (P.law.cellMass_eq k).symm
· rw [if_neg hq1]
have heq : (fun o : Obs d => markedFactorialCoordinate M k a j q o) =
{o : Obs d | o.x = k ∧ o.a = a}.indicator (fun _ => (1 : ℝ)) := by
funext o
simp [markedFactorialCoordinate, hq0, hq1, indicator]
rw [heq, integral_indicator hxa]
have hmass := P.law.arm_outcome_factorization a k Set.univ MeasurableSet.univ
let _ : IsProbabilityMeasure (P.law.outcomeLaw a k) :=
P.law.outcome_isProbability a k
simp [realMass] at hmass
simpa [Measure.real_def, realMass] using hmass.symm
theorem orderedProductMean_markedFactorialCoordinate_exact
Independence across the ordered coordinates gives exactly one normalized outcome mark, one cell selector, and j additional arm-cell selectors.
Formal statement
hypotheses
conclusion
orderedProductMean P.law.observedLaw (markedFactorialCoordinate M k a j)
= (P.law.cellMass k * (if a then P.law.propensity k else 1 - P.law.propensity k) * (P.law.outcomeMean a k / M)) * P.law.cellMass k * (P.law.cellMass k * (if a then P.law.propensity k else 1 - P.law.propensity k)) ^ j
Proof (Lean source)
lemma orderedProductMean_markedFactorialCoordinate_exact
{d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (k : Fin d) (a : Bool) (j : ℕ) :
orderedProductMean P.law.observedLaw
(markedFactorialCoordinate M k a j) =
(P.law.cellMass k *
(if a then P.law.propensity k else 1 - P.law.propensity k) *
(P.law.outcomeMean a k / M)) *
P.law.cellMass k *
(P.law.cellMass k *
(if a then P.law.propensity k else 1 - P.law.propensity k)) ^ j := by
classical
unfold orderedProductMean orderedProductKernel
rw [MeasureTheory.integral_fintype_prod_eq_prod]
simp_rw [integral_markedFactorialCoordinate_exact P k a j]
rw [Fin.prod_univ_succ]
simp only [Fin.val_zero, ↓reduceIte]
rw [Fin.prod_univ_succ]
cases a <;> simp <;> ring
theorem allBlockOrderedMarkedFactorial_expectation_eq_orderedProductMean
If the factorial order fits in the estimation block, an all-block marked factorial is exactly unbiased for its coordinatewise product-law moment whenever its order fits in the estimation block.
Formal statement
hypotheses
conclusion
∫ ω : ℕ → Obs d, allBlockOrderedMarkedFactorial M (fun i : Fin m ↦ ω i) k a j ∂(Measure.infinitePi fun _ : ℕ ↦ P.law.observedLaw)
= orderedProductMean P.law.observedLaw (markedFactorialCoordinate M k a j)
Proof (Lean source)
lemma allBlockOrderedMarkedFactorial_expectation_eq_orderedProductMean
{d m : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (k : Fin d) (a : Bool) (j : ℕ)
(hjm : j + 2 ≤ m) :
∫ ω : ℕ → Obs d,
allBlockOrderedMarkedFactorial M (fun i : Fin m ↦ ω i) k a j
∂(Measure.infinitePi fun _ : ℕ ↦ P.law.observedLaw) =
Causalean.Stat.orderedProductMean P.law.observedLaw
(markedFactorialCoordinate M k a j) := by
let S0 := Causalean.Stat.iidSample_infinitePi P.law.observedLaw
have hstat :
(fun ω : ℕ → Obs d ↦
allBlockOrderedMarkedFactorial M (fun i : Fin m ↦ ω i) k a j) =
Causalean.Stat.normalizedOrderedProductStatistic S0
(markedFactorialCoordinate M k a j) m := by
funext ω
exact allBlockOrderedMarkedFactorial_eq_normalizedOrderedProductStatistic
(m := m) S0 M k a j ω
rw [hstat]
unfold Causalean.Stat.normalizedOrderedProductStatistic
Causalean.Stat.orderedProductMean
have hcard : card (Fin (j + 2)) ≤ m := by simpa using hjm
exact integral_normalizedFiniteKernelStatistic S0 hcard
(measurable_orderedProductKernel_markedFactorialCoordinate M k a j)
(integrable_orderedProductKernel_markedFactorialCoordinate P k a j)
theorem integral_allBlockOrderedMarkedFactorial_eq_orderedProductMean
If the factorial order fits in the estimation block, the same exact marked-factorial expectation identity holds directly under the finite product experiment used by the estimator.
Formal statement
hypotheses
conclusion
∫ s : Fin m → Obs d, allBlockOrderedMarkedFactorial M s k a j ∂(productLaw m P.law)
= orderedProductMean P.law.observedLaw (markedFactorialCoordinate M k a j)
Proof (Lean source)
lemma integral_allBlockOrderedMarkedFactorial_eq_orderedProductMean
{d m : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (k : Fin d) (a : Bool) (j : ℕ)
(hjm : j + 2 ≤ m) :
∫ s : Fin m → Obs d, allBlockOrderedMarkedFactorial M s k a j
∂(productLaw m P.law) =
orderedProductMean P.law.observedLaw
(markedFactorialCoordinate M k a j) := by
let S0 := iidSample_infinitePi P.law.observedLaw
have hpush := Causalean.Stat.iidSample_finN_pushforward S0 m
have hprefix : Measurable (fun ω : ℕ → Obs d ↦ fun i : Fin m ↦ ω i) :=
iidSample_finN_measurable S0 m
dsimp [S0, iidSample_infinitePi] at hpush hprefix
unfold productLaw
rw [← hpush, integral_map hprefix.aemeasurable
(measurable_allBlockOrderedMarkedFactorial M k a j).aestronglyMeasurable]
exact allBlockOrderedMarkedFactorial_expectation_eq_orderedProductMean
P k a j hjm
theorem integral_allBlockOrderedMarkedFactorial_exact
If the factorial order fits in the estimation block, the all-block statistic has the paper's exact marked-factorial expectation: one normalized marked arm-cell moment, one cell mass, and j further arm-cell masses.
Formal statement
hypotheses
conclusion
(∫ s : Fin m → Obs d, allBlockOrderedMarkedFactorial M s k a j ∂productLaw m P.law)
= (P.law.cellMass k * (if a then P.law.propensity k else 1 - P.law.propensity k) * (P.law.outcomeMean a k / M)) * P.law.cellMass k * (P.law.cellMass k * (if a then P.law.propensity k else 1 - P.law.propensity k)) ^ j
Proof (Lean source)
lemma integral_allBlockOrderedMarkedFactorial_exact
{d m : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (k : Fin d) (a : Bool) (j : ℕ)
(hjm : j + 2 ≤ m) :
(∫ s : Fin m → Obs d, allBlockOrderedMarkedFactorial M s k a j
∂productLaw m P.law) =
(P.law.cellMass k *
(if a then P.law.propensity k else 1 - P.law.propensity k) *
(P.law.outcomeMean a k / M)) *
P.law.cellMass k *
(P.law.cellMass k *
(if a then P.law.propensity k else 1 - P.law.propensity k)) ^ j := by
rw [integral_allBlockOrderedMarkedFactorial_eq_orderedProductMean P k a j hjm]
exact orderedProductMean_markedFactorialCoordinate_exact P k a j
Helpers.PolynomialUpper.Fallback 3 declarations
If the indicated calibration branch applies, outside the calibrated sample-size/alphabet branch, the declared zero fallback has squared risk at most M².
Formal statement
hypotheses
n d N :
ℕ
epsilon M sigma rho :
ℝ
P :
ModelClass d epsilon M sigma
hbranch :
¬(N ≤ n ∧ (d : ℝ) ≤ rho * n * logEN n)
conclusion
mse P.law (rawPolyEstimator (n := n) (d := d) N rho M) ≤ M ^ 2
Proof (Lean source)
lemma polyEstimator_fallback_mse_le {n d N : ℕ} {epsilon M sigma rho : ℝ}
(P : ModelClass d epsilon M sigma)
(hbranch : ¬(N ≤ n ∧ (d : ℝ) ≤ rho * n * logEN n)) :
mse P.law (rawPolyEstimator (n := n) (d := d) N rho M) ≤ M ^ 2 := by
have hraw : |rawAteFormula P.law| ≤ M := by
let Q : UnrestrictedClass d epsilon M :=
{ law := P.law
epsilon_pos := P.epsilon_pos
epsilon_lt_half := P.epsilon_lt_half
M_ge_one := P.M_ge_one
consistency := P.consistency
exchangeability := P.exchangeability
overlap := P.overlap
mean_normalization := P.mean_normalization
second_moment := P.second_moment }
exact scale_sanity.1 Q |>.2.1
have hsqAbs : |rawAteFormula P.law| ^ 2 ≤ M ^ 2 :=
(sq_le_sq₀ (abs_nonneg _) (le_trans zero_le_one P.M_ge_one)).2 hraw
have hsq : (rawAteFormula P.law) ^ 2 ≤ M ^ 2 := by
simpa [sq_abs] using hsqAbs
unfold mse rawPolyEstimator
simp_rw [if_neg hbranch]
simp only [zero_sub, neg_sq]
simpa using hsq
If the tuning radius satisfies the stated restriction, for fixed positive alphabet cutoff, the zero fallback already satisfies the paper's capped polynomial rate on every uncalibrated branch.
Formal statement
hypotheses
N :
ℕ
rho :
ℝ
hrho :
0 < rho
conclusion
∃ C : ℝ
if
0 < C ∧ ∀ {n d : ℕ} {epsilon M sigma : ℝ}, 0 < n
and
0 < d
and
¬(N ≤ n ∧ (d : ℝ) ≤ rho * n * logEN n)
then
∀ P : ModelClass d epsilon M sigma,
mse P.law (rawPolyEstimator (n := n) (d := d) N rho M)
≤ C * M ^ 2 * (1 / (n : ℝ) + min 1 (polynomialComponent n d))
Proof (Lean source)
lemma polyEstimator_uncalibrated_rate {N : ℕ} {rho : ℝ} (hrho : 0 < rho) :
∃ C : ℝ, 0 < C ∧
∀ {n d : ℕ} {epsilon M sigma : ℝ},
0 < n → 0 < d →
¬(N ≤ n ∧ (d : ℝ) ≤ rho * n * logEN n) →
∀ P : ModelClass d epsilon M sigma,
mse P.law (rawPolyEstimator (n := n) (d := d) N rho M) ≤
C * M ^ 2 *
(1 / (n : ℝ) + min 1 (polynomialComponent n d)) := by
let q : ℝ := min 1 (rho ^ 2)
have hq : 0 < q := lt_min zero_lt_one (sq_pos_of_pos hrho)
let C : ℝ := (N : ℝ) + 1 + 1 / q
have hC : 0 < C := by
dsimp [C]
positivity
refine ⟨C, hC, ?_⟩
intro n d epsilon M sigma hn hd hbranch P
have hnR : 0 < (n : ℝ) := by exact_mod_cast hn
have hdR : 0 < (d : ℝ) := by exact_mod_cast hd
have hlog : 0 < logEN n := by
rw [logEN, Real.log_mul (Real.exp_ne_zero 1) hnR.ne', Real.log_exp]
have hn_one : 1 ≤ (n : ℝ) := by exact_mod_cast hn
have hlogn : 0 ≤ log (n : ℝ) := Real.log_nonneg hn_one
linarith
have hpoly : 0 ≤ polynomialComponent n d := by
unfold polynomialComponent
positivity
have hrate : 0 ≤ 1 / (n : ℝ) + min 1 (polynomialComponent n d) := by
positivity
have hfallback := polyEstimator_fallback_mse_le P hbranch
apply hfallback.trans
have hC0 : 0 ≤ C := hC.le
have hM2 : 0 ≤ M ^ 2 := sq_nonneg M
suffices hone : 1 ≤ C *
(1 / (n : ℝ) + min 1 (polynomialComponent n d)) by
calc
M ^ 2 = M ^ 2 * 1 := by ring
_ ≤ M ^ 2 * (C *
(1 / (n : ℝ) + min 1 (polynomialComponent n d))) :=
mul_le_mul_of_nonneg_left hone hM2
_ = C * M ^ 2 *
(1 / (n : ℝ) + min 1 (polynomialComponent n d)) := by ring
rcases not_and_or.mp hbranch with hsmall | hlarge
· have hnN : n < N := Nat.lt_of_not_ge hsmall
have hnC : (n : ℝ) ≤ C := by
have hnNreal : (n : ℝ) < N := by exact_mod_cast hnN
dsimp [C]
have hqinv : 0 < 1 / q := by positivity
linarith
have hone' : 1 ≤ C * (1 / (n : ℝ)) := by
rw [mul_one_div]
exact (le_div_iff₀ hnR).2 (by simpa [one_mul] using hnC)
have hmin0 : 0 ≤ min 1 (polynomialComponent n d) :=
le_min zero_le_one hpoly
nlinarith
· have hlarge' : rho * (n : ℝ) * logEN n < (d : ℝ) :=
lt_of_not_ge hlarge
have hleft : 0 < rho * (n : ℝ) * logEN n :=
mul_pos (mul_pos hrho hnR) hlog
have hfactor : 0 <
((d : ℝ) - rho * (n : ℝ) * logEN n) *
((d : ℝ) + rho * (n : ℝ) * logEN n) :=
mul_pos (sub_pos.mpr hlarge') (add_pos hdR hleft)
have hsquare : rho ^ 2 * (n : ℝ) ^ 2 * logEN n ^ 2 < (d : ℝ) ^ 2 := by
nlinarith
have hden : 0 < (n : ℝ) ^ 2 * logEN n ^ 2 := mul_pos (sq_pos_of_pos hnR)
(sq_pos_of_pos hlog)
have hrho_poly : rho ^ 2 < polynomialComponent n d := by
unfold polynomialComponent
rw [lt_div_iff₀ hden]
nlinarith
have hqpoly : q ≤ min 1 (polynomialComponent n d) := by
exact min_le_min (le_refl 1) hrho_poly.le
have hCq : 1 / q ≤ C := by
dsimp [C]
have hN0 : 0 ≤ (N : ℝ) := Nat.cast_nonneg N
linarith
have hone' : 1 ≤ C * min 1 (polynomialComponent n d) := by
calc
1 = (1 / q) * q := by field_simp [hq.ne']
_ ≤ C * min 1 (polynomialComponent n d) :=
mul_le_mul hCq hqpoly hq.le (by positivity)
have hninv0 : 0 ≤ 1 / (n : ℝ) := by positivity
nlinarith
If the target lies in the clipping interval, clipping a normalized estimate to [-1,1] cannot increase squared loss against a normalized target already in that interval.
Formal statement
Proof (Lean source)
lemma clip_normalized_sq_error_le (x t : ℝ) (ht : t ∈ Icc (-1 : ℝ) 1) :
(clip (-1) 1 x - t) ^ 2 ≤ (x - t) ^ 2 := by
rcases ht with ⟨htl, htu⟩
unfold clip
by_cases hxlow : x ≤ -1
· rw [min_eq_right (hxlow.trans (by norm_num : (-1 : ℝ) ≤ 1)),
max_eq_left hxlow]
nlinarith [sq_nonneg (x - t)]
· have hxlow' : -1 ≤ x := le_of_not_ge hxlow
rw [max_eq_right (le_min (by norm_num) hxlow')]
by_cases hxhigh : x ≤ 1
· rw [min_eq_right hxhigh]
· rw [min_eq_left (le_of_not_ge hxhigh)]
nlinarith [sq_nonneg (x - t)]
Helpers.PolynomialUpper.FixedBranchAssembly 3 declarations
def polynomialFixedBranchNormalizedError
The normalized error of a deterministic heavy set and its complementary light set, both evaluated on one independent block.
Definition (Lean source)
noncomputable def polynomialFixedBranchNormalizedError {d m K : ℕ}
(P : RealLaw d) (M B : ℝ) (H : Finset (Fin d))
(z : Fin m → Obs d) : ℝ :=
(fixedStratumMarkedRatio (fun o : Obs d => o.x) (fun o => o.a)
(fun o => o.y) H z -
fixedStratumMarkedTarget P.observedLaw (fun o : Obs d => o.x)
(fun o => o.a) (fun o => o.y) H) / M +
(allBlockMarkedPolynomialSum M B K (univ \ H) z -
∑ k ∈ univ \ H, P.cellMass k * cellEffect P k / M)
theorem polynomialFixedBranchNormalizedError_sq_integrable
Every deterministic normalized branch has an integrable square under the finite product law.
Formal statement
hypotheses
conclusion
Integrable (fun z : Fin m → Obs d => (polynomialFixedBranchNormalizedError (K := K) P.law M B H z) ^ 2) (productLaw m P.law)
Proof (Lean source)
lemma polynomialFixedBranchNormalizedError_sq_integrable {d m K : ℕ}
{epsilon M sigma B : ℝ} (P : ModelClass d epsilon M sigma)
(H : Finset (Fin d)) :
Integrable (fun z : Fin m → Obs d =>
(polynomialFixedBranchNormalizedError (K := K) P.law M B H z) ^ 2)
(productLaw m P.law) := by
let heavy : (Fin m → Obs d) → ℝ := fun z =>
(fixedStratumMarkedRatio (fun o : Obs d => o.x) (fun o => o.a)
(fun o => o.y) H z -
fixedStratumMarkedTarget P.law.observedLaw (fun o : Obs d => o.x)
(fun o => o.a) (fun o => o.y) H) / M
let light : (Fin m → Obs d) → ℝ := fun z =>
allBlockMarkedPolynomialSum M B K (univ \ H) z -
∑ k ∈ univ \ H, P.law.cellMass k * cellEffect P.law k / M
have htuple : Measurable (fun o : Obs d => (o.x, o.a, o.y)) :=
measurable_iff_comap_le.mpr le_rfl
have hx : Measurable (fun o : Obs d => o.x) := measurable_fst.comp htuple
have ha : Measurable (fun o : Obs d => o.a) :=
measurable_fst.comp (measurable_snd.comp htuple)
have hy : Measurable (fun o : Obs d => o.y) :=
measurable_snd.comp (measurable_snd.comp htuple)
have hheavyLp : MemLp heavy 2 (productLaw m P.law) := by
have hratio : MemLp (fun z : Fin m → Obs d =>
fixedStratumMarkedRatio (fun o : Obs d => o.x) (fun o => o.a)
(fun o => o.y) H z -
fixedStratumMarkedTarget P.law.observedLaw
(fun o : Obs d => o.x) (fun o => o.a) (fun o => o.y) H)
2 (productLaw m P.law) := by
unfold productLaw
exact (fixedStratumMarkedRatio_memLp_two (m := m) P.law.observedLaw
(fun o : Obs d => o.x) (fun o => o.a) (fun o => o.y)
(polynomialSupportedCenter P.law) H hx ha hy
(polynomialSupportedCenter_residual_memLp P)).sub (memLp_const _)
simpa [heavy, div_eq_mul_inv, mul_comm] using hratio.const_mul (M⁻¹)
have hlightLp : MemLp light 2 (productLaw m P.law) :=
(memLp_allBlockMarkedPolynomialSum_finite P (univ \ H)).sub
(memLp_const _)
change Integrable (fun z => (heavy z + light z) ^ 2) _
exact (hheavyLp.add hlightLp).integrable_sq
The fixed-heavy marked-ratio bound and fixed-light factorial-moment bound combine into a single deterministic-selector risk inequality.
Formal statement
conclusion
∀ epsilon : ℝ,
0 < epsilon → epsilon < 1 / 2 → ∃ C_epsilon : ℝ, 0 < C_epsilon ∧ ∀ d m K : ℕ, ∀ M sigma B lowerBand : ℝ, ∀ P : ModelClass d epsilon M sigma, ∀ H : Finset (Fin d), 2
≤ K → 0 < B → 0 < lowerBand → (∀ k ∈ H, lowerBand ≤ P.law.cellMass k) → (∀ k ∉ H, P.law.cellMass k ≤ B / 4) → 4 * (K + 2) ^ 2 ≤ m → (4 : ℝ) * (K + 2) / m ≤ 3 * B / 4 → ∫ z : Fin m → Obs d, (polynomialFixedBranchNormalizedError (K := K) P.law M B H z) ^ 2 ∂(productLaw m P.law) ≤ 8 * (8 * (∑ k ∈ H, P.law.cellMass k) / (safeSampleSize m * epsilon) + 6 / safeSampleSize m + 4 * (lowerMassMissingEnvelope P.law.observedLaw (fun o : Obs d => o.x) m epsilon lowerBand H) ^ 2)
+ 2 * (C_epsilon / m + C_epsilon * 6 ^ (2 * K) * ((d : ℝ) * B ^ 2 + (d : ℝ) ^ 2 * K ^ 2 * B ^ 2 / m) + (((univ \ H).card : ℝ) * (B / (epsilon * (K : ℝ) ^ 2))) ^ 2)
Proof (Lean source)
lemma polynomialFixedBranchNormalizedError_sq_integral_le :
∀ epsilon : ℝ, 0 < epsilon → epsilon < 1 / 2 →
∃ C_epsilon : ℝ, 0 < C_epsilon ∧
∀ d m K : ℕ, ∀ M sigma B lowerBand : ℝ,
∀ P : ModelClass d epsilon M sigma, ∀ H : Finset (Fin d),
2 ≤ K → 0 < B → 0 < lowerBand →
(∀ k ∈ H, lowerBand ≤ P.law.cellMass k) →
(∀ k ∉ H, P.law.cellMass k ≤ B / 4) →
4 * (K + 2) ^ 2 ≤ m →
(4 : ℝ) * (K + 2) / m ≤ 3 * B / 4 →
∫ z : Fin m → Obs d,
(polynomialFixedBranchNormalizedError (K := K) P.law M B H z) ^ 2
∂(productLaw m P.law) ≤
8 * (8 * (∑ k ∈ H, P.law.cellMass k) /
(safeSampleSize m * epsilon) +
6 / safeSampleSize m +
4 * (lowerMassMissingEnvelope P.law.observedLaw
(fun o : Obs d => o.x) m epsilon lowerBand H) ^ 2) +
2 * (C_epsilon / m + C_epsilon * 6 ^ (2 * K) *
((d : ℝ) * B ^ 2 + (d : ℝ) ^ 2 * K ^ 2 * B ^ 2 / m) +
(((univ \ H).card : ℝ) *
(B / (epsilon * (K : ℝ) ^ 2))) ^ 2) := by
intro epsilon hepsilon hepsilon_half
obtain ⟨C_epsilon, hC, hlight⟩ :=
fixedLightMarkedPolynomial_error_sq_le epsilon hepsilon hepsilon_half
refine ⟨C_epsilon, hC, ?_⟩
intro d m K M sigma B lowerBand P H hK hB hlower hheavy hlightMass hm hshift
let heavy : (Fin m → Obs d) → ℝ := fun z =>
(fixedStratumMarkedRatio (fun o : Obs d => o.x) (fun o => o.a)
(fun o => o.y) H z -
fixedStratumMarkedTarget P.law.observedLaw (fun o : Obs d => o.x)
(fun o => o.a) (fun o => o.y) H) / M
let light : (Fin m → Obs d) → ℝ := fun z =>
allBlockMarkedPolynomialSum M B K (univ \ H) z -
∑ k ∈ univ \ H,
P.law.cellMass k * cellEffect P.law k / M
have hM : M ≠ 0 := ne_of_gt (lt_of_lt_of_le zero_lt_one P.M_ge_one)
have hheavyRisk := fixedHeavyMarkedRatio_error_sq_le (m := m) P H hheavy
have hmass (k : Fin d) :
categoryMass P.law.observedLaw
(fun o : Obs d => o.x) k = P.law.cellMass k := by
simpa [categoryMass, categoryEvent, groupEvent, realMass] using
(P.law.cellMass_eq k).symm
simp_rw [hmass] at hheavyRisk
have hheavyNorm :
∫ z : Fin m → Obs d, (heavy z) ^ 2 ∂(productLaw m P.law) ≤
4 * (8 * (∑ k ∈ H, P.law.cellMass k) /
(safeSampleSize m * epsilon) +
6 / safeSampleSize m +
4 * (lowerMassMissingEnvelope P.law.observedLaw
(fun o : Obs d => o.x) m epsilon lowerBand H) ^ 2) := by
dsimp [heavy]
simp_rw [div_pow]
simp_rw [div_eq_mul_inv]
rw [integral_mul_const]
have hM2 : 0 < M ^ 2 := sq_pos_of_ne_zero hM
let R : ℝ := 8 * (∑ k ∈ H, P.law.cellMass k) /
(safeSampleSize m * epsilon) +
6 / safeSampleSize m +
4 * (lowerMassMissingEnvelope P.law.observedLaw
(fun o : Obs d => o.x) m epsilon lowerBand H) ^ 2
have hrisk :
∫ z : Fin m → Obs d,
(fixedStratumMarkedRatio (fun o : Obs d => o.x) (fun o => o.a)
(fun o => o.y) H z -
fixedStratumMarkedTarget P.law.observedLaw
(fun o : Obs d => o.x) (fun o => o.a)
(fun o => o.y) H) ^ 2 ∂(productLaw m P.law) ≤
4 * M ^ 2 * R := by
simpa [R] using hheavyRisk
change (∫ z : Fin m → Obs d,
(fixedStratumMarkedRatio (fun o : Obs d => o.x) (fun o => o.a)
(fun o => o.y) H z -
fixedStratumMarkedTarget P.law.observedLaw
(fun o : Obs d => o.x) (fun o => o.a)
(fun o => o.y) H) ^ 2 ∂(productLaw m P.law)) * (M ^ 2)⁻¹ ≤ 4 * R
calc
(∫ z : Fin m → Obs d,
(fixedStratumMarkedRatio (fun o : Obs d => o.x) (fun o => o.a)
(fun o => o.y) H z -
fixedStratumMarkedTarget P.law.observedLaw
(fun o : Obs d => o.x) (fun o => o.a)
(fun o => o.y) H) ^ 2 ∂(productLaw m P.law)) * (M ^ 2)⁻¹ ≤
(4 * M ^ 2 * R) * (M ^ 2)⁻¹ :=
mul_le_mul_of_nonneg_right hrisk (inv_nonneg.mpr (sq_nonneg M))
_ = 4 * R * (M ^ 2 * (M ^ 2)⁻¹) := by ring
_ = 4 * R := by rw [mul_inv_cancel₀ hM2.ne', mul_one]
have hlightSet : ∀ k ∈ univ \ H,
P.law.cellMass k ≤ B / 4 := by
intro k hk
exact hlightMass k (Finset.mem_sdiff.mp hk).2
have hlightRisk := hlight d m K M sigma B P (univ \ H)
hK hB hlightSet hm hshift
have htuple : Measurable (fun o : Obs d => (o.x, o.a, o.y)) :=
measurable_iff_comap_le.mpr le_rfl
have hx : Measurable (fun o : Obs d => o.x) := measurable_fst.comp htuple
have ha : Measurable (fun o : Obs d => o.a) :=
measurable_fst.comp (measurable_snd.comp htuple)
have hy : Measurable (fun o : Obs d => o.y) :=
measurable_snd.comp (measurable_snd.comp htuple)
have hheavyLp : MemLp heavy 2 (productLaw m P.law) := by
have hratio : MemLp (fun z : Fin m → Obs d =>
fixedStratumMarkedRatio (fun o : Obs d => o.x) (fun o => o.a)
(fun o => o.y) H z -
fixedStratumMarkedTarget P.law.observedLaw
(fun o : Obs d => o.x) (fun o => o.a) (fun o => o.y) H)
2 (productLaw m P.law) := by
unfold productLaw
exact (fixedStratumMarkedRatio_memLp_two (m := m) P.law.observedLaw
(fun o : Obs d => o.x) (fun o => o.a) (fun o => o.y)
(polynomialSupportedCenter P.law) H hx ha hy
(polynomialSupportedCenter_residual_memLp P)).sub
(memLp_const (fixedStratumMarkedTarget P.law.observedLaw
(fun o : Obs d => o.x) (fun o => o.a) (fun o => o.y) H))
simpa [heavy, div_eq_mul_inv, mul_comm] using hratio.const_mul (M⁻¹)
have hlightLp : MemLp light 2 (productLaw m P.law) :=
(memLp_allBlockMarkedPolynomialSum_finite P (univ \ H)).sub
(memLp_const _)
have hsumInt : Integrable (fun z => (heavy z + light z) ^ 2)
(productLaw m P.law) := (hheavyLp.add hlightLp).integrable_sq
have hrightInt : Integrable
(fun z => 2 * heavy z ^ 2 + 2 * light z ^ 2)
(productLaw m P.law) :=
hheavyLp.integrable_sq.const_mul 2 |>.add
(hlightLp.integrable_sq.const_mul 2)
change (∫ z : Fin m → Obs d, (heavy z + light z) ^ 2
∂(productLaw m P.law)) ≤ _
calc
(∫ z : Fin m → Obs d, (heavy z + light z) ^ 2
∂(productLaw m P.law)) ≤
∫ z : Fin m → Obs d, 2 * heavy z ^ 2 + 2 * light z ^ 2
∂(productLaw m P.law) := by
apply integral_mono hsumInt hrightInt
intro z
nlinarith [sq_nonneg (heavy z - light z)]
_ = 2 * (∫ z : Fin m → Obs d, heavy z ^ 2
∂(productLaw m P.law)) +
2 * (∫ z : Fin m → Obs d, light z ^ 2
∂(productLaw m P.law)) := by
rw [integral_add, integral_const_mul, integral_const_mul]
· exact hheavyLp.integrable_sq.const_mul 2
· exact hlightLp.integrable_sq.const_mul 2
_ ≤ _ := by nlinarith [hheavyNorm, hlightRisk]
Helpers.PolynomialUpper.FixedBranchRate 1 declarations
theorem polynomial_fixedBranch_uniform_bound
every fixed heavy-set branch of the polynomial estimator obeys the stated uniform normalized-risk bound.
Formal statement
conclusion
∀ epsilon : ℝ,
0 < epsilon → epsilon < 1 / 2 → ∃ C_epsilon : ℝ, 0 < C_epsilon ∧ ∀ n d : ℕ, ∀ M sigma lowerBand upperBand : ℝ, ∀ P : ModelClass d epsilon M sigma, ∀ H : Finset (Fin d), polynomialSelectorEligible P.law lowerBand upperBand H → 2
≤ polynomialDegree n → 0 < lowerBand → 2 ≤ n → upperBand ≤ (4096 * logEN n / (n - n / 2 : ℕ)) / 4 → 4 * (polynomialDegree n + 2) ^ 2 ≤ n
- n / 2 → (4 : ℝ) * (polynomialDegree n + 2) / (n - n / 2 : ℕ)
≤ 3 * (4096 * logEN n / (n - n / 2 : ℕ)) / 4 → ∫ z : Fin (n - n / 2) → Obs d, (polynomialFixedBranchNormalizedError (K := polynomialDegree n) P.law M (4096 * logEN n / (n - n / 2 : ℕ)) H z) ^ 2 ∂(productLaw (n - n / 2) P.law) ≤ 8 * (8 / (((n - n / 2 : ℕ) : ℝ) * epsilon) + 6 / (n - n / 2 : ℕ) + 4 * ((d : ℝ) / (((((n - n / 2 - 2 : ℕ) : ℝ) / 2 * epsilon) ^ 2) * lowerBand)) ^ 2)
+ 2 * (C_epsilon / (n - n / 2 : ℕ) + C_epsilon * 6 ^ (2 * polynomialDegree n) * ((d : ℝ) * (4096 * logEN n / (n - n / 2 : ℕ)) ^ 2 + (d : ℝ) ^ 2 * polynomialDegree n ^ 2 * (4096 * logEN n / (n - n / 2 : ℕ)) ^ 2 / (n - n / 2 : ℕ)) + ((d : ℝ) * ((4096 * logEN n / (n - n / 2 : ℕ)) / (epsilon * (polynomialDegree n : ℝ) ^ 2))) ^ 2)
Proof (Lean source)
lemma polynomial_fixedBranch_uniform_bound :
∀ epsilon : ℝ, 0 < epsilon → epsilon < 1 / 2 →
∃ C_epsilon : ℝ, 0 < C_epsilon ∧
∀ n d : ℕ, ∀ M sigma lowerBand upperBand : ℝ,
∀ P : ModelClass d epsilon M sigma, ∀ H : Finset (Fin d),
polynomialSelectorEligible P.law lowerBand upperBand H →
2 ≤ polynomialDegree n → 0 < lowerBand → 2 ≤ n →
upperBand ≤ (4096 * logEN n / (n - n / 2 : ℕ)) / 4 →
4 * (polynomialDegree n + 2) ^ 2 ≤ n - n / 2 →
(4 : ℝ) * (polynomialDegree n + 2) / (n - n / 2 : ℕ) ≤
3 * (4096 * logEN n / (n - n / 2 : ℕ)) / 4 →
∫ z : Fin (n - n / 2) → Obs d,
(polynomialFixedBranchNormalizedError (K := polynomialDegree n)
P.law M (4096 * logEN n / (n - n / 2 : ℕ)) H z) ^ 2
∂(productLaw (n - n / 2) P.law) ≤
8 * (8 / (((n - n / 2 : ℕ) : ℝ) * epsilon) +
6 / (n - n / 2 : ℕ) +
4 * ((d : ℝ) /
(((((n - n / 2 - 2 : ℕ) : ℝ) / 2 * epsilon) ^ 2) *
lowerBand)) ^ 2) +
2 * (C_epsilon / (n - n / 2 : ℕ) +
C_epsilon * 6 ^ (2 * polynomialDegree n) *
((d : ℝ) * (4096 * logEN n / (n - n / 2 : ℕ)) ^ 2 +
(d : ℝ) ^ 2 * polynomialDegree n ^ 2 *
(4096 * logEN n / (n - n / 2 : ℕ)) ^ 2 /
(n - n / 2 : ℕ)) +
((d : ℝ) * ((4096 * logEN n / (n - n / 2 : ℕ)) /
(epsilon * (polynomialDegree n : ℝ) ^ 2))) ^ 2) := by
intro epsilon hepsilon hepsilon_half
obtain ⟨C, hC, hfixed⟩ :=
polynomialFixedBranchNormalizedError_sq_integral_le epsilon hepsilon
hepsilon_half
refine ⟨C, hC, ?_⟩
intro n d M sigma lowerBand upperBand P H helig hK hlower hn
hupper hm hshift
let m := n - n / 2
let B := 4096 * logEN n / (n - n / 2 : ℕ)
have hmpos : 0 < m := by dsimp [m]; omega
have hm3 : 3 ≤ m := by
have hk4 : 4 ≤ polynomialDegree n + 2 := by omega
have hsquare : 16 ≤ (polynomialDegree n + 2) ^ 2 := by
nlinarith [Nat.mul_self_le_mul_self hk4]
have h64 : 64 ≤ 4 * (polynomialDegree n + 2) ^ 2 := by omega
exact (by omega : 3 ≤ 64).trans (h64.trans hm)
have hB : 0 < B := by
dsimp [B]
exact polynomial_lightScale_pos (by omega)
have hheavy : ∀ k ∈ H, lowerBand ≤ P.law.cellMass k := helig.1
have hlight : ∀ k ∉ H, P.law.cellMass k ≤ B / 4 := by
intro k hk
exact (helig.2 k hk).trans hupper
have hbase := hfixed d m (polynomialDegree n) M sigma B lowerBand P H
hK hB hlower hheavy hlight hm hshift
apply hbase.trans
have hmass : ∑ k ∈ H, P.law.cellMass k ≤ 1 := by
rw [← sum_cellMass_eq_one P.law]
exact Finset.sum_le_sum_of_subset_of_nonneg (Finset.subset_univ H)
(fun k _ _ => (P.law.cellMass_range k).1)
have hcardH : (H.card : ℝ) ≤ d := by
have hc := Finset.card_le_card (Finset.subset_univ H)
simpa using hc
have hcardLight : ((univ \ H).card : ℝ) ≤ d := by
have hc := Finset.card_le_card (sdiff_subset (s := univ) (t := H))
simpa using hc
have hden : 0 < (((((m - 2 : ℕ) : ℝ) / 2 * epsilon) ^ 2) * lowerBand) := by
have : 0 < m - 2 := by omega
positivity
have henv : lowerMassMissingEnvelope P.law.observedLaw
(fun o : Obs d => o.x) m epsilon lowerBand H ≤
(d : ℝ) / (((((m - 2 : ℕ) : ℝ) / 2 * epsilon) ^ 2) * lowerBand) := by
rw [lowerMassMissingEnvelope_eq_of_pos P.law.observedLaw
(fun o : Obs d => o.x) H hm3 hepsilon hlower]
exact (div_le_div_iff_of_pos_right hden).2 hcardH
have henv0 : 0 ≤ lowerMassMissingEnvelope P.law.observedLaw
(fun o : Obs d => o.x) m epsilon lowerBand H := by
rw [lowerMassMissingEnvelope_eq_of_pos P.law.observedLaw
(fun o : Obs d => o.x) H hm3 hepsilon hlower]
positivity
have hsafe : safeSampleSize m = (m : ℝ) := by
unfold safeSampleSize
rw [max_eq_right (by omega)]
rw [hsafe]
dsimp [m, B] at hmass hcardLight henv henv0 hden ⊢
gcongr
· nlinarith
Helpers.PolynomialUpper.FixedLightRisk 6 declarations
theorem integrable_allBlockOrderedMarkedFactorial
Each finite-product marked factorial is integrable; this is the finite-law counterpart of the infinite-IID MemLp certificate used in the covariance proof.
Formal statement
hypotheses
conclusion
Proof (Lean source)
lemma integrable_allBlockOrderedMarkedFactorial
{d m : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (k : Fin d) (a : Bool) (j : ℕ) :
Integrable (fun s : Fin m → Obs d ↦
allBlockOrderedMarkedFactorial M s k a j) (productLaw m P.law) := by
let S0 := iidSample_infinitePi P.law.observedLaw
have hpush := Causalean.Stat.iidSample_finN_pushforward S0 m
have hprefix : Measurable (fun ω : ℕ → Obs d ↦ fun i : Fin m ↦ ω i) :=
iidSample_finN_measurable S0 m
dsimp [S0, iidSample_infinitePi] at hpush hprefix
unfold productLaw
rw [← hpush]
apply (integrable_map_measure
(measurable_allBlockOrderedMarkedFactorial M k a j).aestronglyMeasurable
hprefix.aemeasurable).2
exact (memLp_allBlockOrderedMarkedFactorial P k a j).integrable one_le_two
theorem integral_allBlockLightPolynomialTerm_eq_population
If the outcome bound is positive and the polynomial degree fits the sample block, the exact marked-factorial expectation reconstructs the deterministic population polynomial for one light cell.
Formal statement
hypotheses
conclusion
∫ s : Fin m → Obs d, allBlockLightPolynomialTerm M B K s k ∂(productLaw m P.law)
= polynomialLightCellPopulationTerm P.law M B K k
Proof (Lean source)
lemma integral_allBlockLightPolynomialTerm_eq_population
{d m K : ℕ} {epsilon M sigma B : ℝ}
(P : ModelClass d epsilon M sigma) (k : Fin d)
(hB : 0 < B) (hKm : K ≤ m) :
∫ s : Fin m → Obs d, allBlockLightPolynomialTerm M B K s k
∂(productLaw m P.law) =
polynomialLightCellPopulationTerm P.law M B K k := by
unfold allBlockLightPolynomialTerm polynomialLightCellPopulationTerm
rw [integral_finsetSum]
· dsimp only
simp only [if_true, Bool.false_eq_true, if_false]
rw [Finset.mul_sum, Finset.mul_sum, ← Finset.sum_sub_distrib]
apply Finset.sum_congr rfl
intro j hj
have hjm : j + 2 ≤ m := by
have := Finset.mem_range.mp hj
omega
rw [integral_const_mul,
integral_sub (integrable_allBlockOrderedMarkedFactorial P k true j)
(integrable_allBlockOrderedMarkedFactorial P k false j),
integral_allBlockOrderedMarkedFactorial_exact P k true j hjm,
integral_allBlockOrderedMarkedFactorial_exact P k false j hjm]
simp only [if_true, Bool.false_eq_true, if_false]
simp_rw [div_pow]
rw [pow_succ]
field_simp [hB.ne']
· intro j hj
exact ((integrable_allBlockOrderedMarkedFactorial P k true j).sub
(integrable_allBlockOrderedMarkedFactorial P k false j)).const_mul _
theorem memLp_allBlockLightPolynomialTerm_finite
A finite-product light-cell polynomial is square-integrable.
Formal statement
hypotheses
conclusion
Proof (Lean source)
lemma memLp_allBlockLightPolynomialTerm_finite
{d m K : ℕ} {epsilon M sigma B : ℝ}
(P : ModelClass d epsilon M sigma) (k : Fin d) :
MemLp (fun s : Fin m → Obs d ↦
allBlockLightPolynomialTerm M B K s k) 2 (productLaw m P.law) := by
let S0 := iidSample_infinitePi P.law.observedLaw
have hpush := Causalean.Stat.iidSample_finN_pushforward S0 m
have hprefix : Measurable (fun ω : ℕ → Obs d ↦ fun i : Fin m ↦ ω i) :=
iidSample_finN_measurable S0 m
dsimp [S0, iidSample_infinitePi] at hpush hprefix
unfold productLaw
rw [← hpush]
have heq : (fun s : Fin m → Obs d ↦
allBlockLightPolynomialTerm M B K s k) =
allBlockMarkedPolynomialSum M B K {k} := by
funext s
simp [allBlockMarkedPolynomialSum]
apply (memLp_map_measure_iff
(by
rw [heq]
exact (measurable_allBlockMarkedPolynomialSum M B K {k}).aestronglyMeasurable)
hprefix.aemeasurable).2
exact memLp_allBlockLightPolynomialTerm P k
theorem integral_allBlockMarkedPolynomialSum_eq_population
If the outcome bound is positive and the polynomial degree fits the sample block, summing the cellwise exact expectations reconstructs the population polynomial over a deterministic light set.
Formal statement
hypotheses
conclusion
∫ s : Fin m → Obs d, allBlockMarkedPolynomialSum M B K S s ∂(productLaw m P.law)
= ∑ k ∈ S, polynomialLightCellPopulationTerm P.law M B K k
Proof (Lean source)
lemma integral_allBlockMarkedPolynomialSum_eq_population
{d m K : ℕ} {epsilon M sigma B : ℝ}
(P : ModelClass d epsilon M sigma) (S : Finset (Fin d))
(hB : 0 < B) (hKm : K ≤ m) :
∫ s : Fin m → Obs d, allBlockMarkedPolynomialSum M B K S s
∂(productLaw m P.law) =
∑ k ∈ S, polynomialLightCellPopulationTerm P.law M B K k := by
unfold allBlockMarkedPolynomialSum
rw [integral_finsetSum]
· exact Finset.sum_congr rfl fun k _ ↦
integral_allBlockLightPolynomialTerm_eq_population P k hB hKm
· intro k _
exact (memLp_allBlockLightPolynomialTerm_finite P k).integrable one_le_two
theorem memLp_allBlockMarkedPolynomialSum_finite
The deterministic finite-product light sum is square-integrable.
Formal statement
hypotheses
conclusion
MemLp (allBlockMarkedPolynomialSum (m := m) M B K S) 2 (productLaw m P.law)
Proof (Lean source)
lemma memLp_allBlockMarkedPolynomialSum_finite
{d m K : ℕ} {epsilon M sigma B : ℝ}
(P : ModelClass d epsilon M sigma) (S : Finset (Fin d)) :
MemLp (allBlockMarkedPolynomialSum (m := m) M B K S) 2
(productLaw m P.law) := by
unfold allBlockMarkedPolynomialSum
apply memLp_finsetSum
intro k _
exact memLp_allBlockLightPolynomialTerm_finite P k
theorem fixedLightMarkedPolynomial_error_sq_le
The exact expectation, Chebyshev bias, and marked-factorial covariance bound combine into the fixed-light-set squared-risk inequality.
Formal statement
conclusion
∀ epsilon : ℝ,
0 < epsilon → epsilon < 1 / 2 → ∃ C_epsilon : ℝ, 0 < C_epsilon ∧ ∀ d m K : ℕ, ∀ M sigma B : ℝ, ∀ P : ModelClass d epsilon M sigma, ∀ S : Finset (Fin d), 2
≤ K → 0 < B → (∀ k ∈ S, P.law.cellMass k ≤ B / 4) → 4 * (K + 2) ^ 2 ≤ m → (4 : ℝ) * (K + 2) / m ≤ 3 * B / 4 → ∫ s : Fin m → Obs d, (allBlockMarkedPolynomialSum M B K S s - ∑ k ∈ S, P.law.cellMass k * cellEffect P.law k / M) ^ 2 ∂(productLaw m P.law) ≤ C_epsilon / m
+ C_epsilon * 6 ^ (2 * K) * ((d : ℝ) * B ^ 2 + (d : ℝ) ^ 2 * K ^ 2 * B ^ 2 / m)
+ ((S.card : ℝ) * (B / (epsilon * (K : ℝ) ^ 2))) ^ 2
Proof (Lean source)
lemma fixedLightMarkedPolynomial_error_sq_le :
∀ epsilon : ℝ, 0 < epsilon → epsilon < 1 / 2 →
∃ C_epsilon : ℝ, 0 < C_epsilon ∧
∀ d m K : ℕ, ∀ M sigma B : ℝ,
∀ P : ModelClass d epsilon M sigma, ∀ S : Finset (Fin d),
2 ≤ K → 0 < B →
(∀ k ∈ S, P.law.cellMass k ≤ B / 4) →
4 * (K + 2) ^ 2 ≤ m →
(4 : ℝ) * (K + 2) / m ≤ 3 * B / 4 →
∫ s : Fin m → Obs d,
(allBlockMarkedPolynomialSum M B K S s -
∑ k ∈ S, P.law.cellMass k * cellEffect P.law k / M) ^ 2
∂(productLaw m P.law) ≤
C_epsilon / m + C_epsilon * 6 ^ (2 * K) *
((d : ℝ) * B ^ 2 + (d : ℝ) ^ 2 * K ^ 2 * B ^ 2 / m) +
((S.card : ℝ) * (B / (epsilon * (K : ℝ) ^ 2))) ^ 2 := by
intro epsilon hepsilon hepsilon_half
obtain ⟨C_epsilon, hC, hcovariance⟩ :=
linear_mark_factorial_covariance epsilon hepsilon hepsilon_half
refine ⟨C_epsilon, hC, ?_⟩
intro d m K M sigma B P S hK hB hlight hm hshift
let est : (Fin m → Obs d) → ℝ :=
allBlockMarkedPolynomialSum M B K S
let target : ℝ :=
∑ k ∈ S, P.law.cellMass k * cellEffect P.law k / M
have hKm : K ≤ m := by
have haux : K ≤ 4 * (K + 2) ^ 2 := by nlinarith
omega
have hest : MemLp est 2 (productLaw m P.law) :=
memLp_allBlockMarkedPolynomialSum_finite P S
have hmse :
(∫ s, (est s - target) ^ 2 ∂(productLaw m P.law)) =
variance est (productLaw m P.law) +
((∫ s, est s ∂(productLaw m P.law)) - target) ^ 2 := by
have hsq : Integrable (fun s ↦ est s ^ 2) (productLaw m P.law) :=
hest.integrable_sq
have hint : Integrable est (productLaw m P.law) :=
hest.integrable one_le_two
have hlinear : Integrable (fun s ↦ 2 * target * est s)
(productLaw m P.law) := hint.const_mul _
rw [variance_eq_sub hest]
calc
∫ s, (est s - target) ^ 2 ∂(productLaw m P.law) =
∫ s, (est s ^ 2 - 2 * target * est s) + target ^ 2
∂(productLaw m P.law) := by
apply integral_congr_ae
filter_upwards with s
ring
_ = (∫ s, est s ^ 2 - 2 * target * est s
∂(productLaw m P.law)) +
(∫ _s, target ^ 2 ∂(productLaw m P.law)) :=
integral_add (hsq.sub hlinear) (integrable_const _)
_ = (∫ s, est s ^ 2 ∂(productLaw m P.law)) -
(∫ s, 2 * target * est s ∂(productLaw m P.law)) +
target ^ 2 := by
rw [integral_sub hsq hlinear, integral_const, probReal_univ, one_smul]
_ = (∫ s, est s ^ 2 ∂(productLaw m P.law)) -
2 * target * (∫ s, est s ∂(productLaw m P.law)) +
target ^ 2 := by rw [integral_const_mul]
_ = (∫ s, (est ^ 2) s ∂(productLaw m P.law)) -
(∫ s, est s ∂(productLaw m P.law)) ^ 2 +
((∫ s, est s ∂(productLaw m P.law)) - target) ^ 2 := by
simp only [Pi.pow_apply]
ring
have hmean :
(∫ s, est s ∂(productLaw m P.law)) =
∑ k ∈ S, polynomialLightCellPopulationTerm P.law M B K k := by
exact integral_allBlockMarkedPolynomialSum_eq_population P S hB hKm
have hbiasEq :
(∫ s, est s ∂(productLaw m P.law)) - target =
-polynomialFixedLightPopulationBias P.law M B K S := by
rw [hmean]
unfold target polynomialFixedLightPopulationBias
rw [Finset.sum_sub_distrib]
ring
have hlightB : ∀ k ∈ S, P.law.cellMass k ≤ B := by
intro k hk
exact (hlight k hk).trans (by linarith)
have habs := polynomialFixedLightPopulationBias_abs_le
P S (show 0 < K by omega) hB hlightB
have hbound0 : 0 ≤ (S.card : ℝ) *
(B / (epsilon * (K : ℝ) ^ 2)) := by positivity
have hbiasSq :
((∫ s, est s ∂(productLaw m P.law)) - target) ^ 2 ≤
((S.card : ℝ) * (B / (epsilon * (K : ℝ) ^ 2))) ^ 2 := by
rw [hbiasEq]
simp only [neg_sq]
rw [sq_le_sq]
rw [abs_of_nonneg hbound0]
exact habs
have hvariance := hcovariance d m K M sigma B P S hK hB hlight hm hshift
change (∫ s, (est s - target) ^ 2 ∂(productLaw m P.law)) ≤ _
rw [hmse]
linarith
Helpers.PolynomialUpper.FoldRiskBridge 10 declarations
def polynomialFoldBEquiv
The balanced estimation fold has the canonical finite index type of its declared cardinality.
Definition (Lean source)
noncomputable def polynomialFoldBEquiv {n d : ℕ} (P : RealLaw d) :
Fin (n - n / 2) ≃o (polynomialBalancedSplit P).foldB n :=
((polynomialBalancedSplit P).foldB n).orderIsoOfFin (by
rw [(polynomialBalancedSplit P).foldB_card]
rfl)
def polynomialFoldBReindex
Reindex an estimation-fold tuple by the canonical finite type with the same cardinality.
Definition (Lean source)
noncomputable def polynomialFoldBReindex {n d : ℕ} (P : RealLaw d)
(x : (polynomialBalancedSplit P).foldB n → Obs d) :
Fin (n - n / 2) → Obs d :=
fun j => x (polynomialFoldBEquiv P j)
def polynomialFoldBAsFin
Regard an estimation-fold index as an index of the original finite sample.
Definition (Lean source)
def polynomialFoldBAsFin {n d : ℕ} (P : RealLaw d)
(i : (polynomialBalancedSplit P).foldB n) : Fin n :=
⟨i.val, by
have hmem := i.property
change i.val ∈ (range n).filter (fun q => n / 2 ≤ q) at hmem
exact Finset.mem_range.mp (Finset.mem_filter.mp hmem).1⟩
theorem rebuildPolynomialEstimationSample_foldBAsFin
Rebuilding a full tuple and then restricting to an estimation-fold index returns the original fold observation.
Formal statement
hypotheses
n d :
ℕ
P :
RealLaw d
base :
Obs d
x :
(polynomialBalancedSplit P).foldB n → Obs d
i :
(polynomialBalancedSplit P).foldB n
conclusion
rebuildPolynomialEstimationSample P base x (polynomialFoldBAsFin P i) = x i
Proof (Lean source)
lemma rebuildPolynomialEstimationSample_foldBAsFin {n d : ℕ}
(P : RealLaw d) (base : Obs d)
(x : (polynomialBalancedSplit P).foldB n → Obs d)
(i : (polynomialBalancedSplit P).foldB n) :
rebuildPolynomialEstimationSample P base x (polynomialFoldBAsFin P i) = x i := by
have hmem := i.property
change i.val ∈ (range n).filter (fun q => n / 2 ≤ q) at hmem
have hge := (Finset.mem_filter.mp hmem).2
unfold rebuildPolynomialEstimationSample
change (if h : n / 2 ≤ i.val then
x ⟨i.val, Finset.mem_filter.mpr
⟨Finset.mem_range.mpr (polynomialFoldBAsFin P i).isLt, h⟩⟩
else base) = x i
rw [dif_pos hge]
theorem map_polynomialFoldBReindex_iid_eq_productLaw
Under the infinite iid realization, the reindexed balanced estimation fold has exactly the finite product law of size n - n / 2.
Formal statement
hypotheses
n d :
ℕ
P :
RealLaw d
conclusion
Measure.map (fun ω : ℕ → Obs d => polynomialFoldBReindex P (fun i : (polynomialBalancedSplit P).foldB n => ω i)) (Measure.infinitePi fun _ : ℕ => P.observedLaw)
= productLaw (n - n / 2) P
Proof (Lean source)
lemma map_polynomialFoldBReindex_iid_eq_productLaw {n d : ℕ}
(P : RealLaw d) :
Measure.map
(fun ω : ℕ → Obs d => polynomialFoldBReindex P
(fun i : (polynomialBalancedSplit P).foldB n => ω i))
(Measure.infinitePi fun _ : ℕ => P.observedLaw) =
productLaw (n - n / 2) P := by
let S := iidSample_infinitePi P.observedLaw
let e := polynomialFoldBEquiv (n := n) P
let YB : (ℕ → Obs d) → (polynomialBalancedSplit P).foldB n → Obs d :=
fun ω i => S.Z i ω
let T : ((polynomialBalancedSplit P).foldB n → Obs d) ≃ᵐ
(Fin (n - n / 2) → Obs d) :=
MeasurableEquiv.piCongrLeft (fun _ : Fin (n - n / 2) => Obs d)
e.symm.toEquiv
have hYB : Measurable YB := by
apply measurable_pi_lambda
intro i
exact S.meas i
have heq :
(fun ω : ℕ → Obs d => polynomialFoldBReindex P
(fun i : (polynomialBalancedSplit P).foldB n => ω i)) = T ∘ YB := by
funext ω j
change S.Z (e j) ω = T (YB ω) j
simpa [T, YB] using
(MeasurableEquiv.piCongrLeft_apply_apply
(e := e.symm.toEquiv)
(β := fun _ : Fin (n - n / 2) => Obs d)
(x := fun i : (polynomialBalancedSplit P).foldB n => S.Z i ω)
(i := e j)).symm
rw [heq, ← Measure.map_map T.measurable hYB]
rw [oneShot_iid S (polynomialBalancedSplit P) n]
unfold productLaw
simpa [T] using Measure.pi_map_piCongrLeft
(e := e.symm.toEquiv) (β := fun _ : Fin (n - n / 2) => Obs d)
(μ := fun _ : Fin (n - n / 2) => P.observedLaw)
theorem estimationArmCount_rebuild_eq_reindex
Every estimation-block arm/cell count is the corresponding count on the canonically reindexed fold tuple.
Formal statement
hypotheses
conclusion
estimationArmCount (rebuildPolynomialEstimationSample P base x) a k
= groupArmCount (fun o : Obs d => o.x) (fun o => o.a) (polynomialFoldBReindex P x) a k
Proof (Lean source)
lemma estimationArmCount_rebuild_eq_reindex {n d : ℕ} (P : RealLaw d)
(base : Obs d) (x : (polynomialBalancedSplit P).foldB n → Obs d)
(a : Bool) (k : Fin d) :
estimationArmCount (rebuildPolynomialEstimationSample P base x) a k =
groupArmCount (fun o : Obs d => o.x) (fun o => o.a)
(polynomialFoldBReindex P x) a k := by
classical
unfold estimationArmCount groupArmCount
symm
apply Finset.card_bij
(fun j _ => ⟨(polynomialFoldBEquiv P j).val,
by
have hmem := (polynomialFoldBEquiv P j).property
change (polynomialFoldBEquiv P j).val ∈
(range n).filter (fun i => n / 2 ≤ i) at hmem
exact Finset.mem_range.mp (Finset.mem_filter.mp hmem).1⟩)
· intro j hj
simp only [mem_filter, Finset.mem_univ, true_and] at hj ⊢
have hmem := (polynomialFoldBEquiv P j).property
change (polynomialFoldBEquiv P j).val ∈
(range n).filter (fun i => n / 2 ≤ i) at hmem
have hge := (Finset.mem_filter.mp hmem).2
simp only [inPilot, rebuildPolynomialEstimationSample, hge, dite_true]
have hxeq :
x ⟨(polynomialFoldBEquiv P j).val, by
exact Finset.mem_filter.mpr ⟨(Finset.mem_filter.mp hmem).1, hge⟩⟩ =
x (polynomialFoldBEquiv P j) := by
congr 1
refine ⟨?_, ?_⟩
· simp [Nat.not_lt.mpr hge]
rw [hxeq]
simpa [polynomialFoldBReindex] using hj
· intro j₁ _ j₂ _ h
apply (polynomialFoldBEquiv P).injective
apply Subtype.ext
exact Fin.ext_iff.mp h
· intro i hi
simp only [mem_filter, Finset.mem_univ, true_and] at hi
have hge : n / 2 ≤ i.val := by
simpa [inPilot, Bool.not_eq_true] using hi.1
let ib : (polynomialBalancedSplit P).foldB n :=
⟨i.val, Finset.mem_filter.mpr ⟨Finset.mem_range.mpr i.isLt, hge⟩⟩
let j := (polynomialFoldBEquiv P).symm ib
refine ⟨j, ?_, ?_⟩
· simp only [mem_filter, Finset.mem_univ, true_and]
simpa [polynomialFoldBReindex, j, ib,
rebuildPolynomialEstimationSample, hge] using hi.2
· apply Fin.ext
change (polynomialFoldBEquiv P j).val = i.val
simp [j, ib]
theorem estimationArmSum_rebuild_eq_reindex
Every estimation-block marked outcome sum is the corresponding supported mark sum on the canonically reindexed fold tuple.
Formal statement
hypotheses
conclusion
estimationArmSum (rebuildPolynomialEstimationSample P base x) a k
= armMarkSum (fun o : Obs d => o.x) (fun o => o.a) (fun o => o.y) (polynomialFoldBReindex P x) a k
Proof (Lean source)
lemma estimationArmSum_rebuild_eq_reindex {n d : ℕ} (P : RealLaw d)
(base : Obs d) (x : (polynomialBalancedSplit P).foldB n → Obs d)
(a : Bool) (k : Fin d) :
estimationArmSum (rebuildPolynomialEstimationSample P base x) a k =
armMarkSum
(fun o : Obs d => o.x) (fun o => o.a) (fun o => o.y)
(polynomialFoldBReindex P x) a k := by
classical
unfold estimationArmSum
unfold armMarkSum
supportedArmMark
armCategoryEvent
armGroupEvent indicator
simp only [Set.mem_ofPred_eq]
rw [← Finset.sum_filter]
simp_rw [← Finset.sum_filter]
symm
apply Finset.sum_bij
(fun j _ => polynomialFoldBAsFin P (polynomialFoldBEquiv P j))
· intro j hj
simp only [mem_filter, Finset.mem_univ, true_and] at hj ⊢
have hmem := (polynomialFoldBEquiv P j).property
change (polynomialFoldBEquiv P j).val ∈
(range n).filter (fun i => n / 2 ≤ i) at hmem
have hge := (Finset.mem_filter.mp hmem).2
refine ⟨?_, ?_⟩
· change (!decide ((polynomialFoldBEquiv P j).val < n / 2)) = true
simp [Nat.not_lt.mpr hge]
· rw [rebuildPolynomialEstimationSample_foldBAsFin]
simpa [polynomialFoldBReindex] using hj
· intro j₁ _ j₂ _ h
apply (polynomialFoldBEquiv P).injective
apply Subtype.ext
exact Fin.ext_iff.mp h
· intro i hi
simp only [mem_filter, Finset.mem_univ, true_and] at hi
have hge : n / 2 ≤ i.val := by
simpa [inPilot, Bool.not_eq_true] using hi.1
let ib : (polynomialBalancedSplit P).foldB n :=
⟨i.val, Finset.mem_filter.mpr ⟨Finset.mem_range.mpr i.isLt, hge⟩⟩
let j := (polynomialFoldBEquiv P).symm ib
refine ⟨j, ?_, ?_⟩
· simp only [mem_filter, Finset.mem_univ, true_and]
simpa [polynomialFoldBReindex, j, ib,
rebuildPolynomialEstimationSample, hge] using hi.2
· apply Fin.ext
change (polynomialFoldBEquiv P j).val = i.val
simp [j, ib]
· intro j _
rw [rebuildPolynomialEstimationSample_foldBAsFin]
rfl
theorem estimationCellCount_rebuild_eq_reindex
The estimation-block cell count is the generic cell count on the reindexed fold.
Formal statement
hypotheses
conclusion
estimationCellCount (rebuildPolynomialEstimationSample P base x) k
= groupCount (fun o : Obs d => o.x) (fun o => o.a) (polynomialFoldBReindex P x) k
Proof (Lean source)
lemma estimationCellCount_rebuild_eq_reindex {n d : ℕ} (P : RealLaw d)
(base : Obs d) (x : (polynomialBalancedSplit P).foldB n → Obs d)
(k : Fin d) :
estimationCellCount (rebuildPolynomialEstimationSample P base x) k =
groupCount (fun o : Obs d => o.x) (fun o => o.a)
(polynomialFoldBReindex P x) k := by
unfold estimationCellCount groupCount
rw [estimationArmCount_rebuild_eq_reindex,
estimationArmCount_rebuild_eq_reindex]
theorem estimationArmMean_rebuild_eq_reindex
The zero-safe estimation-block arm mean is the generic totalized arm mean on the reindexed fold.
Formal statement
hypotheses
conclusion
estimationArmMean (rebuildPolynomialEstimationSample P base x) a k
= totalizedArmMean (fun o : Obs d => o.x) (fun o => o.a) (fun o => o.y) (polynomialFoldBReindex P x) a k
Proof (Lean source)
lemma estimationArmMean_rebuild_eq_reindex {n d : ℕ} (P : RealLaw d)
(base : Obs d) (x : (polynomialBalancedSplit P).foldB n → Obs d)
(a : Bool) (k : Fin d) :
estimationArmMean (rebuildPolynomialEstimationSample P base x) a k =
totalizedArmMean
(fun o : Obs d => o.x) (fun o => o.a) (fun o => o.y)
(polynomialFoldBReindex P x) a k := by
unfold estimationArmMean
totalizedArmMean
categoryArmCount
rw [estimationArmCount_rebuild_eq_reindex,
estimationArmSum_rebuild_eq_reindex]
split_ifs with h
· rw [div_eq_mul_inv, mul_comm]
· rfl
theorem heavyEmpiricalTerm_rebuild_eq_reindex
The concrete heavy-cell term is exactly the normalized fixed-stratum term on the reindexed estimation fold.
Formal statement
hypotheses
conclusion
heavyEmpiricalTerm M (rebuildPolynomialEstimationSample P base x) k
= (categoryCount (fun o : Obs d => o.x) (fun o => o.a) (polynomialFoldBReindex P x) k : ℝ) / (n - n / 2 : ℕ) * ((totalizedArmMean (fun o : Obs d => o.x) (fun o => o.a) (fun o => o.y) (polynomialFoldBReindex P x) true k - totalizedArmMean (fun o : Obs d => o.x) (fun o => o.a) (fun o => o.y) (polynomialFoldBReindex P x) false k) / M)
Proof (Lean source)
lemma heavyEmpiricalTerm_rebuild_eq_reindex {n d : ℕ} (P : RealLaw d)
(base : Obs d) (x : (polynomialBalancedSplit P).foldB n → Obs d)
(M : ℝ) (k : Fin d) :
heavyEmpiricalTerm M (rebuildPolynomialEstimationSample P base x) k =
(categoryCount
(fun o : Obs d => o.x) (fun o => o.a)
(polynomialFoldBReindex P x) k : ℝ) / (n - n / 2 : ℕ) *
((totalizedArmMean
(fun o : Obs d => o.x) (fun o => o.a) (fun o => o.y)
(polynomialFoldBReindex P x) true k -
totalizedArmMean
(fun o : Obs d => o.x) (fun o => o.a) (fun o => o.y)
(polynomialFoldBReindex P x) false k) / M) := by
unfold heavyEmpiricalTerm estimationBlockSize
rw [estimationCellCount_rebuild_eq_reindex,
estimationArmMean_rebuild_eq_reindex,
estimationArmMean_rebuild_eq_reindex]
rfl
Helpers.PolynomialUpper.Heavy 9 declarations
structure PolynomialCellAggregate
The cell aggregates available after the single pass through the sample. No raw observation is retained by the post-aggregation evaluator.
def aggregatePolynomialSample
The concrete aggregation pass used by the polynomial estimator.
Definition (Lean source)
noncomputable def aggregatePolynomialSample {n d : ℕ}
(sample : Fin n → Obs d) : PolynomialCellAggregate d where
pilot := pilotCount sample
count := fun a k => estimationArmCount sample a k
outcomeSum := fun a k => estimationArmSum sample a k
inductive AggregatedArithmeticProgram
A small executable arithmetic language over the aggregated cell table. Its only inputs are counts and outcome sums; loops are represented explicitly by finite sums, so their arithmetic cost is determined by syntax.
Definition (Lean source)
inductive AggregatedArithmeticProgram (d : ℕ) where
| const (x : ℝ)
| pilot (k : Fin d)
| count (a : Bool) (k : Fin d)
| outcomeSum (a : Bool) (k : Fin d)
| add (p q : AggregatedArithmeticProgram d)
| sub (p q : AggregatedArithmeticProgram d)
| mul (p q : AggregatedArithmeticProgram d)
| div (p q : AggregatedArithmeticProgram d)
| minimum (p q : AggregatedArithmeticProgram d)
| maximum (p q : AggregatedArithmeticProgram d)
| iteLt (p q yes no : AggregatedArithmeticProgram d)
| sumCells (body : Fin d → AggregatedArithmeticProgram d)
| sumTerms (r : ℕ) (body : Fin r → AggregatedArithmeticProgram d)
| armDescFactorial (a : Bool) (k : Fin d) (offset order : ℕ)
| cellSub (k : Fin d) (offset : ℕ)
def eval
Operational semantics of the post-aggregation arithmetic language.
Definition (Lean source)
noncomputable def AggregatedArithmeticProgram.eval {d : ℕ}
(input : PolynomialCellAggregate d) : AggregatedArithmeticProgram d → ℝ
| .const x => x
| .pilot k => input.pilot k
| .count a k => input.count a k
| .outcomeSum a k => input.outcomeSum a k
| .add p q => p.eval input + q.eval input
| .sub p q => p.eval input - q.eval input
| .mul p q => p.eval input * q.eval input
| .div p q => p.eval input / q.eval input
| .minimum p q => min (p.eval input) (q.eval input)
| .maximum p q => max (p.eval input) (q.eval input)
| .iteLt p q yes no => if p.eval input < q.eval input then yes.eval input else no.eval input
| .sumCells body => ∑ k, (body k).eval input
| .sumTerms _ body => ∑ j, (body j).eval input
| .armDescFactorial a k offset order =>
((input.count a k - offset).descFactorial order : ℕ)
| .cellSub k offset =>
((input.count false k + input.count true k - offset : ℕ) : ℝ)
def operationCount
Structural arithmetic cost of an executable program. Input reads and constants are free; every arithmetic/comparison node and every finite-sum accumulation is charged.
Definition (Lean source)
def AggregatedArithmeticProgram.operationCount {d : ℕ} :
AggregatedArithmeticProgram d → ℕ
| .const _ | .pilot _ | .count _ _ | .outcomeSum _ _ => 0
| .add p q | .sub p q | .mul p q | .div p q | .minimum p q | .maximum p q =>
1 + p.operationCount + q.operationCount
| .iteLt p q yes no =>
1 + p.operationCount + q.operationCount +
max yes.operationCount no.operationCount
| .sumCells body => d + ∑ k, (body k).operationCount
| .sumTerms r body => r + ∑ j, (body j).operationCount
| .armDescFactorial _ _ _ order => order
| .cellSub _ _ => 1
structure AggregatedPolynomialProgram
A genuine post-aggregation program certificate: executing its syntax on the actual aggregate table computes the handle-indexed estimator for every sample.
Definition (Lean source)
parameters
fields
code :
computesEstimator :
∀ sample : Fin n → Obs d,
code.eval (aggregatePolynomialSample sample) = polyEstimator handle M sample
def polynomialOperationBudget
The proof-local numerical budget used to certify the concrete program.
Definition (Lean source)
noncomputable def polynomialOperationBudget (n d : ℕ) : ℕ :=
128 * (d + 1) * (polynomialDegree n + 1) ^ 2
def PolynomialExactComplexityCertificate
Internal finite-sample strengthening with the proof's explicit numerical constant. This is not the public complexity claim.
Definition (Lean source)
def PolynomialExactComplexityCertificate (n d : ℕ)
(handle : PolynomialHandle) (M : ℝ) : Prop :=
∃ program : AggregatedPolynomialProgram n d handle M,
program.code.operationCount ≤ polynomialOperationBudget n d
def PolynomialComplexityBound
The estimator family has post-aggregation arithmetic complexity O(d K²), with one constant uniform in the sample size, alphabet size, and outcome scale.
Definition (Lean source)
def PolynomialComplexityBound (handle : PolynomialHandle) : Prop :=
∃ C : ℕ, 0 < C ∧
∀ n d : ℕ, ∀ M : ℝ,
∃ program : AggregatedPolynomialProgram n d handle M,
program.code.operationCount ≤
C * (d + 1) * (polynomialDegree n + 1) ^ 2
Helpers.PolynomialUpper.HeavyRisk 8 declarations
def polynomialSupportedCenter
The arm-cell center agrees with the model mean on supported cells and is zero on null cells, making the fixed-stratum center envelope total.
Definition (Lean source)
noncomputable def polynomialSupportedCenter {d : ℕ} (P : RealLaw d)
(a : Bool) (k : Fin d) : ℝ :=
if 0 < P.cellMass k then P.outcomeMean a k else 0
theorem polynomialSupportedCenter_abs_le
The support-totalized center obeys the model's half-scale envelope on every cell, including null cells.
Formal statement
hypotheses
conclusion
|polynomialSupportedCenter P.law a k| ≤ M
Proof (Lean source)
lemma polynomialSupportedCenter_abs_le {d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (a : Bool) (k : Fin d) :
|polynomialSupportedCenter P.law a k| ≤ M := by
unfold polynomialSupportedCenter
split_ifs with hk
· exact (P.mean_normalization a k hk).trans (by linarith [P.M_ge_one])
· simp
exact le_trans zero_le_one P.M_ge_one
theorem polynomialSupportedCenter_residual_memLp
Supported residuals about the totalized centers are square-integrable under the observed law.
Formal statement
hypotheses
conclusion
MemLp (supportedArmResidual (fun o : Obs d => o.x) (fun o => o.a) (fun o => o.y) (polynomialSupportedCenter P.law) a k) 2 P.law.observedLaw
Proof (Lean source)
lemma polynomialSupportedCenter_residual_memLp {d : ℕ}
{epsilon M sigma : ℝ} (P : ModelClass d epsilon M sigma)
(a : Bool) (k : Fin d) :
MemLp (supportedArmResidual (fun o : Obs d => o.x) (fun o => o.a)
(fun o => o.y) (polynomialSupportedCenter P.law) a k) 2
P.law.observedLaw := by
let f := supportedArmResidual (fun o : Obs d => o.x) (fun o => o.a)
(fun o => o.y) P.law.outcomeMean a k
let g := supportedArmResidual (fun o : Obs d => o.x) (fun o => o.a)
(fun o => o.y) (polynomialSupportedCenter P.law) a k
have hf : MemLp f 2 P.law.observedLaw := test_memLp P a k
have hfg : f =ᵐ[P.law.observedLaw] g := by
by_cases hk : 0 < P.law.cellMass k
· filter_upwards with o
by_cases ho : o.x = k ∧ o.a = a
· rcases ho with ⟨rfl, hoa⟩
simp [f, g, supportedArmResidual, supportedArmGroupResidual,
armGroupEvent, hoa]
unfold armGroupResidual
simp [polynomialSupportedCenter, hk]
· simp [f, g, supportedArmResidual, supportedArmGroupResidual,
armGroupEvent, ho]
· have hkzero : P.law.cellMass k = 0 :=
le_antisymm (not_lt.mp hk) (P.law.cellMass_range k).1
have hnull := observed_arm_cell_measure_eq_zero_of_cellMass_eq_zero
P.law a k hkzero
filter_upwards [measure_eq_zero_iff_ae_notMem.mp hnull] with o ho
simp [f, g, supportedArmResidual, supportedArmGroupResidual,
armGroupEvent, ho]
have hgmeas : AEStronglyMeasurable g P.law.observedLaw := hf.1.congr hfg
exact hf.congr_norm hgmeas (hfg.mono fun _ h => by rw [h])
theorem polynomialSupportedCenter_centered_integral
Each support-totalized arm-cell residual has zero integral.
Formal statement
hypotheses
conclusion
∫ o in armCategoryEvent (fun o : Obs d => o.x) (fun o => o.a) a k, (o.y - polynomialSupportedCenter P.law a k) ∂P.law.observedLaw
= 0
Proof (Lean source)
lemma polynomialSupportedCenter_centered_integral {d : ℕ}
{epsilon M sigma : ℝ} (P : ModelClass d epsilon M sigma)
(a : Bool) (k : Fin d) :
∫ o in armCategoryEvent (fun o : Obs d => o.x) (fun o => o.a) a k,
(o.y - polynomialSupportedCenter P.law a k) ∂P.law.observedLaw = 0 := by
by_cases hk : 0 < P.law.cellMass k
· simpa [polynomialSupportedCenter, hk, armCategoryEvent, armGroupEvent] using
observed_arm_cell_centered_integral_eq_zero P a k
· have hkzero : P.law.cellMass k = 0 :=
le_antisymm (not_lt.mp hk) (P.law.cellMass_range k).1
have hnull := observed_arm_cell_measure_eq_zero_of_cellMass_eq_zero
P.law a k hkzero
simpa [armCategoryEvent, armGroupEvent] using
(setIntegral_measure_zero
(fun o : Obs d => o.y - polynomialSupportedCenter P.law a k) hnull)
theorem polynomialSupportedCenter_centered_sq_le
Each support-totalized arm-cell residual obeys the conditional second-moment envelope with the exact observed arm-cell mass.
Formal statement
hypotheses
conclusion
∫ o in armCategoryEvent (fun o : Obs d => o.x) (fun o => o.a) a k, (o.y - polynomialSupportedCenter P.law a k) ^ 2 ∂P.law.observedLaw
≤ armCategoryMass P.law.observedLaw (fun o : Obs d => o.x) (fun o => o.a) a k * M ^ 2
Proof (Lean source)
lemma polynomialSupportedCenter_centered_sq_le {d : ℕ}
{epsilon M sigma : ℝ} (P : ModelClass d epsilon M sigma)
(a : Bool) (k : Fin d) :
∫ o in armCategoryEvent (fun o : Obs d => o.x) (fun o => o.a) a k,
(o.y - polynomialSupportedCenter P.law a k) ^ 2 ∂P.law.observedLaw ≤
armCategoryMass P.law.observedLaw (fun o : Obs d => o.x)
(fun o => o.a) a k * M ^ 2 := by
rw [show armCategoryMass P.law.observedLaw (fun o : Obs d => o.x)
(fun o => o.a) a k =
P.law.cellMass k *
(if a then P.law.propensity k else 1 - P.law.propensity k) by
change realMass P.law.observedLaw
(armGroupEvent (fun o : Obs d => o.x) (fun o => o.a) a k) = _
exact test_arm_mass P.law a k]
by_cases hk : 0 < P.law.cellMass k
· simpa [polynomialSupportedCenter, hk, armCategoryEvent, armGroupEvent] using
observed_arm_cell_centered_sq_integral_le P a k
· have hkzero : P.law.cellMass k = 0 :=
le_antisymm (not_lt.mp hk) (P.law.cellMass_range k).1
have hnull := observed_arm_cell_measure_eq_zero_of_cellMass_eq_zero
P.law a k hkzero
rw [hkzero, zero_mul, zero_mul]
rw [show armCategoryEvent (fun o : Obs d => o.x) (fun o => o.a) a k =
{o : Obs d | o.x = k ∧ o.a = a} by rfl,
setIntegral_measure_zero _ hnull]
theorem polynomialPopulationArmMean_eq_outcomeMean
If the stated condition on the cell holds, on every supported cell, the generic fixed-stratum population arm mean is exactly the model's declared conditional outcome mean.
Formal statement
hypotheses
conclusion
populationArmMean P.law.observedLaw (fun o : Obs d => o.x) (fun o => o.a) (fun o => o.y) a k
= P.law.outcomeMean a k
Proof (Lean source)
lemma polynomialPopulationArmMean_eq_outcomeMean {d : ℕ}
{epsilon M sigma : ℝ} (P : ModelClass d epsilon M sigma)
(a : Bool) (k : Fin d) (hk : 0 < P.law.cellMass k) :
populationArmMean P.law.observedLaw (fun o : Obs d => o.x)
(fun o => o.a) (fun o => o.y) a k = P.law.outcomeMean a k := by
have htuple : Measurable (fun o : Obs d => (o.x, o.a, o.y)) :=
measurable_iff_comap_le.mpr le_rfl
have hy : Measurable (fun o : Obs d => o.y) :=
measurable_snd.comp (measurable_snd.comp htuple)
let c : ℝ := P.law.cellMass k *
(if a then P.law.propensity k else 1 - P.law.propensity k)
have hc : armCategoryMass P.law.observedLaw (fun o : Obs d => o.x)
(fun o => o.a) a k = c := by
change realMass P.law.observedLaw
(armGroupEvent (fun o : Obs d => o.x) (fun o => o.a) a k) = c
simpa [c] using test_arm_mass P.law a k
have hc0 : 0 ≤ c := by
dsimp [c]
apply mul_nonneg (P.law.cellMass_range k).1
split <;> simp_all [(P.law.propensity_range k).1,
(P.law.propensity_range k).2]
have hcpos : 0 < c := by
rcases P.overlap k hk with ⟨hlower, hupper⟩
cases a <;> simp [c] <;> nlinarith [P.epsilon_pos]
have hmap := observed_arm_cell_outcome_measure P.law a k
unfold populationArmMean
rw [if_pos (hc ▸ hcpos), hc]
have hmeas : AEStronglyMeasurable (fun y : ℝ => y)
(Measure.map (fun o : Obs d => o.y)
(P.law.observedLaw.restrict
(armCategoryEvent (fun o : Obs d => o.x) (fun o => o.a) a k))) :=
measurable_id.aestronglyMeasurable
rw [← integral_map hy.aemeasurable hmeas]
change c⁻¹ * (∫ y, y ∂Measure.map (fun o : Obs d => o.y)
(P.law.observedLaw.restrict {o : Obs d | o.x = k ∧ o.a = a})) = _
rw [hmap, integral_smul_measure]
rw [show P.law.cellMass k *
(if a then P.law.propensity k else 1 - P.law.propensity k) = c by rfl]
simp only [smul_eq_mul]
change c⁻¹ * ((ofReal c).toReal *
(∫ y, y ∂P.law.outcomeLaw a k)) = _
rw [ENNReal.toReal_ofReal hc0, ← P.law.outcomeMean_eq]
field_simp [hcpos.ne']
theorem polynomialFixedStratumMarkedTarget_eq_cellEffectSum
If the selected heavy set is fixed, for a deterministic supported heavy set, the generic marked-ratio target is the corresponding cell-mass-weighted sum of model treatment effects.
Formal statement
hypotheses
conclusion
fixedStratumMarkedTarget P.law.observedLaw (fun o : Obs d => o.x) (fun o => o.a) (fun o => o.y) H
= ∑ k ∈ H, P.law.cellMass k * cellEffect P.law k
Proof (Lean source)
lemma polynomialFixedStratumMarkedTarget_eq_cellEffectSum {d : ℕ}
{epsilon M sigma : ℝ} (P : ModelClass d epsilon M sigma)
(H : Finset (Fin d)) (hH : ∀ k ∈ H, 0 < P.law.cellMass k) :
fixedStratumMarkedTarget P.law.observedLaw
(fun o : Obs d => o.x) (fun o => o.a) (fun o => o.y) H =
∑ k ∈ H, P.law.cellMass k * cellEffect P.law k := by
unfold fixedStratumMarkedTarget fixedStratumArmTarget cellEffect
have hmass (k : Fin d) :
categoryMass P.law.observedLaw (fun o : Obs d => o.x) k =
P.law.cellMass k := by
simpa [categoryMass, categoryEvent, groupEvent, realMass] using
(P.law.cellMass_eq k).symm
simp_rw [hmass]
rw [← Finset.sum_sub_distrib]
apply Finset.sum_congr rfl
intro k hk
rw [polynomialPopulationArmMean_eq_outcomeMean P true k (hH k hk),
polynomialPopulationArmMean_eq_outcomeMean P false k (hH k hk)]
ring
theorem fixedHeavyMarkedRatio_error_sq_le
If the truncation threshold satisfies its stated bound, for every deterministic heavy set whose cells have mass at least B, the fixed-heavy marked-ratio score satisfies the generic boundary-safe parametric and missing-arm risk bound under the real-outcome model assumptions.
Formal statement
hypotheses
conclusion
∫ z : Fin m → Obs d, (fixedStratumMarkedRatio (fun o : Obs d => o.x) (fun o => o.a) (fun o => o.y) H z - fixedStratumMarkedTarget P.law.observedLaw (fun o : Obs d => o.x) (fun o => o.a) (fun o => o.y) H) ^ 2 ∂(productLaw m P.law)
≤ 4 * M ^ 2 * (8 * (∑ k ∈ H, categoryMass P.law.observedLaw (fun o : Obs d => o.x) k) / (safeSampleSize m * epsilon) + 6 / safeSampleSize m + 4 * (lowerMassMissingEnvelope P.law.observedLaw (fun o : Obs d => o.x) m epsilon B H) ^ 2)
Proof (Lean source)
lemma fixedHeavyMarkedRatio_error_sq_le {d m : ℕ}
{epsilon M sigma B : ℝ} (P : ModelClass d epsilon M sigma)
(H : Finset (Fin d)) (hB : ∀ k ∈ H, B ≤ P.law.cellMass k) :
∫ z : Fin m → Obs d,
(fixedStratumMarkedRatio (fun o : Obs d => o.x) (fun o => o.a)
(fun o => o.y) H z -
fixedStratumMarkedTarget P.law.observedLaw
(fun o : Obs d => o.x) (fun o => o.a) (fun o => o.y) H) ^ 2
∂(productLaw m P.law) ≤
4 * M ^ 2 *
(8 * (∑ k ∈ H, categoryMass P.law.observedLaw
(fun o : Obs d => o.x) k) /
(safeSampleSize m * epsilon) +
6 / safeSampleSize m +
4 * (lowerMassMissingEnvelope P.law.observedLaw
(fun o : Obs d => o.x) m epsilon B H) ^ 2) := by
have htuple : Measurable (fun o : Obs d => (o.x, o.a, o.y)) :=
measurable_iff_comap_le.mpr le_rfl
have hx : Measurable (fun o : Obs d => o.x) := measurable_fst.comp htuple
have ha : Measurable (fun o : Obs d => o.a) :=
measurable_fst.comp (measurable_snd.comp htuple)
have hy : Measurable (fun o : Obs d => o.y) :=
measurable_snd.comp (measurable_snd.comp htuple)
have hmass (k : Fin d) :
categoryMass P.law.observedLaw (fun o : Obs d => o.x) k =
P.law.cellMass k := by
simpa [categoryMass, categoryEvent, groupEvent, realMass] using
(P.law.cellMass_eq k).symm
have hoverlap : ∀ k, 0 < categoryMass P.law.observedLaw
(fun o : Obs d => o.x) k → ∀ a,
epsilon * categoryMass P.law.observedLaw (fun o : Obs d => o.x) k ≤
armCategoryMass P.law.observedLaw (fun o : Obs d => o.x)
(fun o => o.a) a k := by
intro k hk a
rw [hmass] at hk ⊢
rw [show armCategoryMass P.law.observedLaw (fun o : Obs d => o.x)
(fun o => o.a) a k = P.law.cellMass k *
(if a then P.law.propensity k else 1 - P.law.propensity k) by
change realMass P.law.observedLaw
(armGroupEvent (fun o : Obs d => o.x) (fun o => o.a) a k) = _
exact test_arm_mass P.law a k]
have hov := P.overlap k hk
cases a <;> simp only [Bool.false_eq_true, ↓reduceIte] <;>
nlinarith [P.law.cellMass_range k]
rw [show productLaw m P.law =
Measure.pi (fun _ : Fin m => P.law.observedLaw) from rfl]
apply integral_fixedStratumMarkedRatio_error_sq_le
P.law.observedLaw (fun o : Obs d => o.x) (fun o => o.a)
(fun o => o.y) (polynomialSupportedCenter P.law) H M epsilon B
hx ha hy
· exact polynomialSupportedCenter_residual_memLp P
· exact polynomialSupportedCenter_centered_integral P
· exact polynomialSupportedCenter_centered_sq_le P
· exact polynomialSupportedCenter_abs_le P
· exact P.epsilon_pos
· exact hoverlap
· intro k hk
simpa [hmass] using hB k hk
Helpers.PolynomialUpper.Pilot 10 declarations
Population-mass lower band certified for cells selected as heavy.
Definition (Lean source)
noncomputable def polynomialPilotLowerBand (n : ℕ) : ℝ :=
128 * logEN n / (n / 2 : ℕ)
Population-mass upper band certified for cells selected as light.
Definition (Lean source)
noncomputable def polynomialPilotUpperBand (n : ℕ) : ℝ :=
512 * logEN n / (n / 2 : ℕ)
theorem polynomialPilotUpperBand_le_lightScale_quarter
If the sample size satisfies the stated lower bound, from sample size two onward, the pilot light-band lies below one quarter of the polynomial scale used on the independent estimation half.
Formal statement
hypotheses
n :
ℕ
hn :
2 ≤ n
conclusion
polynomialPilotUpperBand n ≤ (4096 * logEN n / (n - n / 2 : ℕ)) / 4
Proof (Lean source)
lemma polynomialPilotUpperBand_le_lightScale_quarter {n : ℕ} (hn : 2 ≤ n) :
polynomialPilotUpperBand n ≤
(4096 * logEN n / (n - n / 2 : ℕ)) / 4 := by
have hnpos : 0 < n := by omega
have hm0 : 0 < n / 2 := Nat.div_pos (by omega) (by norm_num)
have hm1 : 0 < n - n / 2 := Nat.sub_pos_of_lt (Nat.div_lt_self hnpos (by norm_num))
have hlog : 0 < logEN n := by
have hnR : 0 < (n : ℝ) := by exact_mod_cast hnpos
rw [logEN, Real.log_mul (Real.exp_ne_zero 1) hnR.ne', Real.log_exp]
have hone : 1 ≤ (n : ℝ) := by exact_mod_cast hnpos
nlinarith [Real.log_nonneg hone]
have hhalf : n - n / 2 ≤ 2 * (n / 2) := by omega
have hm0R : 0 < ((n / 2 : ℕ) : ℝ) := by exact_mod_cast hm0
have hm1R : 0 < ((n - n / 2 : ℕ) : ℝ) := by exact_mod_cast hm1
unfold polynomialPilotUpperBand
have hrhs : 4096 * logEN n / ((n - n / 2 : ℕ) : ℝ) / 4 =
1024 * logEN n / ((n - n / 2 : ℕ) : ℝ) := by ring
rw [hrhs]
apply (div_le_div_iff₀ hm0R hm1R).2
have hhalfR : ((n - n / 2 : ℕ) : ℝ) ≤
2 * ((n / 2 : ℕ) : ℝ) := by exact_mod_cast hhalf
nlinarith
theorem polynomialPilotCount_eq_finiteCategoryPilotCount
The estimator's pilot count is exactly the generic finite-category count on the first half of the infinite iid realization.
Formal statement
hypotheses
conclusion
pilotCount (fun i : Fin n => ω i) k
= pilotCategoryCount (iidSample_infinitePi P.observedLaw) (fun o : Obs d => o.x) (range (n / 2)) k ω
Proof (Lean source)
lemma polynomialPilotCount_eq_finiteCategoryPilotCount {n d : ℕ}
(P : RealLaw d) (k : Fin d) (ω : ℕ → Obs d) :
pilotCount (fun i : Fin n => ω i) k =
pilotCategoryCount
(iidSample_infinitePi P.observedLaw)
(fun o : Obs d => o.x) (range (n / 2)) k ω := by
unfold pilotCount pilotCategoryCount inPilot
apply Finset.card_bij (fun i _ => i.val)
· intro i hi
simp only [mem_filter, Finset.mem_univ, true_and,
decide_eq_true_eq] at hi
exact Finset.mem_filter.mpr ⟨Finset.mem_range.mpr hi.1, hi.2⟩
· intro i₁ _ i₂ _ hij
exact Fin.ext hij
· intro j hj
simp only [mem_filter, Finset.mem_range] at hj
let i : Fin n := ⟨j, lt_of_lt_of_le hj.1 (Nat.div_le_self n 2)⟩
refine ⟨i, ?_, rfl⟩
simp only [mem_filter, Finset.mem_univ, true_and,
decide_eq_true_eq]
exact ⟨hj.1, hj.2⟩
def polynomialPilotGood
Simultaneous pilot event: every selected-heavy cell has the stated lower mass and every selected-light cell has the stated upper mass.
Definition (Lean source)
noncomputable def polynomialPilotGood {n d : ℕ} (P : RealLaw d)
(lowerBand upperBand : ℝ) : Set (ℕ → Obs d) :=
finiteCategoryPilotGood
(iidSample_infinitePi P.observedLaw)
(fun o : Obs d => o.x) (range (n / 2))
(256 * logEN n) lowerBand upperBand
theorem polynomialPilotGood_measurable
The paper-local simultaneous pilot event is measurable.
Formal statement
hypotheses
conclusion
MeasurableSet (polynomialPilotGood (n := n) P lowerBand upperBand)
Proof (Lean source)
lemma polynomialPilotGood_measurable {n d : ℕ} (P : RealLaw d)
(lowerBand upperBand : ℝ) :
MeasurableSet (polynomialPilotGood (n := n) P lowerBand upperBand) := by
apply measurableSet_finiteCategoryPilotGood
exact measurable_fst.comp (measurable_iff_comap_le.mpr le_rfl)
theorem polynomialPilotGood_selectedHeavy_lower
If the pilot event is good and the stated condition on the cell holds, on the simultaneous pilot event, every cell selected by the concrete heavy threshold has population mass at least the declared lower band.
Formal statement
hypotheses
n d :
ℕ
P :
RealLaw d
lowerBand upperBand :
ℝ
omega :
ℕ → Obs d
hgood :
omega ∈ polynomialPilotGood (n := n) P lowerBand upperBand
k :
Fin d
hk :
conclusion
lowerBand ≤ P.cellMass k
Proof (Lean source)
lemma polynomialPilotGood_selectedHeavy_lower {n d : ℕ} (P : RealLaw d)
{lowerBand upperBand : ℝ} {omega : ℕ → Obs d}
(hgood : omega ∈ polynomialPilotGood (n := n) P lowerBand upperBand)
(k : Fin d)
(hk : 256 * logEN n < pilotCount (fun i : Fin n => omega i) k) :
lowerBand ≤ P.cellMass k := by
have hkCount :
256 * logEN n <
(pilotCategoryCount
(iidSample_infinitePi P.observedLaw)
(fun o : Obs d => o.x) (range (n / 2)) k omega : ℝ) := by
simpa [polynomialPilotCount_eq_finiteCategoryPilotCount P k omega] using hk
have hkSelected : k ∈ pilotSelected
(iidSample_infinitePi P.observedLaw)
(fun o : Obs d => o.x) (range (n / 2))
(256 * logEN n) omega := by
simpa [pilotSelected] using hkCount
have hmass := hgood.1 k hkSelected
have hset : (fun o : Obs d => o.x) ⁻¹' ({k} : Set (Fin d)) =
{o : Obs d | o.x = k} := by ext o; simp
unfold categoryMass at hmass
rw [hset] at hmass
rw [P.cellMass_eq k]
simpa [polynomialPilotGood, categoryMass, Measure.real,
realMass] using hmass
theorem polynomialPilotGood_selectedLight_upper
If the pilot event is good and the stated condition on the cell holds, on the simultaneous pilot event, every cell rejected by the concrete heavy threshold has population mass at most the declared upper band.
Formal statement
hypotheses
n d :
ℕ
P :
RealLaw d
lowerBand upperBand :
ℝ
omega :
ℕ → Obs d
hgood :
omega ∈ polynomialPilotGood (n := n) P lowerBand upperBand
k :
Fin d
hk :
conclusion
P.cellMass k ≤ upperBand
Proof (Lean source)
lemma polynomialPilotGood_selectedLight_upper {n d : ℕ} (P : RealLaw d)
{lowerBand upperBand : ℝ} {omega : ℕ → Obs d}
(hgood : omega ∈ polynomialPilotGood (n := n) P lowerBand upperBand)
(k : Fin d)
(hk : ¬ 256 * logEN n < pilotCount (fun i : Fin n => omega i) k) :
P.cellMass k ≤ upperBand := by
have hkCount : ¬
256 * logEN n <
(pilotCategoryCount
(iidSample_infinitePi P.observedLaw)
(fun o : Obs d => o.x) (range (n / 2)) k omega : ℝ) := by
simpa [polynomialPilotCount_eq_finiteCategoryPilotCount P k omega] using hk
have hkRejected : k ∉ pilotSelected
(iidSample_infinitePi P.observedLaw)
(fun o : Obs d => o.x) (range (n / 2))
(256 * logEN n) omega := by
simpa [pilotSelected] using hkCount
have hmass := hgood.2 k hkRejected
have hset : (fun o : Obs d => o.x) ⁻¹' ({k} : Set (Fin d)) =
{o : Obs d | o.x = k} := by ext o; simp
unfold categoryMass at hmass
rw [hset] at hmass
rw [P.cellMass_eq k]
simpa [polynomialPilotGood, categoryMass, Measure.real,
realMass] using hmass
theorem polynomialPilotGood_compl_probability_le
If the sample is nonempty, the generic logarithmic two-sided pilot bound specialized to the exact threshold and balanced first block used by rawPolyEstimator.
Formal statement
hypotheses
Proof (Lean source)
lemma polynomialPilotGood_compl_probability_le {n d : ℕ} (P : RealLaw d)
(hn : 0 < n) (lowerBand upperBand : ℝ) :
(Measure.infinitePi fun _ : ℕ => P.observedLaw).real
(polynomialPilotGood (n := n) P lowerBand upperBand)ᶜ ≤
(d : ℝ) *
(exp (-log 2 * (256 * logEN n) +
((range (n / 2)).card : ℝ) * lowerBand) +
exp (log 2 * (256 * logEN n) -
((range (n / 2)).card : ℝ) * upperBand / 2)) := by
unfold polynomialPilotGood
have hnR : 0 < (n : ℝ) := by exact_mod_cast hn
have hlog : 0 < logEN n := by
rw [logEN, Real.log_mul (Real.exp_ne_zero 1) hnR.ne', Real.log_exp]
have hn_one : 1 ≤ (n : ℝ) := by exact_mod_cast hn
nlinarith [Real.log_nonneg hn_one]
have ht : 0 < 256 * logEN n := by positivity
simpa using finiteCategoryPilot_bad_probability_log_two
(iidSample_infinitePi P.observedLaw)
(label := fun o : Obs d => o.x)
(measurable_fst.comp (measurable_iff_comap_le.mpr le_rfl))
(range (n / 2)) (t := 256 * logEN n) ht lowerBand upperBand
theorem polynomialPilotGood_compl_probability_calibrated_le
If the sample size satisfies the stated lower bound, at the declared pilot bands, the two Chernoff exponents both dominate 32 * log(en), giving a directly usable bad-selector probability bound.
Formal statement
hypotheses
conclusion
(Measure.infinitePi fun _ : ℕ => P.observedLaw).real (polynomialPilotGood (n := n) P (polynomialPilotLowerBand n) (polynomialPilotUpperBand n))ᶜ
Proof (Lean source)
lemma polynomialPilotGood_compl_probability_calibrated_le {n d : ℕ}
(P : RealLaw d) (hn : 2 ≤ n) :
(Measure.infinitePi fun _ : ℕ => P.observedLaw).real
(polynomialPilotGood (n := n) P
(polynomialPilotLowerBand n) (polynomialPilotUpperBand n))ᶜ ≤
2 * (d : ℝ) * exp (-32 * logEN n) := by
have hnpos : 0 < n := by omega
have hm0 : 0 < n / 2 := Nat.div_pos (by omega) (by norm_num)
have hm0R : 0 < ((n / 2 : ℕ) : ℝ) := by exact_mod_cast hm0
have hnR : 0 < (n : ℝ) := by exact_mod_cast hnpos
have hlog : 0 < logEN n := by
rw [logEN, Real.log_mul (Real.exp_ne_zero 1) hnR.ne', Real.log_exp]
have hone : 1 ≤ (n : ℝ) := by exact_mod_cast hnpos
nlinarith [Real.log_nonneg hone]
have hbase := polynomialPilotGood_compl_probability_le P hnpos
(polynomialPilotLowerBand n) (polynomialPilotUpperBand n)
apply hbase.trans
simp only [Finset.card_range]
unfold polynomialPilotLowerBand polynomialPilotUpperBand
have hcancelLower :
((n / 2 : ℕ) : ℝ) * (128 * logEN n / ((n / 2 : ℕ) : ℝ)) =
128 * logEN n := by field_simp
have hcancelUpper :
((n / 2 : ℕ) : ℝ) * (512 * logEN n / ((n / 2 : ℕ) : ℝ)) / 2 =
256 * logEN n := by field_simp; ring
rw [hcancelLower, hcancelUpper]
have hlogTwoLower : (5 / 8 : ℝ) ≤ log 2 := by
exact le_trans (by norm_num) Real.log_two_gt_d9.le
have hlogTwoUpper : log 2 ≤ (7 / 8 : ℝ) := by
exact Real.log_two_lt_d9.le.trans (by norm_num)
have hfirst :
-log 2 * (256 * logEN n) + 128 * logEN n ≤
-32 * logEN n := by nlinarith
have hsecond :
log 2 * (256 * logEN n) - 256 * logEN n ≤
-32 * logEN n := by nlinarith
have hexpFirst := Real.exp_le_exp.mpr hfirst
have hexpSecond := Real.exp_le_exp.mpr hsecond
have hd0 : 0 ≤ (d : ℝ) := by positivity
calc
(d : ℝ) *
(exp (-log 2 * (256 * logEN n) + 128 * logEN n) +
exp (log 2 * (256 * logEN n) - 256 * logEN n)) ≤
(d : ℝ) * (exp (-32 * logEN n) +
exp (-32 * logEN n)) := by gcongr
_ = 2 * (d : ℝ) * exp (-32 * logEN n) := by ring
Helpers.PolynomialUpper.RateAlgebra 2 declarations
theorem polynomial_linear_dimension_term_le_rate Lemma polynomial_linear_dimension_term_le_rate in the paper ↗
If the sample is nonempty and the logarithmic scale satisfies its stated bound and the stated growth condition holds, the first coefficient-growth term is absorbed by the parametric and quadratic-alphabet rates once its scalar logarithmic factor is at most the square-root sample scale.
Formal statement
hypotheses
A n d L :
ℝ
hn :
0 < n
hL :
0 < L
hgrowth :
A ^ 2 * L ^ 6 ≤ n
conclusion
A * d * L ^ 2 / n ^ 2 ≤ 1 / n + d ^ 2 / (n ^ 2 * L ^ 2)
Proof (Lean source)
lemma polynomial_linear_dimension_term_le_rate
{A n d L : ℝ} (hn : 0 < n) (hL : 0 < L)
(hgrowth : A ^ 2 * L ^ 6 ≤ n) :
A * d * L ^ 2 / n ^ 2 ≤
1 / n + d ^ 2 / (n ^ 2 * L ^ 2) := by
have hL0 : 0 ≤ L := hL.le
have hAL : (A * L ^ 4) ^ 2 ≤ n * L ^ 2 := by
calc
(A * L ^ 4) ^ 2 = (A ^ 2 * L ^ 6) * L ^ 2 := by ring
_ ≤ n * L ^ 2 := mul_le_mul_of_nonneg_right hgrowth (sq_nonneg L)
have hyoung : A * d * L ^ 4 ≤ n * L ^ 2 + d ^ 2 := by
have hsquare := sq_nonneg (A * L ^ 4 - d)
nlinarith [mul_nonneg hn.le (sq_nonneg L)]
have hden : 0 < n ^ 2 * L ^ 2 := mul_pos (sq_pos_of_pos hn) (sq_pos_of_pos hL)
calc
A * d * L ^ 2 / n ^ 2 = (A * d * L ^ 4) / (n ^ 2 * L ^ 2) := by
field_simp [hn.ne', hL.ne']
_ ≤ (n * L ^ 2 + d ^ 2) / (n ^ 2 * L ^ 2) :=
(div_le_div_iff_of_pos_right hden).2 hyoung
_ = 1 / n + d ^ 2 / (n ^ 2 * L ^ 2) := by
field_simp [hn.ne', hL.ne']
theorem polynomial_quadratic_dimension_term_le_rate Lemma polynomial_quadratic_dimension_term_le_rate in the paper ↗
If the sample is nonempty and the logarithmic scale satisfies its stated bound and the stated growth condition holds, the second coefficient-growth term is absorbed directly by the quadratic-alphabet rate under the corresponding logarithmic growth bound.
Formal statement
hypotheses
A n d L :
ℝ
hn :
0 < n
hL :
0 < L
hgrowth :
A * L ^ 6 ≤ n
conclusion
A * d ^ 2 * L ^ 4 / n ^ 3 ≤ d ^ 2 / (n ^ 2 * L ^ 2)
Proof (Lean source)
lemma polynomial_quadratic_dimension_term_le_rate
{A n d L : ℝ} (hn : 0 < n) (hL : 0 < L)
(hgrowth : A * L ^ 6 ≤ n) :
A * d ^ 2 * L ^ 4 / n ^ 3 ≤ d ^ 2 / (n ^ 2 * L ^ 2) := by
have hden : 0 < n ^ 3 * L ^ 2 := mul_pos (pow_pos hn 3) (sq_pos_of_pos hL)
have hd2 : 0 ≤ d ^ 2 := sq_nonneg d
have hnum : A * d ^ 2 * L ^ 6 ≤ n * d ^ 2 := by
simpa [mul_assoc, mul_left_comm, mul_comm] using
mul_le_mul_of_nonneg_right hgrowth hd2
calc
A * d ^ 2 * L ^ 4 / n ^ 3 =
(A * d ^ 2 * L ^ 6) / (n ^ 3 * L ^ 2) := by
field_simp [hn.ne', hL.ne']
_ ≤ (n * d ^ 2) / (n ^ 3 * L ^ 2) :=
(div_le_div_iff_of_pos_right hden).2 hnum
_ = d ^ 2 / (n ^ 2 * L ^ 2) := by
field_simp [hn.ne', hL.ne']
Helpers.PolynomialUpper.RateClosure 5 declarations
theorem polynomial_missing_term_le_component
If the sample size satisfies the stated lower bound and the overlap constant is positive, the missing-mass term is bounded by the polynomial rate component.
Formal statement
hypotheses
n d :
ℕ
epsilon :
ℝ
hn :
8 ≤ n
hepsilon :
0 < epsilon
conclusion
((d : ℝ) / (((((n - n / 2 - 2 : ℕ) : ℝ) / 2 * epsilon) ^ 2) * polynomialPilotLowerBand n)) ^ 2
≤ (1 / (16 * epsilon ^ 4)) * polynomialComponent n d
Proof (Lean source)
lemma polynomial_missing_term_le_component {n d : ℕ} {epsilon : ℝ}
(hn : 8 ≤ n) (hepsilon : 0 < epsilon) :
((d : ℝ) /
(((((n - n / 2 - 2 : ℕ) : ℝ) / 2 * epsilon) ^ 2) *
polynomialPilotLowerBand n)) ^ 2 ≤
(1 / (16 * epsilon ^ 4)) * polynomialComponent n d := by
have hnpos : 0 < n := by omega
have hnR : 0 < (n : ℝ) := by exact_mod_cast hnpos
have hL : 0 < logEN n := zero_lt_one.trans_le (one_le_logEN hnpos)
have hD := polynomial_pilot_denominator_ge (epsilon := epsilon) hn hepsilon
change 4 * (n : ℝ) * epsilon ^ 2 * logEN n ≤
(((((n - n / 2 - 2 : ℕ) : ℝ) / 2 * epsilon) ^ 2) *
polynomialPilotLowerBand n) at hD
have hDpos : 0 < (((((n - n / 2 - 2 : ℕ) : ℝ) / 2 * epsilon) ^ 2) *
polynomialPilotLowerBand n) := lt_of_lt_of_le (by positivity) hD
have hfrac : (d : ℝ) /
(((((n - n / 2 - 2 : ℕ) : ℝ) / 2 * epsilon) ^ 2) *
polynomialPilotLowerBand n) ≤
(d : ℝ) / (4 * (n : ℝ) * epsilon ^ 2 * logEN n) := by
exact div_le_div_of_nonneg_left (by positivity) (by positivity) hD
have hfrac0 : 0 ≤ (d : ℝ) /
(((((n - n / 2 - 2 : ℕ) : ℝ) / 2 * epsilon) ^ 2) *
polynomialPilotLowerBand n) := by positivity
have hsquare := sq_le_sq₀ hfrac0 (by positivity) |>.2 hfrac
apply hsquare.trans_eq
unfold polynomialComponent
field_simp [hnR.ne', hL.ne', hepsilon.ne']
ring
theorem polynomial_bias_term_le_component
If the sample is nonempty and the overlap constant is positive and the polynomial or elbow parameter satisfies its stated bound, the polynomial approximation bias term is bounded by the polynomial rate component.
Formal statement
hypotheses
n d :
ℕ
epsilon :
ℝ
hn :
0 < n
hepsilon :
0 < epsilon
hK :
2 ≤ polynomialDegree n
conclusion
((d : ℝ) * ((4096 * logEN n / (n - n / 2 : ℕ)) / (epsilon * (polynomialDegree n : ℝ) ^ 2))) ^ 2
≤ (32768 / (epsilon * polynomialAlpha0 ^ 2)) ^ 2 * polynomialComponent n d
Proof (Lean source)
lemma polynomial_bias_term_le_component {n d : ℕ} {epsilon : ℝ}
(hn : 0 < n) (hepsilon : 0 < epsilon)
(hK : 2 ≤ polynomialDegree n) :
((d : ℝ) * ((4096 * logEN n / (n - n / 2 : ℕ)) /
(epsilon * (polynomialDegree n : ℝ) ^ 2))) ^ 2 ≤
(32768 / (epsilon * polynomialAlpha0 ^ 2)) ^ 2 *
polynomialComponent n d := by
have hnR : 0 < (n : ℝ) := by exact_mod_cast hn
have hL : 0 < logEN n := zero_lt_one.trans_le (one_le_logEN hn)
have ha := polynomialAlpha0_pos
have hB := polynomial_lightScale_le hn
have hKlower := polynomialDegree_cast_lower hK
have hKsq : (polynomialAlpha0 * logEN n / 2) ^ 2 ≤
(polynomialDegree n : ℝ) ^ 2 := by
have hleft : 0 ≤ polynomialAlpha0 * logEN n / 2 := by positivity
exact (sq_le_sq₀ hleft (by positivity)).2 hKlower
have hden : 0 < epsilon * (polynomialDegree n : ℝ) ^ 2 := by
have : 0 < (polynomialDegree n : ℝ) := by exact_mod_cast (by omega : 0 < polynomialDegree n)
positivity
have hq : 4096 * logEN n / (n - n / 2 : ℕ) /
(epsilon * (polynomialDegree n : ℝ) ^ 2) ≤
32768 / (epsilon * polynomialAlpha0 ^ 2 * (n : ℝ) * logEN n) := by
apply (div_le_iff₀ hden).2
calc
4096 * logEN n / (n - n / 2 : ℕ) ≤
8192 * logEN n / (n : ℝ) := hB
_ = (32768 / (epsilon * polynomialAlpha0 ^ 2 * (n : ℝ) * logEN n)) *
(epsilon * (polynomialAlpha0 * logEN n / 2) ^ 2) := by
field_simp [hnR.ne', hL.ne', hepsilon.ne', ha.ne']
ring
_ ≤ (32768 / (epsilon * polynomialAlpha0 ^ 2 * (n : ℝ) * logEN n)) *
(epsilon * (polynomialDegree n : ℝ) ^ 2) := by
gcongr
have hmul := mul_le_mul_of_nonneg_left hq (by positivity : (0 : ℝ) ≤ d)
have hsquare := (sq_le_sq₀ (by positivity) (by positivity)).2 hmul
apply hsquare.trans_eq
unfold polynomialComponent
field_simp [hnR.ne', hL.ne', hepsilon.ne', ha.ne']
theorem polynomial_linear_light_term_le_rate
If the sample is nonempty and the stated llarge condition holds, the linear light-cell variance term is bounded by the target polynomial rate.
Formal statement
hypotheses
conclusion
(6 : ℝ) ^ (2 * polynomialDegree n) * (d : ℝ) * (4096 * logEN n / (n - n / 2 : ℕ)) ^ 2
≤ 8192 ^ 2 * (1 / (n : ℝ) + polynomialComponent n d)
Proof (Lean source)
lemma polynomial_linear_light_term_le_rate {n d : ℕ} (hn : 0 < n)
(hLlarge : 240 ≤ logEN n) :
(6 : ℝ) ^ (2 * polynomialDegree n) * (d : ℝ) *
(4096 * logEN n / (n - n / 2 : ℕ)) ^ 2 ≤
8192 ^ 2 * (1 / (n : ℝ) + polynomialComponent n d) := by
have hnR : 0 < (n : ℝ) := by exact_mod_cast hn
have hL : 0 < logEN n := by linarith
have hB := polynomial_lightScale_le hn
have hB0 : 0 ≤ 4096 * logEN n / (n - n / 2 : ℕ) := by positivity
have hBsq := (sq_le_sq₀ hB0 (by positivity)).2 hB
have hgrowth : ((6 : ℝ) ^ (2 * polynomialDegree n)) ^ 2 *
logEN n ^ 6 ≤ n := by
rw [show ((6 : ℝ) ^ (2 * polynomialDegree n)) ^ 2 =
(6 : ℝ) ^ (4 * polynomialDegree n) by
calc
((6 : ℝ) ^ (2 * polynomialDegree n)) ^ 2 =
(6 : ℝ) ^ ((2 * polynomialDegree n) * 2) :=
(pow_mul _ _ _).symm
_ = (6 : ℝ) ^ (4 * polynomialDegree n) := by
congr 1
omega]
exact polynomial_log_growth_four hn hLlarge
have hcore := polynomial_linear_dimension_term_le_rate
(A := (6 : ℝ) ^ (2 * polynomialDegree n))
(n := (n : ℝ)) (d := (d : ℝ)) (L := logEN n) hnR hL hgrowth
calc
(6 : ℝ) ^ (2 * polynomialDegree n) * (d : ℝ) *
(4096 * logEN n / (n - n / 2 : ℕ)) ^ 2 ≤
(6 : ℝ) ^ (2 * polynomialDegree n) * (d : ℝ) *
(8192 * logEN n / (n : ℝ)) ^ 2 := by gcongr
_ = 8192 ^ 2 * ((6 : ℝ) ^ (2 * polynomialDegree n) *
(d : ℝ) * logEN n ^ 2 / (n : ℝ) ^ 2) := by
field_simp [hnR.ne']
_ ≤ 8192 ^ 2 * (1 / (n : ℝ) +
(d : ℝ) ^ 2 / ((n : ℝ) ^ 2 * logEN n ^ 2)) := by gcongr
_ = 8192 ^ 2 * (1 / (n : ℝ) + polynomialComponent n d) := by
rfl
theorem polynomial_quadratic_light_term_le_component
If the sample is nonempty and the stated llarge condition holds, the quadratic light-cell variance term is bounded by the polynomial rate component.
Formal statement
hypotheses
conclusion
(6 : ℝ) ^ (2 * polynomialDegree n) * (d : ℝ) ^ 2 * polynomialDegree n ^ 2 * (4096 * logEN n / (n - n / 2 : ℕ)) ^ 2 / (n - n / 2 : ℕ)
≤ (2 * 8192 ^ 2) * polynomialComponent n d
Proof (Lean source)
lemma polynomial_quadratic_light_term_le_component {n d : ℕ} (hn : 0 < n)
(hLlarge : 240 ≤ logEN n) :
(6 : ℝ) ^ (2 * polynomialDegree n) * (d : ℝ) ^ 2 *
polynomialDegree n ^ 2 *
(4096 * logEN n / (n - n / 2 : ℕ)) ^ 2 /
(n - n / 2 : ℕ) ≤
(2 * 8192 ^ 2) * polynomialComponent n d := by
have hnR : 0 < (n : ℝ) := by exact_mod_cast hn
have hL : 0 < logEN n := by linarith
have hmR : 0 < ((n - n / 2 : ℕ) : ℝ) := by
exact_mod_cast polynomial_estimationBlock_pos hn
have hK := polynomialDegree_cast_le_logEN hn
have hKsq : (polynomialDegree n : ℝ) ^ 2 ≤ logEN n ^ 2 := by
exact (sq_le_sq₀ (by positivity) hL.le).2 hK
have hB := polynomial_lightScale_le hn
have hBsq := (sq_le_sq₀ (by positivity) (by positivity)).2 hB
have hinv : (1 : ℝ) / (n - n / 2 : ℕ) ≤ 2 / (n : ℝ) := by
apply (div_le_div_iff₀ hmR hnR).2
have htwice : n ≤ 2 * (n - n / 2) := by omega
have htwiceR : (n : ℝ) ≤ 2 * ((n - n / 2 : ℕ) : ℝ) := by
exact_mod_cast htwice
nlinarith
have hgrowth := polynomial_log_growth_two hn hLlarge
have hcore := polynomial_quadratic_dimension_term_le_rate
(A := (6 : ℝ) ^ (2 * polynomialDegree n))
(n := (n : ℝ)) (d := (d : ℝ)) (L := logEN n) hnR hL hgrowth
calc
(6 : ℝ) ^ (2 * polynomialDegree n) * (d : ℝ) ^ 2 *
polynomialDegree n ^ 2 *
(4096 * logEN n / (n - n / 2 : ℕ)) ^ 2 /
(n - n / 2 : ℕ) ≤
(6 : ℝ) ^ (2 * polynomialDegree n) * (d : ℝ) ^ 2 *
logEN n ^ 2 * (8192 * logEN n / (n : ℝ)) ^ 2 *
(2 / (n : ℝ)) := by
rw [div_eq_mul_inv]
gcongr
simpa [one_div] using hinv
_ = (2 * 8192 ^ 2) *
((6 : ℝ) ^ (2 * polynomialDegree n) * (d : ℝ) ^ 2 *
logEN n ^ 4 / (n : ℝ) ^ 3) := by
field_simp [hnR.ne']
_ ≤ (2 * 8192 ^ 2) *
((d : ℝ) ^ 2 / ((n : ℝ) ^ 2 * logEN n ^ 2)) := by gcongr
_ = (2 * 8192 ^ 2) * polynomialComponent n d := by rfl
theorem polynomial_uniformBranch_expression_le_rate
If the sample size satisfies the stated lower bound and the overlap constant is positive and the stated c condition holds and the polynomial or elbow parameter satisfies its stated bound and the stated llarge condition holds, the complete uniform branchwise error expression is bounded by the target polynomial rate.
Formal statement
hypotheses
n d :
ℕ
epsilon C :
ℝ
hn :
8 ≤ n
hepsilon :
0 < epsilon
hC :
0 < C
hK :
2 ≤ polynomialDegree n
hLlarge :
240 ≤ logEN n
conclusion
8 * (8 / (((n - n / 2 : ℕ) : ℝ) * epsilon) + 6 / (n - n / 2 : ℕ) + 4 * ((d : ℝ) / (((((n - n / 2 - 2 : ℕ) : ℝ) / 2 * epsilon) ^ 2) * polynomialPilotLowerBand n)) ^ 2)
+ 2 * (C / (n - n / 2 : ℕ) + C * 6 ^ (2 * polynomialDegree n) * ((d : ℝ) * (4096 * logEN n / (n - n / 2 : ℕ)) ^ 2 + (d : ℝ) ^ 2 * polynomialDegree n ^ 2 * (4096 * logEN n / (n - n / 2 : ℕ)) ^ 2 / (n - n / 2 : ℕ)) + ((d : ℝ) * ((4096 * logEN n / (n - n / 2 : ℕ)) / (epsilon * (polynomialDegree n : ℝ) ^ 2))) ^ 2)
≤ (128 / epsilon + 96 + 2 / epsilon ^ 4 + 4 * C + 6 * C * 8192 ^ 2 + 2 * (32768 / (epsilon * polynomialAlpha0 ^ 2)) ^ 2) * (1 / (n : ℝ) + polynomialComponent n d)
Proof (Lean source)
lemma polynomial_uniformBranch_expression_le_rate {n d : ℕ} {epsilon C : ℝ}
(hn : 8 ≤ n) (hepsilon : 0 < epsilon) (hC : 0 < C)
(hK : 2 ≤ polynomialDegree n) (hLlarge : 240 ≤ logEN n) :
8 * (8 / (((n - n / 2 : ℕ) : ℝ) * epsilon) +
6 / (n - n / 2 : ℕ) +
4 * ((d : ℝ) /
(((((n - n / 2 - 2 : ℕ) : ℝ) / 2 * epsilon) ^ 2) *
polynomialPilotLowerBand n)) ^ 2) +
2 * (C / (n - n / 2 : ℕ) +
C * 6 ^ (2 * polynomialDegree n) *
((d : ℝ) * (4096 * logEN n / (n - n / 2 : ℕ)) ^ 2 +
(d : ℝ) ^ 2 * polynomialDegree n ^ 2 *
(4096 * logEN n / (n - n / 2 : ℕ)) ^ 2 /
(n - n / 2 : ℕ)) +
((d : ℝ) * ((4096 * logEN n / (n - n / 2 : ℕ)) /
(epsilon * (polynomialDegree n : ℝ) ^ 2))) ^ 2) ≤
(128 / epsilon + 96 + 2 / epsilon ^ 4 + 4 * C +
6 * C * 8192 ^ 2 +
2 * (32768 / (epsilon * polynomialAlpha0 ^ 2)) ^ 2) *
(1 / (n : ℝ) + polynomialComponent n d) := by
have hnpos : 0 < n := by omega
have hnR : 0 < (n : ℝ) := by exact_mod_cast hnpos
have hmR : 0 < ((n - n / 2 : ℕ) : ℝ) := by
exact_mod_cast polynomial_estimationBlock_pos hnpos
have htwice : n ≤ 2 * (n - n / 2) := by omega
have htwiceR : (n : ℝ) ≤ 2 * ((n - n / 2 : ℕ) : ℝ) := by
exact_mod_cast htwice
have hinv : (1 : ℝ) / (n - n / 2 : ℕ) ≤ 2 / (n : ℝ) := by
apply (div_le_div_iff₀ hmR hnR).2
nlinarith
have hrate0 : 0 ≤ 1 / (n : ℝ) + polynomialComponent n d := by
unfold polynomialComponent
positivity
have hinvRate : (1 : ℝ) / (n - n / 2 : ℕ) ≤
2 * (1 / (n : ℝ) + polynomialComponent n d) := by
have hc : 0 ≤ polynomialComponent n d := by
unfold polynomialComponent
positivity
have hb : 1 / (n : ℝ) ≤
1 / (n : ℝ) + polynomialComponent n d :=
le_add_of_nonneg_right hc
have hs : (2 : ℝ) * (1 / (n : ℝ)) ≤
2 * (1 / (n : ℝ) + polynomialComponent n d) :=
mul_le_mul_of_nonneg_left hb (by norm_num)
calc
(1 : ℝ) / (n - n / 2 : ℕ) ≤ 2 / (n : ℝ) := hinv
_ = 2 * (1 / (n : ℝ)) := by ring
_ ≤ 2 * (1 / (n : ℝ) + polynomialComponent n d) := hs
have hinvEps : 1 / (((n - n / 2 : ℕ) : ℝ) * epsilon) ≤
(2 / epsilon) * (1 / (n : ℝ) + polynomialComponent n d) := by
have he : 0 < ((n - n / 2 : ℕ) : ℝ) * epsilon := mul_pos hmR hepsilon
have heq : 1 / (((n - n / 2 : ℕ) : ℝ) * epsilon) =
(1 / epsilon) * (1 / ((n - n / 2 : ℕ) : ℝ)) := by
field_simp [hmR.ne', hepsilon.ne']
rw [heq]
have hmul : (1 / epsilon) * (1 / ((n - n / 2 : ℕ) : ℝ)) ≤
(1 / epsilon) *
(2 * (1 / (n : ℝ) + polynomialComponent n d)) :=
mul_le_mul_of_nonneg_left hinvRate (by positivity)
calc
(1 / epsilon) * (1 / ((n - n / 2 : ℕ) : ℝ)) ≤
(1 / epsilon) *
(2 * (1 / (n : ℝ) + polynomialComponent n d)) :=
hmul
_ = (2 / epsilon) *
(1 / (n : ℝ) + polynomialComponent n d) := by ring
have hmissing := polynomial_missing_term_le_component
(n := n) (d := d) hn hepsilon
have hlinear := polynomial_linear_light_term_le_rate
(n := n) (d := d) hnpos hLlarge
have hquad := polynomial_quadratic_light_term_le_component
(n := n) (d := d) hnpos hLlarge
have hbias := polynomial_bias_term_le_component
(n := n) (d := d) hnpos hepsilon hK
have hparam1 : 8 * (8 / (((n - n / 2 : ℕ) : ℝ) * epsilon)) ≤
(128 / epsilon) * (1 / (n : ℝ) + polynomialComponent n d) := by
calc
8 * (8 / (((n - n / 2 : ℕ) : ℝ) * epsilon)) =
64 * (1 / (((n - n / 2 : ℕ) : ℝ) * epsilon)) := by ring
_ ≤ 64 * ((2 / epsilon) *
(1 / (n : ℝ) + polynomialComponent n d)) :=
mul_le_mul_of_nonneg_left hinvEps (by norm_num)
_ = _ := by ring
have hparam2 : 8 * (6 / ((n - n / 2 : ℕ) : ℝ)) ≤
96 * (1 / (n : ℝ) + polynomialComponent n d) := by
calc
8 * (6 / ((n - n / 2 : ℕ) : ℝ)) =
48 * (1 / ((n - n / 2 : ℕ) : ℝ)) := by ring
_ ≤ 48 * (2 * (1 / (n : ℝ) + polynomialComponent n d)) :=
mul_le_mul_of_nonneg_left hinvRate (by norm_num)
_ = _ := by ring
have hmissing' : 8 * 4 * ((d : ℝ) /
(((((n - n / 2 - 2 : ℕ) : ℝ) / 2 * epsilon) ^ 2) *
polynomialPilotLowerBand n)) ^ 2 ≤
(2 / epsilon ^ 4) * (1 / (n : ℝ) + polynomialComponent n d) := by
have he4 : 0 < epsilon ^ 4 := pow_pos hepsilon 4
have hc : 0 ≤ polynomialComponent n d := by
unfold polynomialComponent
positivity
calc
8 * 4 * ((d : ℝ) /
(((((n - n / 2 - 2 : ℕ) : ℝ) / 2 * epsilon) ^ 2) *
polynomialPilotLowerBand n)) ^ 2 =
32 * ((d : ℝ) /
(((((n - n / 2 - 2 : ℕ) : ℝ) / 2 * epsilon) ^ 2) *
polynomialPilotLowerBand n)) ^ 2 := by ring
_ ≤ 32 * ((1 / (16 * epsilon ^ 4)) * polynomialComponent n d) :=
mul_le_mul_of_nonneg_left hmissing (by norm_num)
_ = (2 / epsilon ^ 4) * polynomialComponent n d := by
field_simp [he4.ne']
ring
_ ≤ (2 / epsilon ^ 4) *
(1 / (n : ℝ) + polynomialComponent n d) := by
exact mul_le_mul_of_nonneg_left
(le_add_of_nonneg_left (by positivity : 0 ≤ 1 / (n : ℝ)))
(by positivity)
have hCparam : 2 * (C / ((n - n / 2 : ℕ) : ℝ)) ≤
(4 * C) * (1 / (n : ℝ) + polynomialComponent n d) := by
have := mul_le_mul_of_nonneg_left hinvRate hC.le
rw [show C / ((n - n / 2 : ℕ) : ℝ) =
C * (1 / ((n - n / 2 : ℕ) : ℝ)) by ring]
nlinarith
have hlinear' : 2 * C *
((6 : ℝ) ^ (2 * polynomialDegree n) * (d : ℝ) *
(4096 * logEN n / (n - n / 2 : ℕ)) ^ 2) ≤
(2 * C * 8192 ^ 2) *
(1 / (n : ℝ) + polynomialComponent n d) := by
calc
_ ≤ (2 * C) * (8192 ^ 2 *
(1 / (n : ℝ) + polynomialComponent n d)) :=
mul_le_mul_of_nonneg_left hlinear (mul_nonneg (by norm_num) hC.le)
_ = _ := by ring
have hquad' : 2 * C *
((6 : ℝ) ^ (2 * polynomialDegree n) * (d : ℝ) ^ 2 *
polynomialDegree n ^ 2 *
(4096 * logEN n / (n - n / 2 : ℕ)) ^ 2 /
(n - n / 2 : ℕ)) ≤
(4 * C * 8192 ^ 2) *
(1 / (n : ℝ) + polynomialComponent n d) := by
calc
_ ≤ (2 * C) * ((2 * 8192 ^ 2) * polynomialComponent n d) :=
mul_le_mul_of_nonneg_left hquad (mul_nonneg (by norm_num) hC.le)
_ ≤ (4 * C * 8192 ^ 2) *
(1 / (n : ℝ) + polynomialComponent n d) := by
have hp : 0 ≤ 1 / (n : ℝ) := by positivity
nlinarith
have hbias' : 2 * ((d : ℝ) *
((4096 * logEN n / (n - n / 2 : ℕ)) /
(epsilon * (polynomialDegree n : ℝ) ^ 2))) ^ 2 ≤
(2 * (32768 / (epsilon * polynomialAlpha0 ^ 2)) ^ 2) *
(1 / (n : ℝ) + polynomialComponent n d) := by
have hc := mul_le_mul_of_nonneg_left hbias (by norm_num : (0 : ℝ) ≤ 2)
have hp : 0 ≤ 1 / (n : ℝ) := by positivity
nlinarith
calc
8 * (8 / (((n - n / 2 : ℕ) : ℝ) * epsilon) +
6 / (n - n / 2 : ℕ) +
4 * ((d : ℝ) /
(((((n - n / 2 - 2 : ℕ) : ℝ) / 2 * epsilon) ^ 2) *
polynomialPilotLowerBand n)) ^ 2) +
2 * (C / (n - n / 2 : ℕ) +
C * 6 ^ (2 * polynomialDegree n) *
((d : ℝ) * (4096 * logEN n / (n - n / 2 : ℕ)) ^ 2 +
(d : ℝ) ^ 2 * polynomialDegree n ^ 2 *
(4096 * logEN n / (n - n / 2 : ℕ)) ^ 2 /
(n - n / 2 : ℕ)) +
((d : ℝ) * ((4096 * logEN n / (n - n / 2 : ℕ)) /
(epsilon * (polynomialDegree n : ℝ) ^ 2))) ^ 2) =
8 * (8 / (((n - n / 2 : ℕ) : ℝ) * epsilon)) +
8 * (6 / ((n - n / 2 : ℕ) : ℝ)) +
8 * 4 * ((d : ℝ) /
(((((n - n / 2 - 2 : ℕ) : ℝ) / 2 * epsilon) ^ 2) *
polynomialPilotLowerBand n)) ^ 2 +
2 * (C / ((n - n / 2 : ℕ) : ℝ)) +
2 * C * ((6 : ℝ) ^ (2 * polynomialDegree n) * (d : ℝ) *
(4096 * logEN n / (n - n / 2 : ℕ)) ^ 2) +
2 * C * ((6 : ℝ) ^ (2 * polynomialDegree n) * (d : ℝ) ^ 2 *
polynomialDegree n ^ 2 *
(4096 * logEN n / (n - n / 2 : ℕ)) ^ 2 /
(n - n / 2 : ℕ)) +
2 * ((d : ℝ) * ((4096 * logEN n / (n - n / 2 : ℕ)) /
(epsilon * (polynomialDegree n : ℝ) ^ 2))) ^ 2 := by ring
_ ≤ (128 / epsilon) * (1 / (n : ℝ) + polynomialComponent n d) +
96 * (1 / (n : ℝ) + polynomialComponent n d) +
(2 / epsilon ^ 4) * (1 / (n : ℝ) + polynomialComponent n d) +
(4 * C) * (1 / (n : ℝ) + polynomialComponent n d) +
(2 * C * 8192 ^ 2) * (1 / (n : ℝ) + polynomialComponent n d) +
(4 * C * 8192 ^ 2) * (1 / (n : ℝ) + polynomialComponent n d) +
(2 * (32768 / (epsilon * polynomialAlpha0 ^ 2)) ^ 2) *
(1 / (n : ℝ) + polynomialComponent n d) := by
exact add_le_add (add_le_add (add_le_add (add_le_add
(add_le_add (add_le_add hparam1 hparam2) hmissing') hCparam)
hlinear') hquad') hbias'
_ = _ := by ring
Helpers.PolynomialUpper.SelectionDecomposition 4 declarations The pilot selector partitions the normalized pre-clipping error into a heavy marked-ratio error and a light polynomial error.
The pilot selector partitions the normalized pre-clipping error into a heavy marked-ratio error and a light polynomial error. Keeping this identity separate lets the two fixed-set moment bounds be assembled independently.
def polynomialHeavySelectedError
The normalized estimation error contributed by cells selected as heavy.
Definition (Lean source)
noncomputable def polynomialHeavySelectedError {n d : ℕ} (P : RealLaw d)
(M : ℝ) (sample : Fin n → Obs d) : ℝ :=
∑ k : Fin d,
if 256 * logEN n < pilotCount sample k then
heavyEmpiricalTerm M sample k -
P.cellMass k * cellEffect P k / M
else 0
def polynomialLightSelectedError
The normalized estimation error contributed by cells selected as light.
Definition (Lean source)
noncomputable def polynomialLightSelectedError {n d : ℕ} (P : RealLaw d)
(M : ℝ) (sample : Fin n → Obs d) : ℝ :=
let K := polynomialDegree n
let B := 4096 * logEN n / (n - n / 2 : ℕ)
∑ k : Fin d,
if 256 * logEN n < pilotCount sample k then 0
else lightPolynomialTerm M B K sample k -
P.cellMass k * cellEffect P k / M
theorem polynomialNormalizedSum_sub_target_eq_selectedErrors
The pre-clipping normalized error is exactly the sum of the selected heavy and selected light errors; no probability or moment assumption enters this partition identity.
Formal statement
hypotheses
conclusion
polynomialNormalizedSum M sample - rawAteFormula P / M
= polynomialHeavySelectedError P M sample + polynomialLightSelectedError P M sample
Proof (Lean source)
lemma polynomialNormalizedSum_sub_target_eq_selectedErrors {n d : ℕ}
(P : RealLaw d) (M : ℝ) (sample : Fin n → Obs d) :
polynomialNormalizedSum M sample - rawAteFormula P / M =
polynomialHeavySelectedError P M sample +
polynomialLightSelectedError P M sample := by
unfold polynomialNormalizedSum polynomialHeavySelectedError
polynomialLightSelectedError rawAteFormula
rw [Finset.sum_div, ← Finset.sum_sub_distrib, ← Finset.sum_add_distrib]
apply Finset.sum_congr rfl
intro k _
by_cases hk : 256 * logEN n < pilotCount sample k <;> simp [hk]
theorem polynomialNormalizedSum_error_sq_le_selectedErrors
The exact selector partition yields the standard two-term squared-error bound used to combine the heavy and light risk estimates.
Formal statement
hypotheses
conclusion
(polynomialNormalizedSum M sample - rawAteFormula P / M) ^ 2
≤ 2 * polynomialHeavySelectedError P M sample ^ 2
+ 2 * polynomialLightSelectedError P M sample ^ 2
Proof (Lean source)
lemma polynomialNormalizedSum_error_sq_le_selectedErrors {n d : ℕ}
(P : RealLaw d) (M : ℝ) (sample : Fin n → Obs d) :
(polynomialNormalizedSum M sample - rawAteFormula P / M) ^ 2 ≤
2 * polynomialHeavySelectedError P M sample ^ 2 +
2 * polynomialLightSelectedError P M sample ^ 2 := by
rw [polynomialNormalizedSum_sub_target_eq_selectedErrors]
nlinarith [sq_nonneg
(polynomialHeavySelectedError P M sample -
polynomialLightSelectedError P M sample)]
Helpers.PolynomialUpper.SelectorBridge 3 declarations
theorem polynomialFixedHeavyError_eq_fixedBranchHeavy
If the selected heavy set is fixed, the selector's fixed heavy error equals the heavy component of the corresponding fixed branch.
Formal statement
hypotheses
n d :
ℕ
epsilon M sigma :
ℝ
P :
ModelClass d epsilon M sigma
base :
Obs d
x :
(polynomialBalancedSplit P.law).foldB n → Obs d
hH :
∀ k ∈ H, 0 < P.law.cellMass k
conclusion
polynomialFixedHeavyError P.law base M H x
= (fixedStratumMarkedRatio (fun o : Obs d => o.x) (fun o => o.a) (fun o => o.y) H (polynomialFoldBReindex P.law x) - fixedStratumMarkedTarget P.law.observedLaw (fun o : Obs d => o.x) (fun o => o.a) (fun o => o.y) H) / M
Proof (Lean source)
lemma polynomialFixedHeavyError_eq_fixedBranchHeavy {n d : ℕ}
{epsilon M sigma : ℝ} (P : ModelClass d epsilon M sigma)
(base : Obs d) (H : Finset (Fin d))
(x : (polynomialBalancedSplit P.law).foldB n → Obs d)
(hH : ∀ k ∈ H, 0 < P.law.cellMass k) :
polynomialFixedHeavyError P.law base M H x =
(fixedStratumMarkedRatio (fun o : Obs d => o.x) (fun o => o.a)
(fun o => o.y) H (polynomialFoldBReindex P.law x) -
fixedStratumMarkedTarget P.law.observedLaw (fun o : Obs d => o.x)
(fun o => o.a) (fun o => o.y) H) / M := by
classical
rw [polynomialFixedStratumMarkedTarget_eq_cellEffectSum P H hH]
unfold polynomialFixedHeavyError fixedStratumMarkedRatio fixedStratumArmScore
simp_rw [heavyEmpiricalTerm_rebuild_eq_reindex P.law base x]
simp only [sub_div, mul_sub]
rw [Finset.sum_sub_distrib]
simp_rw [← mul_div_assoc]
rw [Finset.sum_sub_distrib]
rw [← Finset.sum_div, ← Finset.sum_div]
simp_rw [Finset.sum_div]
theorem polynomialFixedLightError_eq_fixedBranchLight
the selector's fixed light error equals the light component of the corresponding fixed branch.
Formal statement
hypotheses
conclusion
polynomialFixedLightError P base M H x
= allBlockMarkedPolynomialSum M (4096 * logEN n / (n - n / 2 : ℕ)) (polynomialDegree n) (univ \ H) (polynomialFoldBReindex P x)
- ∑ k ∈ univ \ H, P.cellMass k * cellEffect P k / M
Proof (Lean source)
lemma polynomialFixedLightError_eq_fixedBranchLight {n d : ℕ}
(P : RealLaw d) (base : Obs d) (M : ℝ) (H : Finset (Fin d))
(x : (polynomialBalancedSplit P).foldB n → Obs d) :
polynomialFixedLightError P base M H x =
allBlockMarkedPolynomialSum M
(4096 * logEN n / (n - n / 2 : ℕ)) (polynomialDegree n)
(univ \ H) (polynomialFoldBReindex P x) -
∑ k ∈ univ \ H, P.cellMass k * cellEffect P k / M := by
classical
unfold polynomialFixedLightError
dsimp only
calc
(∑ k : Fin d, if k ∈ H then 0 else
lightPolynomialTerm M (4096 * logEN n / (n - n / 2 : ℕ))
(polynomialDegree n) (rebuildPolynomialEstimationSample P base x) k -
P.cellMass k * cellEffect P k / M) =
∑ k ∈ univ \ H,
(lightPolynomialTerm M (4096 * logEN n / (n - n / 2 : ℕ))
(polynomialDegree n) (rebuildPolynomialEstimationSample P base x) k -
P.cellMass k * cellEffect P k / M) := by
rw [Finset.sdiff_eq_filter, Finset.sum_filter]
apply Finset.sum_congr rfl
intro k _
by_cases hk : k ∈ H <;> simp [hk]
_ = (∑ k ∈ univ \ H,
lightPolynomialTerm M (4096 * logEN n / (n - n / 2 : ℕ))
(polynomialDegree n) (rebuildPolynomialEstimationSample P base x) k) -
∑ k ∈ univ \ H, P.cellMass k * cellEffect P k / M := by
rw [Finset.sum_sub_distrib]
_ = _ := by rw [lightPolynomialSum_rebuild_eq_allBlock]
theorem polynomialFixedTotalError_eq_fixedBranch
If the selected heavy set is fixed, the selector's fixed total error equals the error of the corresponding fixed branch.
Formal statement
hypotheses
n d :
ℕ
epsilon M sigma :
ℝ
P :
ModelClass d epsilon M sigma
base :
Obs d
x :
(polynomialBalancedSplit P.law).foldB n → Obs d
hH :
∀ k ∈ H, 0 < P.law.cellMass k
conclusion
polynomialFixedTotalError P.law base M H x
= polynomialFixedBranchNormalizedError P.law M (K := polynomialDegree n) (4096 * logEN n / (n - n / 2 : ℕ)) H (polynomialFoldBReindex P.law x)
Proof (Lean source)
lemma polynomialFixedTotalError_eq_fixedBranch {n d : ℕ}
{epsilon M sigma : ℝ} (P : ModelClass d epsilon M sigma)
(base : Obs d) (H : Finset (Fin d))
(x : (polynomialBalancedSplit P.law).foldB n → Obs d)
(hH : ∀ k ∈ H, 0 < P.law.cellMass k) :
polynomialFixedTotalError P.law base M H x =
polynomialFixedBranchNormalizedError P.law M
(K := polynomialDegree n)
(4096 * logEN n / (n - n / 2 : ℕ)) H
(polynomialFoldBReindex P.law x) := by
rw [polynomialFixedTotalError,
polynomialFixedHeavyError_eq_fixedBranchHeavy P base H x hH,
polynomialFixedLightError_eq_fixedBranchLight]
rfl
Helpers.PolynomialUpper.SelectorRisk 5 declarations
theorem clippedNormalizedError_sq_le_four
If the target lies in the clipping interval, the squared clipped normalized error is at most four.
Formal statement
Proof (Lean source)
lemma clippedNormalizedError_sq_le_four (x t : ℝ) (ht : t ∈ Icc (-1 : ℝ) 1) :
(clip (-1) 1 x - t) ^ 2 ≤ 4 := by
have hc : clip (-1) 1 x ∈ Icc (-1 : ℝ) 1 := by
unfold clip
constructor
· exact le_max_left _ _
· exact max_le (by norm_num) (min_le_left _ _)
have hprod : 0 ≤
(clip (-1) 1 x - t + 2) * (2 - (clip (-1) 1 x - t)) := by
exact mul_nonneg (by linarith [hc.1, ht.2]) (by linarith [hc.2, ht.1])
nlinarith
theorem polynomialClippedBranchError_measurable
the clipped normalized error for a selected polynomial branch is measurable.
Formal statement
hypotheses
conclusion
Measurable (fun x : (polynomialBalancedSplit P).foldB n → Obs d => clip (-1) 1 (t + polynomialFixedTotalError P base M H x) - t)
Proof (Lean source)
lemma polynomialClippedBranchError_measurable {n d : ℕ} (P : RealLaw d)
(base : Obs d) (M : ℝ) (H : Finset (Fin d)) (t : ℝ) :
Measurable (fun x : (polynomialBalancedSplit P).foldB n → Obs d =>
clip (-1) 1 (t + polynomialFixedTotalError P base M H x) - t) := by
have hraw : Measurable (fun x : (polynomialBalancedSplit P).foldB n → Obs d =>
t + polynomialFixedTotalError P base M H x) :=
measurable_const.add
(polynomialFixedTotalError_measurable (n := n) P base M H)
exact (measurable_const.max (measurable_const.min hraw)).sub measurable_const
theorem polynomialPilotFoldGood_compl_probability_le
If the bad-event contribution is bounded as stated, the probability that a pilot fold is bad satisfies the stated exponential bound.
Formal statement
hypotheses
n d :
ℕ
P :
RealLaw d
base :
Obs d
lowerBand upperBand delta :
ℝ
hbad :
(Measure.infinitePi fun _ : ℕ => P.observedLaw).real (polynomialPilotGood (n := n) P lowerBand upperBand)ᶜ
≤ delta
conclusion
(Measure.infinitePi fun _ : ℕ => P.observedLaw).real ((fun omega : ℕ → Obs d => fun i : (polynomialBalancedSplit P).foldA n => omega i) ⁻¹' (polynomialPilotFoldGood P base lowerBand upperBand)ᶜ)
≤ delta
Proof (Lean source)
lemma polynomialPilotFoldGood_compl_probability_le {n d : ℕ}
(P : RealLaw d) (base : Obs d) (lowerBand upperBand delta : ℝ)
(hbad : (Measure.infinitePi fun _ : ℕ => P.observedLaw).real
(polynomialPilotGood (n := n) P lowerBand upperBand)ᶜ ≤ delta) :
(Measure.infinitePi fun _ : ℕ => P.observedLaw).real
((fun omega : ℕ → Obs d =>
fun i : (polynomialBalancedSplit P).foldA n => omega i) ⁻¹'
(polynomialPilotFoldGood P base lowerBand upperBand)ᶜ) ≤ delta := by
apply (measureReal_mono (μ := Measure.infinitePi fun _ : ℕ => P.observedLaw)
(s₂ := (polynomialPilotGood (n := n) P lowerBand upperBand)ᶜ) ?_).trans hbad
intro omega homega hgood
exact homega (polynomialPilotGood_implies_foldGood P base lowerBand upperBand
omega hgood)
theorem polynomial_clipped_finiteSelector_risk
If the stated lower bound holds and the branchwise risk is bounded as stated and the bad-event probability is bounded as stated and each fixed branch has the stated risk bound and the bad-event contribution is bounded as stated, a uniform deterministic fixed-branch bound transfers through the measurable pilot selector. Clipping supplies the global envelope on the bad pilot event.
Formal statement
hypotheses
n d :
ℕ
epsilon M sigma lowerBand upperBand R delta :
ℝ
P :
ModelClass d epsilon M sigma
base :
Obs d
hlower :
0 < lowerBand
hR :
0 ≤ R
hdelta :
0 ≤ delta
hfixed :
if
polynomialSelectorEligible P.law lowerBand upperBand H
and
∫ z : Fin (n - n / 2)
then
Obs d, (polynomialFixedBranchNormalizedError (K := polynomialDegree n) P.law M (4096 * logEN n / (n - n / 2 : ℕ)) H z) ^ 2 ∂(productLaw (n - n / 2) P.law)
≤ R
hbad :
(Measure.infinitePi fun _ : ℕ => P.law.observedLaw).real (polynomialPilotGood (n := n) P.law lowerBand upperBand)ᶜ
≤ delta
conclusion
∫ omega : ℕ → Obs d, (clip (-1) 1 (polynomialNormalizedSum M (fun i : Fin n => omega i)) - rawAteFormula P.law / M) ^ 2 ∂(Measure.infinitePi fun _ : ℕ => P.law.observedLaw)
≤ R + 4 * delta
Proof (Lean source)
lemma polynomial_clipped_finiteSelector_risk {n d : ℕ}
{epsilon M sigma lowerBand upperBand R delta : ℝ}
(P : ModelClass d epsilon M sigma) (base : Obs d)
(hlower : 0 < lowerBand) (hR : 0 ≤ R) (hdelta : 0 ≤ delta)
(hfixed : ∀ H : Finset (Fin d),
polynomialSelectorEligible P.law lowerBand upperBand H →
∫ z : Fin (n - n / 2) → Obs d,
(polynomialFixedBranchNormalizedError (K := polynomialDegree n)
P.law M (4096 * logEN n / (n - n / 2 : ℕ)) H z) ^ 2
∂(productLaw (n - n / 2) P.law) ≤ R)
(hbad : (Measure.infinitePi fun _ : ℕ => P.law.observedLaw).real
(polynomialPilotGood (n := n) P.law lowerBand upperBand)ᶜ ≤ delta) :
∫ omega : ℕ → Obs d,
(clip (-1) 1 (polynomialNormalizedSum M
(fun i : Fin n => omega i)) - rawAteFormula P.law / M) ^ 2
∂(Measure.infinitePi fun _ : ℕ => P.law.observedLaw) ≤ R + 4 * delta := by
let mu := Measure.infinitePi fun _ : ℕ => P.law.observedLaw
let pilot : (ℕ → Obs d) →
((polynomialBalancedSplit P.law).foldA n → Obs d) :=
fun omega i => omega i
let tail : (ℕ → Obs d) →
((polynomialBalancedSplit P.law).foldB n → Obs d) :=
fun omega i => omega i
let select := polynomialPilotHeavySet (n := n) P.law base
let t := rawAteFormula P.law / M
let err : Finset (Fin d) →
((polynomialBalancedSplit P.law).foldB n → Obs d) → ℝ :=
fun H x => clip (-1) 1 (t + polynomialFixedTotalError P.law base M H x) - t
have ht : t ∈ Icc (-1 : ℝ) 1 := modelClass_normalized_ate_mem_Icc P
have hpilot : Measurable pilot := by
apply measurable_pi_lambda
intro i
exact measurable_pi_apply (i : ℕ)
have htail : Measurable tail := by
apply measurable_pi_lambda
intro i
exact measurable_pi_apply (i : ℕ)
have herr : ∀ H, Measurable (err H) := fun H =>
polynomialClippedBranchError_measurable P.law base M H t
have hbranchInt : ∀ H, polynomialSelectorEligible P.law lowerBand upperBand H →
Integrable (fun x => (err H x) ^ 2) (mu.map tail) := by
intro H _
apply Integrable.of_bound (C := 4)
· exact ((herr H).pow_const 2).aestronglyMeasurable
· filter_upwards with x
rw [Real.norm_eq_abs, abs_of_nonneg (sq_nonneg _)]
exact clippedNormalizedError_sq_le_four _ _ ht
have hbranch : ∀ H, polynomialSelectorEligible P.law lowerBand upperBand H →
∫ x, (err H x) ^ 2 ∂(mu.map tail) ≤ R := by
intro H hH
have hpos : ∀ k ∈ H, 0 < P.law.cellMass k := fun k hk =>
hlower.trans_le (hH.1 k hk)
have hrawInt := polynomialFixedBranchNormalizedError_sq_integrable
(m := n - n / 2) (K := polynomialDegree n)
(B := 4096 * logEN n / (n - n / 2 : ℕ)) P H
let reindex := polynomialFoldBReindex (n := n) P.law
have hreindex : Measurable reindex := by
apply measurable_pi_lambda
intro j
exact measurable_pi_apply (polynomialFoldBEquiv P.law j)
have hmap : (mu.map tail).map reindex =
productLaw (n - n / 2) P.law := by
rw [Measure.map_map hreindex htail]
exact map_polynomialFoldBReindex_iid_eq_productLaw P.law
have hrawFold : Integrable (fun x =>
(polynomialFixedBranchNormalizedError (K := polynomialDegree n)
P.law M (4096 * logEN n / (n - n / 2 : ℕ)) H
(reindex x)) ^ 2) (mu.map tail) := by
have hi : Integrable (fun z =>
(polynomialFixedBranchNormalizedError (K := polynomialDegree n)
P.law M (4096 * logEN n / (n - n / 2 : ℕ)) H z) ^ 2)
((mu.map tail).map reindex) := by
rw [hmap]
exact hrawInt
simpa [Function.comp_def] using hi.comp_measurable hreindex
apply (integral_mono (hbranchInt H hH) hrawFold ?_).trans ?_
intro x
dsimp [err]
rw [polynomialFixedTotalError_eq_fixedBranch P base H x hpos]
have hc := clip_normalized_sq_error_le
(t + polynomialFixedBranchNormalizedError (K := polynomialDegree n)
P.law M (4096 * logEN n / (n - n / 2 : ℕ)) H
(polynomialFoldBReindex P.law x)) t ht
simpa [reindex] using hc
let f : (Fin (n - n / 2) → Obs d) → ℝ := fun z =>
(polynomialFixedBranchNormalizedError (K := polynomialDegree n)
P.law M (4096 * logEN n / (n - n / 2 : ℕ)) H z) ^ 2
have hfsm : AEStronglyMeasurable f ((mu.map tail).map reindex) := by
rw [hmap]
exact hrawInt.aestronglyMeasurable
calc
(∫ x, f (reindex x) ∂(mu.map tail)) =
∫ z, f z ∂((mu.map tail).map reindex) :=
(integral_map hreindex.aemeasurable hfsm).symm
_ = ∫ z, f z ∂productLaw (n - n / 2) P.law := by rw [hmap]
_ ≤ R := hfixed H hH
have hsel := IndepFun.integral_finiteSelector_sq_le_add_bad
((polynomialBalancedSplit P.law).folds_indep n)
hpilot htail (polynomialPilotHeavySet_measurable P.law base) herr
(polynomialPilotFoldGood_measurable P.law base lowerBand upperBand)
hR (by norm_num : (0 : ℝ) ≤ 4)
(fun a ha => ha) hbranchInt hbranch
(fun omega => clippedNormalizedError_sq_le_four _ _ ht)
(polynomialPilotFoldGood_compl_probability_le P.law base lowerBand upperBand
delta hbad)
calc
(∫ omega : ℕ → Obs d,
(clip (-1) 1 (polynomialNormalizedSum M
(fun i : Fin n => omega i)) - rawAteFormula P.law / M) ^ 2
∂(Measure.infinitePi fun _ : ℕ => P.law.observedLaw)) =
∫ omega : ℕ → Obs d,
(err (select (pilot omega)) (tail omega)) ^ 2 ∂mu := by
apply integral_congr_ae
filter_upwards with omega
dsimp [err, select, pilot, tail, t]
rw [polynomialFixedTotalError_iidPrefix]
congr 2
ring
_ ≤ R + 4 * delta := by
simpa [mu, pilot, tail, iidSample_infinitePi] using hsel
theorem mse_polyEstimator_le_of_clipped_stream
If the indicated calibration branch applies and the stated stream condition holds, the polynomial estimator's mean squared error is bounded by the corresponding clipped-stream normalized risk after rescaling.
Formal statement
hypotheses
n d N :
ℕ
epsilon M sigma rho R :
ℝ
P :
ModelClass d epsilon M sigma
hbranch :
N ≤ n ∧ (d : ℝ) ≤ rho * n * logEN n
hstream :
∫ omega : ℕ → Obs d, (clip (-1) 1 (polynomialNormalizedSum M (fun i : Fin n => omega i)) - rawAteFormula P.law / M) ^ 2 ∂(Measure.infinitePi fun _ : ℕ => P.law.observedLaw)
≤ R
conclusion
mse P.law (rawPolyEstimator (n := n) (d := d) N rho M) ≤ M ^ 2 * R
Proof (Lean source)
lemma mse_polyEstimator_le_of_clipped_stream {n d N : ℕ}
{epsilon M sigma rho R : ℝ} (P : ModelClass d epsilon M sigma)
(hbranch : N ≤ n ∧ (d : ℝ) ≤ rho * n * logEN n)
(hstream : ∫ omega : ℕ → Obs d,
(clip (-1) 1 (polynomialNormalizedSum M
(fun i : Fin n => omega i)) - rawAteFormula P.law / M) ^ 2
∂(Measure.infinitePi fun _ : ℕ => P.law.observedLaw) ≤ R) :
mse P.law (rawPolyEstimator (n := n) (d := d) N rho M) ≤ M ^ 2 * R := by
let S := Causalean.Stat.iidSample_infinitePi P.law.observedLaw
let pref : (ℕ → Obs d) → (Fin n → Obs d) := fun omega i => omega i
let f : (Fin n → Obs d) → ℝ := fun sample =>
(clip (-1) 1 (polynomialNormalizedSum M sample) -
rawAteFormula P.law / M) ^ 2
have hM : M ≠ 0 := ne_of_gt (lt_of_lt_of_le zero_lt_one P.M_ge_one)
have hpoly : Measurable (rawPolyEstimator (n := n) (d := d) N rho M) :=
(polyEstimator_admissible (n := n) (d := d) (N := N) (rho := rho)
(le_trans zero_le_one P.M_ge_one)).1
have hclip : Measurable (fun sample : Fin n → Obs d =>
clip (-1) 1 (polynomialNormalizedSum M sample)) := by
have heq : (fun sample : Fin n → Obs d =>
clip (-1) 1 (polynomialNormalizedSum M sample)) =
fun sample => (rawPolyEstimator N rho M sample) / M := by
funext sample
rw [polyEstimator_eq_scaled_clip_of_calibrated hbranch]
field_simp [hM]
rw [heq]
exact hpoly.div measurable_const
have hf : Measurable f := ((hclip.sub measurable_const).pow_const 2)
have hpref : Measurable pref :=
Causalean.Stat.iidSample_finN_measurable S n
have hmap : (Measure.infinitePi fun _ : ℕ => P.law.observedLaw).map pref =
productLaw n P.law := by
unfold productLaw
exact Causalean.Stat.iidSample_finN_pushforward S n
have hprod : ∫ sample, f sample ∂productLaw n P.law ≤ R := by
rw [← hmap, integral_map hpref.aemeasurable hf.aestronglyMeasurable]
exact hstream
have hid (sample : Fin n → Obs d) :
(rawPolyEstimator N rho M sample - rawAteFormula P.law) ^ 2 =
M ^ 2 * f sample := by
rw [polyEstimator_eq_scaled_clip_of_calibrated hbranch]
dsimp [f]
field_simp [hM]
unfold mse
simp_rw [hid]
rw [integral_const_mul]
exact mul_le_mul_of_nonneg_left hprod (sq_nonneg M)
Helpers.PolynomialUpper.SplitBridge 26 declarations
def polynomialBalancedSplit
The deterministic half-split used by the concrete polynomial estimator, packaged as a generic one-shot split of the infinite iid realization.
Definition (Lean source)
noncomputable def polynomialBalancedSplit {d : ℕ} (P : RealLaw d) :
OneShotSplit
(iidSample_infinitePi P.observedLaw) where
n₁ n := n / 2
bound n := Nat.div_le_self n 2
grow := Nat.tendsto_div_const_atTop (by norm_num)
cogrow := by
show Tendsto (fun n : ℕ => n - n / 2) atTop atTop
apply Filter.tendsto_atTop_mono
(f := fun n : ℕ => n / 2) (g := fun n => n - n / 2)
· intro n
omega
· exact Nat.tendsto_div_const_atTop (by norm_num)
def rebuildPolynomialPilotSample
Rebuild a full sample from its pilot fold, filling the unused estimation positions by an arbitrary fixed observation.
Definition (Lean source)
def rebuildPolynomialPilotSample {n d : ℕ} (P : RealLaw d) (base : Obs d)
(x : (polynomialBalancedSplit P).foldA n → Obs d) : Fin n → Obs d :=
fun i => if h : i.val < n / 2 then
x ⟨i.val, by
simpa [foldA, polynomialBalancedSplit] using h⟩
else base
def rebuildPolynomialEstimationSample
Rebuild a full sample from its estimation fold, filling the unused pilot positions by an arbitrary fixed observation.
Definition (Lean source)
def rebuildPolynomialEstimationSample {n d : ℕ} (P : RealLaw d) (base : Obs d)
(x : (polynomialBalancedSplit P).foldB n → Obs d) : Fin n → Obs d :=
fun i => if h : n / 2 ≤ i.val then
x ⟨i.val, Finset.mem_filter.mpr ⟨Finset.mem_range.mpr i.isLt, h⟩⟩
else base
theorem rebuildPolynomialEstimationSample_measurable
Rebuilding the full tuple from the estimation fold is measurable.
Formal statement
hypotheses
conclusion
Measurable (rebuildPolynomialEstimationSample (n := n) P base)
Proof (Lean source)
lemma rebuildPolynomialEstimationSample_measurable {n d : ℕ} (P : RealLaw d)
(base : Obs d) :
Measurable (rebuildPolynomialEstimationSample (n := n) P base) := by
apply measurable_pi_lambda
intro i
by_cases hi : n / 2 ≤ i.val
· simp only [rebuildPolynomialEstimationSample, dif_pos hi]
exact measurable_pi_apply _
· simp only [rebuildPolynomialEstimationSample, dif_neg hi]
exact measurable_const
theorem pilotCount_rebuildPolynomialPilot
Rebuilding from the pilot fold preserves every concrete pilot count.
Formal statement
hypotheses
conclusion
pilotCount (rebuildPolynomialPilotSample P base (fun i : (polynomialBalancedSplit P).foldA n => omega i)) k
= pilotCount (fun i : Fin n => omega i) k
Proof (Lean source)
lemma pilotCount_rebuildPolynomialPilot {n d : ℕ} (P : RealLaw d)
(base : Obs d) (omega : ℕ → Obs d) (k : Fin d) :
pilotCount (rebuildPolynomialPilotSample P base
(fun i : (polynomialBalancedSplit P).foldA n => omega i)) k =
pilotCount (fun i : Fin n => omega i) k := by
classical
unfold pilotCount
congr 1
ext i
simp only [mem_filter, Finset.mem_univ, true_and]
by_cases hi : i.val < n / 2
· simp [inPilot, rebuildPolynomialPilotSample, hi]
· simp [inPilot, rebuildPolynomialPilotSample, hi]
theorem estimationArmCount_rebuildPolynomialEstimation
Rebuilding from the estimation fold preserves every arm-cell count.
Formal statement
hypotheses
conclusion
estimationArmCount (rebuildPolynomialEstimationSample P base (fun i : (polynomialBalancedSplit P).foldB n => omega i)) a k
= estimationArmCount (fun i : Fin n => omega i) a k
Proof (Lean source)
lemma estimationArmCount_rebuildPolynomialEstimation {n d : ℕ} (P : RealLaw d)
(base : Obs d) (omega : ℕ → Obs d) (a : Bool) (k : Fin d) :
estimationArmCount (rebuildPolynomialEstimationSample P base
(fun i : (polynomialBalancedSplit P).foldB n => omega i)) a k =
estimationArmCount (fun i : Fin n => omega i) a k := by
classical
unfold estimationArmCount
congr 1
ext i
simp only [mem_filter, Finset.mem_univ, true_and]
by_cases hi : n / 2 ≤ i.val
· simp [inPilot, rebuildPolynomialEstimationSample, hi,
Nat.not_lt.mpr hi]
· have hlt : i.val < n / 2 := Nat.lt_of_not_ge hi
simp [inPilot, rebuildPolynomialEstimationSample, hi, hlt]
theorem estimationArmSum_rebuildPolynomialEstimation
Rebuilding from the estimation fold preserves every marked outcome sum.
Formal statement
hypotheses
conclusion
estimationArmSum (rebuildPolynomialEstimationSample P base (fun i : (polynomialBalancedSplit P).foldB n => omega i)) a k
= estimationArmSum (fun i : Fin n => omega i) a k
Proof (Lean source)
lemma estimationArmSum_rebuildPolynomialEstimation {n d : ℕ} (P : RealLaw d)
(base : Obs d) (omega : ℕ → Obs d) (a : Bool) (k : Fin d) :
estimationArmSum (rebuildPolynomialEstimationSample P base
(fun i : (polynomialBalancedSplit P).foldB n => omega i)) a k =
estimationArmSum (fun i : Fin n => omega i) a k := by
classical
unfold estimationArmSum
apply Finset.sum_congr rfl
intro i _
by_cases hi : n / 2 ≤ i.val
· simp [inPilot, rebuildPolynomialEstimationSample, hi,
Nat.not_lt.mpr hi]
· have hlt : i.val < n / 2 := Nat.lt_of_not_ge hi
simp [inPilot, rebuildPolynomialEstimationSample, hi, hlt]
theorem orderedMarkedFactorial_rebuildPolynomialEstimation
The aggregate one-mark factorial is a function of the estimation fold alone.
Formal statement
hypotheses
conclusion
orderedMarkedFactorial M (rebuildPolynomialEstimationSample P base (fun i : (polynomialBalancedSplit P).foldB n => omega i)) k a j
= orderedMarkedFactorial M (fun i : Fin n => omega i) k a j
Proof (Lean source)
lemma orderedMarkedFactorial_rebuildPolynomialEstimation {n d : ℕ}
(P : RealLaw d) (base : Obs d) (omega : ℕ → Obs d)
(M : ℝ) (k : Fin d) (a : Bool) (j : ℕ) :
orderedMarkedFactorial M (rebuildPolynomialEstimationSample P base
(fun i : (polynomialBalancedSplit P).foldB n => omega i)) k a j =
orderedMarkedFactorial M (fun i : Fin n => omega i) k a j := by
unfold orderedMarkedFactorial estimationCellCount
rw [estimationArmSum_rebuildPolynomialEstimation P base omega,
estimationArmCount_rebuildPolynomialEstimation P base omega,
estimationArmCount_rebuildPolynomialEstimation P base omega,
estimationArmCount_rebuildPolynomialEstimation P base omega]
theorem lightPolynomialTerm_rebuildPolynomialEstimation
Every fixed light-cell polynomial contribution depends only on the estimation fold.
Formal statement
hypotheses
conclusion
lightPolynomialTerm M B K (rebuildPolynomialEstimationSample P base (fun i : (polynomialBalancedSplit P).foldB n => omega i)) k
= lightPolynomialTerm M B K (fun i : Fin n => omega i) k
Proof (Lean source)
lemma lightPolynomialTerm_rebuildPolynomialEstimation {n d : ℕ}
(P : RealLaw d) (base : Obs d) (omega : ℕ → Obs d)
(M B : ℝ) (K : ℕ) (k : Fin d) :
lightPolynomialTerm M B K (rebuildPolynomialEstimationSample P base
(fun i : (polynomialBalancedSplit P).foldB n => omega i)) k =
lightPolynomialTerm M B K (fun i : Fin n => omega i) k := by
unfold lightPolynomialTerm
simp_rw [orderedMarkedFactorial_rebuildPolynomialEstimation P base omega]
theorem heavyEmpiricalTerm_rebuildPolynomialEstimation
Every fixed heavy-cell marked-ratio contribution depends only on the estimation fold.
Formal statement
hypotheses
conclusion
heavyEmpiricalTerm M (rebuildPolynomialEstimationSample P base (fun i : (polynomialBalancedSplit P).foldB n => omega i)) k
= heavyEmpiricalTerm M (fun i : Fin n => omega i) k
Proof (Lean source)
lemma heavyEmpiricalTerm_rebuildPolynomialEstimation {n d : ℕ}
(P : RealLaw d) (base : Obs d) (omega : ℕ → Obs d)
(M : ℝ) (k : Fin d) :
heavyEmpiricalTerm M (rebuildPolynomialEstimationSample P base
(fun i : (polynomialBalancedSplit P).foldB n => omega i)) k =
heavyEmpiricalTerm M (fun i : Fin n => omega i) k := by
unfold heavyEmpiricalTerm estimationCellCount estimationArmMean
rw [estimationArmCount_rebuildPolynomialEstimation P base omega false k,
estimationArmCount_rebuildPolynomialEstimation P base omega true k,
estimationArmSum_rebuildPolynomialEstimation P base omega false k,
estimationArmSum_rebuildPolynomialEstimation P base omega true k]
def polynomialPilotHeavySet
The finite heavy-set branch selected by a rebuilt pilot fold.
Definition (Lean source)
noncomputable def polynomialPilotHeavySet {n d : ℕ} (P : RealLaw d)
(base : Obs d) (x : (polynomialBalancedSplit P).foldA n → Obs d) :
Finset (Fin d) :=
Finset.univ.filter fun k =>
256 * logEN n < pilotCount (rebuildPolynomialPilotSample P base x) k
theorem polynomialPilotHeavySet_measurable
The finite heavy-set selector is measurable as a function of the pilot fold. Its value depends only on the finite cell labels, not on outcomes.
Formal statement
hypotheses
conclusion
Measurable (polynomialPilotHeavySet (n := n) P base)
Proof (Lean source)
lemma polynomialPilotHeavySet_measurable {n d : ℕ} (P : RealLaw d)
(base : Obs d) :
Measurable (polynomialPilotHeavySet (n := n) P base) := by
let design : ((polynomialBalancedSplit P).foldA n → Obs d) →
((polynomialBalancedSplit P).foldA n → Fin d × Bool) :=
fun x i => ((x i).x, (x i).a)
let select : ((polynomialBalancedSplit P).foldA n → Fin d × Bool) →
Finset (Fin d) := fun z =>
Finset.univ.filter fun k =>
256 * logEN n < (Finset.univ.filter fun i : Fin n =>
inPilot i ∧
(if h : i.val < n / 2 then
(z ⟨i.val, by
simpa [foldA,
polynomialBalancedSplit] using h⟩).1
else base.x) = k).card
have hdesign : Measurable design := by
apply measurable_pi_lambda
intro i
have hobs : Measurable (fun o : Obs d => (o.x, o.a, o.y)) :=
measurable_iff_comap_le.mpr le_rfl
have hx : Measurable (fun o : Obs d => o.x) := measurable_fst.comp hobs
have ha : Measurable (fun o : Obs d => o.a) :=
measurable_fst.comp (measurable_snd.comp hobs)
exact (hx.prodMk ha).comp (measurable_pi_apply i)
have hselect : Measurable select := measurable_of_finite _
have heq : polynomialPilotHeavySet (n := n) P base = select ∘ design := by
funext x
ext k
simp only [polynomialPilotHeavySet, select, design, Function.comp_apply,
mem_filter, Finset.mem_univ, true_and]
change (256 * logEN n <
((Finset.univ.filter fun i : Fin n => inPilot i ∧
(rebuildPolynomialPilotSample P base x i).x = k).card : ℝ)) ↔
256 * logEN n <
((Finset.univ.filter fun i : Fin n => inPilot i ∧
(if h : i.val < n / 2 then (x ⟨i.val, by
simpa [foldA,
polynomialBalancedSplit] using h⟩).x else base.x) = k).card : ℝ)
have hfilter :
(Finset.univ.filter fun i : Fin n => inPilot i ∧
(rebuildPolynomialPilotSample P base x i).x = k) =
Finset.univ.filter fun i : Fin n => inPilot i ∧
(if h : i.val < n / 2 then (x ⟨i.val, by
simpa [foldA,
polynomialBalancedSplit] using h⟩).x else base.x) = k := by
ext i
simp only [mem_filter, Finset.mem_univ, true_and]
by_cases hi : i.val < n / 2 <;>
simp [rebuildPolynomialPilotSample, hi]
rw [hfilter]
rw [heq]
exact hselect.comp hdesign
def polynomialSelectorEligible
A deterministic selector value is eligible when every selected-heavy cell lies above the lower pilot band and every complementary light cell lies below the upper pilot band.
Definition (Lean source)
def polynomialSelectorEligible {d : ℕ} (P : RealLaw d)
(lowerBand upperBand : ℝ) (H : Finset (Fin d)) : Prop :=
(∀ k ∈ H, lowerBand ≤ P.cellMass k) ∧
(∀ k ∉ H, P.cellMass k ≤ upperBand)
def polynomialPilotFoldGood
The fold-level good event is the measurable preimage of the eligible finite selector values.
Definition (Lean source)
def polynomialPilotFoldGood {n d : ℕ} (P : RealLaw d) (base : Obs d)
(lowerBand upperBand : ℝ) :
Set ((polynomialBalancedSplit P).foldA n → Obs d) :=
{x | polynomialSelectorEligible P lowerBand upperBand
(polynomialPilotHeavySet P base x)}
theorem polynomialPilotFoldGood_measurable
Eligibility of the pilot-selected heavy set is a measurable event on the finite pilot fold.
Formal statement
hypotheses
conclusion
MeasurableSet (polynomialPilotFoldGood (n := n) P base lowerBand upperBand)
Proof (Lean source)
lemma polynomialPilotFoldGood_measurable {n d : ℕ} (P : RealLaw d)
(base : Obs d) (lowerBand upperBand : ℝ) :
MeasurableSet (polynomialPilotFoldGood (n := n) P base lowerBand upperBand) := by
classical
let eligible : Finset (Fin d) → Bool := fun H =>
decide (polynomialSelectorEligible P lowerBand upperBand H)
have heligible : Measurable eligible := measurable_of_finite _
have heq : polynomialPilotFoldGood (n := n) P base lowerBand upperBand =
(eligible ∘ polynomialPilotHeavySet P base) ⁻¹' {true} := by
ext x
simp [polynomialPilotFoldGood, eligible]
rw [heq]
exact (heligible.comp (polynomialPilotHeavySet_measurable P base))
(measurableSet_singleton true)
def polynomialFixedHeavyError
Normalized heavy error for a deterministic pilot-selected set, evaluated only on the independent estimation fold.
Definition (Lean source)
noncomputable def polynomialFixedHeavyError {n d : ℕ} (P : RealLaw d)
(base : Obs d) (M : ℝ) (H : Finset (Fin d))
(x : (polynomialBalancedSplit P).foldB n → Obs d) : ℝ :=
∑ k ∈ H, (
heavyEmpiricalTerm M (rebuildPolynomialEstimationSample P base x) k -
P.cellMass k * cellEffect P k / M)
def polynomialFixedLightError
Normalized light error for the complementary deterministic set, evaluated only on the independent estimation fold.
Definition (Lean source)
noncomputable def polynomialFixedLightError {n d : ℕ} (P : RealLaw d)
(base : Obs d) (M : ℝ) (H : Finset (Fin d))
(x : (polynomialBalancedSplit P).foldB n → Obs d) : ℝ :=
let K := polynomialDegree n
let B := 4096 * logEN n / (n - n / 2 : ℕ)
∑ k : Fin d, if k ∈ H then 0 else
lightPolynomialTerm M B K (rebuildPolynomialEstimationSample P base x) k -
P.cellMass k * cellEffect P k / M
theorem polynomialFixedHeavyError_measurable
Every deterministic heavy-branch error is measurable on the independent estimation fold.
Formal statement
hypotheses
conclusion
Measurable (polynomialFixedHeavyError (n := n) P base M H)
Proof (Lean source)
lemma polynomialFixedHeavyError_measurable {n d : ℕ} (P : RealLaw d)
(base : Obs d) (M : ℝ) (H : Finset (Fin d)) :
Measurable (polynomialFixedHeavyError (n := n) P base M H) := by
unfold polynomialFixedHeavyError
apply measurable_sum
intro k _
exact ((measurable_heavyEmpiricalTerm M k).comp
(rebuildPolynomialEstimationSample_measurable P base)).sub measurable_const
theorem polynomialFixedLightError_measurable
Every deterministic complementary light-branch error is measurable on the independent estimation fold.
Formal statement
hypotheses
conclusion
Measurable (polynomialFixedLightError (n := n) P base M H)
Proof (Lean source)
lemma polynomialFixedLightError_measurable {n d : ℕ} (P : RealLaw d)
(base : Obs d) (M : ℝ) (H : Finset (Fin d)) :
Measurable (polynomialFixedLightError (n := n) P base M H) := by
unfold polynomialFixedLightError
apply Finset.measurable_sum
intro k _
by_cases hk : k ∈ H
· simp only [if_pos hk]
exact measurable_const
· simp only [if_neg hk]
exact ((measurable_lightPolynomialTerm M
(4096 * logEN n / (n - n / 2 : ℕ)) (polynomialDegree n) k).comp
(rebuildPolynomialEstimationSample_measurable P base)).sub measurable_const
def polynomialFixedTotalError
The fixed-selector branch error combines the heavy score on the selected set with the polynomial score on its complement.
Definition (Lean source)
noncomputable def polynomialFixedTotalError {n d : ℕ} (P : RealLaw d)
(base : Obs d) (M : ℝ) (H : Finset (Fin d))
(x : (polynomialBalancedSplit P).foldB n → Obs d) : ℝ :=
polynomialFixedHeavyError P base M H x +
polynomialFixedLightError P base M H x
theorem polynomialFixedTotalError_measurable
Every fixed-selector total error is measurable on the estimation fold.
Formal statement
hypotheses
conclusion
Measurable (polynomialFixedTotalError (n := n) P base M H)
Proof (Lean source)
lemma polynomialFixedTotalError_measurable {n d : ℕ} (P : RealLaw d)
(base : Obs d) (M : ℝ) (H : Finset (Fin d)) :
Measurable (polynomialFixedTotalError (n := n) P base M H) := by
exact (polynomialFixedHeavyError_measurable P base M H).add
(polynomialFixedLightError_measurable P base M H)
theorem polynomialPilotHeavySet_iidPrefix
On the iid realization, the fold-valued selector is exactly the heavy set appearing in the concrete full-sample estimator.
Formal statement
hypotheses
conclusion
Proof (Lean source)
lemma polynomialPilotHeavySet_iidPrefix {n d : ℕ} (P : RealLaw d)
(base : Obs d) (omega : ℕ → Obs d) :
polynomialPilotHeavySet P base
(fun i : (polynomialBalancedSplit P).foldA n => omega i) =
Finset.univ.filter (fun k =>
256 * logEN n < pilotCount (fun i : Fin n => omega i) k) := by
ext k
simp only [polynomialPilotHeavySet, mem_filter, Finset.mem_univ,
true_and]
rw [pilotCount_rebuildPolynomialPilot P base omega k]
theorem polynomialPilotGood_implies_foldGood
If the pilot event is good, the simultaneous pilot sandwich event from the probability bound implies eligibility of the concrete finite selector on the pilot fold.
Formal statement
hypotheses
n d :
ℕ
P :
RealLaw d
base :
Obs d
lowerBand upperBand :
ℝ
omega :
ℕ → Obs d
hgood :
omega ∈ polynomialPilotGood (n := n) P lowerBand upperBand
conclusion
(fun i : (polynomialBalancedSplit P).foldA n => omega i) ∈ polynomialPilotFoldGood P base lowerBand upperBand
Proof (Lean source)
lemma polynomialPilotGood_implies_foldGood {n d : ℕ} (P : RealLaw d)
(base : Obs d) (lowerBand upperBand : ℝ) (omega : ℕ → Obs d)
(hgood : omega ∈ polynomialPilotGood (n := n) P lowerBand upperBand) :
(fun i : (polynomialBalancedSplit P).foldA n => omega i) ∈
polynomialPilotFoldGood P base lowerBand upperBand := by
change polynomialSelectorEligible P lowerBand upperBand
(polynomialPilotHeavySet P base
(fun i : (polynomialBalancedSplit P).foldA n => omega i))
unfold polynomialSelectorEligible
rw [polynomialPilotHeavySet_iidPrefix P base omega]
constructor
· intro k hk
exact polynomialPilotGood_selectedHeavy_lower P hgood k
(by simpa using (Finset.mem_filter.mp hk).2)
· intro k hk
apply polynomialPilotGood_selectedLight_upper P hgood k
intro hselected
exact hk (Finset.mem_filter.mpr ⟨Finset.mem_univ k, hselected⟩)
theorem polynomialFixedHeavyError_iidPrefix
Selecting the fixed-heavy branch with the pilot fold recovers the concrete selected-heavy error on the iid prefix.
Formal statement
hypotheses
conclusion
polynomialFixedHeavyError P base M (polynomialPilotHeavySet P base (fun i : (polynomialBalancedSplit P).foldA n => omega i)) (fun i : (polynomialBalancedSplit P).foldB n => omega i)
= polynomialHeavySelectedError P M (fun i : Fin n => omega i)
Proof (Lean source)
lemma polynomialFixedHeavyError_iidPrefix {n d : ℕ} (P : RealLaw d)
(base : Obs d) (omega : ℕ → Obs d) (M : ℝ) :
polynomialFixedHeavyError P base M
(polynomialPilotHeavySet P base
(fun i : (polynomialBalancedSplit P).foldA n => omega i))
(fun i : (polynomialBalancedSplit P).foldB n => omega i) =
polynomialHeavySelectedError P M (fun i : Fin n => omega i) := by
classical
unfold polynomialFixedHeavyError polynomialHeavySelectedError
rw [polynomialPilotHeavySet_iidPrefix P base omega]
rw [Finset.sum_filter]
apply Finset.sum_congr rfl
intro k _
rw [heavyEmpiricalTerm_rebuildPolynomialEstimation P base omega M k]
theorem polynomialFixedLightError_iidPrefix
Selecting the complementary fixed-light branch with the pilot fold recovers the concrete selected-light error on the iid prefix.
Formal statement
hypotheses
conclusion
polynomialFixedLightError P base M (polynomialPilotHeavySet P base (fun i : (polynomialBalancedSplit P).foldA n => omega i)) (fun i : (polynomialBalancedSplit P).foldB n => omega i)
= polynomialLightSelectedError P M (fun i : Fin n => omega i)
Proof (Lean source)
lemma polynomialFixedLightError_iidPrefix {n d : ℕ} (P : RealLaw d)
(base : Obs d) (omega : ℕ → Obs d) (M : ℝ) :
polynomialFixedLightError P base M
(polynomialPilotHeavySet P base
(fun i : (polynomialBalancedSplit P).foldA n => omega i))
(fun i : (polynomialBalancedSplit P).foldB n => omega i) =
polynomialLightSelectedError P M (fun i : Fin n => omega i) := by
classical
unfold polynomialFixedLightError polynomialLightSelectedError
rw [polynomialPilotHeavySet_iidPrefix P base omega]
apply Finset.sum_congr rfl
intro k _
rw [lightPolynomialTerm_rebuildPolynomialEstimation P base omega]
by_cases hk : 256 * logEN n < pilotCount (fun i : Fin n => omega i) k <;>
simp [hk]
theorem polynomialFixedTotalError_iidPrefix
Selecting a fixed total branch with the pilot fold recovers the complete normalized pre-clipping error on the iid prefix.
Formal statement
hypotheses
conclusion
polynomialFixedTotalError P base M (polynomialPilotHeavySet P base (fun i : (polynomialBalancedSplit P).foldA n => omega i)) (fun i : (polynomialBalancedSplit P).foldB n => omega i)
Proof (Lean source)
lemma polynomialFixedTotalError_iidPrefix {n d : ℕ} (P : RealLaw d)
(base : Obs d) (omega : ℕ → Obs d) (M : ℝ) :
polynomialFixedTotalError P base M
(polynomialPilotHeavySet P base
(fun i : (polynomialBalancedSplit P).foldA n => omega i))
(fun i : (polynomialBalancedSplit P).foldB n => omega i) =
polynomialNormalizedSum M (fun i : Fin n => omega i) -
rawAteFormula P / M := by
rw [polynomialNormalizedSum_sub_target_eq_selectedErrors,
polynomialFixedTotalError, polynomialFixedHeavyError_iidPrefix,
polynomialFixedLightError_iidPrefix]
Helpers.RadialChannelKernel 4 declarations This file constructs the hypothesis-independent one-record channel used by the radial converse and proves that it is a Markov kernel throughout the declared radius range.
The observed Bernoulli contraction kernel
This file constructs the hypothesis-independent one-record channel used by the radial converse and proves that it is a Markov kernel throughout the declared radius range.
The success probability of the paper's binary contraction channel.
Definition (Lean source)
noncomputable def bernoulliContractionSuccess (sigma : ℝ) (b : Bool) : ℝ :=
1 / 2 + sigma / 2 * ((if b then 1 else 0) - 1 / 2)
theorem bernoulliContractionSuccess_mem_unitInterval
If the heterogeneity radius is nonnegative and the heterogeneity radius is at most two, for a radius parameter between zero and two, both conditional success probabilities are valid Bernoulli parameters.
Formal statement
hypotheses
conclusion
bernoulliContractionSuccess sigma b ∈ Icc (0 : ℝ) 1
Proof (Lean source)
lemma bernoulliContractionSuccess_mem_unitInterval {sigma : ℝ}
(hs0 : 0 ≤ sigma) (hs2 : sigma ≤ 2) (b : Bool) :
bernoulliContractionSuccess sigma b ∈ Icc (0 : ℝ) 1 := by
cases b <;> simp [bernoulliContractionSuccess] <;> constructor <;> linarith
def bernoulliContractionObservedKernel
The one-record radial channel preserves the padded cell and treatment, draws the contracted Bernoulli response, and sends its atoms to ±M/2.
Definition (Lean source)
noncomputable def bernoulliContractionObservedKernel {m d : ℕ}
(pad : Fin m → Fin d) (M sigma : ℝ) :
Kernel (CausalSmith.Stat.DiscreteAteMinimaxLoggap.Obs m) (Obs d) where
toFun z := Measure.map
(fun r : ℝ => (⟨pad z.1, z.2.1, M * (r - 1 / 2)⟩ : Obs d))
(bernoulliLaw
(bernoulliContractionSuccess sigma z.2.2))
measurable' := by
exact measurable_of_finite
(fun z : CausalSmith.Stat.DiscreteAteMinimaxLoggap.Obs m => Measure.map
(fun r : ℝ => (⟨pad z.1, z.2.1, M * (r - 1 / 2)⟩ : Obs d))
(bernoulliLaw
(bernoulliContractionSuccess sigma z.2.2)))
theorem bernoulliContractionObservedKernel_isMarkovKernel
If the heterogeneity radius is nonnegative and the heterogeneity radius is at most two, the observed Bernoulli contraction is a Markov kernel for every declared radius 0 ≤ sigma ≤ 2.
Formal statement
hypotheses
conclusion
IsMarkovKernel (bernoulliContractionObservedKernel pad M sigma)
Proof (Lean source)
lemma bernoulliContractionObservedKernel_isMarkovKernel {m d : ℕ}
(pad : Fin m → Fin d) (M sigma : ℝ)
(hs0 : 0 ≤ sigma) (hs2 : sigma ≤ 2) :
IsMarkovKernel (bernoulliContractionObservedKernel pad M sigma) := by
constructor
intro z
have hp := bernoulliContractionSuccess_mem_unitInterval hs0 hs2 z.2.2
letI : IsProbabilityMeasure
(bernoulliLaw
(bernoulliContractionSuccess sigma z.2.2)) :=
bernoulliLaw_isProbabilityMeasure hp.1 hp.2
change IsProbabilityMeasure (Measure.map
(fun r : ℝ => (⟨pad z.1, z.2.1, M * (r - 1 / 2)⟩ : Obs d))
(bernoulliLaw
(bernoulliContractionSuccess sigma z.2.2)))
have hmeas : Measurable
(fun r : ℝ => (⟨pad z.1, z.2.1, M * (r - 1 / 2)⟩ : Obs d)) := by
rw [measurable_comap_iff]
change Measurable (fun r : ℝ => (pad z.1, z.2.1, M * (r - 1 / 2)))
fun_prop
exact Measure.isProbabilityMeasure_map hmeas.aemeasurable
Helpers.RadialContractedBinary 9 declarations This file constructs the binary observation law obtained by passing the source response through the paper's hypothesis-independent Bernoulli contraction.
Contracted binary source law for the radial converse
This file constructs the binary observation law obtained by passing the source response through the paper's hypothesis-independent Bernoulli contraction.
Conditional law of the contracted response bit given the source bit.
Definition (Lean source)
noncomputable def radialContractionPMF (sigma : ℝ) (b : Bool)
(hs0 : 0 ≤ sigma) (hs2 : sigma ≤ 2) : PMF Bool :=
PMF.ofFintype
(fun b' => ofReal
(if b' then bernoulliContractionSuccess sigma b
else 1 - bernoulliContractionSuccess sigma b)) (by
rw [Fintype.sum_bool]
simp only [if_true]
rw [← ENNReal.ofReal_add]
· norm_num
· exact (bernoulliContractionSuccess_mem_unitInterval hs0 hs2 b).1
· exact sub_nonneg.mpr
(bernoulliContractionSuccess_mem_unitInterval hs0 hs2 b).2)
theorem radialContractionPMF_true_toReal
If the heterogeneity radius is nonnegative and the heterogeneity radius is at most two, the success atom of the contraction PMF has the declared channel mass.
Formal statement
hypotheses
conclusion
Proof (Lean source)
lemma radialContractionPMF_true_toReal {sigma : ℝ} (b : Bool)
(hs0 : 0 ≤ sigma) (hs2 : sigma ≤ 2) :
((radialContractionPMF sigma b hs0 hs2) true).toReal =
bernoulliContractionSuccess sigma b := by
rw [radialContractionPMF, PMF.ofFintype_apply]
exact ENNReal.toReal_ofReal
(bernoulliContractionSuccess_mem_unitInterval hs0 hs2 b).1
theorem radialContractionPMF_false_toReal
If the heterogeneity radius is nonnegative and the heterogeneity radius is at most two, the failure atom of the contraction PMF has the complementary mass.
Formal statement
hypotheses
conclusion
((radialContractionPMF sigma b hs0 hs2) false).toReal
= 1 - bernoulliContractionSuccess sigma b
Proof (Lean source)
lemma radialContractionPMF_false_toReal {sigma : ℝ} (b : Bool)
(hs0 : 0 ≤ sigma) (hs2 : sigma ≤ 2) :
((radialContractionPMF sigma b hs0 hs2) false).toReal =
1 - bernoulliContractionSuccess sigma b := by
rw [radialContractionPMF, PMF.ofFintype_apply]
exact ENNReal.toReal_ofReal (sub_nonneg.mpr
(bernoulliContractionSuccess_mem_unitInterval hs0 hs2 b).2)
The source observation law after retaining (X,A) and independently drawing the contracted response bit conditional on the source response.
Definition (Lean source)
noncomputable def radialContractedBinaryLaw {d : ℕ} (P : BinLaw d)
(sigma : ℝ) (hs0 : 0 ≤ sigma) (hs2 : sigma ≤ 2) : BinLaw d where
pmf := P.pmf.bind fun z =>
(radialContractionPMF sigma z.2.2 hs0 hs2).map
(fun b' => (⟨z.1, z.2.1, b'⟩ : BinObs d))
theorem radialContractedBinaryLaw_cellTreatment_margin
If the heterogeneity radius is nonnegative and the heterogeneity radius is at most two, the contraction leaves the joint (X,A) margin exactly unchanged.
Formal statement
hypotheses
d :
ℕ
P :
BinLaw d
sigma :
ℝ
hs0 :
0 ≤ sigma
hs2 :
sigma ≤ 2
conclusion
(radialContractedBinaryLaw P sigma hs0 hs2).pmf.map (fun z => (z.1, z.2.1))
= P.pmf.map (fun z => (z.1, z.2.1))
Proof (Lean source)
lemma radialContractedBinaryLaw_cellTreatment_margin {d : ℕ} (P : BinLaw d)
(sigma : ℝ) (hs0 : 0 ≤ sigma) (hs2 : sigma ≤ 2) :
(radialContractedBinaryLaw P sigma hs0 hs2).pmf.map
(fun z => (z.1, z.2.1)) =
P.pmf.map (fun z => (z.1, z.2.1)) := by
rw [radialContractedBinaryLaw, PMF.map_bind]
simp_rw [PMF.map_comp]
have hmap (z : BinObs d) :
map ((fun w : BinObs d => (w.1, w.2.1)) ∘
fun b' => (⟨z.1, z.2.1, b'⟩ : BinObs d))
(radialContractionPMF sigma z.2.2 hs0 hs2) =
PMF.pure (z.1, z.2.1) := by
rw [← PMF.map_const]
congr 1
simp_rw [hmap]
change P.pmf.bind (PMF.pure ∘ fun z : BinObs d => (z.1, z.2.1)) = _
exact PMF.bind_pure_comp _ _
theorem radialContractedBinaryLaw_jointMass
If the heterogeneity radius is nonnegative and the heterogeneity radius is at most two, each output atom is the source-bit mixture prescribed by the common Bernoulli contraction channel.
Formal statement
hypotheses
conclusion
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass (radialContractedBinaryLaw P sigma hs0 hs2) k a b'
= ∑ b : Bool, CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass P k a b * (if b' then bernoulliContractionSuccess sigma b else 1 - bernoulliContractionSuccess sigma b)
Proof (Lean source)
lemma radialContractedBinaryLaw_jointMass {d : ℕ} (P : BinLaw d) (sigma : ℝ)
(hs0 : 0 ≤ sigma) (hs2 : sigma ≤ 2) (k : Fin d) (a b' : Bool) :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass
(radialContractedBinaryLaw P sigma hs0 hs2) k a b' =
∑ b : Bool,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass P k a b *
(if b' then bernoulliContractionSuccess sigma b
else 1 - bernoulliContractionSuccess sigma b) := by
unfold CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass
rw [radialContractedBinaryLaw, PMF.bind_apply]
rw [tsum_fintype]
simp_rw [PMF.map_apply, tsum_fintype]
classical
rw [Fintype.sum_prod_type, Finset.sum_eq_single k]
· cases a <;> cases b' <;>
simp [Fintype.sum_prod_type, Fintype.sum_bool] <;>
rw [ENNReal.toReal_add, ENNReal.toReal_mul, ENNReal.toReal_mul] <;>
simp [Fintype.sum_prod_type, radialContractionPMF_true_toReal,
radialContractionPMF_false_toReal, PMF.apply_ne_top]
all_goals apply ENNReal.mul_ne_top <;> apply PMF.apply_ne_top
· intro j _ hj
simp [hj, Ne.symm hj]
· simp
theorem radialContractedBinaryLaw_armMass
If the heterogeneity radius is nonnegative and the heterogeneity radius is at most two, summing over the contracted response shows that the channel preserves each cell-treatment mass exactly.
Formal statement
hypotheses
conclusion
CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass (radialContractedBinaryLaw P sigma hs0 hs2) k a
= CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P k a
Proof (Lean source)
lemma radialContractedBinaryLaw_armMass {d : ℕ} (P : BinLaw d) (sigma : ℝ)
(hs0 : 0 ≤ sigma) (hs2 : sigma ≤ 2) (k : Fin d) (a : Bool) :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass
(radialContractedBinaryLaw P sigma hs0 hs2) k a =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P k a := by
unfold CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass
simp_rw [radialContractedBinaryLaw_jointMass]
rw [Finset.sum_comm]
apply Finset.sum_congr rfl
intro b _
rw [Fintype.sum_bool]
unfold bernoulliContractionSuccess
simp only [if_true, Bool.false_eq_true, if_false]
ring
theorem radialContractedBinaryLaw_cellMass
If the heterogeneity radius is nonnegative and the heterogeneity radius is at most two, the Bernoulli contraction preserves every source cell mass.
Formal statement
hypotheses
conclusion
CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass (radialContractedBinaryLaw P sigma hs0 hs2) k
= CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P k
Proof (Lean source)
lemma radialContractedBinaryLaw_cellMass {d : ℕ} (P : BinLaw d) (sigma : ℝ)
(hs0 : 0 ≤ sigma) (hs2 : sigma ≤ 2) (k : Fin d) :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass
(radialContractedBinaryLaw P sigma hs0 hs2) k =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P k := by
unfold CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass
change (∑ a : Bool, CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass
(radialContractedBinaryLaw P sigma hs0 hs2) k a) = _
simp_rw [radialContractedBinaryLaw_armMass]
rfl
theorem radialContractedBinaryLaw_overlap
If the heterogeneity radius is nonnegative and the heterogeneity radius is at most two and the source law satisfies the stated model condition, because both cell and cell-treatment masses are preserved, contraction preserves the strong-overlap certificate.
Formal statement
hypotheses
d :
ℕ
epsilon sigma :
ℝ
P :
BinLaw d
hs0 :
0 ≤ sigma
hs2 :
sigma ≤ 2
hP :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.Overlap epsilon P
conclusion
CausalSmith.Stat.DiscreteAteMinimaxLoggap.Overlap epsilon (radialContractedBinaryLaw P sigma hs0 hs2)
Proof (Lean source)
lemma radialContractedBinaryLaw_overlap {d : ℕ} {epsilon sigma : ℝ} (P : BinLaw d)
(hs0 : 0 ≤ sigma) (hs2 : sigma ≤ 2)
(hP : CausalSmith.Stat.DiscreteAteMinimaxLoggap.Overlap epsilon P) :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.Overlap epsilon
(radialContractedBinaryLaw P sigma hs0 hs2) := by
intro k hk
unfold CausalSmith.Stat.DiscreteAteMinimaxLoggap.propensity
rw [radialContractedBinaryLaw_armMass, radialContractedBinaryLaw_cellMass]
exact hP k (by simpa [radialContractedBinaryLaw_cellMass] using hk)
Helpers.RadialFiniteSampleScale 1 declarations Finite-sample radial transport scale
Finite-sample radial transport scale
If the sample is nonempty and the sampling budget satisfies the stated lower bound, below a fixed sample-size cutoff, the one-arm parametric source bound dominates the capped radial term after Bernoulli-channel scaling.
Formal statement
hypotheses
n d N :
ℕ
hn :
0 < n
hN :
n ≤ N
M sigma :
ℝ
conclusion
(1 / (800 * (N : ℝ))) * M ^ 2 * sigma ^ 2 * min 1 (polynomialComponent n d)
≤ (M * sigma / 2) ^ 2 * (1 / (200 * (n : ℝ)))
Proof (Lean source)
lemma finiteSampleRadial_transportScale {n d N : ℕ}
(hn : 0 < n) (hN : n ≤ N) (M sigma : ℝ) :
(1 / (800 * (N : ℝ))) * M ^ 2 * sigma ^ 2 *
min 1 (polynomialComponent n d) ≤
(M * sigma / 2) ^ 2 * (1 / (200 * (n : ℝ))) := by
have hnR : 0 < (n : ℝ) := by exact_mod_cast hn
have hNpos : 0 < (N : ℝ) := lt_of_lt_of_le hnR (by exact_mod_cast hN)
have hrecip : 1 / (N : ℝ) ≤ 1 / (n : ℝ) := by
exact one_div_le_one_div_of_le hnR (by exact_mod_cast hN)
have hmin : min 1 (polynomialComponent n d) ≤ 1 := min_le_left _ _
have hnonneg : 0 ≤ M ^ 2 * sigma ^ 2 := mul_nonneg (sq_nonneg _) (sq_nonneg _)
have hcoef : 0 ≤ (1 / (800 * (N : ℝ))) * M ^ 2 * sigma ^ 2 := by
positivity
calc
(1 / (800 * (N : ℝ))) * M ^ 2 * sigma ^ 2 *
min 1 (polynomialComponent n d) ≤
(1 / (800 * (N : ℝ))) * M ^ 2 * sigma ^ 2 := by
simpa using mul_le_mul_of_nonneg_left hmin hcoef
_ ≤ (1 / (800 * (n : ℝ))) * M ^ 2 * sigma ^ 2 := by
have hc : 1 / (800 * (N : ℝ)) ≤ 1 / (800 * (n : ℝ)) := by
simpa [one_div, mul_inv_rev] using
(mul_le_mul_of_nonneg_left hrecip (by norm_num : (0 : ℝ) ≤ 1 / 800))
exact mul_le_mul_of_nonneg_right
(mul_le_mul_of_nonneg_right hc (sq_nonneg M)) (sq_nonneg sigma)
_ = (M * sigma / 2) ^ 2 * (1 / (200 * (n : ℝ))) := by ring
Helpers.RadialFullCoupling 4 declarations This module proves the finite PMF identity behind the full-data radial-channel certificate, including the zero-mass-cell boundary case.
Full-data coupling under the radial Bernoulli channel
This module proves the finite PMF identity behind the full-data radial-channel certificate, including the zero-mass-cell boundary case.
def contractedFullPMF
This is the full-data probability mass function obtained after radial Bernoulli contraction.
Definition (Lean source)
noncomputable def contractedFullPMF {m : ℕ} (P : BinLaw m) (sigma : ℝ)
(hs0 : 0 ≤ sigma) (hs2 : sigma ≤ 2) : PMF (BinaryFullObs m) :=
(binaryIndependentFullPMF P).bind fun z =>
(radialContractionPMF sigma z.b0 hs0 hs2).bind fun c0 =>
(radialContractionPMF sigma z.b1 hs0 hs2).map fun c1 =>
⟨z.x, z.a, c0, c1⟩
theorem binaryOutcomePMF_radialContracted
If the heterogeneity radius is nonnegative and the heterogeneity radius is at most two and the transport scale satisfies the stated condition, the outcome distribution of a radially contracted binary law is the corresponding Bernoulli contraction distribution.
Formal statement
hypotheses
m :
ℕ
P :
BinLaw m
sigma :
ℝ
hs0 :
0 ≤ sigma
hs2 :
sigma ≤ 2
k :
Fin m
a :
ha :
0 < CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P k a
conclusion
binaryOutcomePMF (radialContractedBinaryLaw P sigma hs0 hs2) a k
= (binaryOutcomePMF P a k).bind fun b => radialContractionPMF sigma b hs0 hs2
Proof (Lean source)
lemma binaryOutcomePMF_radialContracted {m : ℕ} (P : BinLaw m)
(sigma : ℝ) (hs0 : 0 ≤ sigma) (hs2 : sigma ≤ 2) (k : Fin m)
(a : Bool) (ha : 0 < CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P k a) :
binaryOutcomePMF (radialContractedBinaryLaw P sigma hs0 hs2) a k =
(binaryOutcomePMF P a k).bind fun b =>
radialContractionPMF sigma b hs0 hs2 := by
apply PMF.ext
intro c
suffices hreal :
(binaryOutcomePMF (radialContractedBinaryLaw P sigma hs0 hs2) a k c).toReal =
(((binaryOutcomePMF P a k).bind fun b =>
radialContractionPMF sigma b hs0 hs2) c).toReal by
rw [ENNReal.toReal_eq_toReal_iff] at hreal
rcases hreal with h | h | h
· exact h
· exact elim ((PMF.apply_ne_top _ _) h.2)
· exact elim ((PMF.apply_ne_top _ _) h.1)
cases c
· rw [binaryOutcomePMF_false_toReal, PMF.bind_apply, tsum_fintype,
ENNReal.toReal_sum]
· rw [Fintype.sum_bool, ENNReal.toReal_mul, ENNReal.toReal_mul,
binaryOutcomePMF_true_toReal, binaryOutcomePMF_false_toReal,
radialContractionPMF_false_toReal,
radialContractionPMF_false_toReal,
radialContractedBinaryLaw_outcomeMean P sigma hs0 hs2 k a ha]
unfold bernoulliContractionSuccess
simp only [if_true, Bool.false_eq_true, if_false]
ring
· intro b _hb
exact ENNReal.mul_ne_top (PMF.apply_ne_top _ _) (PMF.apply_ne_top _ _)
· rw [binaryOutcomePMF_true_toReal, PMF.bind_apply, tsum_fintype,
ENNReal.toReal_sum]
· rw [Fintype.sum_bool, ENNReal.toReal_mul, ENNReal.toReal_mul,
binaryOutcomePMF_true_toReal, binaryOutcomePMF_false_toReal,
radialContractionPMF_true_toReal,
radialContractionPMF_true_toReal,
radialContractedBinaryLaw_outcomeMean P sigma hs0 hs2 k a ha]
unfold bernoulliContractionSuccess
simp only [if_true, Bool.false_eq_true, if_false]
ring
· intro b _hb
exact ENNReal.mul_ne_top (PMF.apply_ne_top _ _) (PMF.apply_ne_top _ _)
theorem contractedFullPMF_eq_independent
If the overlap constant is positive and the heterogeneity radius is nonnegative and the heterogeneity radius is at most two, the contracted full-data distribution equals the independent coupling generated from the contracted observed law.
Formal statement
hypotheses
n m :
ℕ
epsilon :
ℝ
P :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n m epsilon
he0 :
0 < epsilon
sigma :
ℝ
hs0 :
0 ≤ sigma
hs2 :
sigma ≤ 2
conclusion
contractedFullPMF P.1 sigma hs0 hs2
= binaryIndependentFullPMF (radialContractedBinaryLaw P.1 sigma hs0 hs2)
Proof (Lean source)
lemma contractedFullPMF_eq_independent {n m : ℕ} {epsilon : ℝ}
(P : CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n m epsilon)
(he0 : 0 < epsilon) (sigma : ℝ)
(hs0 : 0 ≤ sigma) (hs2 : sigma ≤ 2) :
contractedFullPMF P.1 sigma hs0 hs2 =
binaryIndependentFullPMF (radialContractedBinaryLaw P.1 sigma hs0 hs2) := by
apply PMF.ext
rintro ⟨k, a, c0, c1⟩
by_cases hk : 0 < CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P.1 k
· have hpi := P.2.overlap k hk
have hmne : CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P.1 k ≠ 0 :=
ne_of_gt hk
have htrue_eq :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P.1 k true =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.propensity P.1 k *
CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P.1 k := by
rw [CausalSmith.Stat.DiscreteAteMinimaxLoggap.propensity]
field_simp
have htrue : 0 < CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P.1 k true := by
rw [htrue_eq]
exact mul_pos (lt_of_lt_of_le he0 hpi.1) hk
have hmass_eq :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P.1 k =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P.1 k false +
CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P.1 k true := by
simp [CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass]
ring
have hfalse_eq :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P.1 k false =
(1 - CausalSmith.Stat.DiscreteAteMinimaxLoggap.propensity P.1 k) *
CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P.1 k := by
rw [htrue_eq] at hmass_eq
linarith
have hfalse : 0 < CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P.1 k false := by
rw [hfalse_eq]
exact mul_pos (by linarith [hpi.2]) hk
have hout_true (c : Bool) :
(binaryOutcomePMF
({ pmf := P.1.pmf.bind fun z =>
(radialContractionPMF sigma z.2.2 hs0 hs2).map
(fun b' => (⟨z.1, z.2.1, b'⟩ :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.Obs m)) } : BinLaw m)
true k) c =
(((binaryOutcomePMF P.1 true k).bind fun b =>
radialContractionPMF sigma b hs0 hs2) c) := by
simpa only [radialContractedBinaryLaw] using congrArg (fun q : PMF Bool => q c)
(binaryOutcomePMF_radialContracted P.1 sigma hs0 hs2 k true htrue)
have hout_false (c : Bool) :
(binaryOutcomePMF
({ pmf := P.1.pmf.bind fun z =>
(radialContractionPMF sigma z.2.2 hs0 hs2).map
(fun b' => (⟨z.1, z.2.1, b'⟩ :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.Obs m)) } : BinLaw m)
false k) c =
(((binaryOutcomePMF P.1 false k).bind fun b =>
radialContractionPMF sigma b hs0 hs2) c) := by
simpa only [radialContractedBinaryLaw] using congrArg (fun q : PMF Bool => q c)
(binaryOutcomePMF_radialContracted P.1 sigma hs0 hs2 k false hfalse)
cases a <;> cases c0 <;> cases c1 <;>
simp [contractedFullPMF, binaryIndependentFullPMF, radialContractedBinaryLaw,
binaryIndependentLift, PMF.bind_apply, PMF.map_apply, tsum_fintype,
Fintype.sum_prod_type, Fintype.sum_bool, Finset.sum_eq_single k,
binaryOutcomePMF_radialContracted P.1 sigma hs0 hs2 k true htrue,
binaryOutcomePMF_radialContracted P.1 sigma hs0 hs2 k false hfalse]
all_goals
rw [Finset.sum_eq_single k]
· simp only [Finset.sum_add_distrib, Finset.sum_ite_eq,
Finset.mem_univ, if_true]
simp [hout_true, hout_false, PMF.bind_apply, tsum_fintype,
Fintype.sum_bool] <;> ring
· intro x _hx hxk
simp [hxk, Ne.symm hxk]
· simp
· have hk0 : CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P.1 k = 0 :=
le_antisymm (le_of_not_gt hk)
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass_mem_unitInterval P.1 k).1
have hp (a y : Bool) : P.1.pmf (k, a, y) = 0 := by
have hj :=
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass_eq_zero_of_cellMass_eq_zero
P.1 k hk0 a y
unfold CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass at hj
rw [ENNReal.toReal_eq_zero_iff] at hj
rcases hj with h | h
· exact h
· exact elim ((PMF.apply_ne_top P.1.pmf (k, a, y)) h)
cases a <;> cases c0 <;> cases c1 <;>
simp [contractedFullPMF, binaryIndependentFullPMF, radialContractedBinaryLaw,
binaryIndependentLift, PMF.bind_apply, PMF.map_apply, tsum_fintype,
Fintype.sum_prod_type, Fintype.sum_bool, hp]
all_goals
rw [Finset.sum_eq_single k]
· simp only [Finset.sum_add_distrib, Finset.sum_ite_eq,
Finset.mem_univ, if_true]
simp [hp]
· intro x _hx hxk
simp [hxk, Ne.symm hxk]
· simp
theorem radialPaddedAffine_fullLaw_eq_binaryFullChannel
If the source alphabet embeds in the target alphabet and the overlap constant is positive and the heterogeneity radius is nonnegative and the heterogeneity radius is at most two, the concrete affine radial embedding is exactly the paper's common two-potential-outcome Bernoulli channel applied to the independent source coupling.
Formal statement
hypotheses
n m d :
ℕ
epsilon :
ℝ
P :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n m epsilon
hmd :
m ≤ d
he0 :
0 < epsilon
M sigma :
ℝ
hs0 :
0 ≤ sigma
hs2 :
sigma ≤ 2
conclusion
= binaryFullChannel (fun k : Fin m => ⟨k, lt_of_lt_of_le k.isLt hmd⟩) (bernoulliContractionSuccess sigma) (fun b => M * ((if b then 1 else 0) - 1 / 2)) (binaryIndependentFullPMF P.1).toMeasure
Proof (Lean source)
lemma radialPaddedAffine_fullLaw_eq_binaryFullChannel {n m d : ℕ}
{epsilon : ℝ}
(P : CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n m epsilon)
(hmd : m ≤ d) (he0 : 0 < epsilon) (M sigma : ℝ)
(hs0 : 0 ≤ sigma) (hs2 : sigma ≤ 2) :
(affineBinaryRealLaw M
(binaryPadLaw hmd (radialContractedBinaryLaw P.1 sigma hs0 hs2))).fullLaw =
binaryFullChannel
(fun k : Fin m => ⟨k, lt_of_lt_of_le k.isLt hmd⟩)
(bernoulliContractionSuccess sigma)
(fun b => M * ((if b then 1 else 0) - 1 / 2))
(binaryIndependentFullPMF P.1).toMeasure := by
let P0 : BinLaw m := P.1
change (affineBinaryRealLaw M
(binaryPadLaw hmd (radialContractedBinaryLaw P0 sigma hs0 hs2))).fullLaw =
binaryFullChannel _ _ _ (binaryIndependentFullPMF P0).toMeasure
have hc : contractedFullPMF P0 sigma hs0 hs2 =
binaryIndependentFullPMF (radialContractedBinaryLaw P0 sigma hs0 hs2) := by
simpa [P0] using contractedFullPMF_eq_independent P he0 sigma hs0 hs2
rw [affineBinaryPadded_fullLaw, ← hc]
ext E hE
rw [Measure.map_apply (by fun_prop) hE]
unfold contractedFullPMF
rw [PMF.toMeasure_bind_apply
(p := binaryIndependentFullPMF P0)
(f := fun z =>
(radialContractionPMF sigma z.b0 hs0 hs2).bind fun c0 =>
(radialContractionPMF sigma z.b1 hs0 hs2).map fun c1 =>
⟨z.x, z.a, c0, c1⟩)
(s := (BinaryFullObs.affine M
(fun k : Fin m => ⟨k, lt_of_lt_of_le k.isLt hmd⟩)) ⁻¹' E)
(hE.preimage (by fun_prop))]
unfold binaryFullChannel
simp only [Measure.finsetSum_apply, Measure.smul_apply, smul_eq_mul,
tsum_fintype]
apply Finset.sum_congr rfl
intro z _hz
rw [show (binaryIndependentFullPMF P0).toMeasure {z} =
binaryIndependentFullPMF P0 z by
exact PMF.toMeasure_apply_singleton _ _ (MeasurableSet.singleton z)]
rw [PMF.toMeasure_bind_apply
(p := radialContractionPMF sigma z.b0 hs0 hs2)
(f := fun c0 =>
(radialContractionPMF sigma z.b1 hs0 hs2).map fun c1 =>
⟨z.x, z.a, c0, c1⟩)
(s := (BinaryFullObs.affine M
(fun k : Fin m => ⟨k, lt_of_lt_of_le k.isLt hmd⟩)) ⁻¹' E)
(hE.preimage (by fun_prop)), tsum_fintype]
rw [Finset.mul_sum]
apply Finset.sum_congr rfl
intro c0 _hc0
rw [PMF.toMeasure_map_apply
(f := fun c1 : Bool => (⟨z.x, z.a, c0, c1⟩ : BinaryFullObs m))
(p := radialContractionPMF sigma z.b1 hs0 hs2)
(s := (BinaryFullObs.affine M
(fun k : Fin m => ⟨k, lt_of_lt_of_le k.isLt hmd⟩)) ⁻¹' E)
(measurable_of_finite _) (hE.preimage (by fun_prop))]
rw [PMF.toMeasure_apply (p := radialContractionPMF sigma z.b1 hs0 hs2)
((hE.preimage (by fun_prop)).preimage (measurable_of_finite _)), tsum_fintype]
by_cases h1 : BinaryFullObs.affine M
(fun k : Fin m => ⟨k, lt_of_lt_of_le k.isLt hmd⟩)
(⟨z.x, z.a, c0, true⟩ : BinaryFullObs m) ∈ E <;>
by_cases h0 : BinaryFullObs.affine M
(fun k : Fin m => ⟨k, lt_of_lt_of_le k.isLt hmd⟩)
(⟨z.x, z.a, c0, false⟩ : BinaryFullObs m) ∈ E <;>
simp [BinaryFullObs.affine] at h1 h0 <;>
simp [Measure.dirac_apply' _ hE, radialContractionPMF,
PMF.ofFintype_apply, BinaryFullObs.affine, h1, h0, mul_add, mul_assoc]
Helpers.RadialHardFamily 8 declarations
theorem oneArmMinimaxRisk_hard_family_of_lt Lemma oneArmMinimaxRisk_hard_family_of_lt in the paper ↗
If the alphabet is nonempty and the overlap constant is positive and the overlap constant is below one half and the logarithmic scale satisfies its stated bound, any level strictly below the one-arm minimax risk is attained as a lower bound against every measurable estimator by a control-zero source law.
Formal statement
hypotheses
n d :
ℕ
epsilon L :
ℝ
hd :
0 < d
he0 :
0 < epsilon
he1 :
epsilon < 1 / 2
hL :
L < CausalSmith.Stat.DiscreteAteMinimaxLoggap.oneArmMinimaxRisk n d epsilon
conclusion
∀ est : (Fin n → BinObs d) → ℝ
if
Measurable est
then
∃ P : CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n d epsilon,
L
≤ CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse (CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n) est (CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1)
Proof (Lean source)
lemma oneArmMinimaxRisk_hard_family_of_lt {n d : ℕ} {epsilon L : ℝ}
(hd : 0 < d) (he0 : 0 < epsilon) (he1 : epsilon < 1 / 2)
(hL : L < CausalSmith.Stat.DiscreteAteMinimaxLoggap.oneArmMinimaxRisk n d epsilon) :
∀ est : (Fin n → BinObs d) → ℝ, Measurable est →
∃ P : CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n d epsilon,
L ≤ CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n)
est (CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1) := by
letI : Nonempty (Fin d) := Fin.pos_iff_nonempty.mp hd
obtain ⟨P0, _htau0, _hv0, _hP0⟩ :=
endpoint_null_exact (n := n) hd he0 he1
let hP0class : CausalSmith.Stat.DiscreteAteMinimaxLoggap.ExperimentClass
n epsilon P0.1
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P0.1 n) :=
⟨he0, he1.le, rfl, P0.2.1⟩
let Pzero : CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n d epsilon :=
CausalSmith.Stat.DiscreteAteMinimaxLoggap.eraseControlLaw_controlZero P0.1 hP0class
letI : Nonempty
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n d epsilon) :=
⟨Pzero⟩
intro est hest
let est' : {f : (Fin n → BinObs d) → ℝ // Measurable f} := ⟨est, hest⟩
have hb : BddAbove (range (fun P :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n d epsilon ↦
CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n)
est (CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1))) := by
refine ⟨((∑ sample : Fin n → BinObs d, |est sample|) + 1) ^ 2, ?_⟩
rintro _ ⟨P, rfl⟩
change CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n) est
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1) ≤ _
rw [← CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional_eq_treated_on_controlZero P]
exact CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse_le_estimator_abs_sum_bound
P.1 P.2.overlap est
have hbelow : BddBelow (range (fun e : {f : (Fin n → BinObs d) → ℝ //
Measurable f} ↦
CausalSmith.Stat.DiscreteAteMinimaxLoggap.oneArmWorstCaseMSE
n d epsilon e.1)) := by
refine ⟨0, ?_⟩
rintro _ ⟨e, rfl⟩
unfold CausalSmith.Stat.DiscreteAteMinimaxLoggap.oneArmWorstCaseMSE
have hbe : BddAbove (range (fun P :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n d epsilon ↦
CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n)
e.1 (CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1))) := by
refine ⟨((∑ sample : Fin n → BinObs d, |e.1 sample|) + 1) ^ 2, ?_⟩
rintro _ ⟨P, rfl⟩
change CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n) e.1
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1) ≤ _
rw [← CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional_eq_treated_on_controlZero P]
exact CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse_le_estimator_abs_sum_bound
P.1 P.2.overlap e.1
have hmse0 : 0 ≤ CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw Pzero.1 n)
e.1 (CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional Pzero.1) := by
unfold CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse
exact integral_nonneg fun _ => sq_nonneg _
exact hmse0.trans (le_ciSup hbe Pzero)
have hinf : CausalSmith.Stat.DiscreteAteMinimaxLoggap.oneArmMinimaxRisk n d epsilon ≤
CausalSmith.Stat.DiscreteAteMinimaxLoggap.oneArmWorstCaseMSE
n d epsilon est := by
unfold CausalSmith.Stat.DiscreteAteMinimaxLoggap.oneArmMinimaxRisk
exact ciInf_le hbelow est'
have hsup : L < CausalSmith.Stat.DiscreteAteMinimaxLoggap.oneArmWorstCaseMSE
n d epsilon est := hL.trans_le hinf
unfold CausalSmith.Stat.DiscreteAteMinimaxLoggap.oneArmWorstCaseMSE at hsup
obtain ⟨P, hLP⟩ := (lt_ciSup_iff hb).mp hsup
exact ⟨P, hLP.le⟩
theorem radialSourceHardData_univ_of_strict_oneArm_lower
If the alphabet is nonempty and the overlap constant is positive and the overlap constant is below one half and the strict one-arm lower-bound constant is available and the stated lower bound holds, a strict one-arm minimax lower bound equips the full control-zero source class with the hard-family certificate carried by a radius-channel handle.
Formal statement
hypotheses
n d :
ℕ
epsilon c :
ℝ
hd :
0 < d
he0 :
0 < epsilon
he1 :
epsilon < 1 / 2
hc :
0 < c
hlower :
c * CausalSmith.Stat.DiscreteAteMinimaxLoggap.minimaxRate n d < CausalSmith.Stat.DiscreteAteMinimaxLoggap.oneArmMinimaxRisk n d epsilon
conclusion
RadialSourceHardData (Set.univ : Set (CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n d epsilon))
Proof (Lean source)
lemma radialSourceHardData_univ_of_strict_oneArm_lower {n d : ℕ} {epsilon c : ℝ}
(hd : 0 < d) (he0 : 0 < epsilon) (he1 : epsilon < 1 / 2) (hc : 0 < c)
(hlower : c * CausalSmith.Stat.DiscreteAteMinimaxLoggap.minimaxRate n d <
CausalSmith.Stat.DiscreteAteMinimaxLoggap.oneArmMinimaxRisk n d epsilon) :
RadialSourceHardData
(Set.univ : Set
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n d epsilon)) := by
refine ⟨c, hc, ?_⟩
intro est
obtain ⟨P, hP⟩ := oneArmMinimaxRisk_hard_family_of_lt
hd he0 he1 hlower est.1 est.2
exact ⟨P, Set.mem_univ P, hP⟩
theorem radialSourceHardData_univ_of_oneArm_lower
If the sample is nonempty and the alphabet is nonempty and the overlap constant is positive and the overlap constant is below one half and the transport scale satisfies the stated condition and the stated lower bound holds, a positive non-strict one-arm lower bound yields a hard full source family after halving its constant, which supplies the strict level required by the minimax hard-family extractor.
Formal statement
hypotheses
n d :
ℕ
epsilon a :
ℝ
hn :
0 < n
hd :
0 < d
he0 :
0 < epsilon
he1 :
epsilon < 1 / 2
ha :
0 < a
hlower :
a * CausalSmith.Stat.DiscreteAteMinimaxLoggap.minimaxRate n d
≤ CausalSmith.Stat.DiscreteAteMinimaxLoggap.oneArmMinimaxRisk n d epsilon
conclusion
RadialSourceHardData (Set.univ : Set (CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n d epsilon))
Proof (Lean source)
lemma radialSourceHardData_univ_of_oneArm_lower {n d : ℕ} {epsilon a : ℝ}
(hn : 0 < n) (hd : 0 < d) (he0 : 0 < epsilon) (he1 : epsilon < 1 / 2)
(ha : 0 < a)
(hlower : a * CausalSmith.Stat.DiscreteAteMinimaxLoggap.minimaxRate n d ≤
CausalSmith.Stat.DiscreteAteMinimaxLoggap.oneArmMinimaxRisk n d epsilon) :
RadialSourceHardData
(Set.univ : Set
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n d epsilon)) := by
have hnR : 0 < (n : ℝ) := by exact_mod_cast hn
have hrate : 0 < CausalSmith.Stat.DiscreteAteMinimaxLoggap.minimaxRate n d := by
unfold CausalSmith.Stat.DiscreteAteMinimaxLoggap.minimaxRate
have hone : 0 < 1 / (n : ℝ) := one_div_pos.mpr hnR
have hpoly : 0 ≤
(d : ℝ) ^ 2 / ((n : ℝ) ^ 2 * log n ^ 2) := by positivity
linarith
apply radialSourceHardData_univ_of_strict_oneArm_lower
hd he0 he1 (c := a / 2) (by positivity)
exact (by nlinarith : a / 2 *
CausalSmith.Stat.DiscreteAteMinimaxLoggap.minimaxRate n d <
a * CausalSmith.Stat.DiscreteAteMinimaxLoggap.minimaxRate n d).trans_le hlower
If the sample is nonempty and the alphabet is nonempty and the overlap constant is positive and the overlap constant is below one half and the transport scale satisfies the stated condition and the stated lower bound holds, a positive fixed-sample one-arm lower bound supplies exactly the uniformly bounded source-estimator hardness premise required by randomized kernel transport, after the standard strict half-constant reduction.
Formal statement
hypotheses
n d :
ℕ
epsilon a :
ℝ
hn :
0 < n
hd :
0 < d
he0 :
0 < epsilon
he1 :
epsilon < 1 / 2
ha :
0 < a
hlower :
a * CausalSmith.Stat.DiscreteAteMinimaxLoggap.minimaxRate n d
≤ CausalSmith.Stat.DiscreteAteMinimaxLoggap.oneArmMinimaxRisk n d epsilon
conclusion
∀ sourceEst : (Fin n → BinObs d) → ℝ
if
Measurable sourceEst
and
UniformlyBounded sourceEst
then
∃ P ∈ (Set.univ : Set (CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n d epsilon)),
a / 2 * CausalSmith.Stat.DiscreteAteMinimaxLoggap.minimaxRate n d
≤ sqRisk (CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n) sourceEst (CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1)
Proof (Lean source)
lemma radial_source_risk_of_oneArm_lower {n d : ℕ} {epsilon a : ℝ}
(hn : 0 < n) (hd : 0 < d) (he0 : 0 < epsilon) (he1 : epsilon < 1 / 2)
(ha : 0 < a)
(hlower : a * CausalSmith.Stat.DiscreteAteMinimaxLoggap.minimaxRate n d ≤
CausalSmith.Stat.DiscreteAteMinimaxLoggap.oneArmMinimaxRisk n d epsilon) :
∀ sourceEst : (Fin n → BinObs d) → ℝ,
Measurable sourceEst → UniformlyBounded sourceEst →
∃ P ∈ (Set.univ : Set
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n d epsilon)),
a / 2 * CausalSmith.Stat.DiscreteAteMinimaxLoggap.minimaxRate n d ≤
sqRisk
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n)
sourceEst
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1) := by
have hnR : 0 < (n : ℝ) := by exact_mod_cast hn
have hrate : 0 < CausalSmith.Stat.DiscreteAteMinimaxLoggap.minimaxRate n d := by
unfold CausalSmith.Stat.DiscreteAteMinimaxLoggap.minimaxRate
have hone : 0 < 1 / (n : ℝ) := one_div_pos.mpr hnR
have hpoly : 0 ≤
(d : ℝ) ^ 2 / ((n : ℝ) ^ 2 * log n ^ 2) := by positivity
linarith
have hstrict : a / 2 *
CausalSmith.Stat.DiscreteAteMinimaxLoggap.minimaxRate n d <
CausalSmith.Stat.DiscreteAteMinimaxLoggap.oneArmMinimaxRisk n d epsilon :=
(by nlinarith : a / 2 *
CausalSmith.Stat.DiscreteAteMinimaxLoggap.minimaxRate n d <
a * CausalSmith.Stat.DiscreteAteMinimaxLoggap.minimaxRate n d).trans_le hlower
intro sourceEst hmeas _hbounded
obtain ⟨P, hP⟩ := oneArmMinimaxRisk_hard_family_of_lt
hd he0 he1 hstrict sourceEst hmeas
exact ⟨P, Set.mem_univ P, by
simpa [sqRisk,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse] using hP⟩
theorem radial_source_risk_of_parametric_lower Lemma radial_source_risk_of_parametric_lower in the paper ↗
If the sample is nonempty and the alphabet is nonempty and the overlap constant is positive and the overlap constant is below one half, the one-arm two-point subexperiment supplies the uniformly bounded source-estimator hardness interface at every positive sample size. The strict half-constant permits extraction of an actual source law from the minimax infimum and is the fallback used below the radial asymptotic cutoff.
Formal statement
hypotheses
n d :
ℕ
epsilon :
ℝ
hn :
0 < n
hd :
0 < d
he0 :
0 < epsilon
he1 :
epsilon < 1 / 2
conclusion
∀ sourceEst : (Fin n → BinObs d) → ℝ
if
Measurable sourceEst
and
UniformlyBounded sourceEst
Proof (Lean source)
lemma radial_source_risk_of_parametric_lower {n d : ℕ} {epsilon : ℝ}
(hn : 0 < n) (hd : 0 < d) (he0 : 0 < epsilon)
(he1 : epsilon < 1 / 2) :
∀ sourceEst : (Fin n → BinObs d) → ℝ,
Measurable sourceEst → UniformlyBounded sourceEst →
∃ P ∈ (Set.univ : Set
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n d epsilon)),
1 / (200 * (n : ℝ)) ≤
sqRisk
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n)
sourceEst
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1) := by
have hnR : 0 < (n : ℝ) := by exact_mod_cast hn
have hparam :=
CausalSmith.Stat.DiscreteAteMinimaxLoggap.oneArm_parametric_lower
he0 he1 n d hn hd
have hdenom : 100 * (n : ℝ) < 200 * (n : ℝ) := by nlinarith
have hstrict : 1 / (200 * (n : ℝ)) <
CausalSmith.Stat.DiscreteAteMinimaxLoggap.oneArmMinimaxRisk n d epsilon :=
(one_div_lt_one_div_of_lt (by positivity) hdenom).trans_le hparam
intro sourceEst hmeas _hbounded
obtain ⟨P, hP⟩ := oneArmMinimaxRisk_hard_family_of_lt
hd he0 he1 hstrict sourceEst hmeas
exact ⟨P, Set.mem_univ P, by
simpa [sqRisk,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse] using hP⟩
theorem exists_radial_source_transport_hardness Lemma exists_radial_source_transport_hardness in the paper ↗
If the overlap constant is positive and the overlap constant is below one half, the proved Zeng one-arm theorem yields constants, a cutoff, and the estimator-wise source hardness interface consumed by the Bernoulli channel transport in the radius converse.
Formal statement
hypotheses
epsilon :
ℝ
he0 :
0 < epsilon
he1 :
epsilon < 1 / 2
conclusion
∃ a b : ℝ,
∃ N : ℕ,
0 < a ∧
0 < b ∧
∀ n d : ℕ,
0 < n → 0 < d → N
≤ n → (d : ℝ) ≤ b * n * Real.log n → ∀ sourceEst : (Fin n → BinObs d) → ℝ, Measurable sourceEst → UniformlyBounded sourceEst → ∃ P ∈ (Set.univ : Set (CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n d epsilon)), a / 2 * CausalSmith.Stat.DiscreteAteMinimaxLoggap.minimaxRate n d ≤ sqRisk (CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n) sourceEst (CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1)
Proof (Lean source)
lemma exists_radial_source_transport_hardness {epsilon : ℝ}
(he0 : 0 < epsilon) (he1 : epsilon < 1 / 2) :
∃ a b : ℝ, ∃ N : ℕ, 0 < a ∧ 0 < b ∧
∀ n d : ℕ, 0 < n → 0 < d → N ≤ n →
(d : ℝ) ≤ b * n * Real.log n →
∀ sourceEst : (Fin n → BinObs d) → ℝ,
Measurable sourceEst → UniformlyBounded sourceEst →
∃ P ∈ (Set.univ : Set
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n d epsilon)),
a / 2 * CausalSmith.Stat.DiscreteAteMinimaxLoggap.minimaxRate n d ≤
sqRisk
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n)
sourceEst
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1) := by
obtain ⟨a, b, N, ha, hb, hlower⟩ :=
zengOneArmMinimaxLower epsilon ⟨he0, he1⟩
refine ⟨a, b, N, ha, hb, ?_⟩
intro n d hn hd hN hdRange
exact radial_source_risk_of_oneArm_lower hn hd he0 he1 ha
(hlower n d hd hN hdRange)
theorem radialSourceHardData_univ
If the sample is nonempty and the alphabet is nonempty and the overlap constant is positive and the overlap constant is below one half, for every positive sample size and alphabet, the full control-zero source class is a genuinely hard family. The positive (instance-dependent) constant is obtained by normalizing the universal one-arm parametric lower bound by the strictly positive source rate.
Formal statement
hypotheses
n d :
ℕ
epsilon :
ℝ
hn :
0 < n
hd :
0 < d
he0 :
0 < epsilon
he1 :
epsilon < 1 / 2
conclusion
RadialSourceHardData (Set.univ : Set (CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n d epsilon))
Proof (Lean source)
lemma radialSourceHardData_univ {n d : ℕ} {epsilon : ℝ}
(hn : 0 < n) (hd : 0 < d) (he0 : 0 < epsilon) (he1 : epsilon < 1 / 2) :
RadialSourceHardData
(Set.univ : Set
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n d epsilon)) := by
let r := CausalSmith.Stat.DiscreteAteMinimaxLoggap.minimaxRate n d
have hnR : 0 < (n : ℝ) := by exact_mod_cast hn
have hr : 0 < r := by
dsimp [r]
unfold CausalSmith.Stat.DiscreteAteMinimaxLoggap.minimaxRate
have hone : 0 < 1 / (n : ℝ) := one_div_pos.mpr hnR
have hpoly : 0 ≤
(d : ℝ) ^ 2 / ((n : ℝ) ^ 2 * log n ^ 2) := by positivity
linarith
let c : ℝ := 1 / (200 * (n : ℝ) * r)
have hc : 0 < c := by dsimp [c]; positivity
apply radialSourceHardData_univ_of_strict_oneArm_lower
hd he0 he1 hc
have hparam :=
CausalSmith.Stat.DiscreteAteMinimaxLoggap.oneArm_parametric_lower
he0 he1 n d hn hd
have hcr : c * r = 1 / (200 * (n : ℝ)) := by
dsimp [c]
field_simp [ne_of_gt hnR, ne_of_gt hr]
rw [show CausalSmith.Stat.DiscreteAteMinimaxLoggap.minimaxRate n d = r by rfl,
hcr]
have hdenom : 100 * (n : ℝ) < 200 * (n : ℝ) := by nlinarith
exact (one_div_lt_one_div_of_lt (by positivity) hdenom).trans_le hparam
theorem radialSourceHard_of_source_eq_univ
If the sample is nonempty and the alphabet size satisfies the stated condition and the overlap constant is positive and the overlap constant is below one half and the source parameter set has the stated form, a least-favorable handle whose radial source is the full control-zero class inherits the unconditional hard-family certificate.
Formal statement
hypotheses
n d :
ℕ
epsilon M sigma :
ℝ
H :
LeastFavorableHandle n d epsilon M sigma
hn :
0 < n
hd :
0 < H.radialCap
he0 :
0 < epsilon
he1 :
epsilon < 1 / 2
hsource :
H.radialSource = univ
conclusion
Proof (Lean source)
lemma radialSourceHard_of_source_eq_univ {n d : ℕ} {epsilon M sigma : ℝ}
(H : LeastFavorableHandle n d epsilon M sigma)
(hn : 0 < n) (hd : 0 < H.radialCap)
(he0 : 0 < epsilon) (he1 : epsilon < 1 / 2)
(hsource : H.radialSource = univ) :
RadialSourceHard H := by
rw [RadialSourceHard, hsource]
exact radialSourceHardData_univ hn hd he0 he1
Helpers.RadialMembership 2 declarations
theorem radialPaddedAffine_approximateHomogeneity
If the source alphabet embeds in the target alphabet and the overlap constant is positive and the outcome scale satisfies its stated bound and the heterogeneity radius is nonnegative and the heterogeneity radius is at most two, the contracted control-zero construction has realization-wise radius at most sigma * M, including the zero-radius endpoint.
Formal statement
hypotheses
n m d :
ℕ
epsilon M sigma :
ℝ
P :
ControlZeroLaw n m epsilon
hmd :
m ≤ d
he0 :
0 < epsilon
hM :
0 ≤ M
hs0 :
0 ≤ sigma
hs2 :
sigma ≤ 2
conclusion
ApproximateHomogeneity M sigma (affineBinaryRealLaw M (binaryPadLaw hmd (radialContractedBinaryLaw P.1 sigma hs0 hs2)))
Proof (Lean source)
lemma radialPaddedAffine_approximateHomogeneity {n m d : ℕ}
{epsilon M sigma : ℝ} (P : ControlZeroLaw n m epsilon) (hmd : m ≤ d)
(he0 : 0 < epsilon) (hM : 0 ≤ M)
(hs0 : 0 ≤ sigma) (hs2 : sigma ≤ 2) :
ApproximateHomogeneity M sigma
(affineBinaryRealLaw M
(binaryPadLaw hmd (radialContractedBinaryLaw P.1 sigma hs0 hs2))) := by
intro k hk
have hraw := radialPaddedAffine_rawAteFormula (M := M) P hmd he0 hs0 hs2
by_cases himage : ∃ r : Fin m,
(⟨r, lt_of_lt_of_le r.isLt hmd⟩ : Fin d) = k
· obtain ⟨r, rfl⟩ := himage
have hmass :
0 < CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P.1 r := by
change 0 < CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass
(binaryPadLaw hmd (radialContractedBinaryLaw P.1 sigma hs0 hs2))
⟨r, lt_of_lt_of_le r.isLt hmd⟩ at hk
rw [binaryPadLaw_cellMass_image, radialContractedBinaryLaw_cellMass] at hk
exact hk
have hpi := P.2.overlap r hmass
have htrue :
0 < CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P.1 r true := by
rw [show CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P.1 r true =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.propensity P.1 r *
CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P.1 r by
unfold CausalSmith.Stat.DiscreteAteMinimaxLoggap.propensity
field_simp]
exact mul_pos (he0.trans_le hpi.1) hmass
have hfalse :
0 < CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P.1 r false := by
have hadd :=
CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass_add_eq_cellMass P.1 r
have htrue_eq :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P.1 r true =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.propensity P.1 r *
CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P.1 r := by
unfold CausalSmith.Stat.DiscreteAteMinimaxLoggap.propensity
field_simp
rw [htrue_eq] at hadd
have hpilt :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.propensity P.1 r < 1 := by
linarith
nlinarith
rcases
CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean_mem_unitInterval
P.1 true r with ⟨heta0, heta1⟩
have hpsi0 :
0 ≤ CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1 := by
unfold CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional
exact sum_nonneg fun j _ => mul_nonneg
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass_mem_unitInterval
P.1 j).1
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean_mem_unitInterval
P.1 true j).1
have hpsi1 :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1 ≤ 1 := by
unfold CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional
calc
∑ j : Fin m, CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P.1 j *
CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P.1 true j ≤
∑ j : Fin m,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P.1 j * 1 := by
gcongr with j
· exact
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass_mem_unitInterval
P.1 j).1
· exact
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean_mem_unitInterval
P.1 true j).2
_ = 1 := by
simp [CausalSmith.Stat.DiscreteAteMinimaxLoggap.sum_cellMass_eq_one]
unfold cellDeviation cellEffect
change |M * (_ - 1 / 2) - M * (_ - 1 / 2) - _| ≤ sigma * M
rw [binaryPadLaw_outcomeMean_image, binaryPadLaw_outcomeMean_image,
radialContractedBinaryLaw_outcomeMean P.1 sigma hs0 hs2 r true htrue,
radialContractedBinaryLaw_outcomeMean P.1 sigma hs0 hs2 r false hfalse,
P.2.control_zero r, hraw]
have hscale : 0 ≤ M * sigma / 2 := by positivity
have hdiff :
|CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P.1 true r -
CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1| ≤ 1 :=
(abs_le).2 ⟨by linarith, by linarith⟩
have heq :
M * (1 / 2 + sigma / 2 *
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P.1 true r -
1 / 2) - 1 / 2) -
M * (1 / 2 + sigma / 2 * (0 - 1 / 2) - 1 / 2) -
M * sigma / 2 *
CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1 =
(M * sigma / 2) *
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P.1 true r -
CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1) := by
ring
rw [heq, abs_mul, abs_of_nonneg hscale]
calc
(M * sigma / 2) *
|CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P.1 true r -
CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1| ≤
(M * sigma / 2) * 1 := mul_le_mul_of_nonneg_left hdiff hscale
_ ≤ sigma * M := by nlinarith [mul_nonneg hM hs0]
· have hoff := binaryPadLaw_cellMass_off_image hmd
(radialContractedBinaryLaw P.1 sigma hs0 hs2) k (by simpa using himage)
change CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass
(binaryPadLaw hmd (radialContractedBinaryLaw P.1 sigma hs0 hs2)) k > 0 at hk
linarith
def radialPaddedAffineLaw_model
The concrete padded radial channel is an ambient model-class element.
Definition (Lean source)
noncomputable def radialPaddedAffineLaw_model {n m d : ℕ} (hmd : m ≤ d)
{epsilon M sigma : ℝ} (P : ControlZeroLaw n m epsilon)
(he0 : 0 < epsilon) (he1 : epsilon < 1 / 2) (hM : 1 ≤ M)
(hs0 : 0 ≤ sigma) (hs2 : sigma ≤ 2) :
ModelClass d epsilon M sigma where
law := affineBinaryRealLaw M
(binaryPadLaw hmd (radialContractedBinaryLaw P.1 sigma hs0 hs2))
epsilon_pos := he0
epsilon_lt_half := he1
M_ge_one := hM
sigma_nonneg := hs0
sigma_le_two := hs2
consistency := affineBinaryRealLaw_consistency M _
exchangeability := affineBinaryRealLaw_exchangeability M _
overlap := affineBinaryRealLaw_overlap
(binaryPadLaw_overlap hmd
(radialContractedBinaryLaw_overlap P.1 hs0 hs2 P.2.overlap))
mean_normalization := affineBinaryRealLaw_meanNormalization
(le_trans zero_le_one hM)
second_moment := affineBinaryRealLaw_secondCentralMoment
(le_trans zero_le_one hM)
homogeneity := radialPaddedAffine_approximateHomogeneity P hmd he0
(le_trans zero_le_one hM) hs0 hs2
Helpers.RadialProductDataProcessing 2 declarations This module lifts a one-record common Markov-kernel identity to the finite product experiments and records the resulting data-processing inequality.
Product total-variation contraction for the radial channel
This module lifts a one-record common Markov-kernel identity to the finite product experiments and records the resulting data-processing inequality.
theorem radialProduct_tvDist_le_of_kernel
If the first target law is produced by the channel and the second target law is produced by the channel, a common one-record Markov channel contracts total variation after taking the finite i.i.d. product experiment.
Formal statement
hypotheses
n m d :
ℕ
P0 P1 :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.DiscreteLaw m
Q0 Q1 :
RealLaw d
K :
Kernel (CausalSmith.Stat.DiscreteAteMinimaxLoggap.Obs m) (Obs d)
hQ0 :
Q0.observedLaw = K ∘ₘ CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P0
hQ1 :
Q1.observedLaw = K ∘ₘ CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P1
Proof (Lean source)
lemma radialProduct_tvDist_le_of_kernel {n m d : ℕ}
(P0 P1 : CausalSmith.Stat.DiscreteAteMinimaxLoggap.DiscreteLaw m)
(Q0 Q1 : RealLaw d)
(K : Kernel (CausalSmith.Stat.DiscreteAteMinimaxLoggap.Obs m) (Obs d))
[IsMarkovKernel K]
(hQ0 : Q0.observedLaw = K ∘ₘ
CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P0)
(hQ1 : Q1.observedLaw = K ∘ₘ
CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P1) :
tvDist (productLaw n Q0) (productLaw n Q1) ≤
tvDist
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P0 n)
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P1 n) := by
let Kn := finProductKernel n K
have hprod (P : CausalSmith.Stat.DiscreteAteMinimaxLoggap.DiscreteLaw m) :
Kn ∘ₘ CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P n =
Measure.pi (fun _ : Fin n =>
K ∘ₘ CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P) := by
exact finProductKernel_comp_pi n
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P) K
change tvDist
(Measure.pi (fun _ : Fin n => Q0.observedLaw))
(Measure.pi (fun _ : Fin n => Q1.observedLaw)) ≤
tvDist
(Measure.pi (fun _ : Fin n =>
CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P0))
(Measure.pi (fun _ : Fin n =>
CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P1))
rw [hQ0, hQ1, ← hprod P0, ← hprod P1]
exact
CausalSmith.Stat.DiscreteAteMinimaxLoggap.tvDist_bind_common_kernel_le
_ _ Kn Kn.measurable (fun _ => inferInstance)
theorem radialDataProcessingCertificate_of_kernel
If the target law is the specified transported law, the pointwise observed-law identity for a common radial kernel packages directly as the handle's product data-processing certificate.
Formal statement
hypotheses
n d :
ℕ
epsilon M sigma :
ℝ
H :
LeastFavorableHandle n d epsilon M sigma
K :
Kernel (CausalSmith.Stat.DiscreteAteMinimaxLoggap.Obs H.radialCap) (Obs d)
hQ :
∀ P ∈ H.radialSource,
(H.radialEmbedding P).observedLaw
= K ∘ₘ CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P.1
conclusion
Proof (Lean source)
lemma radialDataProcessingCertificate_of_kernel {n d : ℕ}
{epsilon M sigma : ℝ}
(H : LeastFavorableHandle n d epsilon M sigma)
(K : Kernel
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.Obs H.radialCap) (Obs d))
[IsMarkovKernel K]
(hQ : ∀ P ∈ H.radialSource,
(H.radialEmbedding P).observedLaw =
K ∘ₘ CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P.1) :
DataProcessingCertificate H := by
intro P0 hP0 P1 hP1
exact radialProduct_tvDist_le_of_kernel P0.1 P1.1
(H.radialEmbedding P0) (H.radialEmbedding P1) K
(hQ P0 hP0) (hQ P1 hP1)
Helpers.RadialRateAlgebra 2 declarations
theorem cappedRadial_sourceRate_dominates
If the sample size satisfies the stated lower bound and the alphabet is nonempty and the radial cap satisfies its stated bound and the radial cap is at most one quarter and the scalar satisfies the stated range condition, capping the source alphabet at half of b n log(en) retains, up to the explicit factor b²/16, the ambient capped polynomial component.
Formal statement
Proof (Lean source)
lemma cappedRadial_sourceRate_dominates {n d : ℕ} {b : ℝ}
(hn : 3 ≤ n) (hd : 0 < d) (hb : 0 < b) (hb4 : b ≤ 4)
(hx : 1 ≤ (b / 2) * (n : ℝ) * logEN n) :
let m := min d (max 1
(Nat.floor ((b / 2) * (n : ℝ) * logEN n)))
b ^ 2 / 16 * min 1 (polynomialComponent n d) ≤
CausalSmith.Stat.DiscreteAteMinimaxLoggap.minimaxRate n m := by
dsimp only
let L : ℝ := logEN n
let x : ℝ := (b / 2) * (n : ℝ) * L
let m : ℕ := min d (max 1 (Nat.floor x))
have hnR : 0 < (n : ℝ) := by positivity
have hlogn : 0 < log (n : ℝ) := by
exact Real.log_pos (by exact_mod_cast (show 1 < n by omega))
have hL : 0 < L := by
dsimp [L, logEN]
rw [Real.log_mul (Real.exp_ne_zero 1) hnR.ne', Real.log_exp]
positivity
have hx' : 1 ≤ x := by simpa [x, L] using hx
have hfloor_pos : 0 < Nat.floor x := Nat.floor_pos.mpr hx'
have hmax : max 1 (Nat.floor x) = Nat.floor x := by omega
have hmpos : 0 < m := by
dsimp [m]
rw [hmax]
exact (Nat.lt_min).2 ⟨hd, hfloor_pos⟩
have hfloor_half : x / 2 ≤ (Nat.floor x : ℝ) :=
half_le_natFloor_of_one_le hx'
have hratio : b / 4 * min 1 ((d : ℝ) / ((n : ℝ) * L)) ≤
(m : ℝ) / ((n : ℝ) * L) := by
by_cases hsmall : d ≤ Nat.floor x
· have hm : m = d := by simp [m, hmax, hsmall]
rw [hm]
have hbquarter : b / 4 ≤ 1 := by linarith
have hq : 0 ≤ (d : ℝ) / ((n : ℝ) * L) := by positivity
have hmin0 : 0 ≤ min 1 ((d : ℝ) / ((n : ℝ) * L)) := by positivity
calc
b / 4 * min 1 ((d : ℝ) / ((n : ℝ) * L)) ≤
1 * min 1 ((d : ℝ) / ((n : ℝ) * L)) := by gcongr
_ ≤ (d : ℝ) / ((n : ℝ) * L) := by simpa using
(min_le_right (1 : ℝ) ((d : ℝ) / ((n : ℝ) * L)))
· have hm : m = Nat.floor x := by
simp [m, hmax, Nat.le_of_not_ge hsmall]
have hden : 0 < (n : ℝ) * L := mul_pos hnR hL
have hcap : b / 4 ≤ (m : ℝ) / ((n : ℝ) * L) := by
rw [hm]
apply (le_div_iff₀ hden).2
dsimp [x] at hfloor_half
nlinarith
have hmin : min 1 ((d : ℝ) / ((n : ℝ) * L)) ≤ 1 := min_le_left _ _
exact (mul_le_mul_of_nonneg_left hmin (by positivity : 0 ≤ b / 4)).trans
(by simpa using hcap)
have hsquare : b ^ 2 / 16 *
min 1 ((d : ℝ) ^ 2 / ((n : ℝ) ^ 2 * L ^ 2)) ≤
(m : ℝ) ^ 2 / ((n : ℝ) ^ 2 * L ^ 2) := by
have hq0 : 0 ≤ (d : ℝ) / ((n : ℝ) * L) := by positivity
have hm0 : 0 ≤ (m : ℝ) / ((n : ℝ) * L) := by positivity
have hleft0 : 0 ≤ b / 4 * min 1 ((d : ℝ) / ((n : ℝ) * L)) := by
positivity
have hsq := (sq_le_sq₀ hleft0 hm0).2 hratio
have hminsq : (min 1 ((d : ℝ) / ((n : ℝ) * L))) ^ 2 =
min 1 ((d : ℝ) ^ 2 / ((n : ℝ) ^ 2 * L ^ 2)) := by
have hquot : ((d : ℝ) / ((n : ℝ) * L)) ^ 2 =
(d : ℝ) ^ 2 / ((n : ℝ) ^ 2 * L ^ 2) := by ring
rw [← hquot]
by_cases hq1 : (d : ℝ) / ((n : ℝ) * L) ≤ 1
· rw [min_eq_right hq1, min_eq_right]
nlinarith
· have hq1' : 1 ≤ (d : ℝ) / ((n : ℝ) * L) := le_of_not_ge hq1
have hsq1 : (1 : ℝ) ≤ ((d : ℝ) / ((n : ℝ) * L)) ^ 2 := by
simpa using (sq_le_sq₀ zero_le_one hq0).2 hq1'
rw [min_eq_left hq1', min_eq_left hsq1]
norm_num
rw [mul_pow, hminsq] at hsq
calc
b ^ 2 / 16 * min 1 ((d : ℝ) ^ 2 / ((n : ℝ) ^ 2 * L ^ 2)) =
(b / 4) ^ 2 * min 1 ((d : ℝ) ^ 2 / ((n : ℝ) ^ 2 * L ^ 2)) := by ring
_ ≤ ((m : ℝ) / ((n : ℝ) * L)) ^ 2 := hsq
_ = (m : ℝ) ^ 2 / ((n : ℝ) ^ 2 * L ^ 2) := by ring
have hlog_le : log (n : ℝ) ≤ L := by
dsimp [L, logEN]
rw [Real.log_mul (Real.exp_ne_zero 1) hnR.ne', Real.log_exp]
linarith
have hden_le : (n : ℝ) ^ 2 * log n ^ 2 ≤
(n : ℝ) ^ 2 * L ^ 2 := by
gcongr
have hfrac : (m : ℝ) ^ 2 / ((n : ℝ) ^ 2 * L ^ 2) ≤
(m : ℝ) ^ 2 / ((n : ℝ) ^ 2 * log n ^ 2) := by
exact div_le_div_of_nonneg_left (sq_nonneg (m : ℝ))
(by positivity) hden_le
unfold polynomialComponent
unfold CausalSmith.Stat.DiscreteAteMinimaxLoggap.minimaxRate
change b ^ 2 / 16 * min 1
((d : ℝ) ^ 2 / ((n : ℝ) ^ 2 * L ^ 2)) ≤ _
exact hsquare.trans (hfrac.trans (le_add_of_nonneg_left (by positivity)))
If the sample size satisfies the stated lower bound and the alphabet is nonempty and the transport scale satisfies the stated condition and the radial cap satisfies its stated bound and the radial cap is at most one quarter and the scalar satisfies the stated range condition, after the Bernoulli channel scales the source target by M σ / 2, the capped source lower bound supplies the ambient radial term with the explicit constant a b² / 128.
Formal statement
hypotheses
n d :
ℕ
a b M sigma :
ℝ
hn :
3 ≤ n
hd :
0 < d
ha :
0 ≤ a
hb :
0 < b
hb4 :
b ≤ 4
hx :
1 ≤ (b / 2) * (n : ℝ) * logEN n
Proof (Lean source)
lemma cappedRadial_transportScale {n d : ℕ} {a b M sigma : ℝ}
(hn : 3 ≤ n) (hd : 0 < d) (ha : 0 ≤ a) (hb : 0 < b) (hb4 : b ≤ 4)
(hx : 1 ≤ (b / 2) * (n : ℝ) * logEN n) :
let m := min d (max 1
(floor ((b / 2) * (n : ℝ) * logEN n)))
(a * b ^ 2 / 128) * M ^ 2 * sigma ^ 2 *
min 1 (polynomialComponent n d) ≤
(M * sigma / 2) ^ 2 *
(a / 2 * CausalSmith.Stat.DiscreteAteMinimaxLoggap.minimaxRate n m) := by
dsimp only
have hrate := cappedRadial_sourceRate_dominates hn hd hb hb4 hx
have hnonneg : 0 ≤ (M * sigma / 2) ^ 2 * (a / 2) := by positivity
have hmul := mul_le_mul_of_nonneg_left hrate hnonneg
nlinarith [hmul]
Helpers.RadialTarget 4 declarations This file proves the exact conditional-mean and ATE scaling identities used by the concrete least-favorable radial handle.
Target scaling under the radial Bernoulli channel
This file proves the exact conditional-mean and ATE scaling identities used by the concrete least-favorable radial handle.
theorem radialContractedBinaryLaw_outcomeMean
If the heterogeneity radius is nonnegative and the heterogeneity radius is at most two and the transport scale satisfies the stated condition, on every positive source arm, the common Bernoulli contraction sends the conditional response mean through its declared affine channel.
Formal statement
hypotheses
d :
ℕ
P :
BinLaw d
sigma :
ℝ
hs0 :
0 ≤ sigma
hs2 :
sigma ≤ 2
k :
Fin d
a :
ha :
0 < CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P k a
conclusion
CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean (radialContractedBinaryLaw P sigma hs0 hs2) a k
= 1 / 2
+ sigma / 2 * (CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P a k - 1 / 2)
Proof (Lean source)
lemma radialContractedBinaryLaw_outcomeMean {d : ℕ} (P : BinLaw d) (sigma : ℝ)
(hs0 : 0 ≤ sigma) (hs2 : sigma ≤ 2) (k : Fin d) (a : Bool)
(ha : 0 < CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P k a) :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean
(radialContractedBinaryLaw P sigma hs0 hs2) a k =
1 / 2 + sigma / 2 *
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P a k - 1 / 2) := by
have hane : CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P k a ≠ 0 :=
ne_of_gt ha
have hy :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass P k a true =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P a k *
CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P k a := by
rw [CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean]
field_simp
have harm :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P k a =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass P k a false +
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass P k a true := by
simp [CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass]
ring
unfold CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean
rw [radialContractedBinaryLaw_jointMass,
radialContractedBinaryLaw_armMass, Fintype.sum_bool]
unfold bernoulliContractionSuccess
simp only [Bool.false_eq_true, if_false, if_true]
rw [hy]
have hfalse :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.jointMass P k a false =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P k a -
CausalSmith.Stat.DiscreteAteMinimaxLoggap.outcomeMean P a k *
CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P k a := by
linarith [harm, hy]
rw [hfalse]
field_simp
ring
theorem radialContractedBinaryLaw_ateFunctional_controlZero
If the overlap constant is positive and the heterogeneity radius is nonnegative and the heterogeneity radius is at most two, on the control-zero source family, contraction multiplies the binary ATE by exactly sigma / 2, including the zero-radius endpoint.
Formal statement
hypotheses
n d :
ℕ
epsilon sigma :
ℝ
P :
ControlZeroLaw n d epsilon
he0 :
0 < epsilon
hs0 :
0 ≤ sigma
hs2 :
sigma ≤ 2
conclusion
CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional (radialContractedBinaryLaw P.1 sigma hs0 hs2)
= sigma / 2 * CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1
Proof (Lean source)
lemma radialContractedBinaryLaw_ateFunctional_controlZero {n d : ℕ}
{epsilon sigma : ℝ} (P : ControlZeroLaw n d epsilon)
(he0 : 0 < epsilon) (hs0 : 0 ≤ sigma) (hs2 : sigma ≤ 2) :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional
(radialContractedBinaryLaw P.1 sigma hs0 hs2) =
sigma / 2 *
CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1 := by
rw [CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional_eq_weighted_regression
_ (radialContractedBinaryLaw_overlap P.1 hs0 hs2 P.2.overlap)]
unfold CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional
rw [Finset.mul_sum]
apply Finset.sum_congr rfl
intro k _hk
by_cases hk : 0 < CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P.1 k
· have hpi := P.2.overlap k hk
have hmass_ne : CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P.1 k ≠ 0 :=
ne_of_gt hk
have htrue :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P.1 k true =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.propensity P.1 k *
CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P.1 k := by
rw [CausalSmith.Stat.DiscreteAteMinimaxLoggap.propensity]
field_simp
have hmass :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P.1 k =
CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P.1 k false +
CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P.1 k true := by
simp [CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass]
ring
have hprop_pos :
0 < CausalSmith.Stat.DiscreteAteMinimaxLoggap.propensity P.1 k :=
lt_of_lt_of_le he0 hpi.1
have hprop_lt :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.propensity P.1 k < 1 := by
linarith
have htrue_pos :
0 < CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P.1 k true := by
rw [htrue]
exact mul_pos hprop_pos hk
have hfalse :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P.1 k false =
(1 - CausalSmith.Stat.DiscreteAteMinimaxLoggap.propensity P.1 k) *
CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P.1 k := by
rw [htrue] at hmass
linarith
have hfalse_pos :
0 < CausalSmith.Stat.DiscreteAteMinimaxLoggap.armMass P.1 k false := by
rw [hfalse]
exact mul_pos (sub_pos.mpr hprop_lt) hk
rw [radialContractedBinaryLaw_cellMass,
radialContractedBinaryLaw_outcomeMean P.1 sigma hs0 hs2 k true htrue_pos,
radialContractedBinaryLaw_outcomeMean P.1 sigma hs0 hs2 k false hfalse_pos,
P.2.control_zero k]
ring
· have hk0 : CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass P.1 k = 0 :=
le_antisymm (le_of_not_gt hk)
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.cellMass_mem_unitInterval P.1 k).1
rw [radialContractedBinaryLaw_cellMass, hk0]
ring
theorem radialPaddedAffine_rawAteFormula
If the source alphabet embeds in the target alphabet and the overlap constant is positive and the heterogeneity radius is nonnegative and the heterogeneity radius is at most two, zero-mass padding and affine outcome scaling preserve the radial target identity, giving the exact slope used by Markov-kernel risk transport.
Formal statement
hypotheses
n m d :
ℕ
epsilon M sigma :
ℝ
P :
ControlZeroLaw n m epsilon
hmd :
m ≤ d
he0 :
0 < epsilon
hs0 :
0 ≤ sigma
hs2 :
sigma ≤ 2
conclusion
rawAteFormula (affineBinaryRealLaw M (binaryPadLaw hmd (radialContractedBinaryLaw P.1 sigma hs0 hs2)))
= (M * sigma / 2) * CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1
Proof (Lean source)
lemma radialPaddedAffine_rawAteFormula {n m d : ℕ} {epsilon M sigma : ℝ}
(P : ControlZeroLaw n m epsilon) (hmd : m ≤ d)
(he0 : 0 < epsilon) (hs0 : 0 ≤ sigma) (hs2 : sigma ≤ 2) :
rawAteFormula
(affineBinaryRealLaw M
(binaryPadLaw hmd (radialContractedBinaryLaw P.1 sigma hs0 hs2))) =
(M * sigma / 2) *
CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1 := by
rw [rawAteFormula_eq_mul_binaryAte_of_embedding
(affineBinaryRealLaw_embedding M _) (binaryPadLaw_overlap hmd
(radialContractedBinaryLaw_overlap P.1 hs0 hs2 P.2.overlap)),
binaryPadLaw_ateFunctional hmd
(radialContractedBinaryLaw_overlap P.1 hs0 hs2 P.2.overlap),
radialContractedBinaryLaw_ateFunctional_controlZero P he0 hs0 hs2]
ring
theorem targetSeparationCertificate_of_pointwise_targets
If the transport scale satisfies the stated condition and the radial target formula holds and the exact risk-transfer identity holds, pointwise affine target formulas for the radial and exact source families imply the pairwise separation certificate by subtraction.
Formal statement
hypotheses
n d :
ℕ
epsilon M sigma a b :
ℝ
H :
LeastFavorableHandle n d epsilon M sigma
ha :
a = M * H.lambda
hradial :
∀ P ∈ H.radialSource,
rawAteFormula (H.radialEmbedding P)
= a * CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1 + b
hexact :
∀ P ∈ H.exactSource,
rawAteFormula (H.exactEmbedding P)
= M * CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1
conclusion
Proof (Lean source)
lemma targetSeparationCertificate_of_pointwise_targets {n d : ℕ}
{epsilon M sigma a b : ℝ}
(H : LeastFavorableHandle n d epsilon M sigma)
(ha : a = M * H.lambda)
(hradial : ∀ P ∈ H.radialSource,
rawAteFormula (H.radialEmbedding P) =
a * CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1 + b)
(hexact : ∀ P ∈ H.exactSource,
rawAteFormula (H.exactEmbedding P) =
M * CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1) :
TargetSeparationCertificate H := by
constructor
· intro P0 hP0 P1 hP1
rw [hradial P1 hP1, hradial P0 hP0, ha]
ring
· intro P0 hP0 P1 hP1
rw [hexact P1 hP1, hexact P0 hP0]
ring
Helpers.RateAlgebra 6 declarations
The parametric-plus-exact-homogeneity benchmark b.
Definition (Lean source)
noncomputable def baseRate (n d : ℕ) : ℝ :=
1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2
theorem logEN_sq_le_nine_mul
If the sample is nonempty, the squared logarithmic scale is bounded by nine times the sample size.
Formal statement
Proof (Lean source)
lemma logEN_sq_le_nine_mul (n : ℕ) (hn : 0 < n) :
logEN n ^ 2 ≤ 9 * (n : ℝ) := by
have hn_real : 0 < (n : ℝ) := by exact_mod_cast hn
have hn_one : (1 : ℝ) ≤ n := by exact_mod_cast hn
have hlog : log (n : ℝ) ≤ 2 * sqrt (n : ℝ) := by
calc
log (n : ℝ) ≤ (n : ℝ) ^ (1 / 2 : ℝ) / (1 / 2 : ℝ) :=
Real.log_le_rpow_div (le_of_lt hn_real) (by norm_num)
_ = 2 * sqrt (n : ℝ) := by
rw [← Real.sqrt_eq_rpow]
ring
have hsqrt_one : 1 ≤ sqrt (n : ℝ) := by
simpa using Real.sqrt_le_sqrt hn_one
have hsqrt_nonneg : 0 ≤ sqrt (n : ℝ) := Real.sqrt_nonneg _
have hsqrt_sq : sqrt (n : ℝ) ^ 2 = (n : ℝ) :=
Real.sq_sqrt (le_of_lt hn_real)
have hlog_nonneg : 0 ≤ log (n : ℝ) := Real.log_nonneg hn_one
rw [logEN, Real.log_mul (Real.exp_ne_zero 1) hn_real.ne', Real.log_exp]
nlinarith
At radii zero and two, the all-alphabet upper and lower rate expressions are uniformly comparable, radius two gives the unrestricted model class, and the radius-two frontier is exactly the unrestricted endpoint formula.
Formal statement
conclusion
(∃ c C : ℝ, 0 < c ∧ c ≤ C ∧ ∀ n d : ℕ, 0 < n → 0 < d → c * (1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)) ≤ frontierRate n d 0 ∧ frontierRate n d 0 ≤ C * (1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)) ∧ c * (1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)) ≤ converseRate n d 0 ∧ converseRate n d 0 ≤ C * (1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)) ∧ c * (1 / (n : ℝ) + min 1 (polynomialComponent n d)) ≤ frontierRate n d 2 ∧ frontierRate n d 2 ≤ C * (1 / (n : ℝ) + min 1 (polynomialComponent n d)) ∧ c * (1 / (n : ℝ) + min 1 (polynomialComponent n d)) ≤ converseRate n d 2 ∧ converseRate n d 2 ≤ C * (1 / (n : ℝ) + min 1 (polynomialComponent n d))) ∧
(∀ d : ℕ, ∀ epsilon M : ℝ, (∀ P : UnrestrictedClass d epsilon M, ∃ Q : ModelClass d epsilon M 2, Q.law = P.law) ∧ (∀ Q : ModelClass d epsilon M 2, ∃ P : UnrestrictedClass d epsilon M, P.law = Q.law)) ∧
Proof (Lean source)
theorem endpoint_reductions_all_d :
(∃ c C : ℝ, 0 < c ∧ c ≤ C ∧
∀ n d : ℕ, 0 < n → 0 < d →
c * (1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)) ≤
frontierRate n d 0 ∧
frontierRate n d 0 ≤
C * (1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)) ∧
c * (1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)) ≤
converseRate n d 0 ∧
converseRate n d 0 ≤
C * (1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)) ∧
c * (1 / (n : ℝ) + min 1 (polynomialComponent n d)) ≤
frontierRate n d 2 ∧
frontierRate n d 2 ≤
C * (1 / (n : ℝ) + min 1 (polynomialComponent n d)) ∧
c * (1 / (n : ℝ) + min 1 (polynomialComponent n d)) ≤
converseRate n d 2 ∧
converseRate n d 2 ≤
C * (1 / (n : ℝ) + min 1 (polynomialComponent n d))) ∧
(∀ d : ℕ, ∀ epsilon M : ℝ,
(∀ P : UnrestrictedClass d epsilon M,
∃ Q : ModelClass d epsilon M 2, Q.law = P.law) ∧
(∀ Q : ModelClass d epsilon M 2,
∃ P : UnrestrictedClass d epsilon M, P.law = Q.law)) ∧
(∀ n d : ℕ, 0 < n → 0 < d →
frontierRate n d 2 = 1 / (n : ℝ) + min 1 (polynomialComponent n d)) := by
have hfront_two : ∀ n d : ℕ, 0 < n → 0 < d →
frontierRate n d 2 = 1 / (n : ℝ) + min 1 (polynomialComponent n d) := by
intro n d hn hd
have hn_real : 0 < (n : ℝ) := by exact_mod_cast hn
have hd_real : 0 < (d : ℝ) := by exact_mod_cast hd
have hn_sq : 0 < (n : ℝ) ^ 2 := sq_pos_of_pos hn_real
have ht : 0 ≤ (d : ℝ) / (n : ℝ) ^ 2 :=
div_nonneg (le_of_lt hd_real) (le_of_lt hn_sq)
have hcollision_two : 1 ≤ collisionComponent n d 2 := by
unfold collisionComponent
nlinarith
unfold frontierRate
rw [← min_assoc, min_eq_left (le_trans (min_le_left _ _) hcollision_two)]
refine ⟨⟨1 / 10, 10, by norm_num, by norm_num, ?_⟩,
fun d epsilon M => (scale_sanity (d := d) (epsilon := epsilon) (M := M)).2,
hfront_two⟩
intro n d hn hd
have hn_real : 0 < (n : ℝ) := by exact_mod_cast hn
have hd_real : 0 < (d : ℝ) := by exact_mod_cast hd
have hn_sq : 0 < (n : ℝ) ^ 2 := sq_pos_of_pos hn_real
have hlog_one : 1 ≤ logEN n := by
rw [logEN, Real.log_mul (Real.exp_ne_zero 1) hn_real.ne', Real.log_exp]
exact le_add_of_nonneg_right (Real.log_nonneg (by exact_mod_cast hn))
have hlog_sq : 0 < logEN n ^ 2 := sq_pos_of_pos (lt_of_lt_of_le zero_lt_one hlog_one)
have ht : 0 ≤ (d : ℝ) / (n : ℝ) ^ 2 := div_nonneg (le_of_lt hd_real) (le_of_lt hn_sq)
have hu : 0 ≤ polynomialComponent n d := by
unfold polynomialComponent
positivity
have hfront_zero_upper :
frontierRate n d 0 ≤
1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2) := by
unfold frontierRate collisionComponent
gcongr
simpa using min_le_right (polynomialComponent n d)
((d : ℝ) / (n : ℝ) ^ 2)
have hconverse_zero : converseRate n d 0 =
1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2) := by
simp [converseRate]
have hfront_two := hfront_two n d hn hd
have htarget_zero_nonneg :
0 ≤ 1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2) := by positivity
have htarget_two_nonneg :
0 ≤ 1 / (n : ℝ) + min 1 (polynomialComponent n d) := by positivity
have hone_over_nonneg : 0 ≤ 1 / (n : ℝ) := by positivity
by_cases hlarge : logEN n ^ 2 ≤ (d : ℝ)
· have htu : (d : ℝ) / (n : ℝ) ^ 2 ≤ polynomialComponent n d := by
unfold polynomialComponent
rw [div_le_div_iff₀ hn_sq (mul_pos hn_sq hlog_sq)]
have hmul := mul_le_mul_of_nonneg_left hlarge
(mul_nonneg (le_of_lt hd_real) (le_of_lt hn_sq))
nlinarith
have hfront_zero : frontierRate n d 0 =
1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2) := by
unfold frontierRate collisionComponent
have hz : (0 : ℝ) ^ 2 = 0 := by norm_num
rw [hz, zero_add]
change 1 / (n : ℝ) +
min 1 (min (polynomialComponent n d) ((d : ℝ) / (n : ℝ) ^ 2)) = _
rw [min_eq_right htu]
have htmin_le_u : min 1 ((d : ℝ) / (n : ℝ) ^ 2) ≤
min 1 (polynomialComponent n d) := min_le_min_left 1 htu
have hconv_two_lower :
1 / (n : ℝ) + min 1 (polynomialComponent n d) ≤ converseRate n d 2 := by
unfold converseRate
nlinarith [le_min zero_le_one ht, le_min zero_le_one hu]
have hconv_two_upper : converseRate n d 2 ≤
5 * (1 / (n : ℝ) + min 1 (polynomialComponent n d)) := by
unfold converseRate
nlinarith [htmin_le_u, le_min zero_le_one hu]
rw [hfront_zero, hconverse_zero, hfront_two]
refine ⟨by nlinarith, by nlinarith, by nlinarith, by nlinarith,
by nlinarith, by nlinarith, by nlinarith, ?_⟩
nlinarith
· have hd_log : (d : ℝ) < logEN n ^ 2 := lt_of_not_ge hlarge
have hd_nine : (d : ℝ) ≤ 9 * (n : ℝ) :=
le_trans (le_of_lt hd_log) (logEN_sq_le_nine_mul n hn)
have ht_le : (d : ℝ) / (n : ℝ) ^ 2 ≤ 9 * (1 / (n : ℝ)) := by
calc
(d : ℝ) / (n : ℝ) ^ 2 ≤ (9 * (n : ℝ)) / (n : ℝ) ^ 2 := by
gcongr
_ = 9 * (1 / (n : ℝ)) := by field_simp
have hfront_zero_lower : 1 / (n : ℝ) ≤ frontierRate n d 0 := by
unfold frontierRate collisionComponent
have hz : (0 : ℝ) ^ 2 = 0 := by norm_num
rw [hz, zero_add]
change 1 / (n : ℝ) ≤ 1 / (n : ℝ) +
min 1 (min (polynomialComponent n d) ((d : ℝ) / (n : ℝ) ^ 2))
exact le_add_of_nonneg_right (le_min zero_le_one (le_min hu ht))
have htarget_zero_upper :
1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2) ≤
10 * (1 / (n : ℝ)) := by
calc
1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2) ≤
1 / (n : ℝ) + 9 * (1 / (n : ℝ)) :=
add_le_add (le_refl _) (min_le_of_right_le ht_le)
_ = 10 * (1 / (n : ℝ)) := by ring
have htmin_le : min 1 ((d : ℝ) / (n : ℝ) ^ 2) ≤ 9 * (1 / (n : ℝ)) :=
min_le_of_right_le ht_le
have hconv_two_lower :
1 / (n : ℝ) + min 1 (polynomialComponent n d) ≤ converseRate n d 2 := by
unfold converseRate
nlinarith [le_min zero_le_one ht, le_min zero_le_one hu]
have hconv_two_upper : converseRate n d 2 ≤
10 * (1 / (n : ℝ) + min 1 (polynomialComponent n d)) := by
unfold converseRate
nlinarith [htmin_le, le_min zero_le_one hu]
rw [hconverse_zero, hfront_two]
refine ⟨by nlinarith, by nlinarith, by nlinarith, by nlinarith,
by nlinarith, by nlinarith, by nlinarith, hconv_two_upper⟩
Restricted-range exact-homogeneity reduction together with the unrestricted radius-two identity and converse comparison for every positive alphabet size.
Formal statement
conclusion
∀ epsilon : ℝ
if
0 < epsilon
and
epsilon < 1 / 2
then
∃ c_epsilon c C : ℝ,
0 < c_epsilon ∧
c_epsilon ≤ 1 ∧
0 < c ∧
c ≤ C ∧
( ∀ n d : ℕ, 0 < n → 0 < d → (d : ℝ) ≤ c_epsilon * (n : ℝ) ^ 2 / logEN n → c * (1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2) ≤ frontierRate n d 0 ∧ frontierRate n d 0 ≤ C * (1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2) ∧ c * (1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2) ≤ converseRate n d 0 ∧ converseRate n d 0 ≤ C * (1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2) ∧ c * (1 / (n : ℝ) + min 1 (polynomialComponent n d)) ≤ frontierRate n d 2 ∧ frontierRate n d 2 ≤ C * (1 / (n : ℝ) + min 1 (polynomialComponent n d)) ∧ c * (1 / (n : ℝ) + min 1 (polynomialComponent n d)) ≤ converseRate n d 2 ∧ converseRate n d 2 ≤ C * (1 / (n : ℝ) + min 1 (polynomialComponent n d))) ∧
(∀ d : ℕ, ∀ M : ℝ, (∀ P : UnrestrictedClass d epsilon M, ∃ Q : ModelClass d epsilon M 2, Q.law = P.law) ∧ (∀ Q : ModelClass d epsilon M 2, ∃ P : UnrestrictedClass d epsilon M, P.law = Q.law)) ∧
(∀ n d : ℕ, 0 < n → 0 < d → frontierRate n d 2 = 1 / (n : ℝ) + min 1 (polynomialComponent n d) ∧ c * (1 / (n : ℝ) + min 1 (polynomialComponent n d)) ≤ converseRate n d 2 ∧ converseRate n d 2 ≤ C * (1 / (n : ℝ) + min 1 (polynomialComponent n d)))
Proof (Lean source)
theorem endpoint_reductions :
∀ epsilon : ℝ, 0 < epsilon → epsilon < 1 / 2 →
∃ c_epsilon c C : ℝ,
0 < c_epsilon ∧ c_epsilon ≤ 1 ∧
0 < c ∧ c ≤ C ∧
(
∀ n d : ℕ, 0 < n → 0 < d →
(d : ℝ) ≤ c_epsilon * (n : ℝ) ^ 2 / logEN n →
c * (1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2) ≤
frontierRate n d 0 ∧
frontierRate n d 0 ≤
C * (1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2) ∧
c * (1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2) ≤
converseRate n d 0 ∧
converseRate n d 0 ≤
C * (1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2) ∧
c * (1 / (n : ℝ) + min 1 (polynomialComponent n d)) ≤
frontierRate n d 2 ∧
frontierRate n d 2 ≤
C * (1 / (n : ℝ) + min 1 (polynomialComponent n d)) ∧
c * (1 / (n : ℝ) + min 1 (polynomialComponent n d)) ≤
converseRate n d 2 ∧
converseRate n d 2 ≤
C * (1 / (n : ℝ) + min 1 (polynomialComponent n d))) ∧
(∀ d : ℕ, ∀ M : ℝ,
(∀ P : UnrestrictedClass d epsilon M,
∃ Q : ModelClass d epsilon M 2, Q.law = P.law) ∧
(∀ Q : ModelClass d epsilon M 2,
∃ P : UnrestrictedClass d epsilon M, P.law = Q.law)) ∧
(∀ n d : ℕ, 0 < n → 0 < d →
frontierRate n d 2 =
1 / (n : ℝ) + min 1 (polynomialComponent n d) ∧
c * (1 / (n : ℝ) + min 1 (polynomialComponent n d)) ≤
converseRate n d 2 ∧
converseRate n d 2 ≤
C * (1 / (n : ℝ) + min 1 (polynomialComponent n d))) := by
intro epsilon _ _
obtain ⟨⟨c, C, hc, hcC, hall⟩, hclasses, hfront_two⟩ :=
endpoint_reductions_all_d
refine ⟨1, c, C, zero_lt_one, le_rfl, hc, hcC, ?_, ?_, ?_⟩
· intro n d hn hd hd_range
have hn_real : 0 < (n : ℝ) := by exact_mod_cast hn
have hn_one : (1 : ℝ) ≤ n := by
exact_mod_cast (Nat.one_le_iff_ne_zero.mpr hn.ne')
have hlog : 1 ≤ logEN n := by
rw [logEN, Real.log_mul (Real.exp_ne_zero 1) (ne_of_gt hn_real), Real.log_exp]
exact le_add_of_nonneg_right (Real.log_nonneg hn_one)
have hcap : (d : ℝ) ≤ (n : ℝ) ^ 2 := calc
(d : ℝ) ≤ 1 * (n : ℝ) ^ 2 / logEN n := hd_range
_ ≤ 1 * (n : ℝ) ^ 2 :=
div_le_self (by positivity) hlog
_ = (n : ℝ) ^ 2 := one_mul _
have hratio : (d : ℝ) / (n : ℝ) ^ 2 ≤ 1 := by
rw [div_le_one (sq_pos_of_pos hn_real)]
exact hcap
simpa [min_eq_right hratio] using hall n d hn hd
· intro d M
exact hclasses d epsilon M
· intro n d hn hd
have h := hall n d hn hd
exact ⟨hfront_two n d hn hd, h.2.2.2.2.2.2.1, h.2.2.2.2.2.2.2⟩
A sequence lies in the residual shrinking-radius wedge.
Definition (Lean source)
def ResidualWedge (dseq : ℕ → ℕ) (sseq : ℕ → ℝ) : Prop :=
(∀ᶠ t in atTop, 1 ≤ dseq t ∧ 0 ≤ sseq t ∧ sseq t ≤ 2) ∧
Tendsto (fun n => baseRate n (dseq n) / polynomialComponent n (dseq n))
atTop (nhds 0) ∧
Tendsto (fun n => polynomialComponent n (dseq n)) atTop (nhds 0) ∧
Tendsto (fun n => baseRate n (dseq n) / (sseq n) ^ 2) atTop (nhds 0) ∧
Tendsto (fun n => (sseq n) ^ 2) atTop (nhds 0)
Descriptive, never-proved open question. Put L = log(en), b = n⁻¹ + d/n², and u = d²/(n²L²). In the residual wedge b ≪ u ≪ 1 and b ≪ sigma² ≪ 1, ask for either a realization-wise radius-constrained paired-cell moment-matching fuzzy experiment in the model class with ATE separation of order M * min(sigma, d/(nL)) and sample-mixture total variation bounded away from one, or an explicit total estimator with risk strictly smaller than the selector benchmark M² * (b + min(u,sigma²)). The ideal desideratum is an estimator ideally attaining the proved product benchmark M² * (b + sigma²*u). The existing hypothesis-independent channel yields only product separation M * sigma * min(1,d/(nL)) and therefore does not answer the question.
Definition (Lean source)
def ShrinkingRadiusFrontierQuestion : String :=
"Put L := log(en), b := n^-1 + d/n^2, and u := d^2/(n^2 L^2). " ++
"In the residual wedge b << u << 1 and b << sigma^2 << 1, equivalently " ++
"up to boundary constants sqrt(n)L << d << nL and sigma^2 >> b, ask " ++
"whether one can construct a realization-wise radius-constrained paired-cell " ++
"moment-matching fuzzy experiment in P_{d,epsilon,M,sigma} with ATE " ++
"separation of order M*min(sigma,d/(nL)) and sample-mixture total variation bounded " ++
"away from one, or instead construct an explicit total estimator whose risk " ++
"is strictly smaller in order than the selector benchmark " ++
"M^2*(b+min(u,sigma^2)), ideally attaining the proved product benchmark " ++
"M^2*(b+sigma^2*u). The existing hypothesis-independent channel gives only " ++
"the product separation M*sigma*min(1,d/(nL)) and does not answer this question."
T_AteIdentification 19 declarations
theorem fullObs_measurable_x
the full-data covariate coordinate is measurable.
Formal statement
Proof (Lean source)
lemma fullObs_measurable_x {d : ℕ} : Measurable (FullObs.x : FullObs d → Fin d) := by
exact (show Measurable (fun z : FullObs d => (z.x, z.a, z.y0, z.y1, z.y)) by
rw [measurable_iff_comap_le]
rfl).fst
theorem fullObs_measurable_a
the full-data treatment coordinate is measurable.
Formal statement
Proof (Lean source)
lemma fullObs_measurable_a {d : ℕ} : Measurable (FullObs.a : FullObs d → Bool) := by
exact (show Measurable (fun z : FullObs d => (z.x, z.a, z.y0, z.y1, z.y)) by
rw [measurable_iff_comap_le]
rfl).snd.fst
theorem fullObs_measurable_y0
the control potential-outcome coordinate is measurable.
Formal statement
Proof (Lean source)
lemma fullObs_measurable_y0 {d : ℕ} : Measurable (FullObs.y0 : FullObs d → ℝ) := by
exact (show Measurable (fun z : FullObs d => (z.x, z.a, z.y0, z.y1, z.y)) by
rw [measurable_iff_comap_le]
rfl).snd.snd.fst
theorem fullObs_measurable_y1
the treated potential-outcome coordinate is measurable.
Formal statement
Proof (Lean source)
lemma fullObs_measurable_y1 {d : ℕ} : Measurable (FullObs.y1 : FullObs d → ℝ) := by
exact (show Measurable (fun z : FullObs d => (z.x, z.a, z.y0, z.y1, z.y)) by
rw [measurable_iff_comap_le]
rfl).snd.snd.snd.fst
theorem fullObs_measurable_y
the full-data observed-outcome coordinate is measurable.
Formal statement
Proof (Lean source)
lemma fullObs_measurable_y {d : ℕ} : Measurable (FullObs.y : FullObs d → ℝ) := by
exact (show Measurable (fun z : FullObs d => (z.x, z.a, z.y0, z.y1, z.y)) by
rw [measurable_iff_comap_le]
rfl).snd.snd.snd.snd
theorem obs_measurable_x
the observed covariate coordinate is measurable.
Formal statement
Proof (Lean source)
lemma obs_measurable_x {d : ℕ} : Measurable (Obs.x : Obs d → Fin d) := by
exact (show Measurable (fun o : Obs d => (o.x, o.a, o.y)) by
rw [measurable_iff_comap_le]
rfl).fst
theorem obs_measurable_a
the observed treatment coordinate is measurable.
Formal statement
Proof (Lean source)
lemma obs_measurable_a {d : ℕ} : Measurable (Obs.a : Obs d → Bool) := by
exact (show Measurable (fun o : Obs d => (o.x, o.a, o.y)) by
rw [measurable_iff_comap_le]
rfl).snd.fst
theorem obs_measurable_y
the observed outcome coordinate is measurable.
Formal statement
Proof (Lean source)
lemma obs_measurable_y {d : ℕ} : Measurable (Obs.y : Obs d → ℝ) := by
exact (show Measurable (fun o : Obs d => (o.x, o.a, o.y)) by
rw [measurable_iff_comap_le]
rfl).snd.snd
theorem fullObs_measurable_potential
the selected potential-outcome coordinate is measurable.
Formal statement
Proof (Lean source)
lemma fullObs_measurable_potential {d : ℕ} (a : Bool) :
Measurable (fun z : FullObs d => if a then z.y1 else z.y0) := by
cases a <;> simp [fullObs_measurable_y0, fullObs_measurable_y1]
theorem fullObs_observed_measurable
the projection from full data to observed data is measurable.
Formal statement
Proof (Lean source)
lemma fullObs_observed_measurable {d : ℕ} :
Measurable (FullObs.observed : FullObs d → Obs d) := by
have htuple : Measurable (fun z : FullObs d => (z.x, z.a, z.y)) :=
fullObs_measurable_x.prodMk
(fullObs_measurable_a.prodMk fullObs_measurable_y)
simpa [instMeasurableSpaceObs, FullObs.observed,
measurable_iff_comap_le, Function.comp_def] using htuple
theorem full_cell_mass
the full-data probability of a covariate cell equals its observed cell probability.
Formal statement
Proof (Lean source)
lemma full_cell_mass {d : ℕ} (P : RealLaw d) (k : Fin d) :
P.fullLaw {z | z.x = k} = ofReal (P.cellMass k) := by
have hs : MeasurableSet {o : Obs d | o.x = k} :=
obs_measurable_x (measurableSet_singleton k)
rw [P.cellMass_eq, realMass, ENNReal.ofReal_toReal (measure_ne_top _ _)]
rw [← P.observed_margin, Measure.map_apply fullObs_observed_measurable hs]
rfl
theorem full_arm_cell_mass
the full-data probability of an arm-cell event equals the observed arm-cell probability.
Formal statement
Proof (Lean source)
lemma full_arm_cell_mass {d : ℕ} (P : RealLaw d) (a : Bool) (k : Fin d) :
P.fullLaw {z | z.x = k ∧ z.a = a} =
ofReal
(P.cellMass k * (if a then P.propensity k else 1 - P.propensity k)) := by
let q := if a then P.propensity k else 1 - P.propensity k
have hs : MeasurableSet {o : Obs d | o.x = k ∧ o.a = a} := by
convert (obs_measurable_x.prodMk obs_measurable_a)
(measurableSet_singleton (k, a)) using 1 <;> ext o <;> simp
have hfactor := P.arm_outcome_factorization a k Set.univ MeasurableSet.univ
let _ : IsProbabilityMeasure (P.outcomeLaw a k) := P.outcome_isProbability a k
have hout : realMass (P.outcomeLaw a k) Set.univ = 1 := by
simp [realMass]
rw [hout, mul_one] at hfactor
have hfactor' : P.cellMass k * q =
realMass P.observedLaw {o : Obs d | o.x = k ∧ o.a = a} := by
simpa only [q, Set.mem_univ, and_true] using hfactor
have hobs : P.observedLaw {o : Obs d | o.x = k ∧ o.a = a} =
ofReal (P.cellMass k * q) := by
rw [hfactor']
simp only [realMass]
rw [ENNReal.ofReal_toReal (measure_ne_top _ _)]
rw [← P.observed_margin] at hobs
rw [Measure.map_apply fullObs_observed_measurable hs] at hobs
simpa [FullObs.observed, q] using hobs
theorem observed_arm_outcome_measure
the observed arm-cell outcome measure factors into cell probability, arm propensity, and the conditional outcome law.
Formal statement
Proof (Lean source)
lemma observed_arm_outcome_measure {d : ℕ} (P : RealLaw d) (a : Bool) (k : Fin d) :
Measure.map Obs.y
(P.observedLaw.restrict {o | o.x = k ∧ o.a = a}) =
ofReal
(P.cellMass k * (if a then P.propensity k else 1 - P.propensity k)) •
P.outcomeLaw a k := by
let _ : IsProbabilityMeasure (P.outcomeLaw a k) := P.outcome_isProbability a k
let q := if a then P.propensity k else 1 - P.propensity k
have hp : 0 ≤ P.cellMass k := (P.cellMass_range k).1
have hq : 0 ≤ q := by
rcases P.propensity_range k with ⟨hpi0, hpi1⟩
cases a <;> simp [q] <;> linarith
have hc : 0 ≤ P.cellMass k * q := mul_nonneg hp hq
have hE : MeasurableSet {o : Obs d | o.x = k ∧ o.a = a} := by
convert (obs_measurable_x.prodMk obs_measurable_a)
(measurableSet_singleton (k, a)) using 1 <;> ext o <;> simp
ext s hs
rw [Measure.map_apply obs_measurable_y hs,
Measure.restrict_apply (obs_measurable_y hs), Measure.smul_apply]
have hf := P.arm_outcome_factorization a k s hs
have hf' := congrArg ofReal hf
simp only [realMass] at hf'
rw [ENNReal.ofReal_mul hc, ENNReal.ofReal_toReal (measure_ne_top _ _),
ENNReal.ofReal_toReal (measure_ne_top _ _)] at hf'
rw [show Obs.y ⁻¹' s ∩ {o : Obs d | o.x = k ∧ o.a = a} =
{o : Obs d | o.x = k ∧ o.a = a ∧ o.y ∈ s} by
ext o
simp [and_assoc, and_left_comm, and_comm]]
simpa only [q, smul_eq_mul] using hf'.symm
theorem consistency_ae
If consistency holds, consistency implies that the observed outcome equals the treatment-selected potential outcome almost surely.
Formal statement
Proof (Lean source)
lemma consistency_ae {d : ℕ} (P : RealLaw d) (hP : Consistency P) :
(fun z : FullObs d => z.y) =ᵐ[P.fullLaw]
(fun z => if z.a then z.y1 else z.y0) := by
rw [EventuallyEq, ae_iff]
change P.fullLaw {z | z.y ≠ if z.a then z.y1 else z.y0} = 0
exact hP
theorem full_arm_potential_measure
If consistency holds, under exchangeability, the full-data arm-cell potential-outcome measure factors into the arm-cell probability and conditional outcome law.
Formal statement
Proof (Lean source)
lemma full_arm_potential_measure {d : ℕ} (P : RealLaw d)
(hcons : Consistency P) (a : Bool) (k : Fin d) :
Measure.map (fun z : FullObs d => if a then z.y1 else z.y0)
(P.fullLaw.restrict {z | z.x = k ∧ z.a = a}) =
Measure.map Obs.y
(P.observedLaw.restrict {o | o.x = k ∧ o.a = a}) := by
have hObsE : MeasurableSet {o : Obs d | o.x = k ∧ o.a = a} := by
convert (obs_measurable_x.prodMk obs_measurable_a)
(measurableSet_singleton (k, a)) using 1 <;> ext o <;> simp
have hFullE : MeasurableSet {z : FullObs d | z.x = k ∧ z.a = a} := by
convert (fullObs_measurable_x.prodMk fullObs_measurable_a)
(measurableSet_singleton (k, a)) using 1 <;> ext z <;> simp
have hpot_y :
(fun z : FullObs d => if a then z.y1 else z.y0) =ᵐ[
P.fullLaw.restrict {z | z.x = k ∧ z.a = a}] fun z => z.y := by
filter_upwards [ae_restrict_mem hFullE,
ae_restrict_of_ae (consistency_ae P hcons)] with z hz hzy
have hza : z.a = a := hz.2
simpa only [hza] using hzy.symm
rw [Measure.map_congr hpot_y]
have hrestrict :
P.observedLaw.restrict {o : Obs d | o.x = k ∧ o.a = a} =
Measure.map FullObs.observed
(P.fullLaw.restrict {z : FullObs d | z.x = k ∧ z.a = a}) := by
rw [← P.observed_margin, Measure.restrict_map fullObs_observed_measurable hObsE]
rfl
rw [hrestrict, Measure.map_map obs_measurable_y fullObs_observed_measurable]
rfl
theorem full_cell_potential_measure
If the stated condition on the cell holds, under exchangeability and overlap, the full-data cell potential-outcome measure factors into the cell probability and conditional outcome law.
Formal statement
hypotheses
Proof (Lean source)
lemma full_cell_potential_measure {d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (a : Bool) (k : Fin d)
(hk : 0 < P.law.cellMass k) :
Measure.map (fun z : FullObs d => if a then z.y1 else z.y0)
(P.law.fullLaw.restrict {z | z.x = k}) =
ofReal (P.law.cellMass k) • P.law.outcomeLaw a k := by
let pot : FullObs d → ℝ := fun z => if a then z.y1 else z.y0
let q : ℝ := if a then P.law.propensity k else 1 - P.law.propensity k
have hq : 0 < q := by
rcases P.overlap k hk with ⟨hlow, hupp⟩
cases a <;> simp [q] <;> linarith [P.epsilon_pos]
have hc : 0 < P.law.cellMass k * q := mul_pos hk hq
have hpot : Measurable pot := fullObs_measurable_potential a
have hcell : MeasurableSet {z : FullObs d | z.x = k} :=
fullObs_measurable_x (measurableSet_singleton k)
ext s hs
have harmMap := (full_arm_potential_measure P.law P.consistency a k).trans
(observed_arm_outcome_measure P.law a k)
have harm := congrArg (fun mu : Measure ℝ => mu s) harmMap
rw [Measure.map_apply hpot hs, Measure.restrict_apply (hpot hs),
Measure.smul_apply] at harm
have hci := P.exchangeability k a
(if a then univ else s) (if a then s else univ)
(by cases a <;> simp [hs]) (by cases a <;> simp [hs])
have hind :
P.law.fullLaw {z | z.x = k ∧ z.a = a ∧ pot z ∈ s} *
P.law.fullLaw {z | z.x = k} =
P.law.fullLaw {z | z.x = k ∧ z.a = a} *
P.law.fullLaw {z | z.x = k ∧ pot z ∈ s} := by
cases a <;>
simpa [pot, and_assoc, and_left_comm, and_comm] using hci
rw [full_cell_mass P.law k, full_arm_cell_mass P.law a k] at hind
have hleft : P.law.fullLaw {z | z.x = k ∧ z.a = a ∧ pot z ∈ s} =
ofReal (P.law.cellMass k * q) * P.law.outcomeLaw a k s := by
calc
P.law.fullLaw {z | z.x = k ∧ z.a = a ∧ pot z ∈ s} =
P.law.fullLaw (pot ⁻¹' s ∩ {z | z.x = k ∧ z.a = a}) := by
congr 1
ext z
simp [pot, and_assoc, and_left_comm, and_comm]
_ = ofReal (P.law.cellMass k * q) *
P.law.outcomeLaw a k s := by
simpa only [q, smul_eq_mul] using harm
rw [hleft] at hind
have hcancel : ofReal (P.law.cellMass k * q) ≠ 0 := by
exact (ENNReal.ofReal_pos.mpr hc).ne'
have hfinite : ofReal (P.law.cellMass k * q) ≠ ⊤ := ENNReal.ofReal_ne_top
have heq :
ofReal (P.law.cellMass k * q) *
(P.law.fullLaw {z | z.x = k ∧ pot z ∈ s}) =
ofReal (P.law.cellMass k * q) *
(ofReal (P.law.cellMass k) * P.law.outcomeLaw a k s) := by
simpa only [q, mul_assoc, mul_left_comm, mul_comm] using hind.symm
have hdesired : P.law.fullLaw {z | z.x = k ∧ pot z ∈ s} =
ofReal (P.law.cellMass k) * P.law.outcomeLaw a k s := by
apply le_antisymm
· rw [← ENNReal.mul_le_mul_iff_right hcancel hfinite]
exact heq.le
· rw [← ENNReal.mul_le_mul_iff_right hcancel hfinite]
exact heq.ge
rw [Measure.map_apply hpot hs, Measure.restrict_apply (hpot hs),
Measure.smul_apply]
rw [show pot ⁻¹' s ∩ {z : FullObs d | z.x = k} =
{z | z.x = k ∧ pot z ∈ s} by ext z; simp [and_comm]]
simpa only [smul_eq_mul] using hdesired
theorem full_potential_integrable
each potential outcome is integrable under the full-data law.
Formal statement
hypotheses
conclusion
Integrable (fun z : FullObs d => if a then z.y1 else z.y0) P.law.fullLaw
Proof (Lean source)
lemma full_potential_integrable {d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (a : Bool) :
Integrable (fun z : FullObs d => if a then z.y1 else z.y0) P.law.fullLaw := by
let pot : FullObs d → ℝ := fun z => if a then z.y1 else z.y0
have hpot : Measurable pot := fullObs_measurable_potential a
have hcell : ∀ k : Fin d, MeasurableSet {z : FullObs d | z.x = k} :=
fun k => fullObs_measurable_x (measurableSet_singleton k)
have hcellInt : ∀ k : Fin d, IntegrableOn pot {z | z.x = k} P.law.fullLaw := by
intro k
change Integrable pot (P.law.fullLaw.restrict {z | z.x = k})
by_cases hk : 0 < P.law.cellMass k
· have hout := P.identifiedLaw.outcome_integrable a k hk
have hmap : Integrable (fun y : ℝ => y)
(Measure.map pot (P.law.fullLaw.restrict {z | z.x = k})) := by
rw [full_cell_potential_measure P a k hk]
exact hout.smul_measure ENNReal.ofReal_ne_top
refine (hmap.comp_measurable hpot).congr ?_
filter_upwards with z
rfl
· have hmass : P.law.fullLaw {z : FullObs d | z.x = k} = 0 := by
rw [full_cell_mass P.law k]
have hnonneg := (P.law.cellMass_range k).1
have hz : P.law.cellMass k = 0 := le_antisymm (le_of_not_gt hk) hnonneg
simp [hz]
rw [show P.law.fullLaw.restrict {z : FullObs d | z.x = k} = 0 by
exact Measure.restrict_eq_zero.mpr hmass]
exact integrable_zero_measure
rw [← integrableOn_univ (μ := P.law.fullLaw)]
rw [show (Set.univ : Set (FullObs d)) = ⋃ k : Fin d, {z | z.x = k} by
ext z
simp]
rw [integrableOn_finite_iUnion]
exact hcellInt
theorem full_potential_cell_integral
the full-data integral of a potential outcome within a cell equals cell probability times its conditional mean.
Formal statement
hypotheses
conclusion
(∫ z in {z : FullObs d | z.x = k}, (if a then z.y1 else z.y0) ∂P.law.fullLaw)
= P.law.cellMass k * P.law.outcomeMean a k
Proof (Lean source)
lemma full_potential_cell_integral {d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) (a : Bool) (k : Fin d) :
(∫ z in {z : FullObs d | z.x = k},
(if a then z.y1 else z.y0) ∂P.law.fullLaw) =
P.law.cellMass k * P.law.outcomeMean a k := by
let pot : FullObs d → ℝ := fun z => if a then z.y1 else z.y0
have hpot : Measurable pot := fullObs_measurable_potential a
by_cases hk : 0 < P.law.cellMass k
· have hout := P.identifiedLaw.outcome_integrable a k hk
calc
(∫ z in {z : FullObs d | z.x = k}, pot z ∂P.law.fullLaw) =
∫ y : ℝ, y ∂Measure.map pot
(P.law.fullLaw.restrict {z : FullObs d | z.x = k}) := by
have hmeas : AEStronglyMeasurable (fun y : ℝ => y)
(Measure.map pot
(P.law.fullLaw.restrict {z : FullObs d | z.x = k})) :=
measurable_id.aestronglyMeasurable
simpa only [Function.comp_apply] using
(integral_map hpot.aemeasurable hmeas).symm
_ = ∫ y : ℝ, y ∂(ofReal (P.law.cellMass k) •
P.law.outcomeLaw a k) := by
rw [full_cell_potential_measure P a k hk]
_ = P.law.cellMass k * ∫ y : ℝ, y ∂P.law.outcomeLaw a k := by
rw [integral_smul_measure]
simp [(P.law.cellMass_range k).1, smul_eq_mul]
_ = P.law.cellMass k * P.law.outcomeMean a k := by
rw [P.law.outcomeMean_eq]
· have hp0 : P.law.cellMass k = 0 :=
le_antisymm (le_of_not_gt hk) (P.law.cellMass_range k).1
have hmass : P.law.fullLaw {z : FullObs d | z.x = k} = 0 := by
rw [full_cell_mass P.law k, hp0]
simp
rw [MeasureTheory.setIntegral_measure_zero pot hmass, hp0, zero_mul]
theorem ate_identification
Consistency, conditional exchangeability, overlap, and the moment envelope identify the causal ATE from the observed margin by the finite g-formula.
Formal statement
hypotheses
conclusion
causalATE P.law = ateFunctional P.law P.identifiedLaw
Proof (Lean source)
theorem ate_identification {d : ℕ} {epsilon M sigma : ℝ}
(P : ModelClass d epsilon M sigma) :
causalATE P.law = ateFunctional P.law P.identifiedLaw := by
have hcell : ∀ k : Fin d, MeasurableSet
((fun z : FullObs d => z.x) ⁻¹' {k}) :=
fun k => fullObs_measurable_x (measurableSet_singleton k)
have h1 := full_potential_integrable P true
have h0 := full_potential_integrable P false
have h1' : Integrable (fun z : FullObs d => z.y1) P.law.fullLaw := by
simpa using h1
have h0' : Integrable (fun z : FullObs d => z.y0) P.law.fullLaw := by
simpa using h0
have hsum1 := integral_eq_sum_setIntegral_fiber
(μ := P.law.fullLaw) hcell h1'
have hsum0 := integral_eq_sum_setIntegral_fiber
(μ := P.law.fullLaw) hcell h0'
change causalATE P.law = rawAteFormula P.law
unfold causalATE rawAteFormula cellEffect
rw [integral_sub h1' h0', hsum1, hsum0, ← Finset.sum_sub_distrib]
apply Finset.sum_congr rfl
intro k _
have hk1 := full_potential_cell_integral P true k
have hk0 := full_potential_cell_integral P false k
change (∫ z in (fun z : FullObs d => z.x) ⁻¹' {k}, z.y1 ∂P.law.fullLaw) =
P.law.cellMass k * P.law.outcomeMean true k at hk1
change (∫ z in (fun z : FullObs d => z.x) ⁻¹' {k}, z.y0 ∂P.law.fullLaw) =
P.law.cellMass k * P.law.outcomeMean false k at hk0
rw [hk1, hk0]
ring
T_FixedInteriorTightness 23 declarations
Selector benchmark q before saturation.
Definition (Lean source)
noncomputable def selectorBenchmark (n d : ℕ) (sigma : ℝ) : ℝ :=
1 / (n : ℝ) + min (polynomialComponent n d)
(sigma ^ 2 + (d : ℝ) / (n : ℝ) ^ 2)
Product-form converse benchmark ell.
Definition (Lean source)
noncomputable def productBenchmark (n d : ℕ) (sigma : ℝ) : ℝ :=
baseRate n d + sigma ^ 2 * polynomialComponent n d
One of the three regimes in which the proved benchmarks match uniformly.
Definition (Lean source)
def MatchingElbow (K : ℝ) (n d : ℕ) (sigma : ℝ) : Prop :=
1 ≤ polynomialComponent n d ∨
polynomialComponent n d ≤ K * baseRate n d ∨
sigma ^ 2 ≤ K * baseRate n d
Divergence of the selector/converse benchmark ratio along a sequence.
Definition (Lean source)
def BenchmarkRatioDiverges (dseq : ℕ → ℕ) (sseq : ℕ → ℝ) : Prop :=
Tendsto (fun n => frontierRate n (dseq n) (sseq n) /
converseRate n (dseq n) (sseq n)) atTop atTop
Two positive sequences have the same order eventually.
Definition (Lean source)
def EventuallySameOrder (f g : ℕ → ℝ) : Prop :=
∃ c C : ℝ, 0 < c ∧ c ≤ C ∧
∀ᶠ n in atTop, c * g n ≤ f n ∧ f n ≤ C * g n
If the heterogeneity radius is nonnegative and the heterogeneity radius is at most two and the stated lower bound holds and the stated upper bound holds, in the nonsaturated triangular regime, the selector benchmark is within a factor two of the additive benchmark using min (u, sigma²).
Formal statement
hypotheses
n d :
ℕ
sigma :
ℝ
_hsigma_nonneg :
0 ≤ sigma
_hsigma_two :
sigma ≤ 2
hlower :
logEN n ^ 2 < d
hupper :
(d : ℝ) < n * logEN n
conclusion
≤ selectorBenchmark n d sigma ∧
Proof (Lean source)
lemma selectorBenchmark_triangular (n d : ℕ) (sigma : ℝ)
(_hsigma_nonneg : 0 ≤ sigma) (_hsigma_two : sigma ≤ 2)
(hlower : logEN n ^ 2 < d) (hupper : (d : ℝ) < n * logEN n) :
(1 / 2 : ℝ) *
(baseRate n d + min (polynomialComponent n d) (sigma ^ 2)) ≤
selectorBenchmark n d sigma ∧
selectorBenchmark n d sigma ≤
baseRate n d + min (polynomialComponent n d) (sigma ^ 2) := by
have hd : 0 < (d : ℝ) := lt_of_le_of_lt (sq_nonneg (logEN n)) hlower
have hnR : 0 < (n : ℝ) := by
have hprod : 0 < (n : ℝ) * logEN n := hd.trans hupper
have hn0 : 0 ≤ (n : ℝ) := by positivity
by_contra hn_not
have hn_eq : (n : ℝ) = 0 := le_antisymm (le_of_not_gt hn_not) hn0
rw [hn_eq, zero_mul] at hprod
exact (lt_irrefl 0 hprod)
have hn : 0 < n := by exact_mod_cast hnR
have hlog : 0 < logEN n := by
have hn_one : (1 : ℝ) ≤ n := by exact_mod_cast hn
rw [logEN, Real.log_mul (Real.exp_ne_zero 1) hnR.ne', Real.log_exp]
linarith [Real.log_nonneg hn_one]
have hnSq : 0 < (n : ℝ) ^ 2 := sq_pos_of_pos hnR
have hlogSq : 0 < logEN n ^ 2 := sq_pos_of_pos hlog
let a : ℝ := 1 / (n : ℝ)
let t : ℝ := (d : ℝ) / (n : ℝ) ^ 2
let u : ℝ := polynomialComponent n d
let s : ℝ := sigma ^ 2
have ha : 0 ≤ a := by dsimp [a]; positivity
have ht : 0 ≤ t := by dsimp [t]; positivity
have hs : 0 ≤ s := by dsimp [s]; positivity
have htu : t ≤ u := by
dsimp [t, u, polynomialComponent]
rw [div_le_div_iff₀ hnSq (mul_pos hnSq hlogSq)]
have hmul := mul_lt_mul_of_pos_left hlower (mul_pos hd hnSq)
nlinarith
have hu_one : u < 1 := by
dsimp [u, polynomialComponent]
rw [div_lt_one (mul_pos hnSq hlogSq)]
nlinarith [(sq_lt_sq₀ hd.le (mul_nonneg hnR.le hlog.le)).2 hupper]
have hgeneric :
(1 / 2 : ℝ) * (a + t + min u s) ≤ a + min u (s + t) ∧
a + min u (s + t) ≤ a + t + min u s := by
constructor
· by_cases hmid : u ≤ s + t
· rw [min_eq_left hmid]
have hmin : min u s ≤ u := min_le_left _ _
nlinarith
· rw [min_eq_right (le_of_not_ge hmid)]
have hsu : s ≤ u := by nlinarith
rw [min_eq_right hsu]
nlinarith
· by_cases hus : u ≤ s
· rw [min_eq_left hus]
have hmid : min u (s + t) ≤ u := min_le_left _ _
nlinarith
· rw [min_eq_right (le_of_not_ge hus)]
have hmid : min u (s + t) ≤ s + t := min_le_right _ _
nlinarith
simpa [selectorBenchmark, baseRate, a, t, u, s, min_eq_left hu_one.le,
add_assoc] using hgeneric
theorem polynomialComponent_le_baseRate_iff
If the sample is nonempty, the polynomial component is below the base rate exactly at the displayed constant-free algebraic elbow.
Formal statement
hypotheses
n d :
ℕ
hn :
0 < n
conclusion
Proof (Lean source)
lemma polynomialComponent_le_baseRate_iff (n d : ℕ) (hn : 0 < n) :
polynomialComponent n d ≤ baseRate n d ↔
(d : ℝ) ^ 2 ≤ n * logEN n ^ 2 + d * logEN n ^ 2 := by
have hnR : 0 < (n : ℝ) := by exact_mod_cast hn
have hn_one : (1 : ℝ) ≤ n := by exact_mod_cast hn
have hlog : 0 < logEN n := by
rw [logEN, Real.log_mul (Real.exp_ne_zero 1) hnR.ne', Real.log_exp]
linarith [Real.log_nonneg hn_one]
rw [polynomialComponent, div_le_iff₀ (mul_pos (sq_pos_of_pos hnR)
(sq_pos_of_pos hlog))]
have halg : baseRate n d * ((n : ℝ) ^ 2 * logEN n ^ 2) =
(n : ℝ) * logEN n ^ 2 + (d : ℝ) * logEN n ^ 2 := by
unfold baseRate
field_simp [hnR.ne']
rw [halg]
theorem polynomialComponent_le_baseRate_of_le_sqrt
If the sample is nonempty and the alphabet size satisfies the stated condition, the small-alphabet condition d ≤ sqrt(n) log(en) implies the polynomial component is below the base rate.
Formal statement
hypotheses
conclusion
polynomialComponent n d ≤ baseRate n d
Proof (Lean source)
lemma polynomialComponent_le_baseRate_of_le_sqrt (n d : ℕ) (hn : 0 < n)
(hd : (d : ℝ) ≤ sqrt n * logEN n) :
polynomialComponent n d ≤ baseRate n d := by
rw [polynomialComponent_le_baseRate_iff n d hn]
have hnR : 0 ≤ (n : ℝ) := by positivity
have hn_one : (1 : ℝ) ≤ n := by exact_mod_cast hn
have hlog : 0 ≤ logEN n := by
have hnRpos : 0 < (n : ℝ) := by exact_mod_cast hn
rw [logEN, Real.log_mul (Real.exp_ne_zero 1) hnRpos.ne', Real.log_exp]
linarith [Real.log_nonneg hn_one]
have hd0 : 0 ≤ (d : ℝ) := by positivity
have hsqrt : 0 ≤ sqrt (n : ℝ) := Real.sqrt_nonneg _
have hsq := (sq_le_sq₀ hd0 (mul_nonneg hsqrt hlog)).2 hd
rw [mul_pow, Real.sq_sqrt hnR] at hsq
nlinarith [mul_nonneg (show (0 : ℝ) ≤ d by positivity) (sq_nonneg (logEN n))]
theorem polynomialComponent_le_baseRate_of_le_log_sq
If the sample is nonempty and the alphabet size satisfies the stated condition, the regime d ≤ log(en)² is contained in the constant-free algebraic elbow.
Formal statement
hypotheses
conclusion
polynomialComponent n d ≤ baseRate n d
Proof (Lean source)
lemma polynomialComponent_le_baseRate_of_le_log_sq (n d : ℕ) (hn : 0 < n)
(hd : (d : ℝ) ≤ logEN n ^ 2) :
polynomialComponent n d ≤ baseRate n d := by
rw [polynomialComponent_le_baseRate_iff n d hn]
have hd0 : 0 ≤ (d : ℝ) := by positivity
have hn0 : 0 ≤ (n : ℝ) := by positivity
have hlogSq : 0 ≤ logEN n ^ 2 := sq_nonneg _
nlinarith [mul_le_mul_of_nonneg_left hd hd0,
mul_nonneg hn0 hlogSq]
theorem frontierRate_le_converseRate_fixed_radius Lemma frontierRate_le_converseRate_fixed_radius in the paper ↗
If the sample is nonempty and the radius is bounded away from zero and the heterogeneity radius is nonnegative and the heterogeneity radius is at most two and the heterogeneity radius satisfies the stated bound, at a fixed positive radius, the selector benchmark is bounded by a constant multiple of the product-form converse benchmark, uniformly in the alphabet.
Formal statement
hypotheses
n d :
ℕ
sigma sigmaLower :
ℝ
hn :
0 < n
hsigmaLower :
0 < sigmaLower
_hsigma_nonneg :
0 ≤ sigma
_hsigma_two :
sigma ≤ 2
hsigma :
sigmaLower ≤ sigma
conclusion
Proof (Lean source)
lemma frontierRate_le_converseRate_fixed_radius (n d : ℕ) (sigma sigmaLower : ℝ)
(hn : 0 < n) (hsigmaLower : 0 < sigmaLower)
(_hsigma_nonneg : 0 ≤ sigma) (_hsigma_two : sigma ≤ 2)
(hsigma : sigmaLower ≤ sigma) :
frontierRate n d sigma ≤
max 1 (sigmaLower ^ (-2 : ℤ)) * converseRate n d sigma := by
have hnR : 0 < (n : ℝ) := by exact_mod_cast hn
have ha : 0 ≤ 1 / (n : ℝ) := by positivity
have ht : 0 ≤ (d : ℝ) / (n : ℝ) ^ 2 := by positivity
have hu : 0 ≤ polynomialComponent n d := by
unfold polynomialComponent
positivity
have hs : sigmaLower ^ 2 ≤ sigma ^ 2 := by nlinarith
have hsl2 : 0 < sigmaLower ^ 2 := sq_pos_of_pos hsigmaLower
have hcoef_one : 1 ≤ max 1 (sigmaLower ^ (-2 : ℤ)) := le_max_left _ _
have hcoef_inv : (sigmaLower ^ 2)⁻¹ ≤
max 1 (sigmaLower ^ (-2 : ℤ)) := by
simpa [zpow_neg, zpow_two] using
(le_max_right (1 : ℝ) (sigmaLower ^ (-2 : ℤ)))
have hfront : frontierRate n d sigma ≤
1 / (n : ℝ) + min 1 (polynomialComponent n d) := by
unfold frontierRate
gcongr
exact min_le_left _ _
have hscale : min 1 (polynomialComponent n d) ≤
(sigmaLower ^ 2)⁻¹ *
(sigma ^ 2 * min 1 (polynomialComponent n d)) := by
have hm : 0 ≤ min 1 (polynomialComponent n d) := le_min zero_le_one hu
calc
min 1 (polynomialComponent n d) =
(sigmaLower ^ 2)⁻¹ *
(sigmaLower ^ 2 * min 1 (polynomialComponent n d)) := by
field_simp
_ ≤ (sigmaLower ^ 2)⁻¹ *
(sigma ^ 2 * min 1 (polynomialComponent n d)) := by
gcongr
calc
frontierRate n d sigma ≤
1 / (n : ℝ) + min 1 (polynomialComponent n d) := hfront
_ ≤ max 1 (sigmaLower ^ (-2 : ℤ)) * (1 / (n : ℝ)) +
max 1 (sigmaLower ^ (-2 : ℤ)) *
(sigma ^ 2 * min 1 (polynomialComponent n d)) := by
exact add_le_add (by nlinarith [hcoef_one, ha])
(hscale.trans (mul_le_mul_of_nonneg_right hcoef_inv (by positivity)))
_ ≤ max 1 (sigmaLower ^ (-2 : ℤ)) * converseRate n d sigma := by
unfold converseRate
have hc : 0 ≤ max 1 (sigmaLower ^ (-2 : ℤ)) := zero_le_one.trans hcoef_one
have htc : 0 ≤ min 1 ((d : ℝ) / (n : ℝ) ^ 2) := le_min zero_le_one ht
nlinarith
theorem frontierRate_le_converseRate_matching_elbow
If the sample is nonempty and the polynomial or elbow parameter satisfies its stated bound and the matching-elbow condition holds, in any matching elbow, the selector benchmark is bounded by 1 + K times the converse benchmark.
Formal statement
hypotheses
conclusion
frontierRate n d sigma ≤ (1 + K) * converseRate n d sigma
Proof (Lean source)
lemma frontierRate_le_converseRate_matching_elbow (n d : ℕ) (sigma K : ℝ)
(hn : 0 < n) (hK : 0 ≤ K) (helbow : MatchingElbow K n d sigma) :
frontierRate n d sigma ≤ (1 + K) * converseRate n d sigma := by
have hnR : 0 < (n : ℝ) := by exact_mod_cast hn
have ha : 0 < 1 / (n : ℝ) := by positivity
have ht : 0 ≤ (d : ℝ) / (n : ℝ) ^ 2 := by positivity
have hu : 0 ≤ polynomialComponent n d := by
unfold polynomialComponent
positivity
have hs : 0 ≤ sigma ^ 2 := sq_nonneg _
have hm : 0 ≤ min 1 (polynomialComponent n d) := le_min zero_le_one hu
have hconv0 : 0 ≤ converseRate n d sigma := by
unfold converseRate
positivity
rcases helbow with hu_one | hu_base | hs_base
· have hfront : frontierRate n d sigma ≤
1 / (n : ℝ) + min 1 (sigma ^ 2 + (d : ℝ) / (n : ℝ) ^ 2) := by
unfold frontierRate collisionComponent
gcongr
exact min_le_right _ _
have hconv : 1 / (n : ℝ) +
min 1 (sigma ^ 2 + (d : ℝ) / (n : ℝ) ^ 2) ≤
converseRate n d sigma := by
unfold converseRate
by_cases ht_one : (d : ℝ) / (n : ℝ) ^ 2 ≤ 1
· rw [min_eq_right ht_one, min_eq_left hu_one]
nlinarith [min_le_right (1 : ℝ)
(sigma ^ 2 + (d : ℝ) / (n : ℝ) ^ 2)]
· rw [min_eq_left (le_of_not_ge ht_one), min_eq_left hu_one]
nlinarith [min_le_left (1 : ℝ)
(sigma ^ 2 + (d : ℝ) / (n : ℝ) ^ 2)]
exact (hfront.trans hconv).trans
(by nlinarith [hconv0])
· by_cases ht_one : (d : ℝ) / (n : ℝ) ^ 2 ≤ 1
· have hfront : frontierRate n d sigma ≤
1 / (n : ℝ) + polynomialComponent n d := by
unfold frontierRate
nlinarith [min_le_right (1 : ℝ)
(min (polynomialComponent n d) (collisionComponent n d sigma)),
min_le_left (polynomialComponent n d) (collisionComponent n d sigma)]
have hbase : baseRate n d ≤ converseRate n d sigma := by
unfold baseRate converseRate
rw [min_eq_right ht_one]
nlinarith
calc
frontierRate n d sigma ≤ 1 / (n : ℝ) + polynomialComponent n d := hfront
_ ≤ (1 + K) * baseRate n d := by
unfold baseRate at hu_base ⊢
nlinarith [ha.le, ht]
_ ≤ (1 + K) * converseRate n d sigma := by gcongr
· have hfront : frontierRate n d sigma ≤ 1 / (n : ℝ) + 1 := by
unfold frontierRate
gcongr
exact min_le_left _ _
have hconv : 1 / (n : ℝ) + 1 ≤ converseRate n d sigma := by
unfold converseRate
rw [min_eq_left (le_of_not_ge ht_one)]
nlinarith
exact (hfront.trans hconv).trans
(by nlinarith [hconv0])
· by_cases ht_one : (d : ℝ) / (n : ℝ) ^ 2 ≤ 1
· have hfront : frontierRate n d sigma ≤
baseRate n d + sigma ^ 2 := by
unfold frontierRate collisionComponent baseRate
nlinarith [min_le_right (1 : ℝ)
(min (polynomialComponent n d)
(sigma ^ 2 + (d : ℝ) / (n : ℝ) ^ 2)),
min_le_right (polynomialComponent n d)
(sigma ^ 2 + (d : ℝ) / (n : ℝ) ^ 2)]
have hbase : baseRate n d ≤ converseRate n d sigma := by
unfold baseRate converseRate
rw [min_eq_right ht_one]
nlinarith
calc
frontierRate n d sigma ≤ baseRate n d + sigma ^ 2 := hfront
_ ≤ (1 + K) * baseRate n d := by nlinarith [hs_base]
_ ≤ (1 + K) * converseRate n d sigma := by gcongr
· have hfront : frontierRate n d sigma ≤ 1 / (n : ℝ) + 1 := by
unfold frontierRate
gcongr
exact min_le_left _ _
have hconv : 1 / (n : ℝ) + 1 ≤ converseRate n d sigma := by
unfold converseRate
rw [min_eq_left (le_of_not_ge ht_one)]
nlinarith
exact (hfront.trans hconv).trans
(by nlinarith [hconv0])
If the overlap constant is positive and the overlap constant is below one half and the radius is bounded away from zero, the two-sided minimax bracket and fixed-radius benchmark comparison give uniform minimax equivalence at every radius bounded away from zero.
Formal statement
hypotheses
epsilon sigmaLower :
ℝ
hepsilon :
0 < epsilon
hepsilon_half :
epsilon < 1 / 2
hsigmaLower :
0 < sigmaLower
conclusion
∃ c C : ℝ
if
0 < c ∧ c ≤ C ∧ ∀ n d : ℕ, ∀ M sigma : ℝ, 0 < n
and
0 < d
and
1 ≤ M
and
0 ≤ sigma
and
sigmaLower ≤ sigma
and
sigma ≤ 2
then
c * M ^ 2 * frontierRate n d sigma ≤ minimaxRisk n d epsilon M sigma ∧
minimaxRisk n d epsilon M sigma ≤ C * M ^ 2 * frontierRate n d sigma
Proof (Lean source)
lemma fixed_radius_minimax_equivalence (epsilon sigmaLower : ℝ)
(hepsilon : 0 < epsilon) (hepsilon_half : epsilon < 1 / 2)
(hsigmaLower : 0 < sigmaLower) :
∃ c C : ℝ, 0 < c ∧ c ≤ C ∧
∀ n d : ℕ, ∀ M sigma : ℝ,
0 < n → 0 < d → 1 ≤ M → 0 ≤ sigma →
sigmaLower ≤ sigma → sigma ≤ 2 →
c * M ^ 2 * frontierRate n d sigma ≤
minimaxRisk n d epsilon M sigma ∧
minimaxRisk n d epsilon M sigma ≤
C * M ^ 2 * frontierRate n d sigma := by
obtain ⟨c0, C0, hc0, _hc0_one, _hC0_one, hc0C0, hall⟩ :=
two_sided_minimax_bracket_all_d epsilon hepsilon hepsilon_half
let D := max 1 (sigmaLower ^ (-2 : ℤ))
have hD : 0 < D := lt_of_lt_of_le zero_lt_one (le_max_left _ _)
refine ⟨c0 / D, C0, div_pos hc0 hD, ?_, ?_⟩
· exact (div_le_self hc0.le (le_max_left _ _)).trans hc0C0
· intro n d M sigma hn hd hM hsigma_nonneg hsigma hsigma_two
obtain ⟨hlower, hupper⟩ :=
hall n d M sigma hn hd hM (le_trans (le_of_lt hsigmaLower) hsigma) hsigma_two
refine ⟨?_, hupper⟩
have hrate := frontierRate_le_converseRate_fixed_radius n d sigma sigmaLower
hn hsigmaLower hsigma_nonneg hsigma_two hsigma
calc
c0 / D * M ^ 2 * frontierRate n d sigma =
(c0 / D) * (M ^ 2 * frontierRate n d sigma) := by ring
_ ≤ (c0 / D) * (M ^ 2 * (D * converseRate n d sigma)) := by
gcongr
_ = c0 * M ^ 2 * converseRate n d sigma := by
field_simp [D, hD.ne']
<;> ring
_ ≤ minimaxRisk n d epsilon M sigma := hlower
theorem matching_elbow_minimax_equivalence
If the overlap constant is positive and the overlap constant is below one half and the polynomial or elbow parameter satisfies its stated bound, the two-sided minimax bracket and elbow comparison give uniform minimax equivalence in each of the three matching regimes.
Formal statement
hypotheses
epsilon K :
ℝ
hepsilon :
0 < epsilon
hepsilon_half :
epsilon < 1 / 2
hK :
0 ≤ K
conclusion
∃ c C : ℝ
if
0 < c ∧ c ≤ C ∧ ∀ n d : ℕ, ∀ M sigma : ℝ, 0 < n
and
0 < d
and
1 ≤ M
and
0 ≤ sigma
and
sigma ≤ 2
and
MatchingElbow K n d sigma
then
c * M ^ 2 * frontierRate n d sigma ≤ minimaxRisk n d epsilon M sigma ∧
minimaxRisk n d epsilon M sigma ≤ C * M ^ 2 * frontierRate n d sigma
Proof (Lean source)
lemma matching_elbow_minimax_equivalence (epsilon K : ℝ)
(hepsilon : 0 < epsilon) (hepsilon_half : epsilon < 1 / 2) (hK : 0 ≤ K) :
∃ c C : ℝ, 0 < c ∧ c ≤ C ∧
∀ n d : ℕ, ∀ M sigma : ℝ,
0 < n → 0 < d → 1 ≤ M → 0 ≤ sigma → sigma ≤ 2 →
MatchingElbow K n d sigma →
c * M ^ 2 * frontierRate n d sigma ≤
minimaxRisk n d epsilon M sigma ∧
minimaxRisk n d epsilon M sigma ≤
C * M ^ 2 * frontierRate n d sigma := by
obtain ⟨c0, C0, hc0, _hc0_one, _hC0_one, hc0C0, hall⟩ :=
two_sided_minimax_bracket_all_d epsilon hepsilon hepsilon_half
have hD : 0 < 1 + K := by linarith
refine ⟨c0 / (1 + K), C0, div_pos hc0 hD, ?_, ?_⟩
· exact (div_le_self hc0.le (by linarith)).trans hc0C0
· intro n d M sigma hn hd hM hsigma hsigma_two helbow
obtain ⟨hlower, hupper⟩ :=
hall n d M sigma hn hd hM hsigma hsigma_two
refine ⟨?_, hupper⟩
have hrate := frontierRate_le_converseRate_matching_elbow n d sigma K hn hK helbow
calc
c0 / (1 + K) * M ^ 2 * frontierRate n d sigma =
(c0 / (1 + K)) * (M ^ 2 * frontierRate n d sigma) := by ring
_ ≤ (c0 / (1 + K)) *
(M ^ 2 * ((1 + K) * converseRate n d sigma)) := by
gcongr
_ = c0 * M ^ 2 * converseRate n d sigma := by
field_simp [hD.ne']
<;> ring
_ ≤ minimaxRisk n d epsilon M sigma := hlower
theorem logEN_tendsto_atTop
The paper's logarithmic scale diverges with the sample size.
Proof (Lean source)
lemma logEN_tendsto_atTop : Tendsto logEN atTop atTop := by
have hlognat : Tendsto (fun n : ℕ => log (n : ℝ)) atTop atTop :=
Real.tendsto_log_atTop.comp tendsto_natCast_atTop_atTop
have hadd : Tendsto (fun n : ℕ => 1 + log (n : ℝ)) atTop atTop :=
tendsto_const_nhds.add_atTop hlognat
apply hadd.congr'
filter_upwards [eventually_gt_atTop (0 : ℕ)] with n hn
rw [logEN, Real.log_mul (Real.exp_ne_zero 1) (by positivity), Real.log_exp]
theorem logEN_pow_div_tendsto_zero
Every fixed natural power of log(en) is negligible relative to n.
Formal statement
Proof (Lean source)
lemma logEN_pow_div_tendsto_zero (k : ℕ) :
Tendsto (fun n : ℕ => logEN n ^ k / (n : ℝ)) atTop (nhds 0) := by
have harg : Tendsto (fun n : ℕ => exp 1 * (n : ℝ)) atTop atTop :=
(tendsto_natCast_atTop_atTop (R := ℝ)).const_mul_atTop (Real.exp_pos 1)
have hreal : Tendsto (fun x : ℝ => log x ^ k / x) atTop (nhds 0) := by
simpa using Real.tendsto_pow_log_div_mul_add_atTop 1 0 k one_ne_zero
have hmul := (hreal.comp harg).const_mul (exp 1)
have heq : (fun n : ℕ => logEN n ^ k / (n : ℝ)) =ᶠ[atTop]
(fun n => exp 1 *
((fun x : ℝ => log x ^ k / x) (exp 1 * (n : ℝ)))) := by
filter_upwards [eventually_gt_atTop (0 : ℕ)] with n hn
rw [logEN]
field_simp [Real.exp_ne_zero 1,
(by exact_mod_cast hn.ne' : (n : ℝ) ≠ 0)]
simpa using hmul.congr' heq.symm
theorem diagonal_selector_product_formulas
If the sample is nonempty, on the diagonal witness, the selector and product benchmarks have the explicit reciprocal-log formulas used in the rate comparison.
Formal statement
hypotheses
n :
ℕ
hn :
0 < n
conclusion
Proof (Lean source)
lemma diagonal_selector_product_formulas (n : ℕ) (hn : 0 < n) :
selectorBenchmark n n ((logEN n) ^ (-1 / 2 : ℝ)) =
1 / (n : ℝ) + 1 / logEN n ^ 2 ∧
productBenchmark n n ((logEN n) ^ (-1 / 2 : ℝ)) =
2 / (n : ℝ) + 1 / logEN n ^ 3 := by
have hnR : 0 < (n : ℝ) := by exact_mod_cast hn
have hL : 1 ≤ logEN n := by
rw [logEN, Real.log_mul (Real.exp_ne_zero 1) hnR.ne', Real.log_exp]
exact le_add_of_nonneg_right (Real.log_nonneg (by exact_mod_cast hn))
have hL0 : 0 < logEN n := zero_lt_one.trans_le hL
have hbase : baseRate n n = 2 / (n : ℝ) := by
unfold baseRate
field_simp [hnR.ne']
ring
have hpoly : polynomialComponent n n = 1 / logEN n ^ 2 := by
unfold polynomialComponent
field_simp [hnR.ne']
have hsigma : ((logEN n) ^ (-1 / 2 : ℝ)) ^ 2 = (logEN n)⁻¹ := by
rw [← Real.rpow_natCast, ← Real.rpow_mul hL0.le]
norm_num
exact Real.rpow_neg_one _
have hinv : 1 / logEN n ^ 2 ≤ (logEN n)⁻¹ := by
rw [div_le_iff₀ (sq_pos_of_pos hL0)]
field_simp [hL0.ne']
nlinarith
have hmin : 1 / logEN n ^ 2 ≤
(logEN n)⁻¹ + (n : ℝ) / (n : ℝ) ^ 2 :=
hinv.trans (le_add_of_nonneg_right (by positivity))
constructor
· rw [selectorBenchmark, hpoly, hsigma, min_eq_left hmin]
· rw [productBenchmark, hbase, hpoly, hsigma]
field_simp [hL0.ne']
theorem diagonal_residual_wedge
The diagonal sequence d=n, sigma=log(en)^(-1/2) lies in the residual wedge.
Formal statement
conclusion
ResidualWedge (fun n => n) (fun n => (logEN n) ^ (-1 / 2 : ℝ))
Proof (Lean source)
lemma diagonal_residual_wedge : ResidualWedge (fun n => n)
(fun n => (logEN n) ^ (-1 / 2 : ℝ)) := by
have hLinv : Tendsto (fun n => (logEN n)⁻¹) atTop (nhds 0) :=
logEN_tendsto_atTop.inv_tendsto_atTop
have hLinvSq : Tendsto (fun n => (logEN n ^ 2)⁻¹) atTop (nhds 0) := by
have h := hLinv.pow 2
have heq : (fun n => (logEN n)⁻¹ ^ 2) =
(fun n => (logEN n ^ 2)⁻¹) := by
funext n
rw [← inv_pow]
convert h.congr' (Filter.Eventually.of_forall fun n => congrFun heq n) using 1 <;>
norm_num
refine ⟨?_, ?_, ?_, ?_, ?_⟩
· filter_upwards [eventually_ge_atTop 1] with n hn
have hnR : (1 : ℝ) ≤ (n : ℝ) := by exact_mod_cast hn
have hL1 : 1 ≤ logEN n := by
have hnpos : 0 < (n : ℝ) := zero_lt_one.trans_le hnR
rw [logEN, Real.log_mul (Real.exp_ne_zero 1) hnpos.ne', Real.log_exp]
exact le_add_of_nonneg_right (Real.log_nonneg hnR)
have hs0 : 0 ≤ (logEN n) ^ (-1 / 2 : ℝ) :=
Real.rpow_nonneg (zero_le_one.trans hL1) _
have hpow : (logEN n) ^ (-1 / 2 : ℝ) ≤ 1 :=
Real.rpow_le_one_of_one_le_of_nonpos hL1 (by norm_num)
exact ⟨hn, hs0, hpow.trans (by norm_num)⟩
· have h := (logEN_pow_div_tendsto_zero 2).const_mul 2
have heq : (fun n : ℕ => 2 * (logEN n ^ 2 / (n : ℝ))) =ᶠ[atTop]
(fun n => baseRate n n / polynomialComponent n n) := by
filter_upwards [eventually_gt_atTop (0 : ℕ)] with n hn
have hnR : (n : ℝ) ≠ 0 := by exact_mod_cast hn.ne'
unfold baseRate polynomialComponent
field_simp [hnR]
ring
convert h.congr' heq using 1 <;> norm_num
· apply hLinvSq.congr'
filter_upwards [eventually_gt_atTop (0 : ℕ)] with n hn
have hnR : (n : ℝ) ≠ 0 := by exact_mod_cast hn.ne'
unfold polynomialComponent
field_simp [hnR]
· have h := (logEN_pow_div_tendsto_zero 1).const_mul 2
have heq : (fun n : ℕ => 2 * (logEN n ^ 1 / (n : ℝ))) =ᶠ[atTop]
(fun n => baseRate n n / ((logEN n) ^ (-1 / 2 : ℝ)) ^ 2) := by
filter_upwards [eventually_gt_atTop (0 : ℕ)] with n hn
have hnR : 0 < (n : ℝ) := by exact_mod_cast hn
have hnRone : (1 : ℝ) ≤ (n : ℝ) := by exact_mod_cast hn
have hL0 : 0 < logEN n := by
rw [logEN, Real.log_mul (Real.exp_ne_zero 1) hnR.ne', Real.log_exp]
linarith [Real.log_nonneg hnRone]
have hsigma : ((logEN n) ^ (-1 / 2 : ℝ)) ^ 2 = (logEN n)⁻¹ := by
rw [← Real.rpow_natCast, ← Real.rpow_mul hL0.le]
norm_num
exact Real.rpow_neg_one _
rw [hsigma]
unfold baseRate
field_simp [hnR.ne', hL0.ne']
ring
convert h.congr' heq using 1 <;> norm_num
· apply hLinv.congr'
filter_upwards [eventually_ge_atTop 1] with n hn
have hnR : (1 : ℝ) ≤ (n : ℝ) := by exact_mod_cast hn
have hL0 : 0 < logEN n := by
have hnpos : 0 < (n : ℝ) := zero_lt_one.trans_le hnR
rw [logEN, Real.log_mul (Real.exp_ne_zero 1) hnpos.ne', Real.log_exp]
linarith [Real.log_nonneg hnR]
rw [← Real.rpow_natCast, ← Real.rpow_mul hL0.le]
norm_num
exact (Real.rpow_neg_one _).symm
theorem diagonal_benchmark_eventually_same_order
The diagonal selector/product ratio is eventually between one third and twice the logarithmic scale.
Formal statement
conclusion
EventuallySameOrder (fun n => selectorBenchmark n n ((logEN n) ^ (-1 / 2 : ℝ)) / productBenchmark n n ((logEN n) ^ (-1 / 2 : ℝ))) logEN
Proof (Lean source)
lemma diagonal_benchmark_eventually_same_order : EventuallySameOrder
(fun n => selectorBenchmark n n ((logEN n) ^ (-1 / 2 : ℝ)) /
productBenchmark n n ((logEN n) ^ (-1 / 2 : ℝ))) logEN := by
refine ⟨1 / 3, 2, by norm_num, by norm_num, ?_⟩
have hsmall : ∀ᶠ n : ℕ in atTop, logEN n ^ 3 / (n : ℝ) < 1 :=
(logEN_pow_div_tendsto_zero 3).eventually (Iio_mem_nhds (by norm_num))
filter_upwards [eventually_gt_atTop (0 : ℕ), hsmall] with n hn hsmalln
have hnR : 0 < (n : ℝ) := by exact_mod_cast hn
have hnRone : (1 : ℝ) ≤ (n : ℝ) := by exact_mod_cast hn
have hL : 1 ≤ logEN n := by
rw [logEN, Real.log_mul (Real.exp_ne_zero 1) hnR.ne', Real.log_exp]
exact le_add_of_nonneg_right (Real.log_nonneg hnRone)
rw [(diagonal_selector_product_formulas n hn).1,
(diagonal_selector_product_formulas n hn).2]
have hL0 : 0 < logEN n := zero_lt_one.trans_le hL
have hden : 0 < 2 / (n : ℝ) + 1 / logEN n ^ 3 := by positivity
constructor
· rw [le_div_iff₀ hden]
field_simp [hnR.ne', hL0.ne'] at hsmalln ⊢
nlinarith
· rw [div_le_iff₀ hden]
field_simp [hnR.ne', hL0.ne'] at hsmalln ⊢
nlinarith [sq_nonneg (logEN n - 1)]
theorem diagonal_benchmark_ratio_tendsto_atTop
The diagonal selector/product benchmark ratio diverges.
Formal statement
conclusion
Tendsto (fun n => selectorBenchmark n n ((logEN n) ^ (-1 / 2 : ℝ)) / productBenchmark n n ((logEN n) ^ (-1 / 2 : ℝ))) atTop atTop
Proof (Lean source)
lemma diagonal_benchmark_ratio_tendsto_atTop : Tendsto (fun n =>
selectorBenchmark n n ((logEN n) ^ (-1 / 2 : ℝ)) /
productBenchmark n n ((logEN n) ^ (-1 / 2 : ℝ))) atTop atTop := by
obtain ⟨c, _C, hc, _hcC, hlower⟩ := diagonal_benchmark_eventually_same_order
exact tendsto_atTop_mono' atTop (hlower.mono fun n hn => hn.1)
(logEN_tendsto_atTop.const_mul_atTop hc)
theorem benchmark_divergence_eventually_avoids_elbows
If the sequence is eventually in the admissible parameter range and the benchmark ratio diverges and the polynomial or elbow parameter satisfies its stated bound, a diverging benchmark ratio eventually leaves every fixed matching elbow.
Formal statement
hypotheses
dseq :
ℕ → ℕ
sseq :
ℕ → ℝ
hvalid :
∀ᶠ t in atTop, 1 ≤ dseq t ∧ 0 ≤ sseq t ∧ sseq t ≤ 2
hdiv :
BenchmarkRatioDiverges dseq sseq
K :
ℝ
hK :
0 ≤ K
conclusion
∀ᶠ t in atTop, ¬ MatchingElbow K t (dseq t) (sseq t)
Proof (Lean source)
lemma benchmark_divergence_eventually_avoids_elbows
(dseq : ℕ → ℕ) (sseq : ℕ → ℝ)
(hvalid : ∀ᶠ t in atTop, 1 ≤ dseq t ∧ 0 ≤ sseq t ∧ sseq t ≤ 2)
(hdiv : BenchmarkRatioDiverges dseq sseq) (K : ℝ) (hK : 0 ≤ K) :
∀ᶠ t in atTop, ¬ MatchingElbow K t (dseq t) (sseq t) := by
have hlarge : ∀ᶠ t in atTop,
1 + K < frontierRate t (dseq t) (sseq t) /
converseRate t (dseq t) (sseq t) :=
hdiv (eventually_gt_atTop (1 + K))
filter_upwards [hvalid, eventually_gt_atTop (0 : ℕ), hlarge] with n hv hn hlarge_n
intro helbow
have hnR : 0 < (n : ℝ) := by exact_mod_cast hn
have hu : 0 ≤ polynomialComponent n (dseq n) := by
unfold polynomialComponent
positivity
have hconv : 0 < converseRate n (dseq n) (sseq n) := by
unfold converseRate
have ha : 0 < 1 / (n : ℝ) := by positivity
have hdterm : 0 ≤ min 1 ((dseq n : ℝ) / (n : ℝ) ^ 2) := by positivity
have huterm : 0 ≤ min 1 (polynomialComponent n (dseq n)) :=
le_min zero_le_one hu
nlinarith [sq_nonneg (sseq n), mul_nonneg (sq_nonneg (sseq n)) huterm]
have hrate := frontierRate_le_converseRate_matching_elbow
n (dseq n) (sseq n) K hn hK helbow
have hratio : frontierRate n (dseq n) (sseq n) /
converseRate n (dseq n) (sseq n) ≤ 1 + K := by
rw [div_le_iff₀ hconv]
exact hrate
linarith
theorem benchmark_divergence_implies_residual_wedge
If the sequence is eventually in the admissible parameter range and the benchmark ratio diverges, divergence of the all-alphabet benchmark ratio forces all four defining limits of the residual shrinking-radius wedge.
Formal statement
hypotheses
dseq :
ℕ → ℕ
sseq :
ℕ → ℝ
hvalid :
∀ᶠ t in atTop, 1 ≤ dseq t ∧ 0 ≤ sseq t ∧ sseq t ≤ 2
hdiv :
BenchmarkRatioDiverges dseq sseq
conclusion
ResidualWedge dseq sseq
Proof (Lean source)
lemma benchmark_divergence_implies_residual_wedge
(dseq : ℕ → ℕ) (sseq : ℕ → ℝ)
(hvalid : ∀ᶠ t in atTop, 1 ≤ dseq t ∧ 0 ≤ sseq t ∧ sseq t ≤ 2)
(hdiv : BenchmarkRatioDiverges dseq sseq) : ResidualWedge dseq sseq := by
have hbu : Tendsto (fun n => baseRate n (dseq n) /
polynomialComponent n (dseq n)) atTop (nhds 0) := by
refine tendsto_order.2 ⟨?_, ?_⟩
· intro a ha
filter_upwards [hvalid, eventually_gt_atTop (0 : ℕ)] with n hv hn
have hu : 0 < polynomialComponent n (dseq n) := by
unfold polynomialComponent
have hnR : 0 < (n : ℝ) := by exact_mod_cast hn
have hdR : 0 < (dseq n : ℝ) := by exact_mod_cast hv.1
have hL : 0 < logEN n := by
rw [logEN, Real.log_mul (Real.exp_ne_zero 1) hnR.ne', Real.log_exp]
have hnOne : (1 : ℝ) ≤ (n : ℝ) := by exact_mod_cast hn
linarith [Real.log_nonneg hnOne]
positivity
have hb : 0 ≤ baseRate n (dseq n) := by unfold baseRate; positivity
exact ha.trans_le (div_nonneg hb hu.le)
· intro a ha
let K := a⁻¹
have hK : 0 ≤ K := inv_nonneg.mpr ha.le
have havoid := benchmark_divergence_eventually_avoids_elbows
dseq sseq hvalid hdiv K hK
filter_upwards [hvalid, eventually_gt_atTop (0 : ℕ), havoid] with n hv hn hne
have hu : 0 < polynomialComponent n (dseq n) := by
unfold polynomialComponent
have hnR : 0 < (n : ℝ) := by exact_mod_cast hn
have hdR : 0 < (dseq n : ℝ) := by exact_mod_cast hv.1
have hL : 0 < logEN n := by
rw [logEN, Real.log_mul (Real.exp_ne_zero 1) hnR.ne', Real.log_exp]
have hnOne : (1 : ℝ) ≤ (n : ℝ) := by exact_mod_cast hn
linarith [Real.log_nonneg hnOne]
positivity
have hnot : ¬ polynomialComponent n (dseq n) ≤
K * baseRate n (dseq n) := fun h => hne (inr (inl h))
rw [div_lt_iff₀ hu]
dsimp [K] at hnot
have := lt_of_not_ge hnot
field_simp [ha.ne'] at this ⊢
nlinarith
have hbs : Tendsto (fun n => baseRate n (dseq n) / (sseq n) ^ 2)
atTop (nhds 0) := by
refine tendsto_order.2 ⟨?_, ?_⟩
· intro a ha
filter_upwards [hvalid] with n hv
exact ha.trans_le (div_nonneg (by unfold baseRate; positivity) (sq_nonneg _))
· intro a ha
let K := a⁻¹
have hK : 0 ≤ K := inv_nonneg.mpr ha.le
have havoid := benchmark_divergence_eventually_avoids_elbows
dseq sseq hvalid hdiv K hK
filter_upwards [hvalid, havoid] with n hv hne
have hspos : 0 < (sseq n) ^ 2 := by
have hnot : ¬ (sseq n) ^ 2 ≤ K * baseRate n (dseq n) :=
fun h => hne (inr (inr h))
have hb : 0 ≤ baseRate n (dseq n) := by unfold baseRate; positivity
nlinarith [mul_nonneg hK hb]
have hnot : ¬ (sseq n) ^ 2 ≤ K * baseRate n (dseq n) :=
fun h => hne (inr (inr h))
rw [div_lt_iff₀ hspos]
dsimp [K] at hnot
have := lt_of_not_ge hnot
field_simp [ha.ne'] at this ⊢
nlinarith
have hs0 : Tendsto (fun n => (sseq n) ^ 2) atTop (nhds 0) := by
refine tendsto_order.2 ⟨fun a ha => ?_, ?_⟩
· exact Filter.Eventually.of_forall fun n => ha.trans_le (sq_nonneg _)
· intro a ha
let r := Real.sqrt a
have hr : 0 < r := Real.sqrt_pos.2 ha
let D := max 1 (r ^ (-2 : ℤ))
have hlarge : ∀ᶠ n in atTop, D < frontierRate n (dseq n) (sseq n) /
converseRate n (dseq n) (sseq n) := hdiv (eventually_gt_atTop D)
filter_upwards [hvalid, eventually_gt_atTop (0 : ℕ), hlarge] with n hv hn hlarge_n
by_contra hnot
have hrs : r ≤ sseq n := by
have hsnonneg := hv.2.1
nlinarith [Real.sq_sqrt ha.le]
have hrate := frontierRate_le_converseRate_fixed_radius
n (dseq n) (sseq n) r hn hr hv.2.1 hv.2.2 hrs
have hconv : 0 < converseRate n (dseq n) (sseq n) := by
unfold converseRate
have ha_n : 0 < 1 / (n : ℝ) := by positivity
have hu : 0 ≤ polynomialComponent n (dseq n) := by
unfold polynomialComponent
positivity
have hm : 0 ≤ min 1 (polynomialComponent n (dseq n)) :=
le_min zero_le_one hu
have hdterm : 0 ≤ min 1 ((dseq n : ℝ) / (n : ℝ) ^ 2) := by positivity
nlinarith [mul_nonneg (sq_nonneg (sseq n)) hm]
have hratio : frontierRate n (dseq n) (sseq n) /
converseRate n (dseq n) (sseq n) ≤ D := by
rw [div_le_iff₀ hconv]
exact hrate
linarith
have hu0 : Tendsto (fun n => polynomialComponent n (dseq n))
atTop (nhds 0) := by
refine tendsto_order.2 ⟨?_, ?_⟩
· intro a ha
exact Filter.Eventually.of_forall fun n => ha.trans_le (by
unfold polynomialComponent
positivity)
· intro a ha
have havoid0 := benchmark_divergence_eventually_avoids_elbows
dseq sseq hvalid hdiv 0 (le_refl 0)
by_cases ha1 : 1 ≤ a
· filter_upwards [havoid0] with n hne
exact (lt_of_not_ge fun h => hne (inl h)).trans_le ha1
· have ha_lt_one : a < 1 := lt_of_not_ge ha1
have hbu_half : ∀ᶠ n in atTop,
baseRate n (dseq n) / polynomialComponent n (dseq n) < 1 / 2 :=
hbu.eventually (Iio_mem_nhds (by norm_num))
have hs_half : ∀ᶠ n in atTop, (sseq n) ^ 2 < a / 2 :=
hs0.eventually (Iio_mem_nhds (by linarith))
have hlarge : ∀ᶠ n in atTop, a⁻¹ <
frontierRate n (dseq n) (sseq n) /
converseRate n (dseq n) (sseq n) :=
hdiv (eventually_gt_atTop a⁻¹)
filter_upwards [hvalid, eventually_gt_atTop (0 : ℕ), havoid0,
hbu_half, hs_half, hlarge] with n hv hn hne hbu_n hs_n hlarge_n
by_contra hua
have hu_ge : a ≤ polynomialComponent n (dseq n) := le_of_not_gt hua
have hu_lt : polynomialComponent n (dseq n) < 1 :=
lt_of_not_ge fun h => hne (inl h)
have hu_pos : 0 < polynomialComponent n (dseq n) := ha.trans_le hu_ge
have hb_lt : baseRate n (dseq n) < polynomialComponent n (dseq n) / 2 := by
rw [div_lt_iff₀ hu_pos] at hbu_n
nlinarith
have ht_lt : (dseq n : ℝ) / (n : ℝ) ^ 2 <
polynomialComponent n (dseq n) / 2 :=
(show (dseq n : ℝ) / (n : ℝ) ^ 2 ≤ baseRate n (dseq n) by
unfold baseRate
linarith [show 0 ≤ 1 / (n : ℝ) by positivity]).trans_lt hb_lt
have hs_lt : (sseq n) ^ 2 < polynomialComponent n (dseq n) / 2 :=
hs_n.trans_le (div_le_div_of_nonneg_right hu_ge (by norm_num))
have hsum : (sseq n) ^ 2 + (dseq n : ℝ) / (n : ℝ) ^ 2 <
polynomialComponent n (dseq n) := by linarith
have ht_one : (dseq n : ℝ) / (n : ℝ) ^ 2 ≤ 1 := by linarith
have hconv : 0 < converseRate n (dseq n) (sseq n) := by
unfold converseRate
positivity
have hform_front : frontierRate n (dseq n) (sseq n) =
baseRate n (dseq n) + (sseq n) ^ 2 := by
unfold frontierRate collisionComponent baseRate
rw [min_eq_right hsum.le, min_eq_right (by linarith)]
ring
have hform_conv : converseRate n (dseq n) (sseq n) =
baseRate n (dseq n) + (sseq n) ^ 2 *
polynomialComponent n (dseq n) := by
unfold converseRate baseRate
rw [min_eq_right ht_one, min_eq_right hu_lt.le]
have hb0 : 0 ≤ baseRate n (dseq n) := by unfold baseRate; positivity
have hsnonneg : 0 ≤ (sseq n) ^ 2 := sq_nonneg _
have hrate : frontierRate n (dseq n) (sseq n) /
converseRate n (dseq n) (sseq n) ≤ a⁻¹ := by
have hden : 0 < baseRate n (dseq n) + (sseq n) ^ 2 *
polynomialComponent n (dseq n) := hform_conv ▸ hconv
rw [hform_front, hform_conv, div_le_iff₀ hden]
field_simp [ha.ne']
nlinarith [mul_le_mul_of_nonneg_left hu_ge hsnonneg]
linarith
exact ⟨hvalid, hbu, hu0, hbs, hs0⟩
The benchmarks match at every fixed positive radius and in each stated elbow regime; any divergence is confined to the residual wedge, which is nonempty.
Formal statement
conclusion
∀ epsilon : ℝ
if
0 < epsilon
and
epsilon < 1 / 2
then
(∀ n d : ℕ, ∀ sigma : ℝ, 0 ≤ sigma → sigma ≤ 2 → logEN n ^ 2 < d → (d : ℝ) < n * logEN n → (1 / 2 : ℝ) * (baseRate n d + min (polynomialComponent n d) (sigma ^ 2)) ≤ selectorBenchmark n d sigma ∧ selectorBenchmark n d sigma ≤ baseRate n d + min (polynomialComponent n d) (sigma ^ 2) ∧ productBenchmark n d sigma = baseRate n d + sigma ^ 2 * polynomialComponent n d) ∧
(∀ sigmaLower : ℝ, 0 < sigmaLower → sigmaLower ≤ 2 → ∃ c C : ℝ, 0 < c ∧ c ≤ C ∧ ∀ n d : ℕ, ∀ M sigma : ℝ, 0 < n → 0 < d → 1 ≤ M → 0 ≤ sigma → sigmaLower ≤ sigma → sigma ≤ 2 → c * M ^ 2 * frontierRate n d sigma ≤ minimaxRisk n d epsilon M sigma ∧ minimaxRisk n d epsilon M sigma ≤ C * M ^ 2 * frontierRate n d sigma) ∧
(∀ K : ℝ, 0 ≤ K → ∃ c C : ℝ, 0 < c ∧ c ≤ C ∧ ∀ n d : ℕ, ∀ M sigma : ℝ, 0 < n → 0 < d → 1 ≤ M → 0 ≤ sigma → sigma ≤ 2 → MatchingElbow K n d sigma → c * M ^ 2 * frontierRate n d sigma ≤ minimaxRisk n d epsilon M sigma ∧ minimaxRisk n d epsilon M sigma ≤ C * M ^ 2 * frontierRate n d sigma) ∧
(∀ n d : ℕ, 0 < n → (polynomialComponent n d ≤ baseRate n d ↔ (d : ℝ) ^ 2 ≤ n * logEN n ^ 2 + d * logEN n ^ 2)) ∧
(∀ dseq : ℕ → ℕ, ∀ sseq : ℕ → ℝ, (∀ᶠ t in atTop, 1 ≤ dseq t ∧ 0 ≤ sseq t ∧ sseq t ≤ 2) → BenchmarkRatioDiverges dseq sseq → ResidualWedge dseq sseq) ∧
ResidualWedge (fun n => n) (fun n => (logEN n) ^ (-1 / 2 : ℝ)) ∧
EventuallySameOrder (fun n => selectorBenchmark n n ((logEN n) ^ (-1 / 2 : ℝ)) / productBenchmark n n ((logEN n) ^ (-1 / 2 : ℝ))) logEN ∧
Tendsto (fun n => selectorBenchmark n n ((logEN n) ^ (-1 / 2 : ℝ)) / productBenchmark n n ((logEN n) ^ (-1 / 2 : ℝ))) atTop atTop
Proof (Lean source)
theorem fixed_interior_tightness_and_shrinking_radius_gap_all_d :
∀ epsilon : ℝ, 0 < epsilon → epsilon < 1 / 2 →
(∀ n d : ℕ, ∀ sigma : ℝ,
0 ≤ sigma → sigma ≤ 2 →
logEN n ^ 2 < d → (d : ℝ) < n * logEN n →
(1 / 2 : ℝ) *
(baseRate n d + min (polynomialComponent n d) (sigma ^ 2)) ≤
selectorBenchmark n d sigma ∧
selectorBenchmark n d sigma ≤
baseRate n d + min (polynomialComponent n d) (sigma ^ 2) ∧
productBenchmark n d sigma =
baseRate n d + sigma ^ 2 * polynomialComponent n d) ∧
(∀ sigmaLower : ℝ, 0 < sigmaLower → sigmaLower ≤ 2 →
∃ c C : ℝ, 0 < c ∧ c ≤ C ∧
∀ n d : ℕ, ∀ M sigma : ℝ,
0 < n → 0 < d → 1 ≤ M → 0 ≤ sigma →
sigmaLower ≤ sigma → sigma ≤ 2 →
c * M ^ 2 * frontierRate n d sigma ≤
minimaxRisk n d epsilon M sigma ∧
minimaxRisk n d epsilon M sigma ≤
C * M ^ 2 * frontierRate n d sigma) ∧
(∀ K : ℝ, 0 ≤ K →
∃ c C : ℝ, 0 < c ∧ c ≤ C ∧
∀ n d : ℕ, ∀ M sigma : ℝ,
0 < n → 0 < d → 1 ≤ M → 0 ≤ sigma → sigma ≤ 2 →
MatchingElbow K n d sigma →
c * M ^ 2 * frontierRate n d sigma ≤
minimaxRisk n d epsilon M sigma ∧
minimaxRisk n d epsilon M sigma ≤
C * M ^ 2 * frontierRate n d sigma) ∧
(∀ n d : ℕ, 0 < n →
(polynomialComponent n d ≤ baseRate n d ↔
(d : ℝ) ^ 2 ≤ n * logEN n ^ 2 + d * logEN n ^ 2)) ∧
(∀ n d : ℕ, 0 < n →
(d : ℝ) ≤ sqrt n * logEN n →
polynomialComponent n d ≤ baseRate n d) ∧
(∀ n d : ℕ, 0 < n →
(d : ℝ) ≤ logEN n ^ 2 →
polynomialComponent n d ≤ baseRate n d) ∧
(∀ dseq : ℕ → ℕ, ∀ sseq : ℕ → ℝ,
(∀ᶠ t in atTop, 1 ≤ dseq t ∧ 0 ≤ sseq t ∧ sseq t ≤ 2) →
BenchmarkRatioDiverges dseq sseq → ResidualWedge dseq sseq) ∧
ResidualWedge (fun n => n) (fun n => (logEN n) ^ (-1 / 2 : ℝ)) ∧
EventuallySameOrder
(fun n => selectorBenchmark n n ((logEN n) ^ (-1 / 2 : ℝ)) /
productBenchmark n n ((logEN n) ^ (-1 / 2 : ℝ)))
logEN ∧
Tendsto (fun n =>
selectorBenchmark n n ((logEN n) ^ (-1 / 2 : ℝ)) /
productBenchmark n n ((logEN n) ^ (-1 / 2 : ℝ))) atTop atTop := by
intro epsilon hepsilon hepsilon_half
refine ⟨?_,
(fun sigmaLower hsigmaLower _ =>
fixed_radius_minimax_equivalence epsilon sigmaLower hepsilon hepsilon_half
hsigmaLower),
(fun K hK =>
matching_elbow_minimax_equivalence epsilon K hepsilon hepsilon_half hK),
(fun n d hn => polynomialComponent_le_baseRate_iff n d hn),
(fun n d hn hd => polynomialComponent_le_baseRate_of_le_sqrt n d hn hd),
(fun n d hn hd => polynomialComponent_le_baseRate_of_le_log_sq n d hn hd), ?_,
diagonal_residual_wedge, diagonal_benchmark_eventually_same_order,
diagonal_benchmark_ratio_tendsto_atTop⟩
· intro n d sigma hsigma hsigma_two hlower hupper
obtain ⟨hlo, hup⟩ := selectorBenchmark_triangular n d sigma
hsigma hsigma_two hlower hupper
exact ⟨hlo, hup, rfl⟩
· intro dseq sseq hvalid hdiv
exact benchmark_divergence_implies_residual_wedge dseq sseq hvalid hdiv
Restricted-range specialization of the fixed-interior and shrinking-radius phase theorem.
Formal statement
conclusion
∀ epsilon : ℝ
if
0 < epsilon
and
epsilon < 1 / 2
then
∃ c_epsilon : ℝ,
0 < c_epsilon ∧
(∀ n d : ℕ, ∀ sigma : ℝ, 0 ≤ sigma → sigma ≤ 2 → logEN n ^ 2 < d → (d : ℝ) < n * logEN n → (1 / 2 : ℝ) * (baseRate n d + min (polynomialComponent n d) (sigma ^ 2)) ≤ selectorBenchmark n d sigma ∧ selectorBenchmark n d sigma ≤ baseRate n d + min (polynomialComponent n d) (sigma ^ 2) ∧ productBenchmark n d sigma = baseRate n d + sigma ^ 2 * polynomialComponent n d) ∧
(∀ sigmaLower : ℝ, 0 < sigmaLower → sigmaLower ≤ 2 → ∃ c C : ℝ, 0 < c ∧ c ≤ C ∧ ∀ n d : ℕ, ∀ M sigma : ℝ, 0 < n → 0 < d → 1 ≤ M → 0 ≤ sigma → sigmaLower ≤ sigma → sigma ≤ 2 → (d : ℝ) ≤ c_epsilon * (n : ℝ) ^ 2 / logEN n → c * M ^ 2 * frontierRate n d sigma ≤ minimaxRisk n d epsilon M sigma ∧ minimaxRisk n d epsilon M sigma ≤ C * M ^ 2 * frontierRate n d sigma) ∧
(∀ K : ℝ, 0 ≤ K → ∃ c C : ℝ, 0 < c ∧ c ≤ C ∧ ∀ n d : ℕ, ∀ M sigma : ℝ, 0 < n → 0 < d → 1 ≤ M → 0 ≤ sigma → sigma ≤ 2 → MatchingElbow K n d sigma → c * M ^ 2 * frontierRate n d sigma ≤ minimaxRisk n d epsilon M sigma ∧ minimaxRisk n d epsilon M sigma ≤ C * M ^ 2 * frontierRate n d sigma) ∧
(∀ n d : ℕ, 0 < n → (polynomialComponent n d ≤ baseRate n d ↔ (d : ℝ) ^ 2 ≤ n * logEN n ^ 2 + d * logEN n ^ 2)) ∧
(∀ dseq : ℕ → ℕ, ∀ sseq : ℕ → ℝ, (∀ᶠ t in atTop, 1 ≤ dseq t ∧ 0 ≤ sseq t ∧ sseq t ≤ 2) → BenchmarkRatioDiverges dseq sseq → ResidualWedge dseq sseq) ∧
ResidualWedge (fun n => n) (fun n => (logEN n) ^ (-1 / 2 : ℝ)) ∧
EventuallySameOrder (fun n => selectorBenchmark n n ((logEN n) ^ (-1 / 2 : ℝ)) / productBenchmark n n ((logEN n) ^ (-1 / 2 : ℝ))) logEN ∧
Tendsto (fun n => selectorBenchmark n n ((logEN n) ^ (-1 / 2 : ℝ)) / productBenchmark n n ((logEN n) ^ (-1 / 2 : ℝ))) atTop atTop
Proof (Lean source)
theorem fixed_interior_tightness_and_shrinking_radius_gap :
∀ epsilon : ℝ, 0 < epsilon → epsilon < 1 / 2 →
∃ c_epsilon : ℝ, 0 < c_epsilon ∧
(∀ n d : ℕ, ∀ sigma : ℝ,
0 ≤ sigma → sigma ≤ 2 →
logEN n ^ 2 < d → (d : ℝ) < n * logEN n →
(1 / 2 : ℝ) *
(baseRate n d + min (polynomialComponent n d) (sigma ^ 2)) ≤
selectorBenchmark n d sigma ∧
selectorBenchmark n d sigma ≤
baseRate n d + min (polynomialComponent n d) (sigma ^ 2) ∧
productBenchmark n d sigma =
baseRate n d + sigma ^ 2 * polynomialComponent n d) ∧
(∀ sigmaLower : ℝ, 0 < sigmaLower → sigmaLower ≤ 2 →
∃ c C : ℝ, 0 < c ∧ c ≤ C ∧
∀ n d : ℕ, ∀ M sigma : ℝ,
0 < n → 0 < d → 1 ≤ M → -- @realizes M(range [1,infinity))
0 ≤ sigma → sigmaLower ≤ sigma → sigma ≤ 2 →
(d : ℝ) ≤ c_epsilon * (n : ℝ) ^ 2 / logEN n →
c * M ^ 2 * frontierRate n d sigma ≤
minimaxRisk n d epsilon M sigma ∧
minimaxRisk n d epsilon M sigma ≤
C * M ^ 2 * frontierRate n d sigma) ∧
(∀ K : ℝ, 0 ≤ K →
∃ c C : ℝ, 0 < c ∧ c ≤ C ∧
∀ n d : ℕ, ∀ M sigma : ℝ,
0 < n → 0 < d → 1 ≤ M → -- @realizes M(range [1,infinity))
0 ≤ sigma → sigma ≤ 2 →
MatchingElbow K n d sigma →
c * M ^ 2 * frontierRate n d sigma ≤
minimaxRisk n d epsilon M sigma ∧
minimaxRisk n d epsilon M sigma ≤
C * M ^ 2 * frontierRate n d sigma) ∧
(∀ n d : ℕ, 0 < n →
(polynomialComponent n d ≤ baseRate n d ↔
(d : ℝ) ^ 2 ≤ n * logEN n ^ 2 + d * logEN n ^ 2)) ∧
(∀ dseq : ℕ → ℕ, ∀ sseq : ℕ → ℝ,
(∀ᶠ t in atTop, 1 ≤ dseq t ∧ 0 ≤ sseq t ∧ sseq t ≤ 2) →
BenchmarkRatioDiverges dseq sseq → ResidualWedge dseq sseq) ∧
ResidualWedge (fun n => n) (fun n => (logEN n) ^ (-1 / 2 : ℝ)) ∧
EventuallySameOrder
(fun n => selectorBenchmark n n ((logEN n) ^ (-1 / 2 : ℝ)) /
productBenchmark n n ((logEN n) ^ (-1 / 2 : ℝ)))
logEN ∧
Tendsto (fun n =>
selectorBenchmark n n ((logEN n) ^ (-1 / 2 : ℝ)) /
productBenchmark n n ((logEN n) ^ (-1 / 2 : ℝ))) atTop atTop := by
intro epsilon hepsilon hepsilon_half
obtain ⟨htriangular, hfixed, helbows, hexact_algebra, _hsqrt, _hsmall,
hwedge, hwitness, hsame_order, hdiverges⟩ :=
fixed_interior_tightness_and_shrinking_radius_gap_all_d
epsilon hepsilon hepsilon_half
refine ⟨1, zero_lt_one, ?_⟩
exact ⟨htriangular, fun sigmaLower hsigmaLower hsigmaLower_two => by
obtain ⟨c, C, hc, hcC, hall⟩ :=
hfixed sigmaLower hsigmaLower hsigmaLower_two
exact ⟨c, C, hc, hcC,
fun n d M sigma hn hd hM hsigma_nonneg hsigma hsigma_two _ =>
hall n d M sigma hn hd hM hsigma_nonneg hsigma hsigma_two⟩,
fun K hK => by
obtain ⟨c, C, hc, hcC, hall⟩ := helbows K hK
exact ⟨c, C, hc, hcC, fun n d M sigma hn hd hM hsigma hsigma_two helbow =>
hall n d M sigma hn hd hM hsigma hsigma_two helbow⟩,
hexact_algebra, hwedge, hwitness, hsame_order, hdiverges⟩
T_FrontierUpper 5 declarations
Asymptotic comparison with the published collision remainder.
Definition (Lean source)
def PublishedCollisionComparison : Prop :=
(∀ n d : ℕ, ∀ sigma : ℝ,
0 < n → 0 < d → 0 ≤ sigma → sigma ≤ 2 →
min 1 (min (polynomialComponent n d) (collisionComponent n d sigma)) ≤
collisionComponent n d sigma) ∧
∀ dseq : ℕ → ℕ, ∀ sseq : ℕ → ℝ,
(∀ n, 0 < n →
0 < dseq n ∧ 0 ≤ sseq n ∧ sseq n ≤ 2) →
Tendsto (fun n => polynomialComponent n (dseq n) /
collisionComponent n (dseq n) (sseq n)) atTop (nhds 0) →
Tendsto (fun n => (dseq n : ℝ) / ((n : ℝ) * logEN n)) atTop (nhds 0) →
Tendsto (fun n =>
min 1 (min (polynomialComponent n (dseq n))
(collisionComponent n (dseq n) (sseq n))) /
collisionComponent n (dseq n) (sseq n)) atTop (nhds 0)
The selector risk bound without the secondary published-comparison clause.
Formal statement
conclusion
∀ epsilon : ℝ,
0 < epsilon → epsilon < 1 / 2 → ∃ C_epsilon : ℝ, ∃ handle : PolynomialHandle, 0 < C_epsilon ∧ ∀ n d : ℕ, ∀ M sigma : ℝ, 0 < n → 0 < d → ∀ hM : 1
≤ M, 0 ≤ sigma → sigma ≤ 2 → let poly := polyEstimatorElement (n := n) (d := d) handle hM let collision := collisionEstimatorElement (n := n) (d := d) hM let selector := totalSelector hM sigma poly collision Measurable selector.1 ∧
(∀ s : Fin n → Obs d, selector.1 s ∈ Icc (-M) M) ∧
(∀ P : ModelClass d epsilon M sigma, mse P.law selector.1 ≤ C_epsilon * M ^ 2 * frontierRate n d sigma)
Proof (Lean source)
lemma frontier_upper_bound_all_d :
∀ epsilon : ℝ, 0 < epsilon → epsilon < 1 / 2 →
∃ C_epsilon : ℝ, ∃ handle : PolynomialHandle,
0 < C_epsilon ∧
∀ n d : ℕ, ∀ M sigma : ℝ,
0 < n → 0 < d → ∀ hM : 1 ≤ M, 0 ≤ sigma → sigma ≤ 2 →
let poly := polyEstimatorElement (n := n) (d := d) handle hM
let collision := collisionEstimatorElement (n := n) (d := d) hM
let selector := totalSelector hM sigma poly collision
Measurable selector.1 ∧
(∀ s : Fin n → Obs d, selector.1 s ∈ Icc (-M) M) ∧
(∀ P : ModelClass d epsilon M sigma,
mse P.law selector.1 ≤
C_epsilon * M ^ 2 * frontierRate n d sigma) := by
intro epsilon hepsilon hepsilon_half
obtain ⟨C0, handle, hC0, hall⟩ :=
robust_upper_construction_resolution_all_d
epsilon hepsilon hepsilon_half
let C := max 1 C0
have hC : 0 < C := lt_of_lt_of_le zero_lt_one (le_max_left _ _)
have hC0C : C0 ≤ C := le_max_right _ _
refine ⟨C, handle, hC, ?_⟩
intro n d M sigma hn hd hM hsigma hsigma_two
obtain ⟨hpoly_meas, hpoly_mem, hcollision_meas, hcollision_mem, hrisk⟩ :=
hall n d M sigma hn hd hM hsigma hsigma_two
let poly := polyEstimatorElement (n := n) (d := d) handle hM
let collision := collisionEstimatorElement (n := n) (d := d) hM
let selector := totalSelector hM sigma poly collision
have hpoly_nonneg : 0 ≤ polynomialComponent n d := by
unfold polynomialComponent
positivity
have hcollision_nonneg : 0 ≤ collisionComponent n d sigma := by
unfold collisionComponent
positivity
refine ⟨(totalSelector hM sigma poly collision).2.1,
(totalSelector hM sigma poly collision).2.2, ?_⟩
· intro P
rcases hrisk P with ⟨hpoly_risk, hcollision_risk⟩
change mse P.law (totalSelector hM sigma poly collision).1 ≤
C * M ^ 2 * frontierRate n d sigma
unfold totalSelector
by_cases hcollision : collisionComponent n d sigma ≤
min 1 (polynomialComponent n d)
· rw [if_pos hcollision]
dsimp only [collision]
refine hcollision_risk.trans ?_
have hmin : min 1
(min (polynomialComponent n d) (collisionComponent n d sigma)) =
collisionComponent n d sigma := by
rw [min_eq_right (hcollision.trans (min_le_right _ _))]
exact min_eq_right (hcollision.trans (min_le_left _ _))
rw [frontierRate, hmin]
unfold collisionComponent
have hrate : 0 ≤ M ^ 2 *
(1 / (n : ℝ) + (sigma ^ 2 + (d : ℝ) / (n : ℝ) ^ 2)) := by
positivity
nlinarith [mul_le_mul_of_nonneg_right hC0C hrate]
· rw [if_neg hcollision]
unfold mse
by_cases hpoly : polynomialComponent n d <
min 1 (collisionComponent n d sigma)
· simp only [hpoly]
change mse P.law poly.1 ≤ C * M ^ 2 * frontierRate n d sigma
dsimp only [poly]
have hpoly_one : min 1 (polynomialComponent n d) =
polynomialComponent n d :=
min_eq_right (le_of_lt (hpoly.trans_le (min_le_left _ _)))
rw [hpoly_one] at hpoly_risk
refine hpoly_risk.trans ?_
have hmin : min 1
(min (polynomialComponent n d) (collisionComponent n d sigma)) =
polynomialComponent n d := by
rw [min_eq_left (le_of_lt (hpoly.trans_le (min_le_right _ _)))]
exact min_eq_right (le_of_lt (hpoly.trans_le (min_le_left _ _)))
rw [frontierRate, hmin]
have hrate : 0 ≤ M ^ 2 *
(1 / (n : ℝ) + polynomialComponent n d) := by
positivity
nlinarith [mul_le_mul_of_nonneg_right hC0C hrate]
· simp only [hpoly]
change mse P.law (fun _ : Fin n → Obs d => 0) ≤
C * M ^ 2 * frontierRate n d sigma
have hone_poly : 1 ≤ polynomialComponent n d := by
by_contra hnot
have hpoly_lt : polynomialComponent n d < 1 := lt_of_not_ge hnot
have hpc_lt : polynomialComponent n d < collisionComponent n d sigma := by
have := lt_of_not_ge hcollision
rw [min_eq_right hpoly_lt.le] at this
exact this
have hmc_le : min 1 (collisionComponent n d sigma) ≤
polynomialComponent n d := le_of_not_gt hpoly
by_cases hc : collisionComponent n d sigma ≤ 1
· rw [min_eq_right hc] at hmc_le
linarith
· rw [min_eq_left (le_of_not_ge hc)] at hmc_le
linarith
have hone_collision : 1 ≤ collisionComponent n d sigma := by
have hmin_poly : min 1 (polynomialComponent n d) = 1 :=
min_eq_left hone_poly
simpa [hmin_poly] using lt_of_not_ge hcollision |>.le
have hmin : min 1
(min (polynomialComponent n d) (collisionComponent n d sigma)) = 1 := by
exact min_eq_left (le_min hone_poly hone_collision)
let U : UnrestrictedClass d epsilon M :=
{ law := P.law
epsilon_pos := P.epsilon_pos
epsilon_lt_half := P.epsilon_lt_half
M_ge_one := P.M_ge_one
consistency := P.consistency
exchangeability := P.exchangeability
overlap := P.overlap
mean_normalization := P.mean_normalization
second_moment := P.second_moment }
have hate : |rawAteFormula P.law| ≤ M := by
simpa [U] using
(scale_sanity (d := d) (epsilon := epsilon) (M := M)).1 U |>.2.1
have hmse : mse P.law (fun _ : Fin n → Obs d => 0) =
rawAteFormula P.law ^ 2 := by
unfold mse
simp
rw [hmse]
unfold frontierRate
rw [hmin]
have hate_sq : rawAteFormula P.law ^ 2 ≤ M ^ 2 := by
rw [← sq_abs]
exact (sq_le_sq₀ (abs_nonneg _)
(le_trans zero_le_one hM)).2 hate
have hfactor : M ^ 2 ≤ C * M ^ 2 * (1 / (n : ℝ) + 1) := by
have hn_real : 0 < (n : ℝ) := by exact_mod_cast hn
have hC_one : 1 ≤ C := le_max_left _ _
have hM_sq : 0 ≤ M ^ 2 := sq_nonneg M
have hone : 1 ≤ C * (1 / (n : ℝ) + 1) := by
nlinarith [one_div_pos.mpr hn_real]
nlinarith
exact hate_sq.trans hfactor
The total selector achieves the all-alphabet frontier rate.
Formal statement
conclusion
∀ epsilon : ℝ,
0 < epsilon → epsilon < 1 / 2 → ∃ C_epsilon : ℝ, ∃ handle : PolynomialHandle, 0 < C_epsilon ∧ ∀ n d : ℕ, ∀ M sigma : ℝ, 0 < n → 0 < d → ∀ hM : 1
≤ M, 0 ≤ sigma → sigma ≤ 2 → let poly := polyEstimatorElement (n := n) (d := d) handle hM let collision := collisionEstimatorElement (n := n) (d := d) hM let selector := totalSelector hM sigma poly collision Measurable selector.1 ∧
(∀ s : Fin n → Obs d, selector.1 s ∈ Icc (-M) M) ∧
(∀ P : ModelClass d epsilon M sigma, mse P.law selector.1 ≤ C_epsilon * M ^ 2 * frontierRate n d sigma)
Proof (Lean source)
theorem frontier_upper_all_d :
∀ epsilon : ℝ, 0 < epsilon → epsilon < 1 / 2 →
∃ C_epsilon : ℝ, ∃ handle : PolynomialHandle,
0 < C_epsilon ∧ -- @realizes C_{epsilon}(range [1,infinity) after enlargement)
∀ n d : ℕ, ∀ M sigma : ℝ,
0 < n → 0 < d → ∀ hM : 1 ≤ M, 0 ≤ sigma → sigma ≤ 2 →
let poly := polyEstimatorElement (n := n) (d := d) handle hM
let collision := collisionEstimatorElement (n := n) (d := d) hM
let selector := totalSelector hM sigma poly collision
Measurable selector.1 ∧
(∀ s : Fin n → Obs d, selector.1 s ∈ Icc (-M) M) ∧
(∀ P : ModelClass d epsilon M sigma,
mse P.law selector.1 ≤
C_epsilon * M ^ 2 * frontierRate n d sigma) := by
intro epsilon hepsilon hepsilon_half
exact frontier_upper_bound_all_d epsilon hepsilon hepsilon_half
Restricted-dimension selector upper bound.
Formal statement
conclusion
∀ epsilon : ℝ,
0 < epsilon → epsilon < 1 / 2 → ∃ C_epsilon c_epsilon : ℝ, ∃ handle : PolynomialHandle, 0 < C_epsilon ∧ 0 < c_epsilon ∧ ∀ n d : ℕ, ∀ M sigma : ℝ, 0 < n → 0 < d → ∀ hM : 1
≤ M, 0 ≤ sigma → sigma ≤ 2 → (d : ℝ) ≤ c_epsilon * (n : ℝ) ^ 2 / logEN n → let poly := polyEstimatorElement (n := n) (d := d) handle hM let collision := collisionEstimatorElement (n := n) (d := d) hM let selector := totalSelector hM sigma poly collision Measurable selector.1 ∧
(∀ s : Fin n → Obs d, selector.1 s ∈ Icc (-M) M) ∧
(∀ P : ModelClass d epsilon M sigma, mse P.law selector.1 ≤ C_epsilon * M ^ 2 * frontierRate n d sigma)
Proof (Lean source)
theorem frontier_upper :
∀ epsilon : ℝ, 0 < epsilon → epsilon < 1 / 2 →
∃ C_epsilon c_epsilon : ℝ, ∃ handle : PolynomialHandle,
0 < C_epsilon ∧ 0 < c_epsilon ∧
∀ n d : ℕ, ∀ M sigma : ℝ,
0 < n → 0 < d → ∀ hM : 1 ≤ M, 0 ≤ sigma → sigma ≤ 2 →
(d : ℝ) ≤ c_epsilon * (n : ℝ) ^ 2 / logEN n →
let poly := polyEstimatorElement (n := n) (d := d) handle hM
let collision := collisionEstimatorElement (n := n) (d := d) hM
let selector := totalSelector hM sigma poly collision
Measurable selector.1 ∧
(∀ s : Fin n → Obs d, selector.1 s ∈ Icc (-M) M) ∧
(∀ P : ModelClass d epsilon M sigma,
mse P.law selector.1 ≤
C_epsilon * M ^ 2 * frontierRate n d sigma) := by
intro epsilon hepsilon hepsilon_half
obtain ⟨C_epsilon, handle, hC, hbound⟩ :=
frontier_upper_all_d epsilon hepsilon hepsilon_half
refine ⟨C_epsilon, 1, handle, hC, zero_lt_one, ?_⟩
intro n d M sigma hn hd hM hsigma hsigma_two _
exact hbound n d M sigma hn hd hM hsigma hsigma_two
The cited binary collision guarantee and the independent algebraic comparison to its remainder hold simultaneously.
Formal statement
conclusion
∀ epsilon : ℝ
Proof (Lean source)
theorem published_binary_collision_comparison :
∀ epsilon : ℝ, 0 < epsilon → epsilon < 1 / 2 →
ZengBinaryCollisionUpper epsilon →
ZengBinaryCollisionUpper epsilon ∧ PublishedCollisionComparison := by
intro epsilon hepsilon hepsilon_half hcollision
refine ⟨hcollision, ?_⟩
refine ⟨?_, ?_⟩
· intro n d sigma hn hd hsigma hsigma_two
exact le_trans (min_le_right _ _) (min_le_right _ _)
· intro dseq sseq hdomain hpoly hdim
refine squeeze_zero' ?_ ?_ hpoly
· filter_upwards [eventually_gt_atTop (0 : ℕ)] with n hn
have hd : 0 < dseq n := (hdomain n hn).1
have hn_real : 0 < (n : ℝ) := by exact_mod_cast hn
have hd_real : 0 < (dseq n : ℝ) := by exact_mod_cast hd
have hpoly_nonneg : 0 ≤ polynomialComponent n (dseq n) := by
unfold polynomialComponent
positivity
have hcollision_nonneg :
0 ≤ collisionComponent n (dseq n) (sseq n) := by
unfold collisionComponent
positivity
have hnumerator_nonneg :
0 ≤ min 1 (min (polynomialComponent n (dseq n))
(collisionComponent n (dseq n) (sseq n))) := by
positivity
exact div_nonneg hnumerator_nonneg hcollision_nonneg
· filter_upwards [eventually_gt_atTop (0 : ℕ)] with n hn
have hd : 0 < dseq n := (hdomain n hn).1
have hn_real : 0 < (n : ℝ) := by exact_mod_cast hn
have hd_real : 0 < (dseq n : ℝ) := by exact_mod_cast hd
have hcollision_pos :
0 < collisionComponent n (dseq n) (sseq n) := by
unfold collisionComponent
positivity
apply div_le_div_of_nonneg_right _ hcollision_pos.le
exact le_trans (min_le_right _ _) (min_le_left _ _)
T_RadiusChannelConverse 19 declarations
theorem radial_risk_of_kernel_transport
If the outcome scale satisfies its stated bound and the transport scale satisfies the stated condition and the target law is the specified transported law and the target parameter has the stated affine relation and the source parameter set has the stated form and the target separation has the stated scaling, the generic Markov-kernel comparison specializes to the radial source family: once the one-coordinate channel law and affine target identity are available, every bounded ambient estimator is defeated by a radial source member at the transported scale.
Formal statement
hypotheses
n d :
ℕ
epsilon M sigma c L a b :
ℝ
H :
LeastFavorableHandle n d epsilon M sigma
hM :
0 ≤ M
K :
Kernel (BinObs H.radialCap) (Obs d)
ha :
a ≠ 0
hQ :
∀ P ∈ H.radialSource,
(H.radialEmbedding P).observedLaw
= K ∘ₘ CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P.1
htheta :
∀ P ∈ H.radialSource,
rawAteFormula (H.radialEmbedding P)
= a * CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1 + b
hsource :
∀ sourceEst : (Fin n → BinObs H.radialCap) → ℝ
if
Measurable sourceEst
and
UniformlyBounded sourceEst
then
∃ P ∈ H.radialSource,
L
≤ sqRisk (CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n) sourceEst (CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1)
hscale :
c * M ^ 2 * sigma ^ 2 * min 1 (polynomialComponent n d) ≤ a ^ 2 * L
est :
Estimator n d M
conclusion
∃ P ∈ H.radialSource,
c * M ^ 2 * sigma ^ 2 * min 1 (polynomialComponent n d) ≤ mse (H.radialEmbedding P) est.1
Proof (Lean source)
lemma radial_risk_of_kernel_transport {n d : ℕ} {epsilon M sigma c L a b : ℝ}
(H : LeastFavorableHandle n d epsilon M sigma) (hM : 0 ≤ M)
(K : Kernel (BinObs H.radialCap) (Obs d))
[IsMarkovKernel K]
(ha : a ≠ 0)
(hQ : ∀ P ∈ H.radialSource,
(H.radialEmbedding P).observedLaw =
K ∘ₘ CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P.1)
(htheta : ∀ P ∈ H.radialSource,
rawAteFormula (H.radialEmbedding P) =
a * CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1 + b)
(hsource : ∀ sourceEst : (Fin n → BinObs H.radialCap) → ℝ,
Measurable sourceEst → UniformlyBounded sourceEst →
∃ P ∈ H.radialSource,
L ≤ sqRisk
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n)
sourceEst
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1))
(hscale : c * M ^ 2 * sigma ^ 2 * min 1 (polynomialComponent n d) ≤
a ^ 2 * L) (est : Estimator n d M) :
∃ P ∈ H.radialSource,
c * M ^ 2 * sigma ^ 2 * min 1 (polynomialComponent n d) ≤
mse (H.radialEmbedding P) est.1 := by
let I := {P :
CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw
n H.radialCap epsilon // P ∈ H.radialSource}
have htransport :=
forall_estimator_exists_sqRisk_ge_of_kernel_affine_transport_pi
(Iota := I) (n := n)
(P := fun j => CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw j.1.1)
(Q := fun j => (H.radialEmbedding j.1).observedLaw)
K
(theta := fun j =>
CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional j.1.1)
a b L ha
(fun j => hQ j.1 j.2)
(by
intro sourceEst hmeas hbounded
obtain ⟨P, hP, hrisk⟩ := hsource sourceEst hmeas hbounded
exact ⟨⟨P, hP⟩, by
simpa [CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw] using hrisk⟩)
est.1 est.2.1
⟨M, hM, fun x => (abs_le).2 (est.2.2 x)⟩
obtain ⟨P, hrisk⟩ := htransport
refine ⟨P.1, P.2, hscale.trans ?_⟩
rw [← htheta P.1 P.2] at hrisk
simpa [sqRisk, productLaw, mse] using hrisk
theorem exact_risk_of_affine_transport
If the transport scale satisfies the stated condition and the affine observation map has the stated form and the target law is the specified transported law and the target parameter has the stated affine relation and the source parameter set has the stated form and the target separation has the stated scaling, the deterministic affine comparison specializes to the exact source family carried by a least-favorable handle. This is the exact-family analogue of radial_risk_of_kernel_transport and retains the witnessing source law, which is needed by RiskTransferCertificate.
Formal statement
hypotheses
n d :
ℕ
epsilon M sigma c L a b :
ℝ
H :
LeastFavorableHandle n d epsilon M sigma
phi :
BinObs H.exactCap → Obs d
ha :
a ≠ 0
hphi :
Measurable phi
hQ :
∀ P ∈ H.exactSource,
(H.exactEmbedding P).observedLaw
= Measure.map phi (CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P.1)
htheta :
∀ P ∈ H.exactSource,
rawAteFormula (H.exactEmbedding P)
= a * CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1 + b
hsource :
∀ sourceEst : (Fin n → BinObs H.exactCap) → ℝ
if
Measurable sourceEst
then
∃ P ∈ H.exactSource,
L
≤ sqRisk (CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n) sourceEst (CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1)
hscale :
c * M ^ 2 * (1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)) ≤ a ^ 2 * L
est :
Estimator n d M
conclusion
∃ P ∈ H.exactSource,
c * M ^ 2 * (1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)) ≤ mse (H.exactEmbedding P) est.1
Proof (Lean source)
lemma exact_risk_of_affine_transport {n d : ℕ} {epsilon M sigma c L a b : ℝ}
(H : LeastFavorableHandle n d epsilon M sigma)
(phi : BinObs H.exactCap → Obs d) (ha : a ≠ 0)
(hphi : Measurable phi)
(hQ : ∀ P ∈ H.exactSource,
(H.exactEmbedding P).observedLaw =
Measure.map phi
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P.1))
(htheta : ∀ P ∈ H.exactSource,
rawAteFormula (H.exactEmbedding P) =
a * CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1 + b)
(hsource : ∀ sourceEst : (Fin n → BinObs H.exactCap) → ℝ,
Measurable sourceEst →
∃ P ∈ H.exactSource,
L ≤ sqRisk
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n)
sourceEst
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1))
(hscale : c * M ^ 2 *
(1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)) ≤ a ^ 2 * L)
(est : Estimator n d M) :
∃ P ∈ H.exactSource,
c * M ^ 2 *
(1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)) ≤
mse (H.exactEmbedding P) est.1 := by
let I := {P : BinaryExactLaw n H.exactCap epsilon // P ∈ H.exactSource}
have htransport :=
forall_estimator_exists_sqRisk_ge_of_deterministic_affine_transport_pi
(Iota := I) (n := n)
(P := fun j ↦ CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw j.1.1)
(Q := fun j ↦ (H.exactEmbedding j.1).observedLaw)
(theta := fun j ↦
CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional j.1.1)
phi a b L ha hphi
(fun j ↦ hQ j.1 j.2)
(by
intro sourceEst hmeas
obtain ⟨P, hP, hrisk⟩ := hsource sourceEst hmeas
exact ⟨⟨P, hP⟩, by
simpa [CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw] using hrisk⟩)
est.1 est.2.1
obtain ⟨P, hrisk⟩ := htransport
refine ⟨P.1, P.2, hscale.trans ?_⟩
rw [← htheta P.1 P.2] at hrisk
simpa [sqRisk, productLaw, mse] using hrisk
theorem exactRiskTransfer_of_affine_transport
If the transport scale satisfies the stated condition and the affine observation map has the stated form and the target law is the specified transported law and the target parameter has the stated affine relation and the source parameter set has the stated form and the target separation has the stated scaling, pointwise deterministic affine transport packages directly as the exact half of a RiskTransferCertificate.
Formal statement
hypotheses
n d :
ℕ
epsilon M sigma c L a b :
ℝ
H :
LeastFavorableHandle n d epsilon M sigma
phi :
BinObs H.exactCap → Obs d
ha :
a ≠ 0
hphi :
Measurable phi
hQ :
∀ P ∈ H.exactSource,
(H.exactEmbedding P).observedLaw
= Measure.map phi (CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P.1)
htheta :
∀ P ∈ H.exactSource,
rawAteFormula (H.exactEmbedding P)
= a * CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1 + b
hsource :
∀ sourceEst : (Fin n → BinObs H.exactCap) → ℝ
if
Measurable sourceEst
then
∃ P ∈ H.exactSource,
L
≤ sqRisk (CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n) sourceEst (CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1)
hscale :
c * M ^ 2 * (1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)) ≤ a ^ 2 * L
Proof (Lean source)
lemma exactRiskTransfer_of_affine_transport {n d : ℕ}
{epsilon M sigma c L a b : ℝ}
(H : LeastFavorableHandle n d epsilon M sigma)
(phi : BinObs H.exactCap → Obs d) (ha : a ≠ 0)
(hphi : Measurable phi)
(hQ : ∀ P ∈ H.exactSource,
(H.exactEmbedding P).observedLaw =
Measure.map phi
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P.1))
(htheta : ∀ P ∈ H.exactSource,
rawAteFormula (H.exactEmbedding P) =
a * CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1 + b)
(hsource : ∀ sourceEst : (Fin n → BinObs H.exactCap) → ℝ,
Measurable sourceEst →
∃ P ∈ H.exactSource,
L ≤ sqRisk
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n)
sourceEst
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1))
(hscale : c * M ^ 2 *
(1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)) ≤ a ^ 2 * L) :
∀ est : Estimator n d M,
∃ P ∈ H.exactSource,
c * M ^ 2 *
(1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)) ≤
mse (H.exactEmbedding P) est.1 := by
intro est
exact exact_risk_of_affine_transport H phi ha hphi hQ htheta hsource hscale est
theorem riskTransferCertificate_of_kernel_transport
If the outcome scale satisfies its stated bound and the transport scale satisfies the stated condition and the target law is the specified transported law and the target parameter has the stated affine relation and the source parameter set has the stated form and the target separation has the stated scaling and the exact risk-transfer identity holds, a radial Markov-kernel transport and the independently transferred exact family together discharge the two halves of RiskTransferCertificate.
Formal statement
hypotheses
n d :
ℕ
epsilon M sigma c L a b :
ℝ
H :
LeastFavorableHandle n d epsilon M sigma
hM :
0 ≤ M
K :
Kernel (BinObs H.radialCap) (Obs d)
ha :
a ≠ 0
hQ :
∀ P ∈ H.radialSource,
(H.radialEmbedding P).observedLaw
= K ∘ₘ CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P.1
htheta :
∀ P ∈ H.radialSource,
rawAteFormula (H.radialEmbedding P)
= a * CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1 + b
hsource :
∀ sourceEst : (Fin n → BinObs H.radialCap) → ℝ
if
Measurable sourceEst
and
UniformlyBounded sourceEst
then
∃ P ∈ H.radialSource,
L
≤ sqRisk (CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n) sourceEst (CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1)
hscale :
c * M ^ 2 * sigma ^ 2 * min 1 (polynomialComponent n d) ≤ a ^ 2 * L
conclusion
Proof (Lean source)
lemma riskTransferCertificate_of_kernel_transport {n d : ℕ}
{epsilon M sigma c L a b : ℝ}
(H : LeastFavorableHandle n d epsilon M sigma) (hM : 0 ≤ M)
(K : Kernel (BinObs H.radialCap) (Obs d)) [IsMarkovKernel K]
(ha : a ≠ 0)
(hQ : ∀ P ∈ H.radialSource,
(H.radialEmbedding P).observedLaw =
K ∘ₘ CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P.1)
(htheta : ∀ P ∈ H.radialSource,
rawAteFormula (H.radialEmbedding P) =
a * CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1 + b)
(hsource : ∀ sourceEst : (Fin n → BinObs H.radialCap) → ℝ,
Measurable sourceEst → UniformlyBounded sourceEst →
∃ P ∈ H.radialSource,
L ≤ sqRisk
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n)
sourceEst
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1))
(hscale : c * M ^ 2 * sigma ^ 2 * min 1 (polynomialComponent n d) ≤
a ^ 2 * L)
(hexact : ∀ est : Estimator n d M,
∃ P ∈ H.exactSource,
c * M ^ 2 *
(1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)) ≤
mse (H.exactEmbedding P) est.1) :
RiskTransferCertificate c H := by
intro est
exact ⟨radial_risk_of_kernel_transport H hM K ha hQ htheta hsource hscale est,
hexact est⟩
If the heterogeneity radius is nonnegative, for a radius parameter in [0,2], the Bernoulli contraction probability lies in the advertised interval, including both endpoint bits.
Formal statement
hypotheses
conclusion
1 / 2
+ sigma / 2 * ((if b then 1 else 0) - 1 / 2) ∈ Icc (1 / 2 - sigma / 4) (1 / 2 + sigma / 4)
Proof (Lean source)
lemma bernoulliContraction_success_mem {sigma : ℝ}
(hsigma : 0 ≤ sigma) (b : Bool) :
1 / 2 + sigma / 2 * ((if b then 1 else 0) - 1 / 2) ∈
Icc (1 / 2 - sigma / 4) (1 / 2 + sigma / 4) := by
cases b <;> simp only [Bool.false_eq_true, ↓reduceIte]
· constructor <;> linarith
· constructor <;> linarith
theorem leastFavorableFamily_channel_success_mem Lemma leastFavorableFamily_channel_success_mem in the paper ↗
If the specified least-favorable family is available and the heterogeneity radius is nonnegative, the channel formula stored by a least-favorable family automatically discharges the theorem's pointwise probability-range certificate.
Formal statement
hypotheses
n d :
ℕ
epsilon M sigma bRad bExact :
ℝ
H :
LeastFavorableHandle n d epsilon M sigma
hfamily :
leastFavorableAt n d epsilon M sigma bRad bExact H
hsigma :
0 ≤ sigma
conclusion
∀ b, H.channelSuccess b ∈ Icc (1 / 2 - sigma / 4) (1 / 2 + sigma / 4)
Proof (Lean source)
lemma leastFavorableFamily_channel_success_mem {n d : ℕ}
{epsilon M sigma bRad bExact : ℝ}
(H : LeastFavorableHandle n d epsilon M sigma)
(hfamily : leastFavorableAt n d epsilon M sigma bRad bExact H)
(hsigma : 0 ≤ sigma) :
∀ b, H.channelSuccess b ∈
Icc (1 / 2 - sigma / 4) (1 / 2 + sigma / 4) := by
rcases hfamily with
⟨_hbRad, _hbExact, hlambda, _hradialCap, _hexactCap,
_hradialHard, _hexactHard, _hradialCoupling, _hexactCoupling,
hsuccess, _hscaled⟩
intro b
rw [hsuccess b, hlambda]
exact bernoulliContraction_success_mem hsigma b
theorem converseRate_lower_of_base_and_radius
If the base-rate constant is positive and the radius-rate constant is positive and the base lower bound holds and the radial source lower bound holds, two lower bounds against the same minimax risk combine into the displayed three-term converse rate after halving the smaller constant.
Formal statement
hypotheses
n d :
ℕ
M sigma R cBase cRad :
ℝ
hcBase :
0 < cBase
hcRad :
0 < cRad
hbase :
cBase * M ^ 2 * (1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)) ≤ R
hradius :
cRad * M ^ 2 * sigma ^ 2 * min 1 (polynomialComponent n d) ≤ R
conclusion
min cBase cRad / 2 * M ^ 2 * converseRate n d sigma ≤ R
Proof (Lean source)
lemma converseRate_lower_of_base_and_radius {n d : ℕ} {M sigma R cBase cRad : ℝ}
(hcBase : 0 < cBase) (hcRad : 0 < cRad)
(hbase : cBase * M ^ 2 *
(1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)) ≤ R)
(hradius : cRad * M ^ 2 * sigma ^ 2 *
min 1 (polynomialComponent n d) ≤ R) :
min cBase cRad / 2 * M ^ 2 * converseRate n d sigma ≤ R := by
let A : ℝ := 1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)
let B : ℝ := sigma ^ 2 * min 1 (polynomialComponent n d)
have hpoly : 0 ≤ polynomialComponent n d := by
unfold polynomialComponent
positivity
have hA : 0 ≤ A := by
dsimp [A]
positivity
have hB : 0 ≤ B := by
dsimp [B]
positivity
have hc : 0 ≤ min cBase cRad := le_min hcBase.le hcRad.le
have hc_base : min cBase cRad ≤ cBase := min_le_left _ _
have hc_rad : min cBase cRad ≤ cRad := min_le_right _ _
have hM2 : 0 ≤ M ^ 2 := sq_nonneg M
have hbase' : min cBase cRad * M ^ 2 * A ≤ R := by
calc
min cBase cRad * M ^ 2 * A ≤ cBase * M ^ 2 * A := by gcongr
_ ≤ R := by simpa [A] using hbase
have hradius' : min cBase cRad * M ^ 2 * B ≤ R := by
calc
min cBase cRad * M ^ 2 * B ≤ cRad * M ^ 2 * B := by gcongr
_ = cRad * M ^ 2 * sigma ^ 2 *
min 1 (polynomialComponent n d) := by dsimp [B]; ring
_ ≤ R := hradius
have hsum : min cBase cRad * M ^ 2 * (A + B) ≤ 2 * R := by
nlinarith
calc
min cBase cRad / 2 * M ^ 2 * converseRate n d sigma =
(min cBase cRad * M ^ 2 * (A + B)) / 2 := by
unfold converseRate
dsimp [A, B]
ring
_ ≤ (2 * R) / 2 := div_le_div_of_nonneg_right hsum (by norm_num)
_ = R := by ring
theorem minimaxRisk_ge_radius_of_transfer_certificate
If the outcome scale satisfies its stated bound and the transported family belongs to the target model class and the risk-transfer certificate holds, the radial half of a risk-transfer certificate gives a lower bound on the ambient minimax risk once every embedded source law has a class witness.
Formal statement
hypotheses
n d :
ℕ
epsilon M sigma c :
ℝ
H :
LeastFavorableHandle n d epsilon M sigma
hM :
0 ≤ M
hmembership :
htransfer :
conclusion
Proof (Lean source)
lemma minimaxRisk_ge_radius_of_transfer_certificate {n d : ℕ}
{epsilon M sigma c : ℝ} (H : LeastFavorableHandle n d epsilon M sigma)
(hM : 0 ≤ M)
(hmembership : RadialEmbeddingMembership H)
(htransfer : RiskTransferCertificate c H) :
c * M ^ 2 * sigma ^ 2 * min 1 (polynomialComponent n d) ≤
minimaxRisk n d epsilon M sigma := by
let est0 : Estimator n d M :=
⟨fun _ ↦ 0, measurable_const, fun _ ↦ ⟨by linarith, hM⟩⟩
letI : Nonempty (Estimator n d M) := ⟨est0⟩
unfold minimaxRisk
apply le_ciInf
intro est
obtain ⟨P, hP, hrisk⟩ := (htransfer est).1
obtain ⟨Q, hQ⟩ := hmembership P hP
unfold worstCaseMSE
have hbounded : BddAbove (range (fun R : ModelClass d epsilon M sigma ↦
mse R.law est.1)) := by
refine ⟨(2 * M) ^ 2, ?_⟩
rintro _ ⟨R, rfl⟩
exact test_model_mse_le R est
have hriskQ : c * M ^ 2 * sigma ^ 2 * min 1 (polynomialComponent n d) ≤
mse Q.law est.1 := by simpa [hQ] using hrisk
exact hriskQ.trans (le_ciSup hbounded Q)
theorem converseRate_lower_of_exact_transfer_and_handle
If the outcome scale satisfies its stated bound and the base-rate constant is positive and the radius-rate constant is positive and the base lower bound holds and the transported family belongs to the target model class and the risk-transfer certificate holds, the proved all-alphabet exact transfer and a concrete radial handle assemble the full capped converse rate with the minimum of their constants.
Formal statement
hypotheses
n d :
ℕ
epsilon M sigma cBase cRad :
ℝ
hM :
0 ≤ M
hcBase :
0 < cBase
hcRad :
0 < cRad
hbase :
cBase * M ^ 2 * (1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2))
≤ minimaxRisk n d epsilon M sigma
H :
LeastFavorableHandle n d epsilon M sigma
hmembership :
htransfer :
RiskTransferCertificate cRad H
conclusion
Proof (Lean source)
lemma converseRate_lower_of_exact_transfer_and_handle {n d : ℕ}
{epsilon M sigma cBase cRad : ℝ}
(hM : 0 ≤ M) (hcBase : 0 < cBase) (hcRad : 0 < cRad)
(hbase : cBase * M ^ 2 *
(1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)) ≤
minimaxRisk n d epsilon M sigma)
(H : LeastFavorableHandle n d epsilon M sigma)
(hmembership : RadialEmbeddingMembership H)
(htransfer : RiskTransferCertificate cRad H) :
min cBase cRad / 2 * M ^ 2 * converseRate n d sigma ≤
minimaxRisk n d epsilon M sigma := by
exact converseRate_lower_of_base_and_radius hcBase hcRad hbase
(minimaxRisk_ge_radius_of_transfer_certificate H hM hmembership htransfer)
theorem capped_positive_mass_alphabet_bounds Lemma capped_positive_mass_alphabet_bounds in the paper ↗
If the alphabet is nonempty, capping a positive ambient alphabet by a cutoff forced to be at least one always leaves a nonempty source alphabet and never exceeds the ambient one.
Formal statement
Proof (Lean source)
lemma capped_positive_mass_alphabet_bounds {d cutoff : ℕ} (hd : 0 < d) :
0 < min d (max 1 cutoff) ∧ min d (max 1 cutoff) ≤ d := by
constructor
· exact (Nat.lt_min).2
⟨hd, lt_of_lt_of_le Nat.zero_lt_one (Nat.le_max_left 1 cutoff)⟩
· exact min_le_left _ _
theorem radialCap_le_sourceRange
If the radial cap satisfies its stated bound and the sample size satisfies the stated lower bound and the source range is at least one, halving the source alphabet constant absorbs the log (e n) cutoff into the cited log n source range once n ≥ 3; the explicit unit lower bound also absorbs the nonempty-alphabet max 1 guard.
Formal statement
Proof (Lean source)
lemma radialCap_le_sourceRange {n d : ℕ} {b : ℝ}
(hb : 0 < b) (hn : 3 ≤ n) (hone : 1 ≤ b * (n : ℝ) * log n) :
((min d (max 1 (floor ((b / 2) * (n : ℝ) * logEN n))) : ℕ) : ℝ) ≤
b * (n : ℝ) * log n := by
have hnR : (3 : ℝ) ≤ n := by exact_mod_cast hn
have hnpos : (0 : ℝ) < n := by positivity
have hlog3 : 1 ≤ log (3 : ℝ) := by
rw [Real.le_log_iff_exp_le (by norm_num : (0 : ℝ) < 3)]
exact Real.exp_one_lt_d9.le.trans (by norm_num)
have hlog : 1 ≤ log (n : ℝ) :=
hlog3.trans (Real.log_le_log (by norm_num) hnR)
have hlogEN : logEN n ≤ 2 * log (n : ℝ) := by
rw [logEN, Real.log_mul (Real.exp_ne_zero 1) hnpos.ne', Real.log_exp]
linarith
have hx0 : 0 ≤ (b / 2) * (n : ℝ) * logEN n := by
have : 0 ≤ logEN n := by
rw [logEN, Real.log_mul (Real.exp_ne_zero 1) hnpos.ne', Real.log_exp]
positivity
positivity
have hfloor : (floor ((b / 2) * (n : ℝ) * logEN n) : ℝ) ≤
b * (n : ℝ) * log n := by
calc
(floor ((b / 2) * (n : ℝ) * logEN n) : ℝ) ≤
(b / 2) * (n : ℝ) * logEN n := Nat.floor_le hx0
_ ≤ (b / 2) * (n : ℝ) * (2 * log n) := by gcongr
_ = b * (n : ℝ) * log n := by ring
calc
((min d (max 1 (floor ((b / 2) * (n : ℝ) * logEN n))) : ℕ) : ℝ) ≤
(max 1 (floor ((b / 2) * (n : ℝ) * logEN n)) : ℕ) := by
exact_mod_cast min_le_right d _
_ = max (1 : ℝ)
(floor ((b / 2) * (n : ℝ) * logEN n) : ℝ) := by norm_num
_ ≤ b * (n : ℝ) * log n := max_le hone hfloor
theorem exists_capped_radial_transport_package
If the overlap constant is positive and the overlap constant is below one half, the Zeng one-arm constants can be shrunk once so that the paper's capped radial alphabet is nonempty, remains in the source theorem's range, and has exactly the transported product-radius scale needed in the large-sample branch.
Formal statement
hypotheses
epsilon :
ℝ
he0 :
0 < epsilon
he1 :
epsilon < 1 / 2
conclusion
∃ a bRad bCut cRad : ℝ,
∃ N : ℕ,
0 < a ∧
0 < bRad ∧
bRad < bCut ∧
0 < cRad ∧
∀ n d : ℕ,
0 < n → 0 < d → N
m ≤ d ∧
(m : ℝ) ≤ bCut * (n : ℝ) * log n ∧
(∀ sourceEst : (Fin n → BinObs m) → ℝ, Measurable sourceEst → UniformlyBounded sourceEst → ∃ P ∈ (Set.univ : Set (CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw n m epsilon)), a / 2 * CausalSmith.Stat.DiscreteAteMinimaxLoggap.minimaxRate n m ≤ sqRisk (CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n) sourceEst (CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1)) ∧
∀ M sigma : ℝ,
cRad * M ^ 2 * sigma ^ 2 * min 1 (polynomialComponent n d)
≤ (M * sigma / 2) ^ 2 * (a / 2 * CausalSmith.Stat.DiscreteAteMinimaxLoggap.minimaxRate n m)
Proof (Lean source)
lemma exists_capped_radial_transport_package {epsilon : ℝ}
(he0 : 0 < epsilon) (he1 : epsilon < 1 / 2) :
∃ a bRad bCut cRad : ℝ, ∃ N : ℕ,
0 < a ∧ 0 < bRad ∧ bRad < bCut ∧ 0 < cRad ∧
∀ n d : ℕ, 0 < n → 0 < d → N ≤ n →
let m := min d (max 1 (floor (bRad * (n : ℝ) * logEN n)))
0 < m ∧ m ≤ d ∧
(m : ℝ) ≤ bCut * (n : ℝ) * log n ∧
(∀ sourceEst : (Fin n → BinObs m) → ℝ,
Measurable sourceEst → UniformlyBounded sourceEst →
∃ P ∈ (Set.univ : Set
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.ControlZeroLaw
n m epsilon)),
a / 2 * CausalSmith.Stat.DiscreteAteMinimaxLoggap.minimaxRate n m ≤
sqRisk
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n)
sourceEst
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1)) ∧
∀ M sigma : ℝ,
cRad * M ^ 2 * sigma ^ 2 * min 1 (polynomialComponent n d) ≤
(M * sigma / 2) ^ 2 *
(a / 2 *
CausalSmith.Stat.DiscreteAteMinimaxLoggap.minimaxRate n m) := by
obtain ⟨a, b, N0, ha, hb, hlower⟩ :=
CausalSmith.Stat.DiscreteAteMinimaxLoggap.zengOneArmMinimaxLower epsilon
⟨he0, he1⟩
let b0 : ℝ := min b 4
have hb0 : 0 < b0 := lt_min hb (by norm_num)
have hb0b : b0 ≤ b := min_le_left _ _
have hb04 : b0 ≤ 4 := min_le_right _ _
let N : ℕ := max 3 (max N0 (ceil (2 / b0)))
let cRad : ℝ := a * b0 ^ 2 / 128
refine ⟨a, b0 / 2, b, cRad, N, ha, by positivity, ?_, by
dsimp [cRad]
positivity, ?_⟩
· have : b0 ≤ b := hb0b
nlinarith
intro n d hn hd hN
have hn3 : 3 ≤ n := le_trans (Nat.le_max_left _ _) hN
have hN0 : N0 ≤ n :=
le_trans (le_trans (Nat.le_max_left _ _) (Nat.le_max_right 3 _)) hN
have hceilNat : ceil (2 / b0) ≤ n :=
le_trans (le_trans (Nat.le_max_right _ _) (Nat.le_max_right 3 _)) hN
have hceil : 2 / b0 ≤ (ceil (2 / b0) : ℝ) := Nat.le_ceil _
have hceilReal : (ceil (2 / b0) : ℝ) ≤ n := by exact_mod_cast hceilNat
have hnLower : 2 / b0 ≤ (n : ℝ) := hceil.trans hceilReal
have hnR : 0 < (n : ℝ) := by exact_mod_cast hn
have hlogOne : 1 ≤ log (n : ℝ) := by
have hlog3 : 1 ≤ log (3 : ℝ) := by
rw [Real.le_log_iff_exp_le (by norm_num : (0 : ℝ) < 3)]
exact Real.exp_one_lt_d9.le.trans (by norm_num)
exact hlog3.trans (Real.log_le_log (by norm_num) (by exact_mod_cast hn3))
have hone : 1 ≤ b0 * (n : ℝ) * log n := by
have hbn : 1 ≤ b0 * (n : ℝ) := by
have htwo' : 2 ≤ (n : ℝ) * b0 := (div_le_iff₀ hb0).mp hnLower
have htwo : 2 ≤ b0 * (n : ℝ) := by nlinarith
linarith
nlinarith
have hx : 1 ≤ (b0 / 2) * (n : ℝ) * logEN n := by
have hlogEN : 1 ≤ logEN n := by
rw [logEN, Real.log_mul (Real.exp_ne_zero 1) hnR.ne', Real.log_exp]
linarith
have hbn : 1 ≤ (b0 / 2) * (n : ℝ) := by
calc
1 = (b0 / 2) * (2 / b0) := by field_simp [hb0.ne']
_ ≤ (b0 / 2) * (n : ℝ) := mul_le_mul_of_nonneg_left hnLower (by positivity)
nlinarith
let m := min d (max 1 (floor ((b0 / 2) * (n : ℝ) * logEN n)))
have hm : 0 < m ∧ m ≤ d := capped_positive_mass_alphabet_bounds hd
have hmrange0 : (m : ℝ) ≤ b0 * (n : ℝ) * log n :=
radialCap_le_sourceRange hb0 hn3 hone
have hmrange : (m : ℝ) ≤ b * (n : ℝ) * log n := by
calc
(m : ℝ) ≤ b0 * (n : ℝ) * log n := hmrange0
_ ≤ b * (n : ℝ) * log n := by gcongr
have hsource := radial_source_risk_of_oneArm_lower hn hm.1 he0 he1 ha
(hlower n m hm.1 hN0 hmrange)
refine ⟨hm.1, hm.2, hmrange, hsource, ?_⟩
intro M sigma
simpa [m, cRad] using
(cappedRadial_transportScale (n := n) (d := d) (a := a) (b := b0)
(M := M) (sigma := sigma) hn3 hd ha.le hb0 hb04 hx)
theorem exact_source_risk_of_binaryExact_lower Lemma exact_source_risk_of_binaryExact_lower in the paper ↗
If the sample is nonempty and the alphabet is nonempty and the overlap constant is positive and the overlap constant is below one half and the transport scale satisfies the stated condition and the stated lower bound holds, a positive fixed-sample exact-homogeneity minimax lower bound supplies the estimator-wise source hardness interface used by deterministic affine transport, after the same strict half-constant reduction as in the radial family.
Formal statement
hypotheses
n d :
ℕ
epsilon a :
ℝ
hn :
0 < n
hd :
0 < d
he0 :
0 < epsilon
he1 :
epsilon < 1 / 2
ha :
0 < a
hlower :
a * (1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2) ≤ binaryExactMinimaxRisk n d epsilon
conclusion
∀ sourceEst : (Fin n → BinObs d) → ℝ
if
Measurable sourceEst
then
∃ P ∈ (Set.univ : Set (BinaryExactLaw n d epsilon)),
a / 2 * (1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2)
≤ sqRisk (CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n) sourceEst (CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1)
Proof (Lean source)
lemma exact_source_risk_of_binaryExact_lower {n d : ℕ} {epsilon a : ℝ}
(hn : 0 < n) (hd : 0 < d) (he0 : 0 < epsilon) (he1 : epsilon < 1 / 2)
(ha : 0 < a)
(hlower : a * (1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2) ≤
binaryExactMinimaxRisk n d epsilon) :
∀ sourceEst : (Fin n → BinObs d) → ℝ, Measurable sourceEst →
∃ P ∈ (Set.univ : Set (BinaryExactLaw n d epsilon)),
a / 2 * (1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2) ≤
sqRisk
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n)
sourceEst
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1) := by
have hnR : 0 < (n : ℝ) := by exact_mod_cast hn
have hrate : 0 < 1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2 := by
have hone : 0 < 1 / (n : ℝ) := one_div_pos.mpr hnR
have hdim : 0 ≤ (d : ℝ) / (n : ℝ) ^ 2 := by positivity
linarith
have hstrict : a / 2 * (1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2) <
binaryExactMinimaxRisk n d epsilon :=
(by nlinarith : a / 2 * (1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2) <
a * (1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2)).trans_le hlower
intro sourceEst hmeas
obtain ⟨P, hP⟩ := binaryExactMinimaxRisk_hard_family_of_lt
hn hd he0 he1 hstrict sourceEst hmeas
exact ⟨P, Set.mem_univ P, by
simpa [sqRisk,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse] using hP⟩
theorem exactSourceHard_of_source_eq_univ_and_lower
If the sample is nonempty and the alphabet size satisfies the stated condition and the overlap constant is positive and the overlap constant is below one half and the transport scale satisfies the stated condition and the source parameter set has the stated form and the stated lower bound holds, a handle whose exact source is the full exact binary class inherits the hard-family certificate from a positive fixed-sample exact minimax lower bound.
Formal statement
hypotheses
n d :
ℕ
epsilon M sigma a :
ℝ
H :
LeastFavorableHandle n d epsilon M sigma
hn :
0 < n
hd :
0 < H.exactCap
he0 :
0 < epsilon
he1 :
epsilon < 1 / 2
ha :
0 < a
hsource :
H.exactSource = univ
hlower :
a * (1 / (n : ℝ) + (H.exactCap : ℝ) / (n : ℝ) ^ 2)
≤ binaryExactMinimaxRisk n H.exactCap epsilon
conclusion
Proof (Lean source)
lemma exactSourceHard_of_source_eq_univ_and_lower {n d : ℕ}
{epsilon M sigma a : ℝ} (H : LeastFavorableHandle n d epsilon M sigma)
(hn : 0 < n) (hd : 0 < H.exactCap)
(he0 : 0 < epsilon) (he1 : epsilon < 1 / 2) (ha : 0 < a)
(hsource : H.exactSource = univ)
(hlower : a * (1 / (n : ℝ) + (H.exactCap : ℝ) / (n : ℝ) ^ 2) ≤
binaryExactMinimaxRisk n H.exactCap epsilon) :
ExactSourceHard H := by
refine ⟨a / 2, by positivity, ?_⟩
intro est
obtain ⟨P, _hP, hrisk⟩ := exact_source_risk_of_binaryExact_lower
hn hd he0 he1 ha hlower est.1 est.2
refine ⟨P, ?_, ?_⟩
· rw [hsource]
exact Set.mem_univ P
· simpa [sqRisk,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse] using hrisk
theorem exactSourceHard_of_source_eq_univ
If the sample is nonempty and the alphabet size satisfies the stated condition and the overlap constant is positive and the overlap constant is below one half and the source parameter set has the stated form, for every positive sample size and capped alphabet, the full exact binary source class is a genuinely hard family. As in the radial source interface, the constant carried by the structural handle may depend on the concrete instance; the uniform constant used by the eventual risk-transfer certificate is supplied separately by the cited exact lower bound.
Formal statement
hypotheses
n d :
ℕ
epsilon M sigma :
ℝ
H :
LeastFavorableHandle n d epsilon M sigma
hn :
0 < n
hd :
0 < H.exactCap
he0 :
0 < epsilon
he1 :
epsilon < 1 / 2
hsource :
H.exactSource = univ
conclusion
Proof (Lean source)
lemma exactSourceHard_of_source_eq_univ {n d : ℕ}
{epsilon M sigma : ℝ} (H : LeastFavorableHandle n d epsilon M sigma)
(hn : 0 < n) (hd : 0 < H.exactCap)
(he0 : 0 < epsilon) (he1 : epsilon < 1 / 2)
(hsource : H.exactSource = univ) :
ExactSourceHard H := by
let r : ℝ := 1 / (n : ℝ) + (H.exactCap : ℝ) / (n : ℝ) ^ 2
have hnR : 0 < (n : ℝ) := by exact_mod_cast hn
have hr : 0 < r := by
dsimp [r]
positivity
let a : ℝ := 1 / (100 * (n : ℝ) * r)
have ha : 0 < a := by dsimp [a]; positivity
have hlower : a * r ≤ binaryExactMinimaxRisk n H.exactCap epsilon := by
have hparam := binaryExactMinimaxRisk_parametric_lower
hn hd he0 he1
have har : a * r = 1 / (100 * (n : ℝ)) := by
dsimp [a]
field_simp [hnR.ne', hr.ne']
rw [har]
exact hparam
exact exactSourceHard_of_source_eq_univ_and_lower H hn hd he0 he1 ha
hsource (by simpa [r] using hlower)
theorem exact_source_risk_of_parametric_lower Lemma exact_source_risk_of_parametric_lower in the paper ↗
If the sample is nonempty and the alphabet is nonempty and the overlap constant is positive and the overlap constant is below one half, the exact source's two-point subexperiment gives the estimator-wise parametric risk interface at every positive sample size. The strict half-constant is what permits extraction of an actual hard source law from the minimax infimum/supremum.
Formal statement
hypotheses
n d :
ℕ
epsilon :
ℝ
hn :
0 < n
hd :
0 < d
he0 :
0 < epsilon
he1 :
epsilon < 1 / 2
conclusion
∀ sourceEst : (Fin n → BinObs d) → ℝ
if
Measurable sourceEst
then
∃ P : BinaryExactLaw n d epsilon,
1 / (200 * (n : ℝ))
≤ sqRisk (CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n) sourceEst (CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1)
Proof (Lean source)
lemma exact_source_risk_of_parametric_lower {n d : ℕ} {epsilon : ℝ}
(hn : 0 < n) (hd : 0 < d) (he0 : 0 < epsilon)
(he1 : epsilon < 1 / 2) :
∀ sourceEst : (Fin n → BinObs d) → ℝ, Measurable sourceEst →
∃ P : BinaryExactLaw n d epsilon,
1 / (200 * (n : ℝ)) ≤
sqRisk
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n)
sourceEst
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1) := by
have hnR : 0 < (n : ℝ) := by exact_mod_cast hn
have hstrict : 1 / (200 * (n : ℝ)) <
binaryExactMinimaxRisk n d epsilon := by
have hparam := binaryExactMinimaxRisk_parametric_lower hn hd he0 he1
have hdenom : 100 * (n : ℝ) < 200 * (n : ℝ) := by nlinarith
exact (one_div_lt_one_div_of_lt (by positivity) hdenom).trans_le hparam
intro sourceEst hmeas
obtain ⟨P, hP⟩ := binaryExactMinimaxRisk_hard_family_of_lt
hn hd he0 he1 hstrict sourceEst hmeas
exact ⟨P, by
simpa [sqRisk,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse] using hP⟩
theorem exists_capped_exact_transport_package Lemma exists_capped_exact_transport_package in the paper ↗
If the overlap constant is positive and the overlap constant is below one half, the cited exact lower bound, its capped alphabet, and the finite-sample parametric fallback can be normalized to one estimator-wise transport scale valid for every positive sample size and ambient alphabet.
Formal statement
hypotheses
epsilon :
ℝ
he0 :
0 < epsilon
he1 :
epsilon < 1 / 2
conclusion
∃ bExact cExact : ℝ,
∃ NExact : ℕ,
0 < bExact ∧
0 < cExact ∧
∀ n d : ℕ,
m ≤ d ∧
(NExact ≤ n → (m : ℝ) ≤ bExact * (n : ℝ) ^ 2) ∧
∃ L : ℝ,
(∀ sourceEst : (Fin n → BinObs m) → ℝ, Measurable sourceEst → ∃ P : BinaryExactLaw n m epsilon, L ≤ sqRisk (CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n) sourceEst (CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1)) ∧
∀ M : ℝ, cExact * M ^ 2 * (1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)) ≤ M ^ 2 * L
Proof (Lean source)
lemma exists_capped_exact_transport_package {epsilon : ℝ}
(he0 : 0 < epsilon) (he1 : epsilon < 1 / 2) :
∃ bExact cExact : ℝ, ∃ NExact : ℕ,
0 < bExact ∧ 0 < cExact ∧
∀ n d : ℕ, 0 < n → 0 < d →
let m := min d (max 1 (Nat.floor (bExact * (n : ℝ) ^ 2)))
0 < m ∧ m ≤ d ∧
(NExact ≤ n → (m : ℝ) ≤ bExact * (n : ℝ) ^ 2) ∧
∃ L : ℝ,
(∀ sourceEst : (Fin n → BinObs m) → ℝ, Measurable sourceEst →
∃ P : BinaryExactLaw n m epsilon,
L ≤ sqRisk
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n)
sourceEst
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1)) ∧
∀ M : ℝ, cExact * M ^ 2 *
(1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)) ≤ M ^ 2 * L := by
obtain ⟨a, b, N, ha, hb, hexact⟩ :=
zengBinaryExactHomogeneityLower epsilon ⟨he0, he1⟩
let b0 : ℝ := min b (1 / 2)
have hb0 : 0 < b0 := lt_min hb (by norm_num)
have hb0b : b0 ≤ b := min_le_left _ _
have hb0half : b0 ≤ 1 / 2 := min_le_right _ _
let K : ℝ := 1 + (N : ℝ) + 1 / b0
have hK : 0 < K := by dsimp [K]; positivity
let c : ℝ := min (a * b0 / 4) ((1 / 200) / K)
have hc : 0 < c := lt_min (by positivity) (by positivity)
let NExact : ℕ := max N (ceil (1 / b0))
refine ⟨b0, c, NExact, hb0, hc, ?_⟩
intro n d hn hd
have hnR : 0 < (n : ℝ) := by exact_mod_cast hn
have hnOne : (1 : ℝ) ≤ n := by exact_mod_cast (Nat.one_le_iff_ne_zero.mpr hn.ne')
let x : ℝ := b0 * (n : ℝ) ^ 2
let m : ℕ := min d (max 1 (Nat.floor x))
have hm : 0 < m ∧ m ≤ d := capped_positive_mass_alphabet_bounds hd
have hcut : NExact ≤ n → (m : ℝ) ≤ x := by
intro hN
have hceilNat : ceil (1 / b0) ≤ n :=
(Nat.le_max_right _ _).trans hN
have hceil : 1 / b0 ≤ (ceil (1 / b0) : ℝ) := Nat.le_ceil _
have hnLower : 1 / b0 ≤ (n : ℝ) := hceil.trans (by exact_mod_cast hceilNat)
have hxOne : 1 ≤ x := by
dsimp [x]
have : 1 ≤ b0 * (n : ℝ) := by
calc
1 = b0 * (1 / b0) := by field_simp [hb0.ne']
_ ≤ b0 * (n : ℝ) := mul_le_mul_of_nonneg_left hnLower hb0.le
nlinarith [hnOne]
calc
(m : ℝ) ≤ (max 1 (Nat.floor x) : ℕ) := by
exact_mod_cast min_le_right d _
_ = max (1 : ℝ) (Nat.floor x : ℝ) := by norm_num
_ ≤ x := max_le hxOne (Nat.floor_le (by linarith))
refine ⟨hm.1, hm.2, hcut, ?_⟩
by_cases hlarge : N ≤ n ∧ 1 ≤ x
· have hmFloor : m = min d (Nat.floor x) := by
dsimp [m]
rw [max_eq_right (Nat.one_le_iff_ne_zero.mpr
(Nat.ne_of_gt (Nat.floor_pos.mpr hlarge.2)))]
have hmx : (m : ℝ) ≤ x := by
rw [hmFloor]
exact (show ((min d (Nat.floor x) : ℕ) : ℝ) ≤ Nat.floor x by
exact_mod_cast min_le_right d (Nat.floor x)).trans
(Nat.floor_le (by linarith [hlarge.2]))
have hmb : (m : ℝ) ≤ b * (n : ℝ) ^ 2 := by
exact hmx.trans (by dsimp [x]; exact mul_le_mul_of_nonneg_right hb0b (sq_nonneg _))
let sourceRate : ℝ := 1 / (n : ℝ) + (m : ℝ) / (n : ℝ) ^ 2
let L : ℝ := a / 2 * sourceRate
have hsource := exact_source_risk_of_binaryExact_lower hn hm.1 he0 he1 ha
(by simpa [sourceRate] using hexact n m hm.1 hlarge.1 hmb)
have hfloorHalf : x / 2 ≤ (Nat.floor x : ℝ) :=
half_le_natFloor_of_one_le hlarge.2
have hbcoef : b0 / 2 ≤ 1 := by linarith
have honeDiv : 0 ≤ 1 / (n : ℝ) := by positivity
have htarget : b0 / 2 *
(1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)) ≤ sourceRate := by
by_cases hdcap : d ≤ Nat.floor x
· have hmEq : m = d := by rw [hmFloor, min_eq_left hdcap]
have hdRatio : (d : ℝ) / (n : ℝ) ^ 2 ≤ 1 := by
rw [div_le_one (sq_pos_of_pos hnR)]
calc
(d : ℝ) ≤ Nat.floor x := by exact_mod_cast hdcap
_ ≤ x := Nat.floor_le (by linarith [hlarge.2])
_ ≤ (n : ℝ) ^ 2 := by
dsimp [x]
exact mul_le_of_le_one_left (sq_nonneg _) (by linarith)
rw [min_eq_right hdRatio]
dsimp [sourceRate]
rw [hmEq]
have hratePos : 0 ≤ 1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2 := by positivity
simpa using mul_le_mul_of_nonneg_right hbcoef hratePos
· have hmEq : m = Nat.floor x := by
rw [hmFloor, min_eq_right (Nat.le_of_not_ge hdcap)]
have hmLower : b0 / 2 ≤ (m : ℝ) / (n : ℝ) ^ 2 := by
rw [hmEq]
apply (le_div_iff₀ (sq_pos_of_pos hnR)).2
dsimp [x] at hfloorHalf
nlinarith
have hmin := min_le_left (1 : ℝ) ((d : ℝ) / (n : ℝ) ^ 2)
dsimp [sourceRate]
calc
b0 / 2 * (1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)) ≤
b0 / 2 * (1 / (n : ℝ) + 1) := by gcongr
_ ≤ 1 / (n : ℝ) + (m : ℝ) / (n : ℝ) ^ 2 := by
have hfirst : b0 / 2 * (1 / (n : ℝ)) ≤ 1 / (n : ℝ) :=
mul_le_of_le_one_left honeDiv hbcoef
nlinarith
have hsource' : ∀ sourceEst : (Fin n → BinObs m) → ℝ,
Measurable sourceEst → ∃ P : BinaryExactLaw n m epsilon,
L ≤ sqRisk
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw P.1 n)
sourceEst
(CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1) := by
intro sourceEst hmeas
obtain ⟨P, _hP, hrisk⟩ := hsource sourceEst hmeas
exact ⟨P, by simpa [L, sourceRate] using hrisk⟩
refine ⟨L, hsource', ?_⟩
intro M
have hcHigh : c ≤ a * b0 / 4 := min_le_left _ _
let rate : ℝ := 1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)
have hrate0 : 0 ≤ rate := by dsimp [rate]; positivity
dsimp [L]
calc
c * M ^ 2 * rate = M ^ 2 * (c * rate) := by ring
_ ≤ M ^ 2 * ((a * b0 / 4) * rate) := by gcongr
_ = M ^ 2 * (a / 2 * (b0 / 2 * rate)) := by ring
_ ≤ M ^ 2 * (a / 2 * sourceRate) := by gcongr
· let L : ℝ := 1 / (200 * (n : ℝ))
have hsource := exact_source_risk_of_parametric_lower hn hm.1 he0 he1
have hfallback :
1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2) ≤ K / (n : ℝ) := by
have hmin := min_le_left (1 : ℝ) ((d : ℝ) / (n : ℝ) ^ 2)
rcases not_and_or.mp hlarge with hN | hx
· have hnN : (n : ℝ) ≤ N := by exact_mod_cast le_of_lt (Nat.lt_of_not_ge hN)
have hone : 1 ≤ (N : ℝ) / (n : ℝ) := by
apply (le_div_iff₀ hnR).2
simpa using hnN
have hbterm : 0 ≤ 1 / b0 / (n : ℝ) := by positivity
calc
1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2) ≤
1 / (n : ℝ) + 1 := by linarith
_ ≤ 1 / (n : ℝ) + (N : ℝ) / (n : ℝ) + 1 / b0 / (n : ℝ) := by linarith
_ = K / (n : ℝ) := by dsimp [K]; ring
· have hxlt : x < 1 := lt_of_not_ge hx
have hone : 1 ≤ (1 / b0) / (n : ℝ) := by
rw [div_div]
apply (le_div_iff₀ (mul_pos hb0 hnR)).2
dsimp [x] at hxlt
nlinarith [hnOne]
have hNterm : 0 ≤ (N : ℝ) / (n : ℝ) := by positivity
calc
1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2) ≤
1 / (n : ℝ) + 1 := by linarith
_ ≤ 1 / (n : ℝ) + (N : ℝ) / (n : ℝ) + 1 / b0 / (n : ℝ) := by linarith
_ = K / (n : ℝ) := by dsimp [K]; ring
refine ⟨L, hsource, ?_⟩
intro M
have hcLow : c ≤ (1 / 200) / K := min_le_right _ _
let rate : ℝ := 1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)
have hrate0 : 0 ≤ rate := by dsimp [rate]; positivity
dsimp [L]
calc
c * M ^ 2 * rate = M ^ 2 * (c * rate) := by ring
_ ≤ M ^ 2 * (((1 / 200) / K) * rate) := by gcongr
_ ≤ M ^ 2 * (((1 / 200) / K) * (K / (n : ℝ))) := by gcongr
_ = M ^ 2 * (1 / (200 * (n : ℝ))) := by field_simp [hK.ne', hnR.ne']
The hypothesis-independent Bernoulli contraction transfers the one-arm source bound, while the exact-homogeneity source supplies the capped d/n² term.
Formal statement
conclusion
∀ epsilon : ℝ,
0 < epsilon → epsilon < 1 / 2 → ∃ c_epsilon bRad : ℝ, 0 < c_epsilon ∧ c_epsilon
≤ 1 ∧
0 < bRad ∧
∀ n d : ℕ,
∀ M sigma : ℝ,
0 < n → 0 < d → 1
≤ M → 0 ≤ sigma → sigma ≤ 2 → c_epsilon * M ^ 2 * converseRate n d sigma ≤ minimaxRisk n d epsilon M sigma ∧
RadialTargetRiskTransferCertificate n d epsilon M sigma c_epsilon bRad
Proof (Lean source)
theorem radius_channel_converse_all_d :
∀ epsilon : ℝ, 0 < epsilon → epsilon < 1 / 2 →
∃ c_epsilon bRad : ℝ,
0 < c_epsilon ∧ c_epsilon ≤ 1 ∧
0 < bRad ∧
∀ n d : ℕ, ∀ M sigma : ℝ,
0 < n → 0 < d → 1 ≤ M → -- @realizes M(range [1,infinity))
0 ≤ sigma → sigma ≤ 2 →
c_epsilon * M ^ 2 * converseRate n d sigma ≤
minimaxRisk n d epsilon M sigma ∧
RadialTargetRiskTransferCertificate n d epsilon M sigma
c_epsilon bRad := by
intro epsilon hepsilon hepsilon_half
obtain ⟨aRad, bRad, bRadCutoff, cRad, NRad0,
haRad, hbRad, hbRadCutoff, hcRad, hradial⟩ :=
exists_capped_radial_transport_package hepsilon hepsilon_half
obtain ⟨bExact, cExact, NExact,
hbExact, hcExact, hexact⟩ :=
exists_capped_exact_transport_package hepsilon hepsilon_half
obtain ⟨cBase, hcBase, hbase⟩ :=
scaled_binary_exact_lower_transfer_all_d epsilon hepsilon hepsilon_half
let NRad : ℕ := max NRad0 1
have hNRad : 0 < NRad := lt_of_lt_of_le Nat.zero_lt_one (Nat.le_max_right _ _)
let cSmall : ℝ := 1 / (800 * (NRad : ℝ))
have hcSmall : 0 < cSmall := by dsimp [cSmall]; positivity
let cTransfer : ℝ := min (min cRad cSmall) cExact
have hcTransfer : 0 < cTransfer := lt_min (lt_min hcRad hcSmall) hcExact
have hcTransferRad : cTransfer ≤ cRad :=
(min_le_left (min cRad cSmall) cExact).trans (min_le_left _ _)
have hcTransferSmall : cTransfer ≤ cSmall :=
(min_le_left (min cRad cSmall) cExact).trans (min_le_right _ _)
have hcTransferExact : cTransfer ≤ cExact := min_le_right _ _
let cE : ℝ := min (min cBase cTransfer / 2) 1
have hcE : 0 < cE := lt_min (by positivity) (by norm_num)
have hcEOne : cE ≤ 1 := min_le_right _ _
have hcERate : cE ≤ min cBase cTransfer / 2 := min_le_left _ _
have hcETransfer : cE ≤ cTransfer := by
calc
cE ≤ min cBase cTransfer / 2 := hcERate
_ ≤ cTransfer / 2 := div_le_div_of_nonneg_right
(min_le_right _ _) (by norm_num)
_ ≤ cTransfer := by linarith
have hcERad : cE ≤ cRad := by
calc
cE ≤ min cBase cTransfer / 2 := min_le_left _ _
_ ≤ cTransfer := by
have : 0 ≤ min cBase cTransfer := le_min hcBase.le hcTransfer.le
have := min_le_right cBase cTransfer
nlinarith
_ ≤ min cRad cSmall := min_le_left _ _
_ ≤ cRad := min_le_left _ _
have hcESmall : cE ≤ cSmall := by
calc
cE ≤ min cBase cTransfer / 2 := min_le_left _ _
_ ≤ cTransfer := by
have : 0 ≤ min cBase cTransfer := le_min hcBase.le hcTransfer.le
have := min_le_right cBase cTransfer
nlinarith
_ ≤ min cRad cSmall := min_le_left _ _
_ ≤ cSmall := min_le_right _ _
have hcEExact : cE ≤ cExact := by
calc
cE ≤ min cBase cTransfer / 2 := min_le_left _ _
_ ≤ cTransfer := by
have : 0 ≤ min cBase cTransfer := le_min hcBase.le hcTransfer.le
have := min_le_right cBase cTransfer
nlinarith
_ ≤ cExact := min_le_right _ _
refine ⟨cE, bRad, hcE, hcEOne, hbRad, ?_⟩
intro n d M sigma hn hd hM hsigma hsigmaTwo
let mRad := min d (max 1 (floor (bRad * (n : ℝ) * logEN n)))
let mExact := min d (max 1 (floor (bExact * (n : ℝ) ^ 2)))
have hmRad := capped_positive_mass_alphabet_bounds
(d := d) (cutoff := floor (bRad * (n : ℝ) * logEN n)) hd
have hmExact := capped_positive_mass_alphabet_bounds
(d := d) (cutoff := floor (bExact * (n : ℝ) ^ 2)) hd
let padRad : Fin mRad → Fin d := fun k => ⟨k, lt_of_lt_of_le k.isLt hmRad.2⟩
let padExact : Fin mExact → Fin d := fun k => ⟨k, lt_of_lt_of_le k.isLt hmExact.2⟩
let H : LeastFavorableHandle n d epsilon M sigma := {
lambda := sigma / 2
radialCap := mRad
exactCap := mExact
radialSource := univ
exactSource := univ
radialIndex := padRad
exactIndex := padExact
radialIndex_injective := by
intro x y h
apply Fin.ext
simpa [padRad] using congrArg val h
exactIndex_injective := by
intro x y h
apply Fin.ext
simpa [padExact] using congrArg val h
radialEmbedding := fun P => affineBinaryRealLaw M
(binaryPadLaw hmRad.2 (radialContractedBinaryLaw P.1 sigma hsigma hsigmaTwo))
exactEmbedding := fun P => affineBinaryRealLaw M (binaryPadLaw hmExact.2 P.1)
radialCoupling := fun P => (binaryIndependentFullPMF P.1).toMeasure
exactCoupling := fun P => (binaryIndependentFullPMF P.1).toMeasure
radial_mass_preserved := by
intro P _hP k
exact radialPaddedAffine_cellMass_image hmRad.2 M sigma hsigma hsigmaTwo P.1 k
exact_mass_preserved := by
intro P _hP k
exact affineBinaryPadded_cellMass_image hmExact.2 M P.1 k
radial_zero_extension := by
intro P _hP k hk
exact radialPaddedAffine_cellMass_off_image hmRad.2 M sigma hsigma hsigmaTwo P.1 k hk
exact_zero_extension := by
intro P _hP k hk
exact affineBinaryPadded_cellMass_off_image hmExact.2 M P.1 k hk
channelSuccess := bernoulliContractionSuccess sigma
scaledOutcome := fun b => M * ((if b then 1 else 0) - 1 / 2)
radial_channel_law := by
intro P _hP
exact radialPaddedAffine_fullLaw_eq_binaryFullChannel
P hmRad.2 hepsilon M sigma hsigma hsigmaTwo
exact_affine_law := by
intro P _hP
exact affineBinaryPadded_fullLaw hmExact.2 M P.1
}
have hRadCouple : RadialCouplingCertificate H :=
radialCouplingCertificate_of_independentFullPMF H (fun _ => rfl)
have hExactCouple : ExactCouplingCertificate H :=
exactCouplingCertificate_of_independentFullPMF H (fun _ => rfl)
have hfamily : leastFavorableAt n d epsilon M sigma bRad bExact H := by
refine ⟨hbRad, hbExact, rfl, rfl, rfl, ?_, ?_, hRadCouple, hExactCouple, ?_, ?_⟩
· exact radialSourceHard_of_source_eq_univ H hn hmRad.1
hepsilon hepsilon_half rfl
· exact exactSourceHard_of_source_eq_univ H hn hmExact.1
hepsilon hepsilon_half rfl
· intro b
rfl
· intro b
rfl
have hRadMem : RadialEmbeddingMembership H :=
radialEmbeddingMembership_of_contractedPadded H hmRad.2 hepsilon
hepsilon_half hM hsigma hsigmaTwo (fun _ => rfl)
have hExactMem : ExactEmbeddingMembership H :=
exactEmbeddingMembership_of_affinePadded H hmExact.1 hmExact.2
hepsilon hepsilon_half hM (fun _ => rfl)
have hRadius : RadiusCertificate H :=
radiusCertificate_of_radialEmbeddingMembership H hRadMem
have hRadTarget : ∀ P ∈ H.radialSource,
rawAteFormula (H.radialEmbedding P) =
(M * sigma / 2) *
CausalSmith.Stat.DiscreteAteMinimaxLoggap.treatedFunctional P.1 + 0 := by
intro P _hP
simpa using radialPaddedAffine_rawAteFormula P hmRad.2 hepsilon hsigma hsigmaTwo
have hExactTarget : ∀ P ∈ H.exactSource,
rawAteFormula (H.exactEmbedding P) =
M * CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1 := by
intro P _hP
exact affineBinaryPadded_rawAteFormula hmExact.2 P.1 P.2.1
have hTarget : TargetSeparationCertificate H :=
targetSeparationCertificate_of_pointwise_targets H (by simp [H]; ring)
hRadTarget hExactTarget
let K := bernoulliContractionObservedKernel padRad M sigma
letI : IsMarkovKernel K :=
bernoulliContractionObservedKernel_isMarkovKernel padRad M sigma hsigma hsigmaTwo
have hQRad : ∀ P ∈ H.radialSource,
(H.radialEmbedding P).observedLaw =
K ∘ₘ CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P.1 := by
intro P _hP
exact radialPaddedAffine_observedLaw hmRad.2 M sigma hsigma hsigmaTwo P.1
have hProcessing : DataProcessingCertificate H :=
radialDataProcessingCertificate_of_kernel H K hQRad
obtain ⟨_hmEpos, _hmEd, hcutExact, LExact, hsourceExact, hscaleExact⟩ :=
hexact n d hn hd
have hExactTransfer : ∀ est : Estimator n d M,
∃ P ∈ H.exactSource,
cTransfer * M ^ 2 *
(1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2)) ≤
mse (H.exactEmbedding P) est.1 := by
apply exactRiskTransfer_of_affine_transport (c := cTransfer)
(L := LExact) (a := M) (b := 0) H
(fun z : BinObs H.exactCap => affineObserved M (binaryPadObs hmExact.2 z))
(by nlinarith [hM]) (by fun_prop)
· intro P _hP
exact affineBinaryPadded_observedLaw hmExact.2 M P.1
· intro P _hP
simpa using hExactTarget P (Set.mem_univ P)
· simpa [H, mExact] using hsourceExact
· calc
cTransfer * M ^ 2 * _ ≤ cExact * M ^ 2 * _ := by
have hrate : 0 ≤ 1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2) := by positivity
gcongr
_ ≤ M ^ 2 * LExact := hscaleExact M
have hRadTransfer : ∀ est : Estimator n d M,
∃ P ∈ H.radialSource,
cTransfer * M ^ 2 * sigma ^ 2 * min 1 (polynomialComponent n d) ≤
mse (H.radialEmbedding P) est.1 := by
by_cases hsZero : sigma = 0
· intro est
obtain ⟨P, _hP, _hrisk⟩ := radial_source_risk_of_parametric_lower
hn hmRad.1 hepsilon hepsilon_half (fun _ => 0) measurable_const
⟨0, le_rfl, by simp⟩
refine ⟨P, Set.mem_univ P, ?_⟩
subst sigma
norm_num
unfold mse
exact integral_nonneg (fun _ => sq_nonneg _)
· by_cases hnLarge : NRad0 ≤ n
· obtain ⟨_hmRpos, _hmRd, _hcutR, hsourceRad, hscaleRad⟩ :=
hradial n d hn hd hnLarge
exact radial_risk_of_kernel_transport H (by linarith) K
(div_ne_zero (mul_ne_zero (by linarith) hsZero) (by norm_num)) hQRad hRadTarget
(by simpa [H, mRad] using hsourceRad)
(by
calc
cTransfer * M ^ 2 * sigma ^ 2 * min 1 (polynomialComponent n d) ≤
cRad * M ^ 2 * sigma ^ 2 * min 1 (polynomialComponent n d) := by
have hp : 0 ≤ min 1 (polynomialComponent n d) := by
apply le_min (by norm_num)
unfold polynomialComponent
positivity
gcongr
_ ≤ (M * sigma / 2) ^ 2 *
(aRad / 2 *
CausalSmith.Stat.DiscreteAteMinimaxLoggap.minimaxRate n mRad) := by
simpa [mRad] using hscaleRad M sigma)
· have hnSmall : n ≤ NRad := by
exact le_trans (le_of_lt (Nat.lt_of_not_ge hnLarge))
(Nat.le_max_left _ _)
exact radial_risk_of_kernel_transport H (by linarith) K
(div_ne_zero (mul_ne_zero (by linarith) hsZero) (by norm_num)) hQRad hRadTarget
(by
intro sourceEst hmeas _hbounded
exact radial_source_risk_of_parametric_lower
hn hmRad.1 hepsilon hepsilon_half sourceEst hmeas _hbounded)
(by
calc
cTransfer * M ^ 2 * sigma ^ 2 * min 1 (polynomialComponent n d) ≤
cSmall * M ^ 2 * sigma ^ 2 * min 1 (polynomialComponent n d) := by
have hp : 0 ≤ min 1 (polynomialComponent n d) := by
apply le_min (by norm_num)
unfold polynomialComponent
positivity
gcongr
_ ≤ (M * sigma / 2) ^ 2 * (1 / (200 * (n : ℝ))) := by
simpa [cSmall] using finiteSampleRadial_transportScale hn hnSmall M sigma)
have hTransferStrong : RiskTransferCertificate cTransfer H := by
intro est
exact ⟨hRadTransfer est, hExactTransfer est⟩
have hTransfer : RiskTransferCertificate cE H := by
intro est
obtain ⟨hr, he⟩ := hTransferStrong est
refine ⟨⟨hr.choose, hr.choose_spec.1, ?_⟩,
-- … truncated; follow the source link for the rest …
Restricted-range corollary of the all-alphabet radius-channel converse.
Formal statement
conclusion
∀ epsilon : ℝ,
0 < epsilon → epsilon < 1 / 2 → ∃ c_epsilon bRad : ℝ, 0 < c_epsilon ∧ c_epsilon
≤ 1 ∧
0 < bRad ∧
∀ n d : ℕ,
∀ M sigma : ℝ,
0 < n → 0 < d → 1
≤ M → 0 ≤ sigma → sigma ≤ 2 → (d : ℝ) ≤ c_epsilon * (n : ℝ) ^ 2 / logEN n → c_epsilon * M ^ 2 * (1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2 + sigma ^ 2 * min 1 (polynomialComponent n d)) ≤ minimaxRisk n d epsilon M sigma ∧
RadialTargetRiskTransferCertificate n d epsilon M sigma c_epsilon bRad
Proof (Lean source)
theorem radius_channel_converse :
∀ epsilon : ℝ, 0 < epsilon → epsilon < 1 / 2 →
∃ c_epsilon bRad : ℝ,
0 < c_epsilon ∧ c_epsilon ≤ 1 ∧
0 < bRad ∧
∀ n d : ℕ, ∀ M sigma : ℝ,
0 < n → 0 < d → 1 ≤ M → 0 ≤ sigma → sigma ≤ 2 →
(d : ℝ) ≤ c_epsilon * (n : ℝ) ^ 2 / logEN n →
c_epsilon * M ^ 2 *
(1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2 +
sigma ^ 2 * min 1 (polynomialComponent n d)) ≤
minimaxRisk n d epsilon M sigma ∧
RadialTargetRiskTransferCertificate n d epsilon M sigma
c_epsilon bRad := by
intro epsilon hepsilon hepsilon_half
obtain ⟨c_epsilon, bRad, hc, hc_one, hbRad, hall⟩ :=
radius_channel_converse_all_d epsilon hepsilon hepsilon_half
refine ⟨c_epsilon, bRad, hc, hc_one, hbRad, ?_⟩
intro n d M sigma hn hd hM hsigma hsigma_two hd_range
obtain ⟨hrisk, htransfer⟩ := hall n d M sigma hn hd hM hsigma hsigma_two
have hn_real : 0 < (n : ℝ) := by exact_mod_cast hn
have hn_one : (1 : ℝ) ≤ n := by
exact_mod_cast (Nat.one_le_iff_ne_zero.mpr hn.ne')
have hlog : 1 ≤ logEN n := by
rw [logEN, Real.log_mul (Real.exp_ne_zero 1) (ne_of_gt hn_real), Real.log_exp]
exact le_add_of_nonneg_right (Real.log_nonneg hn_one)
have hnumerator :
c_epsilon * (n : ℝ) ^ 2 ≤ (n : ℝ) ^ 2 := by
nlinarith [sq_nonneg (n : ℝ)]
have hcap : (d : ℝ) ≤ (n : ℝ) ^ 2 := calc
(d : ℝ) ≤ c_epsilon * (n : ℝ) ^ 2 / logEN n := hd_range
_ ≤ c_epsilon * (n : ℝ) ^ 2 :=
div_le_self (mul_nonneg (le_of_lt hc) (sq_nonneg (n : ℝ))) hlog
_ ≤ (n : ℝ) ^ 2 := hnumerator
have hratio : (d : ℝ) / (n : ℝ) ^ 2 ≤ 1 := by
rw [div_le_one (sq_pos_of_pos hn_real)]
exact hcap
refine ⟨?_, htransfer⟩
simpa [converseRate, min_eq_right hratio] using hrisk
T_RobustUpperConstruction 2 declarations
The explicit polynomial and collision constructions are admissible and obey their two all-alphabet risk envelopes without an additional logarithmic factor.
Formal statement
conclusion
∀ epsilon : ℝ,
0 < epsilon → epsilon < 1 / 2 → ∃ C_epsilon : ℝ, ∃ handle : PolynomialHandle, 0 < C_epsilon ∧ ∀ n d : ℕ, ∀ M sigma : ℝ, 0 < n → 0 < d → 1
≤ M → 0 ≤ sigma → sigma ≤ 2 → Measurable (polyEstimator handle (n := n) (d := d) M) ∧
(∀ s : Fin n → Obs d, polyEstimator handle (n := n) (d := d) M s ∈ Icc (-M) M) ∧
Measurable (collisionEstimator (n := n) (d := d) M) ∧
(∀ s : Fin n → Obs d, collisionEstimator (n := n) (d := d) M s ∈ Icc (-M) M) ∧
∀ P : ModelClass d epsilon M sigma,
mse P.law (polyEstimator handle (n := n) (d := d) M)
≤ C_epsilon * M ^ 2 * (1 / (n : ℝ) + min 1 (polynomialComponent n d)) ∧
mse P.law (collisionEstimator (n := n) (d := d) M)
≤ C_epsilon * M ^ 2 * (1 / (n : ℝ) + sigma ^ 2 + (d : ℝ) / (n : ℝ) ^ 2)
Proof (Lean source)
theorem robust_upper_construction_resolution_all_d :
∀ epsilon : ℝ, 0 < epsilon → epsilon < 1 / 2 →
∃ C_epsilon : ℝ, ∃ handle : PolynomialHandle,
0 < C_epsilon ∧
∀ n d : ℕ, ∀ M sigma : ℝ,
0 < n → 0 < d → 1 ≤ M → 0 ≤ sigma → sigma ≤ 2 →
Measurable
(polyEstimator handle (n := n) (d := d) M) ∧
(∀ s : Fin n → Obs d,
polyEstimator handle (n := n) (d := d) M s ∈ Icc (-M) M) ∧
Measurable (collisionEstimator (n := n) (d := d) M) ∧
(∀ s : Fin n → Obs d,
collisionEstimator (n := n) (d := d) M s ∈ Icc (-M) M) ∧
∀ P : ModelClass d epsilon M sigma,
mse P.law
(polyEstimator handle (n := n) (d := d) M) ≤
C_epsilon * M ^ 2 *
(1 / (n : ℝ) + min 1 (polynomialComponent n d)) ∧
mse P.law (collisionEstimator (n := n) (d := d) M) ≤
C_epsilon * M ^ 2 *
(1 / (n : ℝ) + sigma ^ 2 + (d : ℝ) / (n : ℝ) ^ 2) := by
intro epsilon hepsilon hepsilon_half
obtain ⟨Cpoly, handle, hCpoly, _hcomplexity, hpoly⟩ :=
continuous_ratio_polynomial_upper_all_d epsilon hepsilon hepsilon_half
obtain ⟨Ccollision, hCcollision, hcollision⟩ :=
continuous_occupancy_collision_upper_all_d
epsilon hepsilon hepsilon_half
refine ⟨max Cpoly Ccollision, handle,
lt_of_lt_of_le hCpoly (le_max_left _ _), ?_⟩
intro n d M sigma hn hd hM hsigma hsigma_two
obtain ⟨hpoly_meas, hpoly_mem, hpoly_risk⟩ :=
hpoly n d M sigma hn hd hM hsigma hsigma_two
obtain ⟨hcollision_meas, hcollision_mem⟩ :=
collisionEstimator_admissible (le_trans zero_le_one hM)
refine ⟨hpoly_meas, hpoly_mem, hcollision_meas, hcollision_mem, ?_⟩
intro P
constructor
· refine hpoly_risk P |>.trans ?_
gcongr
· apply add_nonneg
· positivity
· exact le_min zero_le_one (by
unfold polynomialComponent
positivity)
· exact le_max_left Cpoly Ccollision
· refine hcollision n d M sigma hn hd P |>.trans ?_
gcongr
exact le_max_right Cpoly Ccollision
Restricted-dimension form of the robust upper-construction theorem.
Formal statement
conclusion
∀ epsilon : ℝ,
0 < epsilon → epsilon < 1 / 2 → ∃ C_epsilon c_epsilon : ℝ, ∃ handle : PolynomialHandle, 0 < C_epsilon ∧ 0 < c_epsilon ∧ ∀ n d : ℕ, ∀ M sigma : ℝ, 0 < n → 0 < d → 1
≤ M → 0 ≤ sigma → sigma ≤ 2 → Measurable (polyEstimator handle (n := n) (d := d) M) ∧
(∀ s : Fin n → Obs d, polyEstimator handle (n := n) (d := d) M s ∈ Icc (-M) M) ∧
Measurable (collisionEstimator (n := n) (d := d) M) ∧
(∀ s : Fin n → Obs d, collisionEstimator (n := n) (d := d) M s ∈ Icc (-M) M) ∧
((d : ℝ) ≤ c_epsilon * (n : ℝ) ^ 2 / logEN n → ∀ P : ModelClass d epsilon M sigma, mse P.law (polyEstimator handle (n := n) (d := d) M) ≤ C_epsilon * M ^ 2 * (1 / (n : ℝ) + min 1 (polynomialComponent n d)) ∧ mse P.law (collisionEstimator (n := n) (d := d) M) ≤ C_epsilon * M ^ 2 * (1 / (n : ℝ) + sigma ^ 2 + (d : ℝ) / (n : ℝ) ^ 2))
Proof (Lean source)
theorem robust_upper_construction_resolution :
∀ epsilon : ℝ, 0 < epsilon → epsilon < 1 / 2 →
∃ C_epsilon c_epsilon : ℝ, ∃ handle : PolynomialHandle,
0 < C_epsilon ∧ 0 < c_epsilon ∧
∀ n d : ℕ, ∀ M sigma : ℝ,
0 < n → 0 < d → 1 ≤ M → 0 ≤ sigma → sigma ≤ 2 →
Measurable
(polyEstimator handle (n := n) (d := d) M) ∧
(∀ s : Fin n → Obs d,
polyEstimator handle (n := n) (d := d) M s ∈ Icc (-M) M) ∧
Measurable (collisionEstimator (n := n) (d := d) M) ∧
(∀ s : Fin n → Obs d,
collisionEstimator (n := n) (d := d) M s ∈ Icc (-M) M) ∧
((d : ℝ) ≤ c_epsilon * (n : ℝ) ^ 2 / logEN n →
∀ P : ModelClass d epsilon M sigma,
mse P.law
(polyEstimator handle (n := n) (d := d) M) ≤
C_epsilon * M ^ 2 *
(1 / (n : ℝ) + min 1 (polynomialComponent n d)) ∧
mse P.law (collisionEstimator (n := n) (d := d) M) ≤
C_epsilon * M ^ 2 *
(1 / (n : ℝ) + sigma ^ 2 + (d : ℝ) / (n : ℝ) ^ 2)) := by
intro epsilon hepsilon hepsilon_half
obtain ⟨C_epsilon, handle, hC, hbound⟩ :=
robust_upper_construction_resolution_all_d
epsilon hepsilon hepsilon_half
refine ⟨C_epsilon, 1, handle,
hC, zero_lt_one, ?_⟩
intro n d M sigma hn hd hM hsigma hsigma_two
have hall := hbound n d M sigma hn hd hM hsigma hsigma_two
refine ⟨hall.1, hall.2.1, hall.2.2.1, hall.2.2.2.1, ?_⟩
intro _
exact hall.2.2.2.2
T_TwoSidedMinimaxBracket 3 declarations
All-alphabet two-sided minimax bracket on one and the same real-outcome model class. The two benchmarks are not asserted to be uniformly equivalent for every shrinking interior radius.
Formal statement
conclusion
∀ epsilon : ℝ,
0 < epsilon → epsilon < 1 / 2 → ∃ c_epsilon : ℝ, ∃ C_epsilon : ℝ, 0 < c_epsilon ∧ c_epsilon
≤ 1 ∧
1 ≤ C_epsilon ∧
c_epsilon ≤ C_epsilon ∧
∀ n d : ℕ,
∀ M sigma : ℝ,
0 < n → 0 < d → 1
≤ M → 0 ≤ sigma → sigma ≤ 2 → c_epsilon * M ^ 2 * converseRate n d sigma ≤ minimaxRisk n d epsilon M sigma ∧
minimaxRisk n d epsilon M sigma ≤ C_epsilon * M ^ 2 * frontierRate n d sigma
Proof (Lean source)
theorem two_sided_minimax_bracket_all_d :
∀ epsilon : ℝ, 0 < epsilon → epsilon < 1 / 2 →
∃ c_epsilon : ℝ, -- @realizes c_{\epsilon}(carrier Real)
∃ C_epsilon : ℝ, -- @realizes C_{\epsilon}(carrier Real)
0 < c_epsilon ∧ c_epsilon ≤ 1 ∧ -- @realizes c_{\epsilon}(range (0,1])
1 ≤ C_epsilon ∧ -- @realizes C_{\epsilon}(range [1,infinity))
c_epsilon ≤ C_epsilon ∧
∀ n d : ℕ, ∀ M sigma : ℝ,
0 < n → 0 < d → 1 ≤ M → 0 ≤ sigma → sigma ≤ 2 →
c_epsilon * M ^ 2 * converseRate n d sigma ≤
minimaxRisk n d epsilon M sigma ∧
minimaxRisk n d epsilon M sigma ≤
C_epsilon * M ^ 2 * frontierRate n d sigma := by
intro epsilon hepsilon hepsilon_half
obtain ⟨C0, handle, hC0, hupper⟩ :=
frontier_upper_bound_all_d epsilon hepsilon hepsilon_half
obtain ⟨c, bRad, hc, hc_one, hbRad, hlower⟩ :=
radius_channel_converse_all_d
epsilon hepsilon hepsilon_half
refine ⟨c, max 1 C0, hc, hc_one, le_max_left _ _,
hc_one.trans (le_max_left _ _), ?_⟩
intro n d M sigma hn hd hM hsigma hsigma_two
obtain ⟨hrisk, hradial⟩ :=
hlower n d M sigma hn hd hM hsigma hsigma_two
rcases hradial with
⟨cap, source, pad, embedding, coupling, hchannelData,
_htargetSeparation, _hriskTransfer⟩
rcases hchannelData with
⟨_hcap, _hinjective,
hsourceHard, _hmass, _hzero, _hcoupling, _hchannel,
hmembership, _hradius, _hprocessing⟩
rcases hsourceHard with ⟨cHard, hcHard, hHard⟩
obtain ⟨Psrc, hPsrc, _hPsrcRisk⟩ :=
hHard ⟨fun _ => 0, measurable_const⟩
obtain ⟨Q, _hQ⟩ := hmembership Psrc hPsrc
letI : Nonempty (ModelClass d epsilon M sigma) := ⟨Q⟩
let poly := polyEstimatorElement (n := n) (d := d) handle hM
let collision := collisionEstimatorElement (n := n) (d := d) hM
let selector := totalSelector hM sigma poly collision
obtain ⟨hselector_meas, hselector_mem, hselector_risk⟩ :=
hupper n d M sigma hn hd hM hsigma hsigma_two
have hworst_nonneg (est : Estimator n d M) :
0 ≤ worstCaseMSE n d epsilon M sigma est.1 := by
unfold worstCaseMSE
cases isEmpty_or_nonempty (ModelClass d epsilon M sigma) with
| inl hempty =>
letI := hempty
simp
| inr hnonempty =>
letI := hnonempty
by_cases hbounded : BddAbove (range (fun P : ModelClass d epsilon M sigma =>
mse P.law est.1))
· have hmse : 0 ≤ mse
(Classical.arbitrary (ModelClass d epsilon M sigma)).law est.1 := by
unfold mse
exact integral_nonneg (fun x => sq_nonneg (est.1 x -
rawAteFormula (Classical.arbitrary
(ModelClass d epsilon M sigma)).law))
exact hmse.trans (le_ciSup hbounded (Classical.arbitrary _))
· change 0 ≤ (⨆ P : ModelClass d epsilon M sigma, mse P.law est.1)
rw [show (⨆ P : ModelClass d epsilon M sigma, mse P.law est.1) = sSup ∅ from
csSup_of_not_bddAbove hbounded]
simp
have hb : BddBelow (range (fun est : Estimator n d M =>
worstCaseMSE n d epsilon M sigma est.1)) := by
refine ⟨0, ?_⟩
rintro _ ⟨est, rfl⟩
exact hworst_nonneg est
have hminimax_selector :
minimaxRisk n d epsilon M sigma ≤
worstCaseMSE n d epsilon M sigma selector.1 := by
unfold minimaxRisk
exact ciInf_le hb selector
have hworst_selector :
worstCaseMSE n d epsilon M sigma selector.1 ≤
C0 * M ^ 2 * frontierRate n d sigma := by
unfold worstCaseMSE
apply ciSup_le
intro P
exact hselector_risk P
refine ⟨hrisk, hminimax_selector.trans (hworst_selector.trans ?_)⟩
have hrate : 0 ≤ M ^ 2 * frontierRate n d sigma := by
exact mul_nonneg (sq_nonneg M) (le_of_lt (frontierRate_pos hn))
calc
C0 * M ^ 2 * frontierRate n d sigma =
C0 * (M ^ 2 * frontierRate n d sigma) := by ring
_ ≤ max 1 C0 * (M ^ 2 * frontierRate n d sigma) :=
mul_le_mul_of_nonneg_right (le_max_right 1 C0) hrate
_ = max 1 C0 * M ^ 2 * frontierRate n d sigma := by ring
Restricted-range form of the headline minimax bracket.
Formal statement
conclusion
∀ epsilon : ℝ,
0 < epsilon → epsilon < 1 / 2 → ∃ c_epsilon : ℝ, ∃ C_epsilon : ℝ, 0 < c_epsilon ∧ c_epsilon
≤ 1 ∧
1 ≤ C_epsilon ∧
c_epsilon ≤ C_epsilon ∧
∀ n d : ℕ,
∀ M sigma : ℝ,
0 < n → 0 < d → 1
≤ M → 0 ≤ sigma → sigma ≤ 2 → (d : ℝ) ≤ c_epsilon * (n : ℝ) ^ 2 / logEN n → c_epsilon * M ^ 2 * (1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2 + sigma ^ 2 * min 1 (polynomialComponent n d)) ≤ minimaxRisk n d epsilon M sigma ∧
minimaxRisk n d epsilon M sigma ≤ C_epsilon * M ^ 2 * frontierRate n d sigma
Proof (Lean source)
theorem two_sided_minimax_bracket :
∀ epsilon : ℝ, 0 < epsilon → epsilon < 1 / 2 →
∃ c_epsilon : ℝ, -- @realizes c_{\epsilon}(existential range (0,1])
∃ C_epsilon : ℝ, -- @realizes C_{\epsilon}(existential range [1,infinity))
0 < c_epsilon ∧ c_epsilon ≤ 1 ∧
1 ≤ C_epsilon ∧ c_epsilon ≤ C_epsilon ∧
∀ n d : ℕ, ∀ M sigma : ℝ,
0 < n → 0 < d → 1 ≤ M → 0 ≤ sigma → sigma ≤ 2 →
(d : ℝ) ≤ c_epsilon * (n : ℝ) ^ 2 / logEN n →
c_epsilon * M ^ 2 *
(1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2 +
sigma ^ 2 * min 1 (polynomialComponent n d)) ≤
minimaxRisk n d epsilon M sigma ∧
minimaxRisk n d epsilon M sigma ≤
C_epsilon * M ^ 2 * frontierRate n d sigma := by
intro epsilon hepsilon hepsilon_half
obtain ⟨c_epsilon, C_epsilon, hc, hc_one, hC, hcC, hall⟩ :=
two_sided_minimax_bracket_all_d epsilon hepsilon hepsilon_half
refine ⟨c_epsilon, C_epsilon, hc, hc_one, hC, hcC, ?_⟩
intro n d M sigma hn hd hM hsigma hsigma_two hd_range
obtain ⟨hlower, hupper⟩ := hall n d M sigma hn hd hM hsigma hsigma_two
have hn_real : 0 < (n : ℝ) := by exact_mod_cast hn
have hn_one : (1 : ℝ) ≤ n := by
exact_mod_cast (Nat.one_le_iff_ne_zero.mpr hn.ne')
have hlog : 1 ≤ logEN n := by
rw [logEN, Real.log_mul (Real.exp_ne_zero 1) (ne_of_gt hn_real), Real.log_exp]
exact le_add_of_nonneg_right (Real.log_nonneg hn_one)
have hnumerator :
c_epsilon * (n : ℝ) ^ 2 ≤ (n : ℝ) ^ 2 := by
nlinarith [sq_nonneg (n : ℝ)]
have hcap : (d : ℝ) ≤ (n : ℝ) ^ 2 := calc
(d : ℝ) ≤ c_epsilon * (n : ℝ) ^ 2 / logEN n := hd_range
_ ≤ c_epsilon * (n : ℝ) ^ 2 :=
div_le_self (mul_nonneg (le_of_lt hc) (sq_nonneg (n : ℝ))) hlog
_ ≤ (n : ℝ) ^ 2 := hnumerator
have hratio : (d : ℝ) / (n : ℝ) ^ 2 ≤ 1 := by
rw [div_le_one (sq_pos_of_pos hn_real)]
exact hcap
refine ⟨?_, hupper⟩
simpa [converseRate, min_eq_right hratio] using hlower
The canonical affine embedding realizes the binary subclasses, with strict image inclusion and the lower- and upper-bound transfers assembled here.
Formal statement
conclusion
∀ epsilon : ℝ,
0 < epsilon → epsilon < 1 / 2 → ∃ c_exact c_radius b_radius C_upper : ℝ, ∃ handle : PolynomialHandle, 0 < c_exact ∧ 0 < c_radius ∧ 0 < b_radius ∧ 0 < C_upper ∧ ∀ d : ℕ, ∀ M : ℝ, 0 < d → 1
≤ M → ∃ Phi : BinLaw d → RealLaw d, (∀ P, AffineBinaryEmbedding M P (Phi P)) ∧ (∀ P, CausalSmith.Stat.DiscreteAteMinimaxLoggap.Overlap epsilon P → ∃ Q : UnrestrictedClass d epsilon M, Q.law = Phi P) ∧ (∀ P, BinaryExactHomogeneous epsilon P → ∃ Q : ModelClass d epsilon M 0, Q.law = Phi P) ∧ (∀ sigma : ℝ, 0 ≤ sigma → sigma ≤ 2 → ∃ Q : ModelClass d epsilon M sigma, ∀ P, Phi P ≠ Q.law) ∧ ∀ n : ℕ, ∀ sigma : ℝ, 0 < n → 0 ≤ sigma → sigma ≤ 2 → ((d : ℝ) ≤ c_exact * (n : ℝ) ^ 2 → (∀ est : Estimator n d M, ∃ P : BinaryExactLaw n d epsilon, c_exact * M ^ 2 * (1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2) ≤ mse (Phi P.1) est.1) ∧ c_exact * M ^ 2 * (1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2) ≤ minimaxRisk n d epsilon M sigma) ∧ ((c_radius * M ^ 2 * sigma ^ 2 * min 1 (polynomialComponent n d) ≤ minimaxRisk n d epsilon M sigma) ∧ RadialTargetRiskTransferCertificate n d epsilon M sigma c_radius b_radius) ∧ Measurable (polyEstimator handle (n := n) (d := d) M) ∧ (∀ s : Fin n → Obs d, polyEstimator handle (n := n) (d := d) M s ∈ Icc (-M) M) ∧ (∀ Q : ModelClass d epsilon M sigma, mse Q.law (polyEstimator handle (n := n) (d := d) M) ≤ C_upper * M ^ 2 * (1 / (n : ℝ) + min 1 (polynomialComponent n d)))
Proof (Lean source)
theorem zeng_class_inclusion_and_lower_transfer :
∀ epsilon : ℝ, 0 < epsilon → epsilon < 1 / 2 →
∃ c_exact c_radius b_radius C_upper : ℝ, ∃ handle : PolynomialHandle,
0 < c_exact ∧ 0 < c_radius ∧ 0 < b_radius ∧
0 < C_upper ∧
∀ d : ℕ, ∀ M : ℝ, 0 < d → 1 ≤ M → -- @realizes M(range [1,infinity))
∃ Phi : BinLaw d → RealLaw d,
(∀ P, AffineBinaryEmbedding M P (Phi P)) ∧
(∀ P, CausalSmith.Stat.DiscreteAteMinimaxLoggap.Overlap epsilon P →
∃ Q : UnrestrictedClass d epsilon M, Q.law = Phi P) ∧
(∀ P, BinaryExactHomogeneous epsilon P →
∃ Q : ModelClass d epsilon M 0, Q.law = Phi P) ∧
(∀ sigma : ℝ, 0 ≤ sigma → sigma ≤ 2 →
∃ Q : ModelClass d epsilon M sigma, ∀ P, Phi P ≠ Q.law) ∧
∀ n : ℕ, ∀ sigma : ℝ,
0 < n → 0 ≤ sigma → sigma ≤ 2 →
((d : ℝ) ≤ c_exact * (n : ℝ) ^ 2 →
(∀ est : Estimator n d M,
∃ P : BinaryExactLaw n d epsilon,
c_exact * M ^ 2 *
(1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2) ≤
mse (Phi P.1) est.1) ∧
c_exact * M ^ 2 *
(1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2) ≤
minimaxRisk n d epsilon M sigma) ∧
((c_radius * M ^ 2 * sigma ^ 2 *
min 1 (polynomialComponent n d) ≤
minimaxRisk n d epsilon M sigma) ∧
RadialTargetRiskTransferCertificate n d epsilon M sigma
c_radius b_radius) ∧
Measurable
(polyEstimator handle (n := n) (d := d) M) ∧
(∀ s : Fin n → Obs d,
polyEstimator handle (n := n) (d := d) M s ∈ Icc (-M) M) ∧
(∀ Q : ModelClass d epsilon M sigma,
mse Q.law
(polyEstimator handle (n := n) (d := d) M) ≤
C_upper * M ^ 2 *
(1 / (n : ℝ) + min 1 (polynomialComponent n d))) := by
intro epsilon hepsilon hepsilon_half
obtain ⟨cExact, hcExact, hExact⟩ :=
scaled_binary_exact_lower_transfer_all_d epsilon hepsilon hepsilon_half
obtain ⟨cRadius, bRad, hcRadius, _hcRadiusOne, hbRad, hRadius⟩ :=
radius_channel_converse_all_d epsilon hepsilon hepsilon_half
obtain ⟨CUpper, handle, hCUpper, _hcomplexity, hUpper⟩ :=
continuous_ratio_polynomial_upper_all_d epsilon hepsilon hepsilon_half
let uExact : ℝ := min (1 / 2) (8 * (1 / 2 - epsilon) ^ 2)
have huExact : 0 < uExact := by dsimp [uExact]; positivity
let cWitness : ℝ := min (1 / 100) (uExact ^ 2 / 64) / 4
have hcWitness : 0 < cWitness := by dsimp [cWitness]; positivity
let cOut := min cExact (min cWitness 1)
have hcOut : 0 < cOut := lt_min hcExact (lt_min hcWitness zero_lt_one)
have hcOutExact : cOut ≤ cExact := min_le_left _ _
have hcOutWitness : cOut ≤ cWitness :=
(min_le_right cExact (min cWitness 1)).trans (min_le_left _ _)
have hcOutOne : cOut ≤ 1 :=
(min_le_right cExact (min cWitness 1)).trans (min_le_right _ _)
refine ⟨cOut, cRadius, bRad, CUpper, handle,
hcOut, hcRadius, hbRad, hCUpper, ?_⟩
intro d M hd hM
let k : Fin d := ⟨0, hd⟩
refine ⟨affineBinaryRealLaw M, affineBinaryRealLaw_embedding M, ?_, ?_, ?_, ?_⟩
· intro P hP
exact ⟨affineBinaryRealLaw_unrestricted P hepsilon hepsilon_half hM hP, rfl⟩
· intro P hP
exact ⟨affineBinaryRealLaw_model_zero P hepsilon hepsilon_half hM hP, rfl⟩
· intro sigma hsigma hsigma_two
let Q := testModelClass k epsilon M sigma 0 hepsilon hepsilon_half hM
hsigma hsigma_two (by rw [abs_zero]; positivity)
refine ⟨Q, ?_⟩
intro P
exact affineBinaryRealLaw_ne_testModelLaw k hepsilon hepsilon_half hM
hsigma hsigma_two P
· intro n sigma hn hsigma hsigma_two
have hexact := hExact n d M sigma hn hd hM hsigma hsigma_two
obtain ⟨hradius, hradialTransfer⟩ :=
hRadius n d M sigma hn hd hM hsigma hsigma_two
obtain ⟨hmeas, hmem, hrisk⟩ :=
hUpper n d M sigma hn hd hM hsigma hsigma_two
refine ⟨?_, ⟨?_, hradialTransfer⟩, hmeas, hmem, hrisk⟩
· intro hdRange
have hnR : 0 < (n : ℝ) := by exact_mod_cast hn
have hdOne : (d : ℝ) ≤ (n : ℝ) ^ 2 := calc
(d : ℝ) ≤ cOut * (n : ℝ) ^ 2 := hdRange
_ ≤ 1 * (n : ℝ) ^ 2 := by gcongr
_ = (n : ℝ) ^ 2 := one_mul _
have hratio : (d : ℝ) / (n : ℝ) ^ 2 ≤ 1 := by
rw [div_le_one (sq_pos_of_pos hnR)]
exact hdOne
have hrate : 0 ≤ 1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2 := by
positivity
constructor
· intro est
have hparam := binaryExactMinimaxRisk_parametric_lower
hn hd hepsilon hepsilon_half
have hcollision := binaryExactMinimaxRisk_collision_lower
hn hd hepsilon hepsilon_half hdOne
have hbasePos : 0 < min (1 / 100 : ℝ) (uExact ^ 2 / 64) :=
lt_min (by norm_num) (by positivity)
have hparam' : min (1 / 100 : ℝ) (uExact ^ 2 / 64) *
(1 / (n : ℝ)) ≤ binaryExactMinimaxRisk n d epsilon := by
calc
_ ≤ (1 / 100 : ℝ) * (1 / (n : ℝ)) :=
mul_le_mul_of_nonneg_right (min_le_left _ _) (by positivity)
_ = 1 / (100 * (n : ℝ)) := by ring
_ ≤ binaryExactMinimaxRisk n d epsilon := hparam
have hcollision' : min (1 / 100 : ℝ) (uExact ^ 2 / 64) *
((d : ℝ) / (n : ℝ) ^ 2) ≤
binaryExactMinimaxRisk n d epsilon := by
calc
_ ≤ (uExact ^ 2 / 64) * ((d : ℝ) / (n : ℝ) ^ 2) :=
mul_le_mul_of_nonneg_right (min_le_right _ _) (by positivity)
_ ≤ binaryExactMinimaxRisk n d epsilon := by
simpa [uExact] using hcollision
have hriskPos : 0 < binaryExactMinimaxRisk n d epsilon :=
lt_of_lt_of_le (by positivity) hparam
have hsourceLt : cOut *
(1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2) <
binaryExactMinimaxRisk n d epsilon := by
have hout : cOut ≤
min (1 / 100 : ℝ) (uExact ^ 2 / 64) / 4 := by
simpa [cWitness] using hcOutWitness
have hx : 0 ≤ 1 / (n : ℝ) := by positivity
have hy : 0 ≤ (d : ℝ) / (n : ℝ) ^ 2 := by positivity
have hsum := add_le_add hparam' hcollision'
nlinarith [mul_le_mul_of_nonneg_right hout (add_nonneg hx hy)]
have hhard := binaryExactMinimaxRisk_hard_family_of_lt
hn hd hepsilon hepsilon_half hsourceLt
have hMne : M ≠ 0 := ne_of_gt (lt_of_lt_of_le zero_lt_one hM)
have htransport :=
forall_estimator_exists_sqRisk_ge_of_deterministic_affine_transport_pi
(Iota := BinaryExactLaw n d epsilon) (n := n)
(P := fun P => CausalSmith.Stat.DiscreteAteMinimaxLoggap.obsLaw P.1)
(Q := fun P => (affineBinaryRealLaw M P.1).observedLaw)
(theta := fun P =>
CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1)
(phi := affineObserved M) (a := M) (b := 0)
(L := cOut *
(1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2))
hMne (measurable_affineObserved M)
(fun P => (affineBinaryRealLaw_embedding M P.1).1)
(by
intro sourceEst hsourceEst
obtain ⟨P, hP⟩ := hhard sourceEst hsourceEst
exact ⟨P, by
simpa [sqRisk,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.mse,
CausalSmith.Stat.DiscreteAteMinimaxLoggap.productLaw] using hP⟩)
est.1 est.2.1
obtain ⟨P, hP⟩ := htransport
refine ⟨P, ?_⟩
have htau : rawAteFormula (affineBinaryRealLaw M P.1) =
M * CausalSmith.Stat.DiscreteAteMinimaxLoggap.ateFunctional P.1 :=
rawAteFormula_eq_mul_binaryAte_of_embedding
(affineBinaryRealLaw_embedding M P.1) P.2.1
rw [← htau] at hP
simp only [sqRisk, mse, productLaw] at hP ⊢
rw [show cOut * M ^ 2 *
(1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2) =
M ^ 2 * (cOut *
(1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2)) by ring]
simpa only [add_zero] using hP
· calc
cOut * M ^ 2 * (1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2) ≤
cExact * M ^ 2 * (1 / (n : ℝ) + (d : ℝ) / (n : ℝ) ^ 2) := by
gcongr
_ ≤ minimaxRisk n d epsilon M sigma := by
simpa [min_eq_right hratio] using hexact
· unfold converseRate at hradius
have hpoly : 0 ≤ polynomialComponent n d := by
unfold polynomialComponent
positivity
have hbase : 0 ≤ 1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2) := by
positivity
have hradterm : 0 ≤ sigma ^ 2 * min 1 (polynomialComponent n d) :=
mul_nonneg (sq_nonneg _) (le_min zero_le_one hpoly)
have hcoeff : 0 ≤ cRadius * M ^ 2 :=
mul_nonneg (le_of_lt hcRadius) (sq_nonneg M)
calc
cRadius * M ^ 2 * sigma ^ 2 * min 1 (polynomialComponent n d) =
(cRadius * M ^ 2) *
(sigma ^ 2 * min 1 (polynomialComponent n d)) := by ring
_ ≤ cRadius * M ^ 2 *
(1 / (n : ℝ) + min 1 ((d : ℝ) / (n : ℝ) ^ 2) +
sigma ^ 2 * min 1 (polynomialComponent n d)) := by
apply mul_le_mul_of_nonneg_left _ hcoeff
linarith
_ ≤ minimaxRisk n d epsilon M sigma := hradius