Formalization: Honest Expected Length for Transported Complier Effects with Weak First Stages
The complete Lean development behind this paper — every definition, lemma, and theorem of its module, including helpers the paper text never cites. Identifiers link within this page, into the Causalean library, or out to the official Mathlib docs.
Basic 72 declarations Explicit full-data and observed-data worlds
Explicit full-data and observed-data worlds
def FullData
Full-data coordinate (S,X,D(0),D(1),Y(0),Y(1)).
def SourceObs
Source observation (X,Z,D,Y).
def AssignedFullData
The assigned source overlay (full data,Z).
Definition (Lean source)
def fullS
Returns the indicator for whether a full-data unit belongs to the source population.
def fullX
Returns a full-data unit's covariate value.
def fullD0
Returns a unit's potential treatment receipt under instrument value zero.
def fullD1
Returns a unit's potential treatment receipt under instrument value one.
def fullY0
Returns a unit's potential outcome under treatment receipt zero.
def fullY1
Returns a unit's potential outcome under treatment receipt one.
def boolReal
Converts a binary indicator to its real-valued zero-one representation.
def potentialReceipt
Returns the treatment receipt a unit would have under the specified instrument assignment.
def potentialOutcome
Returns the outcome a unit would have under the specified treatment receipt.
Definition (Lean source)
def potentialOutcome {𝒳 : Type*} (o : FullData 𝒳) (d : Bool) : ℝ :=
if d then fullY1 o else fullY0 o
An array law contains only primitive laws and conditional-mean versions. The assumption atoms below pin every version to its law by integral identities.
Definition (Lean source)
parameters
𝒳 :
Type*
fields
@realizes P_n^F(full-data law carrier; probability/support via FullDataSupport) @realizes P(triangular array primitive)
assignedSourceLaw :
ℕ → Measure (AssignedFullData 𝒳)
@realizes P_S(source assigned-full-data carrier)
propensity :
ℕ → 𝒳 → ℝ
@realizes e(carrier 𝒳→ℝ; range pinned by InstrumentOverlap)
propensity_measurable :
∀ n, Measurable (propensity n)
@realizes Y^Z(z)(assignment-potential-outcome carrier; pinned by IVExclusion)
assignmentOutcome_measurable :
∀ n z, Measurable (assignmentOutcome n z)
assignmentContrast :
ℕ → Bool → 𝒳 → ℝ
assignmentContrast_measurable :
∀ n s, Measurable (assignmentContrast n s)
receiptContrast :
ℕ → Bool → 𝒳 → ℝ
receiptContrast_measurable :
∀ n s, Measurable (receiptContrast n s)
def populationLaw
Population-conditioned full-data law.
Definition (Lean source)
noncomputable def populationLaw (P : TransportedArray 𝒳) (n : ℕ) (s : Bool) :
Measure (FullData 𝒳) :=
ProbabilityTheory.cond (P.fullLaw n) {o | fullS o = s}
def populationXLaw
The covariate marginal in population s.
Definition (Lean source)
noncomputable def populationXLaw (P : TransportedArray 𝒳) (n : ℕ) (s : Bool) :
Measure 𝒳 :=
(populationLaw P n s).map fullX
def targetXLaw
Target covariate law.
Definition (Lean source)
noncomputable def targetXLaw (P : TransportedArray 𝒳) (n : ℕ) : Measure 𝒳 :=
populationXLaw P n false
def derivedAssignmentOutcome
Assignment potential outcome Y(D(z)).
Definition (Lean source)
def derivedAssignmentOutcome (o : FullData 𝒳) (z : Bool) : ℝ :=
potentialOutcome o (potentialReceipt o z)
def observeSource
The observed source record derived from assigned full data.
Definition (Lean source)
def observeSource (q : AssignedFullData 𝒳) : SourceObs 𝒳 :=
(fullX q.1, q.2, potentialReceipt q.1 q.2,
potentialOutcome q.1 (potentialReceipt q.1 q.2))
def sourceObsLaw
Observed source law induced by the assigned full-data overlay.
Definition (Lean source)
noncomputable def sourceObsLaw (P : TransportedArray 𝒳) (n : ℕ) :
Measure (SourceObs 𝒳) :=
(P.assignedSourceLaw n).map observeSource
def sourceXLaw
Source covariate marginal.
Definition (Lean source)
noncomputable def sourceXLaw (P : TransportedArray 𝒳) (n : ℕ) : Measure 𝒳 :=
(sourceObsLaw P n).map (fun o => o.1)
def instrumentScore
Inverse-propensity score multiplier.
Definition (Lean source)
noncomputable def instrumentScore (P : TransportedArray 𝒳) (n : ℕ)
(o : SourceObs 𝒳) : ℝ :=
(if o.2.1 then 1 / P.propensity n o.1 else
-1 / (1 - P.propensity n o.1))
def OutcomeContrastRepresentation
A measurable version of the observed source outcome ITT is characterized by its integrals over every measurable covariate set.
Definition (Lean source)
def OutcomeContrastRepresentation (P : TransportedArray 𝒳) (n : ℕ)
(f : 𝒳 → ℝ) : Prop :=
Measurable f ∧
∀ A, MeasurableSet A →
∫ o in {o | o.1 ∈ A},
instrumentScore P n o * o.2.2.2 ∂sourceObsLaw P n =
∫ x in A, f x ∂sourceXLaw P n
def deltaY
The source outcome ITT is the canonical observed-law representative, rather than an independently supplied model coordinate.
Definition (Lean source)
noncomputable def TransportedArray.deltaY (P : TransportedArray 𝒳)
(n : ℕ) : 𝒳 → ℝ :=
P.assignmentContrast n true
theorem deltaY_measurable
The source conditional outcome contrast is measurable as a function of covariates.
Formal statement
Proof (Lean source)
lemma TransportedArray.deltaY_measurable (P : TransportedArray 𝒳) (n : ℕ) :
Measurable (P.deltaY n) :=
P.assignmentContrast_measurable n true
A measurable version of the observed source receipt ITT is characterized by its integrals over every measurable covariate set.
Definition (Lean source)
def ReceiptContrastRepresentation (P : TransportedArray 𝒳) (n : ℕ)
(f : 𝒳 → ℝ) : Prop :=
Measurable f ∧
∀ A, MeasurableSet A →
∫ o in {o | o.1 ∈ A},
instrumentScore P n o * boolReal o.2.2.1 ∂sourceObsLaw P n =
∫ x in A, f x ∂sourceXLaw P n
def deltaD
The source receipt ITT is the canonical observed-law representative, rather than an independently supplied model coordinate.
Definition (Lean source)
noncomputable def TransportedArray.deltaD (P : TransportedArray 𝒳)
(n : ℕ) : 𝒳 → ℝ :=
P.receiptContrast n true
theorem deltaD_measurable
The source conditional treatment-receipt contrast is measurable as a function of covariates.
Formal statement
Proof (Lean source)
lemma TransportedArray.deltaD_measurable (P : TransportedArray 𝒳) (n : ℕ) :
Measurable (P.deltaD n) :=
P.receiptContrast_measurable n true
def transportWeight
Radon--Nikodym transport weight, real-valued by ENNReal.toReal.
Definition (Lean source)
noncomputable def transportWeight (P : TransportedArray 𝒳) (n : ℕ) : 𝒳 → ℝ :=
fun x => ((targetXLaw P n).rnDeriv (sourceXLaw P n) x).toReal
def kishDispersion
Kish dispersion.
Definition (Lean source)
noncomputable def kishDispersion (P : TransportedArray 𝒳) (n : ℕ) : ℝ :=
∫ x, (transportWeight P n x) ^ 2 ∂sourceXLaw P n
def transportedOutcomeITT
Transported source outcome contrast.
Definition (Lean source)
noncomputable def transportedOutcomeITT (P : TransportedArray 𝒳) (n : ℕ) : ℝ :=
∫ x, transportWeight P n x * P.deltaY n x ∂sourceXLaw P n
def transportedFirstStage
The paper's transported first stage is the source-law moment of the transport weight times the identified conditional receipt contrast.
Definition (Lean source)
noncomputable def transportedFirstStage (P : TransportedArray 𝒳) (n : ℕ) : ℝ :=
∫ x, transportWeight P n x * P.deltaD n x ∂sourceXLaw P n
def effectiveStrength
Effective identification strength.
Definition (Lean source)
noncomputable def effectiveStrength (P : TransportedArray 𝒳) (n : ℕ) : ℝ :=
(n : ℝ) * (transportedFirstStage P n) ^ 2 / kishDispersion P n
theorem index_pos_of_pos_le_effectiveStrength
A positive effective-strength threshold excludes the zero sample index.
Formal statement
hypotheses
conclusion
0 < n
Proof (Lean source)
lemma index_pos_of_pos_le_effectiveStrength (P : TransportedArray 𝒳)
{n : ℕ} {t0 : ℝ} (ht0 : 0 < t0)
(hstrength : t0 ≤ effectiveStrength P n) : 0 < n := by
by_contra hn
have hn0 : n = 0 := Nat.eq_zero_of_not_pos hn
subst n
simp [effectiveStrength] at hstrength
exact (not_lt_of_ge hstrength) ht0
def parameterSpace
Forced parameter range.
def targetCACE
Target complier average causal effect, with Lean's total division convention.
Definition (Lean source)
noncomputable def targetCACE (P : TransportedArray 𝒳) (n : ℕ) : ℝ :=
(∫ o, (fullY1 o - fullY0 o) *
(if fullD1 o = true ∧ fullD0 o = false then 1 else 0)
∂populationLaw P n false) / targetComplierShare P n
def SourceSample
Index-n source sample.
Definition (Lean source)
def TargetSample
Index-n target covariate sample.
Definition (Lean source)
abbrev TargetSample (𝒳 : Type*) (N : ℕ) := Fin N → 𝒳
def TwoSample
Product sample carrier.
Definition (Lean source)
The independent source/target finite-product sampling law.
Definition (Lean source)
noncomputable def twoSampleLaw (P : TransportedArray 𝒳) (N : ℕ → ℕ) (n : ℕ) :
Measure (TwoSample 𝒳 n (N n)) :=
(Measure.pi (fun _ : Fin n => sourceObsLaw P n)).prod
(Measure.pi (fun _ : Fin (N n) => targetXLaw P n))
def FullDataSupport
Full-data probability law and bounded potential outcomes.
Definition (Lean source)
def FullDataSupport (P : TransportedArray 𝒳) (n : ℕ) : Prop :=
IsProbabilityMeasure (P.fullLaw n) ∧ -- @realizes P_n^F(probability measure)
∀ᵐ o : FullData 𝒳 ∂P.fullLaw n,
fullY0 o ∈ Icc (0 : ℝ) 1 ∧ -- @realizes Y(d)(Y(0)∈[0,1])
-- @realizes Y^Z(z)(bounded support inherited from Y(0),Y(1))
fullY1 o ∈ Icc (0 : ℝ) 1 -- @realizes Y(d)(Y(1)∈[0,1])
def PopulationPresence
Both populations have strictly positive full-data mass.
Definition (Lean source)
def PopulationPresence (P : TransportedArray 𝒳) (n : ℕ) : Prop :=
∀ s : Bool, 0 < (P.fullLaw n {o | fullS o = s}).toReal
def TwoSampleArray
The two sample factors are probability laws and N_n/n → c.
Definition (Lean source)
def TwoSampleArray (P : TransportedArray 𝒳) (N : ℕ → ℕ) (c : ℝ) : Prop :=
0 < c ∧ -- @realizes c(c∈(0,∞))
(∀ n, IsProbabilityMeasure (sourceObsLaw P n)) ∧
(∀ n, IsProbabilityMeasure (targetXLaw P n)) ∧
Tendsto (fun n : ℕ => (N n : ℝ) / (n : ℝ)) atTop (𝓝 c)
def InstrumentOverlap
Instrument propensity is in [ε,1-ε] source-almost surely.
Definition (Lean source)
def InstrumentOverlap (P : TransportedArray 𝒳) (n : ℕ) (epsilon : ℝ) : Prop :=
0 < epsilon ∧ epsilon < 1 / 2 ∧
∀ᵐ x ∂sourceXLaw P n,
epsilon ≤ P.propensity n x ∧
P.propensity n x ≤ 1 - epsilon
def SourceAssignmentConsistency
The paper's source assignment and consistency assumption. The assigned overlay is a genuine probability law with the source-population full-data marginal; mapping it through observeSource enforces D=D(Z) and Y=Y(D), while the last clause pins the assignment propensity.
Definition (Lean source)
def SourceAssignmentConsistency (P : TransportedArray 𝒳) (n : ℕ) : Prop :=
IsProbabilityMeasure (P.assignedSourceLaw n) ∧
(P.assignedSourceLaw n).map fst = populationLaw P n true ∧
(∀ A, MeasurableSet A →
(sourceObsLaw P n {o | o.1 ∈ A ∧ o.2.1 = true}).toReal =
∫ x in A, P.propensity n x ∂sourceXLaw P n)
Well-definedness and auxiliary-representation facts carried by the complete model. These pin its observed law and contrast fields to the paper-defined objects; SourceAssignmentConsistency is the paper assumption itself.
Definition (Lean source)
def SourceObservation (P : TransportedArray 𝒳) (n : ℕ) : Prop :=
IsProbabilityMeasure (P.assignedSourceLaw n) ∧
IsProbabilityMeasure (sourceObsLaw P n) ∧ -- @realizes P_S(probability measure)
(∀ᵐ o ∂sourceObsLaw P n,
o.2.2.2 ∈ Icc (0 : ℝ) 1) ∧ -- @realizes Y(observed Y∈[0,1])
-- @realizes O_i^S(observed-Y coordinate lies in [0,1])
(P.assignedSourceLaw n).map fst = populationLaw P n true ∧
(∀ A, MeasurableSet A →
(sourceObsLaw P n {o | o.1 ∈ A ∧ o.2.1 = true}).toReal =
∫ x in A, P.propensity n x ∂sourceXLaw P n) ∧
-- @realizes e(P_S(Z=1|X)=e(X))
(∀ A, MeasurableSet A →
∫ o in {o | o.1 ∈ A},
instrumentScore P n o * o.2.2.2 ∂sourceObsLaw P n =
∫ x in A, P.deltaY n x ∂sourceXLaw P n) ∧
(∀ᵐ x ∂sourceXLaw P n,
P.deltaY n x ∈ Icc (-1 : ℝ) 1) ∧
-- @realizes \Delta_Y(observed conditional outcome ITT in [-1,1])
(∀ A, MeasurableSet A →
∫ o in {o | o.1 ∈ A},
instrumentScore P n o * boolReal o.2.2.1 ∂sourceObsLaw P n =
∫ x in A, P.deltaD n x ∂sourceXLaw P n) ∧
(∀ᵐ x ∂sourceXLaw P n,
P.deltaD n x ∈ Icc (0 : ℝ) 1)
def IVRandomization
Conditional randomization: the assigned-full-data law factors with Bernoulli propensity given the source full-data law.
Definition (Lean source)
def IVRandomization (P : TransportedArray 𝒳) (n : ℕ) : Prop :=
∀ (A : Set (FullData 𝒳)), MeasurableSet A → ∀ z : Bool,
(P.assignedSourceLaw n {q | q.1 ∈ A ∧ q.2 = z}).toReal =
∫ o in A, (if z then P.propensity n (fullX o)
else 1 - P.propensity n (fullX o)) ∂populationLaw P n true
def IVExclusion
Exclusion pins the assignment potential outcome to Y(D(z)) in both populations.
Definition (Lean source)
def IVExclusion (P : TransportedArray 𝒳) (n : ℕ) : Prop :=
∀ s z : Bool, P.assignmentOutcome n z =ᵐ[populationLaw P n s]
fun o => derivedAssignmentOutcome o z
def IVMonotonicity
No defiers in either population.
Definition (Lean source)
def IVMonotonicity (P : TransportedArray 𝒳) (n : ℕ) : Prop :=
∀ s : Bool, ∀ᵐ o ∂populationLaw P n s,
boolReal (fullD0 o) ≤ boolReal (fullD1 o)
def OutcomeTransport
Conditional assignment-outcome contrasts are pinned to each population law and agree target-almost everywhere.
Definition (Lean source)
def OutcomeTransport (P : TransportedArray 𝒳) (n : ℕ) : Prop :=
(∀ s : Bool, ∀ A, MeasurableSet A →
∫ o in {o | fullX o ∈ A},
(P.assignmentOutcome n true o - P.assignmentOutcome n false o)
∂populationLaw P n s =
∫ x in A, P.assignmentContrast n s x ∂populationXLaw P n s) ∧
P.assignmentContrast n false =ᵐ[targetXLaw P n]
P.assignmentContrast n true
def ReceiptTransport
Integrated first-stage transport, with population conditional receipt contrasts pinned by integral identities.
Definition (Lean source)
def ReceiptTransport (P : TransportedArray 𝒳) (n : ℕ) : Prop :=
(∀ s : Bool, ∀ A, MeasurableSet A →
∫ o in {o | fullX o ∈ A},
(boolReal (fullD1 o) - boolReal (fullD0 o))
∂populationLaw P n s =
∫ x in A, P.receiptContrast n s x ∂populationXLaw P n s) ∧
(∫ x, P.receiptContrast n false x ∂targetXLaw P n) =
∫ x, P.receiptContrast n true x ∂targetXLaw P n
def TargetComplierPositivity
The target complier share is positive.
Definition (Lean source)
def TransportDomination
Target covariate law is absolutely continuous with respect to source.
Definition (Lean source)
def WeightEnvelope
The transport density ratio is capped by 2 k_n.
Definition (Lean source)
def WeightEnvelope (P : TransportedArray 𝒳) (k : ℕ → ℕ) (n : ℕ) : Prop :=
∀ᵐ x ∂sourceXLaw P n,
0 ≤ transportWeight P n x ∧ transportWeight P n x ≤ 2 * (k n : ℝ)
def WeightSecondMoment
Kish dispersion is at most the envelope.
Definition (Lean source)
def WeightSecondMoment (P : TransportedArray 𝒳) (k : ℕ → ℕ) (n : ℕ) : Prop :=
kishDispersion P n ≤ (k n : ℝ)
def DegradingArray
The overlap envelope diverges sub-root-n while the first stage vanishes.
Definition (Lean source)
def DegradingArray (P : TransportedArray 𝒳) (k : ℕ → ℕ) : Prop :=
Tendsto (fun n => (k n : ℝ)) atTop atTop ∧
Tendsto (fun n => (k n : ℝ) / sqrt n) atTop (𝓝 0) ∧
Tendsto
(fun n => ∫ x, transportWeight P n x * P.deltaD n x
∂sourceXLaw P n)
atTop (𝓝 0)
def finiteUniformCellLaw
The uniform probability law on a row's injected finite covariate carrier. Equality with this measure is the carrier-exhaustion clause: the experiment is definitionally a pushforward of the discrete Fin m experiment, even though all rows share one arbitrary ambient measurable carrier.
Definition (Lean source)
noncomputable def finiteUniformCellLaw {m : ℕ} (cell : Fin m ↪ 𝒳) : Measure 𝒳 :=
∑ i : Fin m, (m : ENNReal)⁻¹ • Measure.dirac (cell i)
def FiniteCellSource
At index n, cell realizes the paper's literal finite covariate experiment inside the common ambient carrier. The source law is exactly the uniform pushforward from Fin (k n); hence the injected image exhausts the row-law carrier and every ambient point outside it is null.
Definition (Lean source)
def FiniteCellSource (P : TransportedArray 𝒳) (k : ℕ → ℕ) (n : ℕ) : Prop :=
0 < k n ∧
IsProbabilityMeasure (sourceXLaw P n) ∧
∃ cell : Fin (k n) ↪ 𝒳,
(∀ i, MeasurableSet {cell i}) ∧
sourceXLaw P n (range cell) = 1 ∧
(∀ i, (sourceXLaw P n {cell i}).toReal = (k n : ℝ)⁻¹) ∧
sourceXLaw P n = finiteUniformCellLaw cell ∧
(∀ᵐ x ∂sourceXLaw P n, P.propensity n x = 1 / 2)
structure TransportedIVClass
Main transported-IV model class at index n.
Definition (Lean source)
parameters
fields
fullDataSupport :
FullDataSupport P n
populationPresence :
twoSampleArray :
TwoSampleArray P N c
instrumentOverlap :
InstrumentOverlap P n epsilon
sourceObservation :
ivRandomization :
IVRandomization P n
ivExclusion :
IVExclusion P n
ivMonotonicity :
IVMonotonicity P n
outcomeTransport :
OutcomeTransport P n
receiptTransport :
ReceiptTransport P n
targetComplierPositivity :
transportDomination :
weightEnvelope :
WeightEnvelope P k n
weightSecondMoment :
WeightSecondMoment P k n
degradingArray :
DegradingArray P k
def SourceObservationFacts
Derived-facing view of the complete model's observed-law well-definedness and auxiliary-field pinning facts.
Definition (Lean source)
def SourceObservationFacts (P : TransportedArray 𝒳) (n : ℕ) : Prop :=
IsProbabilityMeasure (P.assignedSourceLaw n) ∧
IsProbabilityMeasure (sourceObsLaw P n) ∧ -- @realizes P_S(probability measure)
(∀ᵐ o ∂sourceObsLaw P n,
o.2.2.2 ∈ Icc (0 : ℝ) 1) ∧ -- @realizes Y(observed Y∈[0,1])
-- @realizes O_i^S(observed-Y coordinate lies in [0,1])
(P.assignedSourceLaw n).map fst = populationLaw P n true ∧
(∀ A, MeasurableSet A →
(sourceObsLaw P n {o | o.1 ∈ A ∧ o.2.1 = true}).toReal =
∫ x in A, P.propensity n x ∂sourceXLaw P n) ∧
(∀ A, MeasurableSet A →
∫ o in {o | o.1 ∈ A},
instrumentScore P n o * o.2.2.2 ∂sourceObsLaw P n =
∫ x in A, P.deltaY n x ∂sourceXLaw P n) ∧
(∀ᵐ x ∂sourceXLaw P n,
P.deltaY n x ∈ Icc (-1 : ℝ) 1) ∧
-- @realizes \Delta_Y(observed conditional outcome ITT in [-1,1])
(∀ A, MeasurableSet A →
∫ o in {o | o.1 ∈ A},
instrumentScore P n o * boolReal o.2.2.1 ∂sourceObsLaw P n =
∫ x in A, P.deltaD n x ∂sourceXLaw P n) ∧
(∀ᵐ x ∂sourceXLaw P n,
P.deltaD n x ∈ Icc (0 : ℝ) 1)
The probability, boundedness, and identified-contrast consequences of the complete transported-IV model assumptions.
Formal statement
hypotheses
conclusion
Proof (Lean source)
lemma sourceObservationFacts_of_class
(P : TransportedArray 𝒳) (N k : ℕ → ℕ) (c epsilon : ℝ) (n : ℕ)
(hP : TransportedIVClass P N k c epsilon n) :
SourceObservationFacts P n := by
have hObserve :
Measurable (observeSource : AssignedFullData 𝒳 → SourceObs 𝒳) := by
have hx : Measurable fun q : AssignedFullData 𝒳 => q.1.2.1 := by
fun_prop
have hz : Measurable fun q : AssignedFullData 𝒳 => q.2 := by
fun_prop
have hd : Measurable fun q : AssignedFullData 𝒳 =>
if q.2 then q.1.2.2.2.1 else q.1.2.2.1 := by
exact Measurable.ite
(hz (MeasurableSet.singleton true)) (by fun_prop) (by fun_prop)
have hy : Measurable fun q : AssignedFullData 𝒳 =>
if (if q.2 then q.1.2.2.2.1 else q.1.2.2.1)
then q.1.2.2.2.2.2 else q.1.2.2.2.2.1 := by
exact Measurable.ite
(hd (MeasurableSet.singleton true)) (by fun_prop) (by fun_prop)
exact hx.prodMk (hz.prodMk (hd.prodMk hy))
have hFullX : Measurable (fullX : FullData 𝒳 → 𝒳) := by
unfold fullX
fun_prop
have hSourceX :
sourceXLaw P n = populationXLaw P n true := by
rw [sourceXLaw, sourceObsLaw, Measure.map_map measurable_fst hObserve]
rw [populationXLaw, ← hP.sourceObservation.2.1]
rw [Measure.map_map hFullX measurable_fst]
congr 1
have hAssigned : IsProbabilityMeasure (P.assignedSourceLaw n) :=
hP.sourceObservation.1
have hObserved : IsProbabilityMeasure (sourceObsLaw P n) :=
hP.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (P.assignedSourceLaw n) := hAssigned
letI : IsProbabilityMeasure (sourceObsLaw P n) := hObserved
letI : IsProbabilityMeasure (sourceXLaw P n) := by
unfold sourceXLaw
exact Measure.isProbabilityMeasure_map measurable_fst.aemeasurable
have hPopulation : IsProbabilityMeasure (populationLaw P n true) := by
rw [← hP.sourceObservation.2.1]
exact Measure.isProbabilityMeasure_map measurable_fst.aemeasurable
letI : IsProbabilityMeasure (populationLaw P n true) := hPopulation
have hSupportSource : ∀ᵐ o ∂populationLaw P n true,
fullY0 o ∈ Icc (0 : ℝ) 1 ∧
fullY1 o ∈ Icc (0 : ℝ) 1 :=
ProbabilityTheory.cond_absolutelyContinuous.ae_le hP.fullDataSupport.2
have hAssignedSupport : ∀ᵐ q ∂P.assignedSourceLaw n,
fullY0 q.1 ∈ Icc (0 : ℝ) 1 ∧
fullY1 q.1 ∈ Icc (0 : ℝ) 1 := by
have hs : ∀ᵐ o ∂(P.assignedSourceLaw n).map Prod.fst,
fullY0 o ∈ Icc (0 : ℝ) 1 ∧
fullY1 o ∈ Icc (0 : ℝ) 1 := by
rw [hP.sourceObservation.2.1]
exact hSupportSource
exact ae_of_ae_map measurable_fst.aemeasurable hs
have hObservedBound : ∀ᵐ o ∂sourceObsLaw P n,
o.2.2.2 ∈ Icc (0 : ℝ) 1 := by
unfold sourceObsLaw
apply (ae_map_iff hObserve.aemeasurable
((show Measurable fun o : SourceObs 𝒳 => o.2.2.2 by
fun_prop) measurableSet_Icc)).2
filter_upwards [hAssignedSupport] with q hq
rcases q with ⟨o, z⟩
cases z <;> cases hd0 : fullD0 o <;> cases hd1 : fullD1 o <;>
simp [observeSource, potentialOutcome, potentialReceipt, hd0, hd1]
at hq ⊢ <;> tauto
have hOverlapObs : ∀ᵐ o ∂sourceObsLaw P n,
epsilon ≤ P.propensity n o.1 ∧
P.propensity n o.1 ≤ 1 - epsilon := by
have hx := hP.instrumentOverlap.2.2
unfold sourceXLaw at hx
exact ae_of_ae_map measurable_fst.aemeasurable hx
have hScoreMeasurable : Measurable (instrumentScore P n) := by
have hz : Measurable fun o : SourceObs 𝒳 => o.2.1 := by
fun_prop
have he : Measurable fun o : SourceObs 𝒳 => P.propensity n o.1 :=
(P.propensity_measurable n).comp measurable_fst
unfold instrumentScore
exact Measurable.ite (hz (MeasurableSet.singleton true))
(measurable_const.div he)
(measurable_const.neg.div (measurable_const.sub he))
have hScoreBound : ∀ᵐ o ∂sourceObsLaw P n,
|instrumentScore P n o| ≤ 1 / epsilon := by
filter_upwards [hOverlapObs] with o ho
rcases o with ⟨x, z, d, y⟩
cases z
· simp only [instrumentScore, Bool.false_eq_true, ↓reduceIte, abs_neg,
abs_div, abs_one]
rw [abs_of_pos (sub_pos.mpr (lt_of_le_of_lt ho.2
(by linarith [hP.instrumentOverlap.1])))]
exact one_div_le_one_div_of_le hP.instrumentOverlap.1
(by linarith [ho.2])
· simp only [instrumentScore, ↓reduceIte, abs_div, abs_one]
rw [abs_of_pos
(lt_of_lt_of_le hP.instrumentOverlap.1 ho.1)]
exact one_div_le_one_div_of_le hP.instrumentOverlap.1 ho.1
have hScoreIntegrable
(G : SourceObs 𝒳 → ℝ) (hGmeas : Measurable G)
(hGbound : ∀ᵐ o ∂sourceObsLaw P n, |G o| ≤ 1) :
Integrable (fun o => instrumentScore P n o * G o)
(sourceObsLaw P n) := by
refine Integrable.of_bound
((hScoreMeasurable.mul hGmeas).aestronglyMeasurable)
(1 / epsilon) ?_
filter_upwards [hScoreBound, hGbound] with o hs hG
rw [Real.norm_eq_abs, abs_mul]
calc
|instrumentScore P n o| * |G o| ≤ (1 / epsilon) * 1 :=
mul_le_mul hs hG (abs_nonneg _)
(one_div_nonneg.mpr hP.instrumentOverlap.1.le)
_ = 1 / epsilon := mul_one _
have hOutcomeScoreIntegrable :
Integrable (fun o => instrumentScore P n o * o.2.2.2)
(sourceObsLaw P n) := by
apply hScoreIntegrable (fun o => o.2.2.2) (by fun_prop)
filter_upwards [hObservedBound] with o ho
rw [abs_of_nonneg ho.1]
exact ho.2
have hReceiptMeasurable :
Measurable (fun o : SourceObs 𝒳 => boolReal o.2.2.1) := by
unfold boolReal
have hd : Measurable fun o : SourceObs 𝒳 => o.2.2.1 := by
fun_prop
exact Measurable.ite (hd (MeasurableSet.singleton true))
measurable_const measurable_const
have hReceiptScoreIntegrable :
Integrable (fun o => instrumentScore P n o * boolReal o.2.2.1)
(sourceObsLaw P n) := by
apply hScoreIntegrable _ hReceiptMeasurable
filter_upwards with o
cases o.2.2.1 <;> simp [boolReal]
have hSlice (z : Bool) :
((P.assignedSourceLaw n).restrict {q | q.2 = z}).map Prod.fst =
(populationLaw P n true).withDensity (fun o =>
ofReal
(if z then P.propensity n (fullX o)
else 1 - P.propensity n (fullX o))) := by
let μ := populationLaw P n true
let ν := ((P.assignedSourceLaw n).restrict {q | q.2 = z}).map Prod.fst
let w : FullData 𝒳 → ℝ := fun o =>
if z then P.propensity n (fullX o)
else 1 - P.propensity n (fullX o)
have hwmeas : Measurable w := by
have hecomp : Measurable fun o : FullData 𝒳 =>
P.propensity n (fullX o) :=
(P.propensity_measurable n).comp hFullX
cases z
· simpa [w] using measurable_const.fun_sub hecomp
· simpa [w] using hecomp
have hoverlapPop : ∀ᵐ o ∂μ,
epsilon ≤ P.propensity n (fullX o) ∧
P.propensity n (fullX o) ≤ 1 - epsilon := by
have hx := hP.instrumentOverlap.2.2
rw [hSourceX] at hx
exact ae_of_ae_map hFullX.aemeasurable hx
have hw0 : ∀ᵐ o ∂μ, 0 ≤ w o := by
filter_upwards [hoverlapPop] with o ho
unfold w
cases z
· simp
linarith [hP.instrumentOverlap.1]
· simpa using
(le_trans hP.instrumentOverlap.1.le ho.1)
have hwle : ∀ᵐ o ∂μ, w o ≤ 1 := by
filter_upwards [hoverlapPop] with o ho
unfold w
cases z
· simp
linarith [ho.1, hP.instrumentOverlap.1]
· simpa using
ho.2.trans (sub_le_self 1 hP.instrumentOverlap.1.le)
have hwint : Integrable w μ := by
refine Integrable.of_bound hwmeas.aestronglyMeasurable 1 ?_
filter_upwards [hw0, hwle] with o h0 h1
rw [Real.norm_eq_abs, abs_of_nonneg h0]
exact h1
ext A hA
rw [withDensity_apply _ hA]
rw [← ofReal_integral_eq_lintegral_ofReal hwint.integrableOn
((ae_restrict_iff' hA).2
(Filter.Eventually.mono hw0 fun _ hx _ => hx))]
rw [← ENNReal.ofReal_toReal (measure_ne_top ν A)]
congr 1
rw [Measure.map_apply measurable_fst hA]
rw [Measure.restrict_apply (hA.preimage measurable_fst)]
rw [show Prod.fst ⁻¹' A ∩ {q | q.2 = z} =
{q | q.1 ∈ A ∧ q.2 = z} by
ext q
simp]
simpa [μ, ν, w] using hP.ivRandomization A hA z
have hIPW
(F0 F1 : FullData 𝒳 → ℝ)
(hF0 : Measurable F0) (hF1 : Measurable F1)
(hF0int : Integrable F0 (populationLaw P n true))
(hF1int : Integrable F1 (populationLaw P n true))
(hInt : Integrable (fun q : AssignedFullData 𝒳 =>
if q.2 then
(1 / P.propensity n (fullX q.1)) * F1 q.1
else
(-1 / (1 - P.propensity n (fullX q.1))) * F0 q.1)
(P.assignedSourceLaw n))
(A : Set 𝒳) (hA : MeasurableSet A) :
(∫ q in {q | fullX q.1 ∈ A},
(if q.2 then
(1 / P.propensity n (fullX q.1)) * F1 q.1
else
(-1 / (1 - P.propensity n (fullX q.1))) * F0 q.1)
∂P.assignedSourceLaw n) =
∫ o in {o | fullX o ∈ A}, (F1 o - F0 o)
∂populationLaw P n true := by
let μ := populationLaw P n true
let e : FullData 𝒳 → ℝ := fun o => P.propensity n (fullX o)
let S : Set (AssignedFullData 𝒳) := {q | fullX q.1 ∈ A}
let E : Set (AssignedFullData 𝒳) := {q | q.2 = true}
let W : AssignedFullData 𝒳 → ℝ := fun q =>
if q.2 then (1 / e q.1) * F1 q.1
else (-1 / (1 - e q.1)) * F0 q.1
have hS : MeasurableSet S :=
hA.preimage (hFullX.comp measurable_fst)
have hE : MeasurableSet E :=
measurable_snd (MeasurableSet.singleton true)
have hsplit := integral_add_compl
(μ := (P.assignedSourceLaw n).restrict S) hE hInt.integrableOn
have hemeas : Measurable e :=
(P.propensity_measurable n).comp hFullX
have hoverlapPop : ∀ᵐ o ∂μ,
epsilon ≤ e o ∧ e o ≤ 1 - epsilon := by
have hx := hP.instrumentOverlap.2.2
rw [hSourceX] at hx
exact ae_of_ae_map hFullX.aemeasurable hx
have htrue :
(∫ q in S ∩ E, W q ∂P.assignedSourceLaw n) =
∫ o in {o | fullX o ∈ A}, F1 o ∂μ := by
have hmap :
(∫ o in {o | fullX o ∈ A}, (1 / e o) * F1 o
∂(((P.assignedSourceLaw n).restrict E).map Prod.fst)) =
∫ q in S ∩ E, W q ∂P.assignedSourceLaw n := by
calc
_ = ∫ q in Prod.fst ⁻¹' {o | fullX o ∈ A},
(1 / e q.1) * F1 q.1
∂(P.assignedSourceLaw n).restrict E :=
setIntegral_map (μ := (P.assignedSourceLaw n).restrict E)
(hA.preimage hFullX)
((measurable_const.div hemeas).mul hF1).aestronglyMeasurable
measurable_fst.aemeasurable
_ = _ := by
change (∫ q, (1 / e q.1) * F1 q.1
∂((P.assignedSourceLaw n).restrict E).restrict
-- … truncated; follow the source link for the rest …
theorem sourceAssignmentConsistency_of_class
Complete model membership entails the paper's source assignment atom.
Formal statement
hypotheses
conclusion
Proof (Lean source)
lemma sourceAssignmentConsistency_of_class
(P : TransportedArray 𝒳) (N k : ℕ → ℕ) (c epsilon : ℝ) (n : ℕ)
(hP : TransportedIVClass P N k c epsilon n) :
SourceAssignmentConsistency P n := by
exact hP.sourceObservation
theorem weighted_observed_contrast_eq
A covariate-indexed contrast representation remains valid after multiplication by an integrable measurable covariate weight.
Formal statement
hypotheses
P :
n :
ℕ
F :
SourceObs 𝒳 → ℝ
d w :
𝒳 → ℝ
hprob :
hF :
Integrable F (sourceObsLaw P n)
hd :
Integrable d (sourceXLaw P n)
hdmeas :
hw :
hWF :
Integrable (fun o => w o.1 * F o) (sourceObsLaw P n)
hwd :
Integrable (fun x => w x * d x) (sourceXLaw P n)
hset :
∀ A
if
then
∫ o in {o | o.1 ∈ A}, F o ∂sourceObsLaw P n = ∫ x in A, d x ∂sourceXLaw P n
conclusion
(∫ o, w o.1 * F o ∂sourceObsLaw P n) = ∫ x, w x * d x ∂sourceXLaw P n
Proof (Lean source)
lemma weighted_observed_contrast_eq
(P : TransportedArray 𝒳) (n : ℕ)
(F : SourceObs 𝒳 → ℝ) (d w : 𝒳 → ℝ)
(hprob : IsProbabilityMeasure (sourceObsLaw P n))
(hF : Integrable F (sourceObsLaw P n))
(hd : Integrable d (sourceXLaw P n))
(hdmeas : Measurable d)
(hw : Measurable w)
(hWF : Integrable (fun o => w o.1 * F o) (sourceObsLaw P n))
(hwd : Integrable (fun x => w x * d x) (sourceXLaw P n))
(hset : ∀ A, MeasurableSet A →
∫ o in {o | o.1 ∈ A}, F o ∂sourceObsLaw P n =
∫ x in A, d x ∂sourceXLaw P n) :
(∫ o, w o.1 * F o ∂sourceObsLaw P n) =
∫ x, w x * d x ∂sourceXLaw P n := by
let m0 : MeasurableSpace (SourceObs 𝒳) := inferInstance
let mX : MeasurableSpace (SourceObs 𝒳) :=
comap (fun o => o.1)
(inferInstance : MeasurableSpace 𝒳)
letI : MeasurableSpace (SourceObs 𝒳) := m0
letI : IsProbabilityMeasure (sourceObsLaw P n) := hprob
have hfst :
@Measurable (SourceObs 𝒳) 𝒳 m0 inferInstance (fun o => o.1) :=
measurable_fst
have hm : mX ≤ m0 := hfst.comap_le
have hfstM :
@Measurable (SourceObs 𝒳) 𝒳 mX inferInstance (fun o => o.1) := by
intro A hA
exact MeasurableSpace.measurableSet_comap.mpr ⟨A, hA, rfl⟩
have hdcomp : Integrable (fun o => d o.1) (sourceObsLaw P n) := by
have hdmap : Integrable d
(Measure.map (fun o : SourceObs 𝒳 => o.1) (sourceObsLaw P n)) := by
simpa [sourceXLaw] using hd
simpa [Function.comp_def] using
(integrable_map_measure hdmap.1 hfst.aemeasurable).mp hdmap
have hcond : (fun o => d o.1) =ᵐ[sourceObsLaw P n]
(sourceObsLaw P n)[F | mX] := by
refine ae_eq_condExp_of_forall_setIntegral_eq
(μ := sourceObsLaw P n) (f := F) (g := fun o => d o.1)
hm hF ?_ ?_ ?_
· intro s hs _hfin
exact hdcomp.integrableOn
· intro s hs _hfin
rcases MeasurableSpace.measurableSet_comap.mp hs with
⟨A, hA, rfl⟩
calc
(∫ x in (fun o : SourceObs 𝒳 => o.1) ⁻¹' A,
d x.1 ∂sourceObsLaw P n) =
∫ x in A, d x ∂sourceXLaw P n := by
rw [sourceXLaw]
exact (setIntegral_map hA hd.1 hfst.aemeasurable).symm
_ = ∫ x in (fun o : SourceObs 𝒳 => o.1) ⁻¹' A,
F x ∂sourceObsLaw P n := (hset A hA).symm
· exact (hdmeas.comp hfstM).aestronglyMeasurable
have hwM : @StronglyMeasurable (SourceObs 𝒳) ℝ _ mX
(fun o => w o.1) :=
(hw.comp hfstM).stronglyMeasurable
have hpull := condExp_mul_of_stronglyMeasurable_left
(m := mX) (μ := sourceObsLaw P n) hwM hWF hF
have hpull' :
(sourceObsLaw P n)[(fun o => w o.1 * F o) | mX] =ᵐ[
sourceObsLaw P n]
fun o => w o.1 * (sourceObsLaw P n)[F | mX] o := by
exact hpull
calc
(∫ o, w o.1 * F o ∂sourceObsLaw P n) =
∫ o, (sourceObsLaw P n)[(fun o => w o.1 * F o) | mX] o
∂sourceObsLaw P n := (integral_condExp hm).symm
_ = ∫ o, w o.1 * d o.1 ∂sourceObsLaw P n := by
apply integral_congr_ae
filter_upwards [hpull', hcond] with o hp hc
rw [hp, ← hc]
_ = ∫ x, w x * d x ∂sourceXLaw P n := by
rw [sourceXLaw]
exact (integral_map measurable_fst.aemeasurable hwd.1).symm
theorem transportedFirstStage_eq_weighted_deltaD
The named transported first stage unfolds to the paper's weighted conditional receipt-contrast moment.
Formal statement
hypotheses
P :
k :
ℕ → ℕ
epsilon :
ℝ
n :
ℕ
hFacts :
hOverlap :
InstrumentOverlap P n epsilon
hEnvelope :
WeightEnvelope P k n
conclusion
transportedFirstStage P n = ∫ x, transportWeight P n x * P.deltaD n x ∂sourceXLaw P n
Proof (Lean source)
lemma transportedFirstStage_eq_weighted_deltaD
(P : TransportedArray 𝒳) (k : ℕ → ℕ) (epsilon : ℝ) (n : ℕ)
(hFacts : SourceObservationFacts P n)
(hOverlap : InstrumentOverlap P n epsilon)
(hEnvelope : WeightEnvelope P k n) :
transportedFirstStage P n =
∫ x, transportWeight P n x * P.deltaD n x ∂sourceXLaw P n := by
rfl
The paper-declared ranges of the four scalar functionals. This predicate does not add a model assumption: the companion lemma below derives it from membership in the transported-IV class.
Definition (Lean source)
def TransportedFunctionalRanges (P : TransportedArray 𝒳) (n : ℕ) : Prop :=
transportedOutcomeITT P n ∈ Icc (-1 : ℝ) 1 ∧
-- @realizes \mu_{Y,n}(transported outcome ITT lies in [-1,1])
transportedFirstStage P n ∈ Ioc (0 : ℝ) 1 ∧
-- @realizes \mu_n(transported first stage lies in (0,1])
1 ≤ kishDispersion P n ∧
-- @realizes \kappa_n(Kish dispersion lies in [1,∞))
(0 < n → 0 < effectiveStrength P n)
theorem transportedFunctionalRanges_of_class
Every transported-IV class member satisfies the scalar ranges declared in the paper. In particular these bounds are consequences of normalization, bounded contrasts, complier positivity, and the defining identities rather than additional hypotheses.
Formal statement
hypotheses
conclusion
Proof (Lean source)
-- @realizes \mu_{Y,n}(range derived for every transported-IV class member)
-- @realizes \mu_n(range derived for every transported-IV class member)
-- @realizes \kappa_n(lower bound derived for every transported-IV class member)
-- @realizes t_n(positivity derived on the paper domain n>0)
lemma transportedFunctionalRanges_of_class
(P : TransportedArray 𝒳) (N k : ℕ → ℕ) (c epsilon : ℝ) (n : ℕ)
(hP : TransportedIVClass P N k c epsilon n) :
TransportedFunctionalRanges P n := by
have hSourceFacts :=
sourceObservationFacts_of_class P N k c epsilon n hP
have hTwo := hP.twoSampleArray
have hOverlap := hP.instrumentOverlap
have hRandom := hP.ivRandomization
have hMono := hP.ivMonotonicity
have hOutcome := hP.outcomeTransport
have hReceipt := hP.receiptTransport
have hPositive := hP.targetComplierPositivity
have hDom := hP.transportDomination
have hEnvelope := hP.weightEnvelope
have hSecond := hP.weightSecondMoment
letI : IsProbabilityMeasure (sourceObsLaw P n) := hTwo.2.1 n
letI : IsProbabilityMeasure (sourceXLaw P n) := by
unfold sourceXLaw
exact Measure.isProbabilityMeasure_map measurable_fst.aemeasurable
letI : IsProbabilityMeasure (targetXLaw P n) := hTwo.2.2.1 n
have hWeightMeas : Measurable (transportWeight P n) :=
(Measure.measurable_rnDeriv
(targetXLaw P n) (sourceXLaw P n)).ennreal_toReal
have hWeightMem : MemLp (transportWeight P n) 2 (sourceXLaw P n) := by
refine MemLp.of_bound hWeightMeas.aestronglyMeasurable
(2 * (k n : ℝ)) ?_
filter_upwards [hEnvelope] with x hx
rw [Real.norm_eq_abs, abs_of_nonneg hx.1]
exact hx.2
have hWeightMean :
(∫ x, transportWeight P n x ∂sourceXLaw P n) = 1 := by
have hchange := integral_rnDeriv_smul
(μ := targetXLaw P n) (ν := sourceXLaw P n)
(f := fun _ => (1 : ℝ)) hDom
simpa [transportWeight] using hchange
have hKish : 1 ≤ kishDispersion P n := by
have hv := variance_nonneg (transportWeight P n) (sourceXLaw P n)
rw [variance_eq_sub hWeightMem, hWeightMean] at hv
norm_num at hv ⊢
simpa [kishDispersion] using hv
have hDeltaYSource :
∀ᵐ x ∂sourceXLaw P n, P.deltaY n x ∈ Icc (-1 : ℝ) 1 :=
hSourceFacts.2.2.2.2.2.2.1
have hDeltaDSource :
∀ᵐ x ∂sourceXLaw P n, P.deltaD n x ∈ Icc (0 : ℝ) 1 :=
hSourceFacts.2.2.2.2.2.2.2.2
have hDeltaYTarget :
∀ᵐ x ∂targetXLaw P n, P.deltaY n x ∈ Icc (-1 : ℝ) 1 :=
hDom.ae_le hDeltaYSource
have hDeltaDTarget :
∀ᵐ x ∂targetXLaw P n, P.deltaD n x ∈ Icc (0 : ℝ) 1 :=
hDom.ae_le hDeltaDSource
have hYInt : Integrable (P.deltaY n) (targetXLaw P n) := by
refine Integrable.of_bound
(P.deltaY_measurable n).aestronglyMeasurable 1 ?_
filter_upwards [hDeltaYTarget] with x hx
rw [Real.norm_eq_abs]
exact (abs_le).2 hx
have hDInt : Integrable (P.deltaD n) (targetXLaw P n) := by
refine Integrable.of_bound
(P.deltaD_measurable n).aestronglyMeasurable 1 ?_
filter_upwards [hDeltaDTarget] with x hx
rw [Real.norm_eq_abs, abs_of_nonneg hx.1]
exact hx.2
have hOutcomeChange :
transportedOutcomeITT P n =
∫ x, P.deltaY n x ∂targetXLaw P n := by
unfold transportedOutcomeITT transportWeight
simpa only [smul_eq_mul] using
(integral_rnDeriv_smul
(μ := targetXLaw P n) (ν := sourceXLaw P n)
(f := P.deltaY n) hDom)
have hOutcomeRange :
transportedOutcomeITT P n ∈ Icc (-1 : ℝ) 1 := by
rw [hOutcomeChange]
constructor
· have hconst : Integrable (fun _ : 𝒳 => (-1 : ℝ)) (targetXLaw P n) :=
integrable_const _
simpa using integral_mono_ae hconst hYInt (hDeltaYTarget.mono fun x hx => hx.1)
· have hconst : Integrable (fun _ : 𝒳 => (1 : ℝ)) (targetXLaw P n) :=
integrable_const _
simpa using integral_mono_ae hYInt hconst (hDeltaYTarget.mono fun x hx => hx.2)
clear hOutcomeRange hOutcomeChange hYInt
hDeltaYTarget hDeltaYSource hKish hWeightMean hWeightMem hWeightMeas hDInt
have hMeasObserve :
Measurable (observeSource : AssignedFullData 𝒳 → SourceObs 𝒳) := by
have hx : Measurable fun q : AssignedFullData 𝒳 => q.1.2.1 := by fun_prop
have hz : Measurable fun q : AssignedFullData 𝒳 => q.2 := by fun_prop
have hd : Measurable fun q : AssignedFullData 𝒳 =>
if q.2 then q.1.2.2.2.1 else q.1.2.2.1 := by
exact Measurable.ite
(hz (MeasurableSet.singleton true)) (by fun_prop) (by fun_prop)
have hy : Measurable fun q : AssignedFullData 𝒳 =>
if (if q.2 then q.1.2.2.2.1 else q.1.2.2.1)
then q.1.2.2.2.2.2 else q.1.2.2.2.2.1 := by
exact Measurable.ite
(hd (MeasurableSet.singleton true)) (by fun_prop) (by fun_prop)
exact hx.prodMk (hz.prodMk (hd.prodMk hy))
have hSourceX :
sourceXLaw P n = populationXLaw P n true := by
have hFullX : Measurable (fullX : FullData 𝒳 → 𝒳) := by
unfold fullX
fun_prop
rw [sourceXLaw, sourceObsLaw, Measure.map_map measurable_fst hMeasObserve]
rw [populationXLaw, ← hSourceFacts.2.2.2.1]
rw [Measure.map_map hFullX measurable_fst]
congr 1
have hSlice : ∀ z : Bool,
((P.assignedSourceLaw n).restrict {q | q.2 = z}).map Prod.fst =
(populationLaw P n true).withDensity (fun o =>
ofReal
(if z then P.propensity n (fullX o)
else 1 - P.propensity n (fullX o))) := by
intro z
let μ := populationLaw P n true
let ν := ((P.assignedSourceLaw n).restrict {q | q.2 = z}).map Prod.fst
let w : FullData 𝒳 → ℝ := fun o =>
if z then P.propensity n (fullX o)
else 1 - P.propensity n (fullX o)
have hFullX : Measurable (fullX : FullData 𝒳 → 𝒳) := by
unfold fullX
fun_prop
have hwmeas : Measurable w := by
have hecomp : Measurable fun o : FullData 𝒳 =>
P.propensity n (fullX o) :=
(P.propensity_measurable n).comp hFullX
cases z
· simpa [w] using measurable_const.fun_sub hecomp
· simpa [w] using hecomp
have hoverlapPop : ∀ᵐ o ∂μ,
epsilon ≤ P.propensity n (fullX o) ∧
P.propensity n (fullX o) ≤ 1 - epsilon := by
have hx := hOverlap.2.2
rw [hSourceX] at hx
exact ae_of_ae_map hFullX.aemeasurable hx
have hw0 : ∀ᵐ o ∂μ, 0 ≤ w o := by
filter_upwards [hoverlapPop] with o ho
unfold w
cases z
· simp
linarith [hOverlap.1]
· simpa using (le_trans hOverlap.1.le ho.1)
have hwle : ∀ᵐ o ∂μ, w o ≤ 1 := by
filter_upwards [hoverlapPop] with o ho
unfold w
cases z
· simp
linarith [ho.1, hOverlap.1]
· simpa using ho.2.trans (sub_le_self 1 hOverlap.1.le)
letI : IsProbabilityMeasure (P.assignedSourceLaw n) := hSourceFacts.1
letI : IsProbabilityMeasure μ := by
unfold μ
rw [← hSourceFacts.2.2.2.1]
exact Measure.isProbabilityMeasure_map measurable_fst.aemeasurable
have hwint : Integrable w μ := by
refine Integrable.of_bound hwmeas.aestronglyMeasurable 1 ?_
filter_upwards [hw0, hwle] with o h0 h1
rw [Real.norm_eq_abs, abs_of_nonneg h0]
exact h1
ext A hA
rw [Measure.map_apply measurable_fst hA]
rw [Measure.restrict_apply (hA.preimage measurable_fst)]
rw [show Prod.fst ⁻¹' A ∩ {q | q.2 = z} =
{q | q.1 ∈ A ∧ q.2 = z} by
ext q
simp]
rw [withDensity_apply _ hA]
rw [← ofReal_integral_eq_lintegral_ofReal hwint.integrableOn
((ae_restrict_iff' hA).2
(Filter.Eventually.mono hw0 fun _ hx _ => hx))]
rw [← hRandom A hA z, ENNReal.ofReal_toReal
(measure_ne_top (P.assignedSourceLaw n) _)]
have hFullXMeas : Measurable (fullX : FullData 𝒳 → 𝒳) := by
unfold fullX
fun_prop
have hD0Meas : Measurable fun o : FullData 𝒳 => boolReal (fullD0 o) := by
unfold boolReal
have hd : Measurable (fullD0 : FullData 𝒳 → Bool) := by
unfold fullD0
fun_prop
exact Measurable.ite
(hd (MeasurableSet.singleton true))
measurable_const measurable_const
have hD1Meas : Measurable fun o : FullData 𝒳 => boolReal (fullD1 o) := by
unfold boolReal
have hd : Measurable (fullD1 : FullData 𝒳 → Bool) := by
unfold fullD1
fun_prop
exact Measurable.ite
(hd (MeasurableSet.singleton true))
measurable_const measurable_const
have hOverlapObs : ∀ᵐ o ∂sourceObsLaw P n,
epsilon ≤ P.propensity n o.1 ∧
P.propensity n o.1 ≤ 1 - epsilon := by
have hx := hOverlap.2.2
unfold sourceXLaw at hx
exact ae_of_ae_map measurable_fst.aemeasurable hx
have hScoreMeas : Measurable (instrumentScore P n) := by
unfold instrumentScore
exact Measurable.ite
((by fun_prop : Measurable fun o : SourceObs 𝒳 => o.2.1)
(MeasurableSet.singleton true))
(measurable_const.div ((P.propensity_measurable n).comp measurable_fst))
(measurable_const.neg.div
(measurable_const.sub ((P.propensity_measurable n).comp measurable_fst)))
have hScoreBound : ∀ᵐ o ∂sourceObsLaw P n,
|instrumentScore P n o| ≤ 1 / epsilon := by
filter_upwards [hOverlapObs] with o ho
rcases o with ⟨x, z, d, y⟩
cases z
· simp only [instrumentScore, Bool.false_eq_true, ↓reduceIte, abs_neg,
abs_div, abs_one]
rw [abs_of_pos (sub_pos.mpr (lt_of_le_of_lt ho.2
(by linarith [hOverlap.1])))]
exact one_div_le_one_div_of_le hOverlap.1 (by linarith [ho.2])
· simp only [instrumentScore, ↓reduceIte, abs_div, abs_one]
rw [abs_of_pos (lt_of_lt_of_le hOverlap.1 ho.1)]
exact one_div_le_one_div_of_le hOverlap.1 ho.1
have hReceiptScoreInt :
Integrable (fun o =>
instrumentScore P n o * boolReal o.2.2.1) (sourceObsLaw P n) := by
refine Integrable.of_bound
((hScoreMeas.mul
(Measurable.ite
((by fun_prop : Measurable fun o : SourceObs 𝒳 => o.2.2.1)
(MeasurableSet.singleton true))
measurable_const measurable_const)).aestronglyMeasurable)
(1 / epsilon) ?_
filter_upwards [hScoreBound] with o hs
rw [Real.norm_eq_abs, abs_mul]
cases o.2.2.1
· simp only [boolReal, Bool.false_eq_true, ↓reduceIte, abs_zero, mul_zero]
exact (one_div_pos.mpr hOverlap.1).le
· simpa [boolReal] using hs
have hD0Int :
Integrable (fun o : FullData 𝒳 => boolReal (fullD0 o))
(populationLaw P n true) := by
letI : IsProbabilityMeasure (P.assignedSourceLaw n) := hSourceFacts.1
letI : IsProbabilityMeasure (populationLaw P n true) := by
rw [← hSourceFacts.2.2.2.1]
exact Measure.isProbabilityMeasure_map measurable_fst.aemeasurable
-- … truncated; follow the source link for the rest …
structure FiniteCellClass
Uniform finite-cell subclass, flattened to the sixteen core members. Its finiteCellSource field carries the exact finite pushforward-law exhaustion condition, rather than merely requiring full mass on some subset.
Definition (Lean source)
parameters
fields
fullDataSupport :
FullDataSupport P n
populationPresence :
twoSampleArray :
TwoSampleArray P N c
instrumentOverlap :
InstrumentOverlap P n epsilon
sourceObservation :
ivRandomization :
IVRandomization P n
ivExclusion :
IVExclusion P n
ivMonotonicity :
IVMonotonicity P n
outcomeTransport :
OutcomeTransport P n
receiptTransport :
ReceiptTransport P n
targetComplierPositivity :
transportDomination :
weightEnvelope :
WeightEnvelope P k n
weightSecondMoment :
WeightSecondMoment P k n
degradingArray :
DegradingArray P k
finiteCellSource :
FiniteCellSource P k n
def toTransportedIVClass
Forget the finite-cell restriction.
Definition (Lean source)
def FiniteCellClass.toTransportedIVClass
(h : FiniteCellClass P N k c epsilon n) :
TransportedIVClass P N k c epsilon n :=
{ h with }
Deterministic source/target covariate, weight, and propensity arrays.
Definition (Lean source)
parameters
𝒳 :
Type*
fields
sourceX :
ℕ → Measure 𝒳
@realizes P_S^X(geometry source-law array)
targetX :
ℕ → Measure 𝒳
@realizes P_T(geometry target-law array)
weight :
ℕ → 𝒳 → ℝ
@realizes w(geometry density-ratio array)
propensity :
ℕ → 𝒳 → ℝ
@realizes e(geometry propensity array)
weight_measurable :
∀ n, Measurable (weight n)
propensity_measurable :
∀ n, Measurable (propensity n)
def AdmissibleGeometry
Admissible deterministic geometry class.
Definition (Lean source)
def AdmissibleGeometry (g : Geometry 𝒳) (k : ℕ → ℕ) (epsilon : ℝ) : Prop :=
(∀ n, IsProbabilityMeasure (g.sourceX n)) ∧
(∀ n, IsProbabilityMeasure (g.targetX n)) ∧
0 < epsilon ∧ epsilon < 1 / 2 ∧
(∀ n x,
epsilon ≤ g.propensity n x ∧ g.propensity n x ≤ 1 - epsilon) ∧
-- @realizes e(pointwise geometry propensity in [epsilon,1-epsilon])
(∀ n x, 0 ≤ g.weight n x ∧ g.weight n x ≤ 2 * (k n : ℝ)) ∧
-- @realizes w(pointwise geometry weight in [0,2k_n])
(∀ n, ∫ x, g.weight n x ∂g.sourceX n = 1) ∧
(∀ n, ∫ x, (g.weight n x) ^ 2 ∂g.sourceX n ≤ (k n : ℝ)) ∧
(∀ n A, MeasurableSet A →
g.targetX n A =
ofReal (∫ x in A, g.weight n x ∂g.sourceX n))
def fixedGeometrySlice
Laws whose source law, target law, and propensity equal geometry g at index n, with the law-induced Radon--Nikodym weight equal to the geometry weight source-almost everywhere. The a.e. clause respects the fact that density-ratio versions are determined only up to source-null sets.
Definition (Lean source)
def fixedGeometrySlice (P : TransportedArray 𝒳) (g : Geometry 𝒳)
(N k : ℕ → ℕ) (c epsilon : ℝ) (n : ℕ) : Prop :=
TransportedIVClass P N k c epsilon n ∧
sourceXLaw P n = g.sourceX n ∧
targetXLaw P n = g.targetX n ∧
transportWeight P n =ᵐ[sourceXLaw P n] g.weight n ∧
P.propensity n = g.propensity n
def RegularFiniteCellClass
Regular nonuniform finite-cell subclass on a finite injected support whose image has full source mass.
Definition (Lean source)
def RegularFiniteCellClass (P : TransportedArray 𝒳) (N k : ℕ → ℕ)
(c epsilon cminus cplus : ℝ) (n : ℕ) : Prop :=
TransportedIVClass P N k c epsilon n ∧
0 < k n ∧
0 < cminus ∧ cminus ≤ 1 ∧ 1 ≤ cplus ∧
∃ cell : Fin (k n) ↪ 𝒳,
(∀ i : Fin (k n), MeasurableSet {cell i}) ∧
sourceXLaw P n (range cell) = 1 ∧
∀ i : Fin (k n),
cminus / (k n : ℝ) ≤
(sourceXLaw P n {cell i}).toReal ∧
(sourceXLaw P n {cell i}).toReal ≤ cplus / (k n : ℝ)
Frontier 59 declarations Procedure objects
Procedure objects
def NonnegativeWeight
Oracle weight inputs use a carrier whose elements are nonnegative pointwise, matching the density-ratio space.
Definition (Lean source)
abbrev NonnegativeWeight (𝒳 : Type*) := {w : 𝒳 → ℝ // ∀ x, 0 ≤ w x}
def transportWeightInput
The law-induced Radon--Nikodym weight as an admissible oracle input.
Definition (Lean source)
noncomputable def transportWeightInput (P : TransportedArray 𝒳) (n : ℕ) :
NonnegativeWeight 𝒳 :=
⟨transportWeight P n, fun _ => ENNReal.toReal_nonneg⟩
def TransportWeightVersion
A measurable nonnegative version of a model row's transport density ratio. The paper determines w only source-law almost everywhere, so global oracle honesty and risk range over this entire nonempty fiber rather than silently selecting one Mathlib representative.
Definition (Lean source)
def TransportWeightVersion (P : TransportedArray 𝒳) (n : ℕ) :=
{w : NonnegativeWeight 𝒳 //
Measurable w.1 ∧ w.1 =ᵐ[sourceXLaw P n] transportWeight P n}
def canonicalTransportWeightVersion
The canonical Radon--Nikodym representative witnesses that every version fiber is nonempty.
Definition (Lean source)
noncomputable def canonicalTransportWeightVersion
(P : TransportedArray 𝒳) (n : ℕ) : TransportWeightVersion P n :=
⟨transportWeightInput P n,
(Measure.measurable_rnDeriv (targetXLaw P n) (sourceXLaw P n)).ennreal_toReal,
Filter.Eventually.of_forall fun _ => rfl⟩
def geometryWeightInput
A fixed geometry's declared density-ratio version as an oracle input. Admissible geometries take the first branch, so procedures receive exactly g.weight; the zero fallback only totalizes this definition away from the admissible geometry class.
Definition (Lean source)
noncomputable def geometryWeightInput (g : Geometry 𝒳) (n : ℕ) :
NonnegativeWeight 𝒳 := by
classical
exact if h : ∀ x, 0 ≤ g.weight n x then
⟨g.weight n, h⟩
else
⟨fun _ => 0, fun _ => le_rfl⟩
def OracleInput
Inputs available to an oracle procedure.
Definition (Lean source)
abbrev OracleInput (𝒳 : Type*) (N : ℕ → ℕ) (n : ℕ) :=
TwoSample 𝒳 n (N n) × NonnegativeWeight 𝒳 × (𝒳 → ℝ)
def FiniteCellInput
Inputs available to a feasible finite-cell procedure. FiniteCellSource pins the row-n measure to the image of Fin (k n) inside the arbitrary ambient carrier.
Definition (Lean source)
abbrev FiniteCellInput (𝒳 : Type*) (N _k : ℕ → ℕ) (n : ℕ) :=
TwoSample 𝒳 n (N n)
The realized regular-cell support inside the fixed ambient carrier. The embedding is part of the experiment, so a procedure is never compared across an unrelated alternative support.
Definition (Lean source)
Inputs available to a regular-cell procedure: samples, the true realized support, and the known Fin (k n)-indexed cell-mass and propensity vectors, but no transport weight and no arbitrary ambient extensions of those vectors.
Definition (Lean source)
parameters
fields
sample :
TwoSample 𝒳 n (N n)
design :
RegularCellDesign 𝒳 (k n)
q :
Fin (k n) → ℝ
@realizes q_{x,n}(known source-cell array input)
e :
Fin (k n) → ℝ
Oracle confidence-set sequence with range and sample-by-parameter graph measurability for every fixed ADMISSIBLE deterministic oracle input (w,e), i.e. every measurable pair. The source supplies the oracle the model's own density ratio and propensity, both measurable, so the measurability obligation is imposed exactly on the inputs the source quantifies over; a nonmeasurable w makes the sample-level score itself nonmeasurable, so demanding a measurable graph there would be strictly stronger than the source and in general unsatisfiable.
Definition (Lean source)
parameters
fields
set :
∀ n
if
OracleInput 𝒳 N n
then
Set ℝ
@realizes C_n(random subset of Theta)
subset :
∀ n x, set n x ⊆ parameterSpace
@realizes C_n(C_n⊆Theta)
measurableGraph :
∀ n (w : NonnegativeWeight 𝒳) (e : 𝒳 → ℝ)
@realizes C_n(sample-by-theta measurable for each admissible oracle input)
weightAEInvariant :
∀ n (P : TransportedArray 𝒳)
if
TransportedIVClass P N k c epsilon n
then
∀ w w' : TransportWeightVersion P n,
∀ᵐ s ∂twoSampleLaw P N n, set n (s, w.1, P.propensity n) = set n (s, w'.1, P.propensity n)
Sample-only finite-cell confidence-set sequence on the ambient carrier.
Definition (Lean source)
parameters
𝒳 :
Type*
N k :
ℕ → ℕ
fields
set :
∀ n
if
FiniteCellInput 𝒳 N k n
then
Set ℝ
@realizes C_n(sample-only finite-cell random set)
subset :
∀ n x, set n x ⊆ parameterSpace
measurableGraph :
∀ n (cell : Fin (k n) ↪ 𝒳)
if
(∀ i, MeasurableSet {cell i})
then
MeasurableSet {p : FiniteCellInput 𝒳 N k n × ℝ | ((∀ i, (p.1.1 i).1 ∈ range cell) ∧ (∀ j, p.1.2 j ∈ range cell)) ∧ p.2 ∈ set n p.1}
Finite-cell oracle procedures are global oracle procedures whose output on the realized finite-cell experiment depends on the oracle weight and propensity only through their values on the realized cells.
Definition (Lean source)
parameters
fields
offCellInvariant :
∀ n (cell : Fin (k n) ↪ 𝒳) (w w' : NonnegativeWeight 𝒳) (e e' : 𝒳 → ℝ),
(∀ i, w.1 (cell i) = w'.1 (cell i)) → (∀ i, e (cell i) = e' (cell i)) → ∀ s : TwoSample 𝒳 n (N n), ((∀ i, (s.1 i).1 ∈ range cell) ∧ (∀ j, s.2 j ∈ range cell)) → set n (s, w, e)
= set n (s, w', e')
Regular-cell confidence-set sequence structurally excluding w. Its known q and e inputs are finite vectors indexed by the bundled true support, so there are no ambient off-support extensions to quotient out.
Definition (Lean source)
parameters
𝒳 :
Type*
N k :
ℕ → ℕ
fields
set :
∀ n
if
RegularCellInput 𝒳 N k n
then
Set ℝ
@realizes C_n(regular-cell random set)
subset :
∀ n x, set n x ⊆ parameterSpace
measurableGraph :
∀ n (design : RegularCellDesign 𝒳 (k n)) (q e : Fin (k n) → ℝ),
MeasurableSet {p : TwoSample 𝒳 n (N n) × ℝ | ((∀ i, (p.1.1 i).1 ∈ range design.cell) ∧ (∀ j, p.1.2 j ∈ range design.cell)) ∧ p.2 ∈ set n ⟨p.1, design, q, e⟩}
def setLength
Lebesgue length restricted to the forced parameter interval.
Definition (Lean source)
def BorelParameterSet
Borel subsets of the forced parameter interval.
Definition (Lean source)
def paperSetLength
Lebesgue length on its exact paper domain and codomain.
Definition (Lean source)
noncomputable def paperSetLength (A : BorelParameterSet) : Icc (0 : ℝ) 2 :=
⟨setLength A.1, ENNReal.toReal_nonneg, by
unfold setLength parameterSpace
calc
(volume (A.1 ∩ Icc (-1 : ℝ) 1)).toReal ≤
(volume (Icc (-1 : ℝ) 1)).toReal :=
ENNReal.toReal_mono (by simp [Real.volume_Icc])
(measure_mono Set.inter_subset_right)
_ = 2 := by
rw [Real.volume_Icc, ENNReal.toReal_ofReal (by norm_num)]
norm_num⟩
def oracleSet
Gives the confidence set produced by an oracle procedure from a two-sample dataset, the population transport weight, and the population propensity score.
Definition (Lean source)
noncomputable def oracleSet (C : OracleProcedure 𝒳 N k c epsilon)
(P : TransportedArray 𝒳) (n : ℕ) (s : TwoSample 𝒳 n (N n)) : Set ℝ :=
C.set n (s, transportWeightInput P n, P.propensity n)
def oracleSetAtWeight
Auxiliary oracle evaluation at a declared measurable version in the model row's density-ratio fiber. Intrinsic procedure invariance identifies this with the canonical paper-facing evaluation almost surely.
Definition (Lean source)
noncomputable def oracleSetAtWeight (C : OracleProcedure 𝒳 N k c epsilon)
(P : TransportedArray 𝒳) (n : ℕ) (w : TransportWeightVersion P n)
(s : TwoSample 𝒳 n (N n)) : Set ℝ :=
C.set n (s, w.1, P.propensity n)
def oracleCoverage
Gives the probability that an oracle confidence set contains the target complier average causal effect.
Definition (Lean source)
noncomputable def oracleCoverage (C : OracleProcedure 𝒳 N k c epsilon)
(P : TransportedArray 𝒳) (n : ℕ) : ℝ :=
(twoSampleLaw P N n {s | targetCACE P n ∈ oracleSet C P n s}).toReal
def oracleCoverageAtWeight
Gives the coverage probability of an oracle confidence set when evaluated with a specified admissible transport-weight version.
Definition (Lean source)
noncomputable def oracleCoverageAtWeight
(C : OracleProcedure 𝒳 N k c epsilon)
(P : TransportedArray 𝒳) (n : ℕ)
(w : TransportWeightVersion P n) : ℝ :=
(twoSampleLaw P N n
{s | targetCACE P n ∈ oracleSetAtWeight C P n w s}).toReal
def oracleExpectedLength
The oracle procedure's expected confidence-set length under the two-sample law.
Definition (Lean source)
noncomputable def oracleExpectedLength
(C : OracleProcedure 𝒳 N k c epsilon)
(P : TransportedArray 𝒳) (n : ℕ) : ℝ :=
∫ s, setLength (oracleSet C P n s) ∂twoSampleLaw P N n
def oracleExpectedLengthAtWeight
The oracle procedure's expected confidence-set length when evaluated at a specified admissible transport-weight version.
Definition (Lean source)
noncomputable def oracleExpectedLengthAtWeight
(C : OracleProcedure 𝒳 N k c epsilon)
(P : TransportedArray 𝒳) (n : ℕ)
(w : TransportWeightVersion P n) : ℝ :=
∫ s, setLength (oracleSetAtWeight C P n w s) ∂twoSampleLaw P N n
def fixedGeometryOracleSet
A fixed-geometry oracle is evaluated with the geometry's chosen weight and propensity versions, rather than the canonical rnDeriv representative recovered from each law in the slice.
Definition (Lean source)
noncomputable def fixedGeometryOracleSet
(C : OracleProcedure 𝒳 N k c epsilon)
(g : Geometry 𝒳) (n : ℕ) (s : TwoSample 𝒳 n (N n)) : Set ℝ :=
C.set n (s, geometryWeightInput g n, g.propensity n)
def fixedGeometryOracleCoverage
Gives the coverage probability of an oracle confidence set under a fixed geometry and its declared weight and propensity functions.
Definition (Lean source)
noncomputable def fixedGeometryOracleCoverage
(C : OracleProcedure 𝒳 N k c epsilon)
(g : Geometry 𝒳) (P : TransportedArray 𝒳) (n : ℕ) : ℝ :=
(twoSampleLaw P N n
{s | targetCACE P n ∈ fixedGeometryOracleSet C g n s}).toReal
def fixedGeometryOracleExpectedLength
Gives the expected length of an oracle confidence set under a fixed geometry and its declared weight and propensity functions.
Definition (Lean source)
noncomputable def fixedGeometryOracleExpectedLength
(C : OracleProcedure 𝒳 N k c epsilon) (g : Geometry 𝒳)
(P : TransportedArray 𝒳) (n : ℕ) : ℝ :=
∫ s, setLength (fixedGeometryOracleSet C g n s) ∂twoSampleLaw P N n
def finiteCellCoverage
The finite-cell procedure's coverage probability for the target CACE under the two-sample law.
Definition (Lean source)
noncomputable def finiteCellCoverage (C : FiniteCellProcedure 𝒳 N k)
(n : ℕ) (P : TransportedArray 𝒳) : ℝ :=
(twoSampleLaw P N n {s | targetCACE P n ∈ C.set n s}).toReal
def finiteCellExpectedLength
The finite-cell procedure's expected confidence-set length under the two-sample law.
Definition (Lean source)
noncomputable def finiteCellExpectedLength (C : FiniteCellProcedure 𝒳 N k)
(n : ℕ) (P : TransportedArray 𝒳) : ℝ :=
∫ s, setLength (C.set n s) ∂twoSampleLaw P N n
def finiteCellOracleSet
The confidence set produced by a finite-cell oracle procedure using the population transport weight and propensity.
Definition (Lean source)
noncomputable def finiteCellOracleSet
(C : FiniteCellOracleProcedure 𝒳 N k c epsilon)
(n : ℕ) (P : TransportedArray 𝒳)
(s : TwoSample 𝒳 n (N n)) : Set ℝ :=
C.set n (s, transportWeightInput P n, P.propensity n)
def finiteCellOracleCoverage
The probability that a finite-cell oracle confidence set contains the target CACE.
Definition (Lean source)
noncomputable def finiteCellOracleCoverage
(C : FiniteCellOracleProcedure 𝒳 N k c epsilon)
(n : ℕ) (P : TransportedArray 𝒳) : ℝ :=
(twoSampleLaw P N n
{s | targetCACE P n ∈ finiteCellOracleSet C n P s}).toReal
def finiteCellOracleExpectedLength
The expected length of a finite-cell oracle confidence set under the two-sample law.
Definition (Lean source)
noncomputable def finiteCellOracleExpectedLength
(C : FiniteCellOracleProcedure 𝒳 N k c epsilon)
(n : ℕ) (P : TransportedArray 𝒳) : ℝ :=
∫ s, setLength (finiteCellOracleSet C n P s) ∂twoSampleLaw P N n
The known source-cell mass function.
Definition (Lean source)
noncomputable def sourceCellMass (P : TransportedArray 𝒳) (n : ℕ) : 𝒳 → ℝ :=
fun x => (sourceXLaw P n {x}).toReal
def regularCellDesignOfClass
A regular-class witness packaged as the true support supplied to the procedure.
Definition (Lean source)
noncomputable def regularCellDesignOfClass
(P : TransportedArray 𝒳)
(hP : RegularFiniteCellClass P N k c epsilon cminus cplus n) :
RegularCellDesign 𝒳 (k n) := by
classical
let hw := hP.2.2.2.2.2
exact ⟨Classical.choose hw, (Classical.choose_spec hw).1⟩
def cellVectorExtension
Extend a finite cell vector only for reuse by ambient-function estimator helpers. Procedure inputs expose the finite vector itself, never this arbitrary zero extension.
Definition (Lean source)
noncomputable def cellVectorExtension (design : RegularCellDesign 𝒳 m)
(v : Fin m → ℝ) : 𝒳 → ℝ := by
classical
exact fun x => if h : ∃ i, design.cell i = x then v (choose h) else 0
Constructs the regular-cell procedure input from a model in the regular finite-cell class and a two-sample dataset, including the class-supplied cell design, source-cell masses, and propensity scores.
Definition (Lean source)
noncomputable def regularCellInputOfClass
(P : TransportedArray 𝒳)
(hP : RegularFiniteCellClass P N k c epsilon cminus cplus n)
(s : TwoSample 𝒳 n (N n)) : RegularCellInput 𝒳 N k n :=
let design := regularCellDesignOfClass P hP
⟨s, design,
fun i => sourceCellMass P n (design.cell i),
fun i => P.propensity n (design.cell i)⟩
def regularCellSet
The confidence set produced by a regular-cell procedure from a sample and the class-supplied cell design.
Definition (Lean source)
noncomputable def regularCellSet (C : RegularCellProcedure 𝒳 N k)
(n : ℕ) (P : TransportedArray 𝒳)
(hP : RegularFiniteCellClass P N k c epsilon cminus cplus n)
(s : TwoSample 𝒳 n (N n)) : Set ℝ :=
C.set n (regularCellInputOfClass P hP s)
def regularCellCoverage
The probability that a regular-cell confidence set contains the target CACE over a regular finite-cell model.
Definition (Lean source)
noncomputable def regularCellCoverage (C : RegularCellProcedure 𝒳 N k)
(n : ℕ)
(P : {P : TransportedArray 𝒳 //
RegularFiniteCellClass P N k c epsilon cminus cplus n}) : ℝ :=
(twoSampleLaw P.1 N n
{s | targetCACE P.1 n ∈ regularCellSet C n P.1 P.2 s}).toReal
def regularCellExpectedLength
The expected length of a regular-cell confidence set under the corresponding two-sample law.
Definition (Lean source)
noncomputable def regularCellExpectedLength (C : RegularCellProcedure 𝒳 N k)
(n : ℕ)
(P : {P : TransportedArray 𝒳 //
RegularFiniteCellClass P N k c epsilon cminus cplus n}) : ℝ :=
∫ s, setLength (regularCellSet C n P.1 P.2 s) ∂twoSampleLaw P.1 N n
def coverageInfOrOne
The infimum of rowwise coverage, with the vacuous value one on an empty model row. Coverage takes values in [0,1], so this is the bounded paper-facing counterpart of the mathematical convention inf empty = +infinity.
Definition (Lean source)
noncomputable abbrev coverageInfOrOne {ι : Sort*} (f : ι → ℝ) : ℝ :=
coverageInfOrOne f
def oracleCoverageClassInf
Rowwise worst-case coverage indexed exactly by model rows. The canonical input represents the row's source-a.e. quotient, by intrinsic procedure invariance.
Definition (Lean source)
noncomputable def oracleCoverageClassInf
(C : OracleProcedure 𝒳 N k c epsilon)
(n : ℕ) : ℝ :=
coverageInfOrOne fun P : {P : TransportedArray 𝒳 //
TransportedIVClass P N k c epsilon n} =>
oracleCoverage C P.1 n
def OracleHonest
Oracle procedures have asymptotic uniform coverage over the main class; their structure makes evaluation well-defined on the law-indexed source-a.e. quotient of the declared density-ratio input.
Definition (Lean source)
noncomputable def OracleHonest
(C : OracleProcedure 𝒳 N k c epsilon) : Prop :=
0 < alpha ∧ alpha < 1 ∧ -- @realizes \alpha(noncoverage level in (0,1))
1 - alpha ≤ liminf
(oracleCoverageClassInf N k c epsilon C) atTop
def FiniteCellHonest
Sample-only procedures have asymptotic uniform coverage over the finite-cell subclass.
Definition (Lean source)
noncomputable def FiniteCellHonest
(C : FiniteCellProcedure 𝒳 N k) : Prop :=
0 < alpha ∧ alpha < 1 ∧ -- @realizes \alpha(noncoverage level in (0,1))
1 - alpha ≤ liminf
(fun n => coverageInfOrOne fun P : {P : TransportedArray 𝒳 //
FiniteCellClass P N k c epsilon n} =>
finiteCellCoverage C n P) atTop
Oracle procedures are honest over the full transported-IV class on the same carrier; the risk may subsequently be restricted to cells.
Definition (Lean source)
noncomputable def FiniteCellOracleHonest
(C : FiniteCellOracleProcedure 𝒳 N k c epsilon) : Prop :=
OracleHonest (𝒳 := 𝒳) N k c epsilon alpha C.toOracleProcedure
def FixedGeometryOracleHonest
Oracle procedures honest uniformly over a fixed geometry slice.
Definition (Lean source)
noncomputable def FixedGeometryOracleHonest
(g : {g : Geometry 𝒳 // AdmissibleGeometry g k epsilon})
(C : OracleProcedure 𝒳 N k c epsilon) : Prop :=
0 < alpha ∧ alpha < 1 ∧ -- @realizes \alpha(noncoverage level in (0,1))
1 - alpha ≤ liminf
(fun n => coverageInfOrOne fun P : {P : TransportedArray 𝒳 //
fixedGeometrySlice P g.1 N k c epsilon n} =>
fixedGeometryOracleCoverage C g.1 P n) atTop
def RegularCellHonest
Procedures using known source-cell masses and propensity, but not w, are honest over the regular finite-cell class.
Definition (Lean source)
noncomputable def RegularCellHonest (cminus cplus : ℝ)
(C : RegularCellProcedure 𝒳 N k) : Prop :=
0 < alpha ∧ alpha < 1 ∧ -- @realizes \alpha(noncoverage level in (0,1))
1 - alpha ≤ liminf
(fun n => coverageInfOrOne fun P : {P : TransportedArray 𝒳 //
RegularFiniteCellClass P N k c epsilon cminus cplus n} =>
regularCellCoverage (c := c) (epsilon := epsilon)
(cminus := cminus) (cplus := cplus) C n P) atTop
def PositiveThreshold
Positive frontier thresholds, the exact paper domain of every tagged risk and minimax value function.
Definition (Lean source)
def PositiveThreshold := {t0 : ℝ // 0 < t0}
def frontierRiskRow
Rowwise global oracle risk over exactly the model rows satisfying the strength restriction. There is no additional weight-version supremum.
Definition (Lean source)
noncomputable def frontierRiskRow
(C : OracleProcedure 𝒳 N k c epsilon)
(t0 : ℝ) (n : ℕ) : ℝ :=
⨆ P : {P : TransportedArray 𝒳 //
TransportedIVClass P N k c epsilon n ∧
t0 ≤ effectiveStrength P n},
oracleExpectedLength C P.1 n
def finiteCellOracleRiskRow
The corresponding rowwise risk restricted to the finite-cell submodel.
Definition (Lean source)
noncomputable def finiteCellOracleRiskRow
(C : OracleProcedure 𝒳 N k c epsilon)
(t0 : ℝ) (n : ℕ) : ℝ :=
⨆ P : {P : TransportedArray 𝒳 //
FiniteCellClass P N k c epsilon n ∧
t0 ≤ effectiveStrength P n},
oracleExpectedLength C P.1 n
def frontierRiskTotal
Total computational risk at an arbitrary real threshold.
Definition (Lean source)
noncomputable def frontierRiskTotal
(C : OracleProcedure 𝒳 N k c epsilon) (t0 : ℝ) : ℝ :=
limsup (frontierRiskRow N k c epsilon C t0) atTop
def frontierRisk
Paper-facing risk on the declared positive threshold domain.
Definition (Lean source)
noncomputable def frontierRisk
(C : OracleProcedure 𝒳 N k c epsilon)
(t0 : PositiveThreshold) : ℝ :=
frontierRiskTotal N k c epsilon C t0.1
def finiteCellOracleRisk
Gives the limiting worst-case expected length of an oracle procedure over finite-cell model rows whose effective strength meets the supplied threshold.
Definition (Lean source)
noncomputable def finiteCellOracleRisk
(C : OracleProcedure 𝒳 N k c epsilon)
(t0 : ℝ) : ℝ :=
limsup (finiteCellOracleRiskRow N k c epsilon C t0) atTop
def fixedGeometryRisk
Gives the limiting worst-case expected length of an oracle procedure over model rows in a fixed geometry whose effective strength meets the supplied threshold.
Definition (Lean source)
noncomputable def fixedGeometryRisk (g : Geometry 𝒳)
(C : OracleProcedure 𝒳 N k c epsilon) (t0 : ℝ) : ℝ :=
limsup
(fun n => ⨆ P : {P : TransportedArray 𝒳 //
fixedGeometrySlice P g N k c epsilon n ∧
t0 ≤ effectiveStrength P n},
fixedGeometryOracleExpectedLength C g P n) atTop
The asymptotic worst-case expected length of a regular-cell procedure over regular finite-cell models whose effective strength exceeds a given threshold.
Definition (Lean source)
noncomputable def regularCellRisk (cminus cplus : ℝ)
(C : RegularCellProcedure 𝒳 N k) (t0 : ℝ) : ℝ :=
limsup
(fun n => ⨆ P : {P : TransportedArray 𝒳 //
RegularFiniteCellClass P N k c epsilon cminus cplus n ∧
t0 ≤ effectiveStrength P n},
regularCellExpectedLength (c := c) (epsilon := epsilon)
(cminus := cminus) (cplus := cplus) C n ⟨P.1, P.2.1⟩) atTop
The asymptotic worst-case expected length of a finite-cell procedure over finite-cell models whose effective strength exceeds a given threshold.
Definition (Lean source)
noncomputable def feasibleFiniteCellRisk
(C : FiniteCellProcedure 𝒳 N k)
(t0 : ℝ) : ℝ :=
limsup
(fun n => ⨆ P : {P : TransportedArray 𝒳 //
FiniteCellClass P N k c epsilon n ∧
t0 ≤ effectiveStrength P n},
finiteCellExpectedLength C n P) atTop
def oracleValueTotal
Total computational helper underlying the paper-facing oracle value.
Definition (Lean source)
noncomputable def oracleValueTotal (t0 : ℝ) : ℝ :=
⨅ C : {C : OracleProcedure 𝒳 N k c epsilon //
OracleHonest N k c epsilon alpha C},
frontierRiskTotal N k c epsilon C.1 t0
Oracle minimax expected-length frontier on its declared positive domain.
Definition (Lean source)
noncomputable def oracleValue (t0 : PositiveThreshold) : ℝ :=
oracleValueTotal (𝒳 := 𝒳) N k c epsilon alpha t0.1
def finiteCellOracleValueTotal
Gives the smallest limiting worst-case expected length attainable by an oracle-honest procedure over finite-cell models at the supplied strength threshold.
Definition (Lean source)
noncomputable def finiteCellOracleValueTotal (t0 : ℝ) : ℝ :=
⨅ C : {C : OracleProcedure 𝒳 N k c epsilon //
OracleHonest N k c epsilon alpha C},
finiteCellOracleRisk N k c epsilon C.1 t0
The global oracle-honest procedure class, with risk restricted to the growing finite-cell submodel, on the declared positive threshold domain.
Definition (Lean source)
noncomputable def finiteCellOracleValue (t0 : PositiveThreshold) : ℝ :=
finiteCellOracleValueTotal (𝒳 := 𝒳) N k c epsilon alpha t0.1
def fixedGeometryValueTotal
Total computational helper underlying the fixed-geometry value.
Definition (Lean source)
noncomputable def fixedGeometryValueTotal
(g : {g : Geometry 𝒳 // AdmissibleGeometry g k epsilon})
(t0 : ℝ) : ℝ :=
⨅ C : {C : OracleProcedure 𝒳 N k c epsilon //
FixedGeometryOracleHonest N k c epsilon alpha g C},
fixedGeometryRisk N k c epsilon g.1 C.1 t0
def fixedGeometryValue
Conditional minimax frontier at an admissible deterministic geometry, on the declared positive threshold domain.
Definition (Lean source)
noncomputable def fixedGeometryValue
(g : {g : Geometry 𝒳 // AdmissibleGeometry g k epsilon})
(t0 : PositiveThreshold) : ℝ :=
fixedGeometryValueTotal N k c epsilon alpha g t0.1
Helpers.CellEstimators 6 declarations
def collisionKernel
Symmetric, generally nondegenerate collision kernel.
Definition (Lean source)
noncomputable abbrev collisionKernel (q : 𝒳 → ℝ) (x y : 𝒳) : ℝ :=
collisionKernel q x y
def collisionScale
Ordered-pair collision estimate of Kish dispersion.
Definition (Lean source)
noncomputable abbrev collisionScale (q : 𝒳 → ℝ) {N : ℕ}
(target : Fin N → 𝒳) : ℝ :=
collisionScale q target
def cellMoment
Source-cell inverse-frequency moment estimate.
Definition (Lean source)
noncomputable abbrev cellMoment (q : 𝒳 → ℝ) {n : ℕ}
(sample : SourceSample 𝒳 n) (G : SourceObs 𝒳 → ℝ) (x : 𝒳) : ℝ :=
cellMoment q fst sample G x
def crossAverage
Cross-average using target empirical cell frequencies.
Definition (Lean source)
noncomputable abbrev crossAverage (q : 𝒳 → ℝ) {n N : ℕ}
(source : SourceSample 𝒳 n) (target : TargetSample 𝒳 N)
(G : SourceObs 𝒳 → ℝ) : ℝ :=
crossAverage q fst source target G
Regular-cell inversion rule using cross-averaged outcome and receipt moments and the collision estimate of the transport scale.
Definition (Lean source)
noncomputable def regularCellInversion (q e : 𝒳 → ℝ) {n N : ℕ} (L : ℝ)
(source : SourceSample 𝒳 n) (target : TargetSample 𝒳 N) : Set ℝ :=
if N < 2 then parameterSpace else
{theta | theta ∈ parameterSpace ∧
|crossAverage q source target
(fun o => oracleInstrumentScore e o * o.2.2.2) -
theta * crossAverage q source target
(fun o => oracleInstrumentScore e o * boolReal o.2.2.1)| ≤
L * sqrt ((1 + collisionScale q target) / n)}
Uniform-cell specialization with q_x=1/k and e=1/2.
Definition (Lean source)
noncomputable def finiteCellInversion (k n N : ℕ) (L : ℝ)
(source : SourceSample 𝒳 n) (target : TargetSample 𝒳 N) :
Set ℝ :=
regularCellInversion (fun _ => (k : ℝ)⁻¹) (fun _ => (1 : ℝ) / 2)
L source target
Helpers.Divergence 1 declarations
theorem witness_source_chiSq_bound
Per-source-observation chi-square calibration for the continuum witness.
Formal statement
hypotheses
g :
Geometry 𝒳
k :
ℕ → ℕ
epsilon t0 :
ℝ
hepsilon :
0 < epsilon
hg :
AdmissibleGeometry g k epsilon
n :
ℕ
h :
ℝ
hh :
|h| ≤ 1 / 4
hvalid :
∀ x, 0 ≤ geometryCompliance g t0 n x ∧ geometryCompliance g t0 n x ≤ 1
conclusion
chiSqDiv (sourceObsLaw (geometryWitnessFamily g t0 h) n) (sourceObsLaw (geometryWitnessFamily g t0 0) n)
≤ 8 * geometryMu g t0 n ^ 2 * h ^ 2 / geometryKish g n
Proof (Lean source)
lemma witness_source_chiSq_bound
(g : Geometry 𝒳) (k : ℕ → ℕ) (epsilon t0 : ℝ)
(hepsilon : 0 < epsilon) (hg : AdmissibleGeometry g k epsilon)
(n : ℕ) (h : ℝ) (hh : |h| ≤ 1 / 4)
(hvalid : ∀ x, 0 ≤ geometryCompliance g t0 n x ∧
geometryCompliance g t0 n x ≤ 1) :
chiSqDiv
(sourceObsLaw (geometryWitnessFamily g t0 h) n)
(sourceObsLaw (geometryWitnessFamily g t0 0) n) ≤
8 * geometryMu g t0 n ^ 2 * h ^ 2 / geometryKish g n := by
exact geometryWitnessFamily_source_chiSq_bound
g k epsilon t0 hepsilon hg n h hh hvalid
Helpers.ExpectedLength 2 declarations
theorem expected_setLength_eq_integral_inclusion
Tonelli section identity for a jointly measurable random set.
Formal statement
hypotheses
conclusion
(∫ ω, setLength (C ω) ∂Q) = ∫ u in parameterSpace, (Q {ω | u ∈ C ω}).toReal
Proof (Lean source)
lemma expected_setLength_eq_integral_inclusion
(Q : Measure Ω) [IsFiniteMeasure Q] (C : Ω → Set ℝ)
(hgraph : MeasurableSet {p : Ω × ℝ | p.2 ∈ C p.1})
(hsub : ∀ ω, C ω ⊆ parameterSpace) :
(∫ ω, setLength (C ω) ∂Q) =
∫ u in parameterSpace, (Q {ω | u ∈ C ω}).toReal := by
clear hsub
simpa [setLength, restrictedSetVolume] using
expected_restrictedSetVolume_eq_integral_inclusion
(Q := Q) C parameterSpace hgraph measurableSet_Icc
(by simp [parameterSpace, Real.volume_Icc])
theorem coverage_tv_expectedLength_lower
Uniform coverage on an interval and a TV comparison with its center force positive expected length at the center law.
Formal statement
hypotheses
Q :
ℝ → Measure Ω
C :
Ω → Set ℝ
I :
Set ℝ
coverage tv :
ℝ
hQ :
∀ u, IsProbabilityMeasure (Q u)
hcover :
∀ u ∈ I, coverage ≤ (Q u {ω | u ∈ C ω}).toReal
htv :
∀ u ∈ I, tvDist (Q u) (Q 0) ≤ tv
hgraph :
MeasurableSet {p : Ω × ℝ | p.2 ∈ C p.1}
hsub :
∀ ω, C ω ⊆ parameterSpace
hI :
hI_sub :
I ⊆ parameterSpace
Proof (Lean source)
lemma coverage_tv_expectedLength_lower
(Q : ℝ → Measure Ω) (C : Ω → Set ℝ) (I : Set ℝ)
(coverage tv : ℝ)
(hQ : ∀ u, IsProbabilityMeasure (Q u))
(hcover : ∀ u ∈ I, coverage ≤ (Q u {ω | u ∈ C ω}).toReal)
(htv : ∀ u ∈ I, tvDist (Q u) (Q 0) ≤ tv)
(hgraph : MeasurableSet {p : Ω × ℝ | p.2 ∈ C p.1})
(hsub : ∀ ω, C ω ⊆ parameterSpace)
(hI : MeasurableSet I) (hI_sub : I ⊆ parameterSpace) :
(volume I).toReal * (coverage - tv) ≤
∫ ω, setLength (C ω) ∂Q 0 := by
clear hsub
simpa [setLength, restrictedSetVolume] using
coverage_tv_expectedRestrictedVolume_lower
Q C parameterSpace I 0 coverage tv hQ hcover htv hgraph
measurableSet_Icc (by simp [parameterSpace, Real.volume_Icc]) hI hI_sub
Helpers.FiniteCellBridge 6 declarations
def finiteCellProcedure
The sample-only uniform-cell specialization of the regular-cell inversion procedure.
Definition (Lean source)
noncomputable def finiteCellProcedure (N k : ℕ → ℕ) (L : ℝ) :
FiniteCellProcedure 𝒳 N k where
set n input := finiteCellInversion (k n) n (N n) L input.1 input.2
subset n input := by
intro theta htheta
by_cases hsmall : N n < 2
· simpa [finiteCellInversion, regularCellInversion, hsmall] using htheta
· simp only [finiteCellInversion, regularCellInversion, hsmall,
↓reduceIte] at htheta
exact htheta.1
measurableGraph n cell hcell := by
simpa [finiteCellInversion] using
regularCellInversion_measurableGraph_on_cells N k L n cell hcell
(fun _ : 𝒳 => (k n : ℝ)⁻¹) (fun _ : 𝒳 => (1 : ℝ) / 2)
measurable_const measurable_const
theorem toRegularFiniteCellClass
A uniform finite-cell law belongs to the regular finite-cell class with unit lower and upper cell-mass constants at any smaller overlap radius.
Formal statement
hypotheses
P :
N k :
ℕ → ℕ
c epsilon epsilonBar :
ℝ
n :
ℕ
hP :
FiniteCellClass P N k c epsilon n
hBar :
0 < epsilonBar ∧ epsilonBar < 1 / 2
conclusion
RegularFiniteCellClass P N k c epsilonBar 1 1 n
Proof (Lean source)
lemma FiniteCellClass.toRegularFiniteCellClass
{P : TransportedArray 𝒳} {N k : ℕ → ℕ} {c epsilon epsilonBar : ℝ}
{n : ℕ} (hP : FiniteCellClass P N k c epsilon n)
(hBar : 0 < epsilonBar ∧ epsilonBar < 1 / 2) :
RegularFiniteCellClass P N k c epsilonBar 1 1 n := by
have hoverlap : InstrumentOverlap P n epsilonBar := by
refine ⟨hBar.1, hBar.2, ?_⟩
filter_upwards [hP.finiteCellSource.2.2.choose_spec.2.2.2.2] with x hx
rw [hx]
constructor <;> linarith [hBar.2]
have hIV : TransportedIVClass P N k c epsilonBar n :=
{ hP with instrumentOverlap := hoverlap }
obtain ⟨cell, hcell, hrange, hatom, _, _⟩ :=
hP.finiteCellSource.2.2
refine ⟨hIV, hP.finiteCellSource.1, by norm_num, by norm_num,
by norm_num, cell, hcell, hrange, ?_⟩
intro i
rw [hatom i]
constructor <;> simp [div_eq_mul_inv]
theorem finiteCellProcedure_eq_regularCellSet_ae
On a uniform finite-cell law, the regular procedure fed the law's cell masses and propensity agrees almost surely with the sample-only specialization.
Formal statement
hypotheses
conclusion
∀ᵐ s ∂twoSampleLaw P N n,
(finiteCellProcedure (𝒳 := 𝒳) N k L).set n s
= regularCellSet (regularCellProcedure (𝒳 := 𝒳) N k L) n P (hP.toRegularFiniteCellClass ⟨hP.instrumentOverlap.1, hP.instrumentOverlap.2.1⟩) s
Proof (Lean source)
lemma finiteCellProcedure_eq_regularCellSet_ae
{c epsilon : ℝ} (N k : ℕ → ℕ) (L : ℝ) (n : ℕ)
(P : TransportedArray 𝒳)
(hP : FiniteCellClass P N k c epsilon n) :
∀ᵐ s ∂twoSampleLaw P N n,
(finiteCellProcedure (𝒳 := 𝒳) N k L).set n s =
regularCellSet (regularCellProcedure (𝒳 := 𝒳) N k L) n P
(hP.toRegularFiniteCellClass
⟨hP.instrumentOverlap.1, hP.instrumentOverlap.2.1⟩) s := by
classical
let hPreg := hP.toRegularFiniteCellClass
⟨hP.instrumentOverlap.1, hP.instrumentOverlap.2.1⟩
let design := regularCellDesignOfClass P hPreg
let cell := design.cell
have hcell : ∀ i, MeasurableSet {cell i} := design.measurableCell
have hrange : sourceXLaw P n (range cell) = 1 := by
exact (Classical.choose_spec hPreg.2.2.2.2.2).2.1
have hatom : ∀ i, sourceCellMass P n (cell i) = (k n : ℝ)⁻¹ := by
intro i
have hb := (Classical.choose_spec hPreg.2.2.2.2.2).2.2 i
change sourceCellMass P n
((choose hPreg.2.2.2.2.2) i) = (k n : ℝ)⁻¹
exact le_antisymm (by simpa [sourceCellMass, one_div] using hb.2)
(by simpa [sourceCellMass, one_div] using hb.1)
have hprop := hP.finiteCellSource.2.2.choose_spec.2.2.2.2
letI : IsProbabilityMeasure (sourceXLaw P n) :=
hP.finiteCellSource.2.1
letI : IsProbabilityMeasure (sourceObsLaw P n) :=
hP.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (targetXLaw P n) :=
hP.twoSampleArray.2.2.1 n
have hrangeMeas : MeasurableSet (range cell) := by
rw [show range cell = ⋃ i, {cell i} by
ext x
simp]
exact MeasurableSet.iUnion hcell
have hrangeCompl : sourceXLaw P n (range cell)ᶜ = 0 := by
rw [measure_compl hrangeMeas (measure_ne_top _ _), hrange, measure_univ]
simp
have htargetRange :
∀ᵐ x ∂targetXLaw P n, x ∈ range cell := by
have hc := hP.transportDomination hrangeCompl
change range cell ∈ ae (targetXLaw P n)
rw [mem_ae_iff]
exact hc
have hsourceProp :
∀ᵐ o ∂sourceObsLaw P n, P.propensity n o.1 = 1 / 2 := by
unfold sourceXLaw at hprop
exact ae_of_ae_map measurable_fst.aemeasurable hprop
let SG : Set (SourceSample 𝒳 n) :=
{source | (∀ i, (source i).1 ∈ range cell) ∧
∀ i, P.propensity n (source i).1 = 1 / 2}
let TG : Set (TargetSample 𝒳 (N n)) :=
{target | ∀ j, target j ∈ range cell}
have hSGmeas : MeasurableSet SG := by
rw [show SG = (⋂ i, {source | (source i).1 ∈ range cell}) ∩
⋂ i, {source | P.propensity n (source i).1 = 1 / 2} by
ext source
simp [SG]]
exact (MeasurableSet.iInter fun i =>
hrangeMeas.preimage
(measurable_fst.comp (measurable_pi_apply i))).inter
(MeasurableSet.iInter fun i =>
(MeasurableSet.singleton (1 / 2 : ℝ)).preimage
((P.propensity_measurable n).comp
(measurable_fst.comp (measurable_pi_apply i))))
have hTGmeas : MeasurableSet TG := by
rw [show TG = ⋂ j, {target | target j ∈ range cell} by
ext target
simp [TG]]
exact MeasurableSet.iInter fun j =>
hrangeMeas.preimage (measurable_pi_apply j)
have hSGAE :
∀ᵐ source ∂Measure.pi (fun _ : Fin n => sourceObsLaw P n),
source ∈ SG := by
have hsourceXAE : ∀ᵐ x ∂sourceXLaw P n, x ∈ range cell :=
(mem_ae_iff_prob_eq_one hrangeMeas).2 hrange
have hsourceObsAE :
∀ᵐ o ∂sourceObsLaw P n, o.1 ∈ range cell := by
unfold sourceXLaw at hsourceXAE
exact ae_of_ae_map measurable_fst.aemeasurable hsourceXAE
filter_upwards
[Measure.ae_pi_le_pi
(Filter.eventually_pi fun _ : Fin n => hsourceObsAE),
Measure.ae_pi_le_pi
(Filter.eventually_pi fun _ : Fin n => hsourceProp)] with source hs hp
exact ⟨hs, hp⟩
have hTGAE :
∀ᵐ target ∂Measure.pi (fun _ : Fin (N n) => targetXLaw P n),
target ∈ TG := by
apply Measure.ae_pi_le_pi
exact Filter.eventually_pi fun _ => htargetRange
have hSGone :
Measure.pi (fun _ : Fin n => sourceObsLaw P n) SG = 1 :=
(mem_ae_iff_prob_eq_one hSGmeas).1 hSGAE
have hTGone :
Measure.pi (fun _ : Fin (N n) => targetXLaw P n) TG = 1 :=
(mem_ae_iff_prob_eq_one hTGmeas).1 hTGAE
have hprod :
twoSampleLaw P N n (SG ×ˢ TG) = 1 := by
unfold twoSampleLaw
rw [Measure.prod_prod, hSGone, hTGone]
simp
letI : IsProbabilityMeasure (twoSampleLaw P N n) := by
unfold twoSampleLaw
infer_instance
have hprodAE : ∀ᵐ s ∂twoSampleLaw P N n, s ∈ SG ×ˢ TG := by
exact (mem_ae_iff_prob_eq_one (hSGmeas.prod hTGmeas)).2 hprod
filter_upwards [hprodAE] with s hs
have hmass (j : Fin (N n)) :
sourceCellMass P n (s.2 j) = (k n : ℝ)⁻¹ := by
obtain ⟨i, hi⟩ := hs.2 j
rw [← hi]
exact hatom i
have hprop' (i : Fin n) :
P.propensity n (s.1 i).1 = 1 / 2 :=
hs.1.2 i
change finiteCellInversion (k n) n (N n) L s.1 s.2 =
regularCellInversion
(cellVectorExtension design
(fun i => sourceCellMass P n (design.cell i)))
(cellVectorExtension design
(fun i => P.propensity n (design.cell i))) L s.1 s.2
have hqExt (j : Fin (N n)) :
cellVectorExtension design
(fun i => sourceCellMass P n (design.cell i)) (s.2 j) =
(k n : ℝ)⁻¹ := by
obtain ⟨i, hi⟩ := hs.2 j
rw [← hi, cellVectorExtension_apply_cell]
exact hatom i
have heExt (i : Fin n) :
cellVectorExtension design
(fun j => P.propensity n (design.cell j)) (s.1 i).1 =
1 / 2 := by
obtain ⟨j, hj⟩ := hs.1.1 i
calc
cellVectorExtension design
(fun j => P.propensity n (design.cell j)) (s.1 i).1 =
P.propensity n (s.1 i).1 := by
rw [← hj, cellVectorExtension_apply_cell]
_ = 1 / 2 := hprop' i
unfold finiteCellInversion
have hinv :
regularCellInversion
(cellVectorExtension design
(fun i => sourceCellMass P n (design.cell i)))
(cellVectorExtension design
(fun i => P.propensity n (design.cell i))) L s.1 s.2 =
regularCellInversion (fun _ : 𝒳 => (k n : ℝ)⁻¹)
(fun _ : 𝒳 => 1 / 2) L s.1 s.2 := by
unfold regularCellInversion crossAverage collisionScale
crossAverage cellMoment
collisionScale collisionKernel
oracleInstrumentScore
simp_rw [hqExt, heExt]
exact hinv.symm
theorem finiteCellProcedure_coverage_eq_regular
On a uniform finite-cell law, the sample-only and regular specialized procedures have the same coverage.
Formal statement
hypotheses
conclusion
finiteCellCoverage (finiteCellProcedure (𝒳 := 𝒳) N k L) n P
= regularCellCoverage (regularCellProcedure (𝒳 := 𝒳) N k L) n ⟨P, hP.toRegularFiniteCellClass ⟨hP.instrumentOverlap.1, hP.instrumentOverlap.2.1⟩⟩
Proof (Lean source)
lemma finiteCellProcedure_coverage_eq_regular
{c epsilon : ℝ} (N k : ℕ → ℕ) (L : ℝ) (n : ℕ)
(P : TransportedArray 𝒳) (hP : FiniteCellClass P N k c epsilon n) :
finiteCellCoverage (finiteCellProcedure (𝒳 := 𝒳) N k L) n P =
regularCellCoverage (regularCellProcedure (𝒳 := 𝒳) N k L) n
⟨P, hP.toRegularFiniteCellClass
⟨hP.instrumentOverlap.1, hP.instrumentOverlap.2.1⟩⟩ := by
unfold finiteCellCoverage regularCellCoverage
congr 1
apply measure_congr
filter_upwards [finiteCellProcedure_eq_regularCellSet_ae N k L n P hP]
with s hs
change (targetCACE P n ∈
(finiteCellProcedure (𝒳 := 𝒳) N k L).set n s) =
(targetCACE P n ∈
regularCellSet (regularCellProcedure (𝒳 := 𝒳) N k L) n P
(hP.toRegularFiniteCellClass
⟨hP.instrumentOverlap.1, hP.instrumentOverlap.2.1⟩) s)
rw [hs]
theorem finiteCellProcedure_expectedLength_eq_regular
On a uniform finite-cell law, the sample-only and regular specialized procedures have the same expected set length.
Formal statement
hypotheses
conclusion
finiteCellExpectedLength (finiteCellProcedure (𝒳 := 𝒳) N k L) n P
= regularCellExpectedLength (regularCellProcedure (𝒳 := 𝒳) N k L) n ⟨P, hP.toRegularFiniteCellClass ⟨hP.instrumentOverlap.1, hP.instrumentOverlap.2.1⟩⟩
Proof (Lean source)
lemma finiteCellProcedure_expectedLength_eq_regular
{c epsilon : ℝ} (N k : ℕ → ℕ) (L : ℝ) (n : ℕ)
(P : TransportedArray 𝒳) (hP : FiniteCellClass P N k c epsilon n) :
finiteCellExpectedLength (finiteCellProcedure (𝒳 := 𝒳) N k L) n P =
regularCellExpectedLength
(regularCellProcedure (𝒳 := 𝒳) N k L) n
⟨P, hP.toRegularFiniteCellClass
⟨hP.instrumentOverlap.1, hP.instrumentOverlap.2.1⟩⟩ := by
unfold finiteCellExpectedLength regularCellExpectedLength
apply integral_congr_ae
filter_upwards [finiteCellProcedure_eq_regularCellSet_ae N k L n P hP]
with s hs
rw [hs]
theorem finiteCellProcedure_upper_bridge
Honesty and worst-case expected length transfer from the unit-regular procedure to its sample-only uniform-cell specialization.
Formal statement
hypotheses
N k :
ℕ → ℕ
c epsilon epsilonBar alpha L :
ℝ
hBar :
0 < epsilonBar ∧ epsilonBar < 1 / 2
hClass :
∀ n, ∃ P : TransportedArray 𝒳, FiniteCellClass P N k c epsilon n
Creg :
RegularCellProcedure 𝒳 N k
hSetEq :
∀ n P
if
(hP : FiniteCellClass P N k c epsilon n)
then
∀ᵐ s ∂twoSampleLaw P N n,
(finiteCellProcedure (𝒳 := 𝒳) N k L).set n s
= regularCellSet Creg n P (hP.toRegularFiniteCellClass hBar) s
hHonest :
RegularCellHonest N k c epsilonBar alpha 1 1 Creg
conclusion
FiniteCellHonest N k c epsilon alpha (finiteCellProcedure (𝒳 := 𝒳) N k L) ∧
∀ t0,
feasibleFiniteCellRisk N k c epsilon (finiteCellProcedure (𝒳 := 𝒳) N k L) t0
≤ regularCellRisk N k c epsilonBar 1 1 Creg t0
Proof (Lean source)
lemma finiteCellProcedure_upper_bridge
(N k : ℕ → ℕ) (c epsilon epsilonBar alpha L : ℝ)
(hBar : 0 < epsilonBar ∧ epsilonBar < 1 / 2)
(hClass : ∀ n, ∃ P : TransportedArray 𝒳,
FiniteCellClass P N k c epsilon n)
(Creg : RegularCellProcedure 𝒳 N k)
(hSetEq : ∀ n P, (hP : FiniteCellClass P N k c epsilon n) →
∀ᵐ s ∂twoSampleLaw P N n,
(finiteCellProcedure (𝒳 := 𝒳) N k L).set n s =
regularCellSet Creg n P
(hP.toRegularFiniteCellClass hBar) s)
(hHonest : RegularCellHonest N k c epsilonBar alpha 1 1
Creg) :
FiniteCellHonest N k c epsilon alpha
(finiteCellProcedure (𝒳 := 𝒳) N k L) ∧
∀ t0,
feasibleFiniteCellRisk N k c epsilon
(finiteCellProcedure (𝒳 := 𝒳) N k L) t0 ≤
regularCellRisk N k c epsilonBar 1 1
Creg t0 := by
classical
let Cfin := finiteCellProcedure (𝒳 := 𝒳) N k L
have hcoverageEq : ∀ n P, (hP : FiniteCellClass P N k c epsilon n) →
finiteCellCoverage Cfin n P = regularCellCoverage Creg n
⟨P, hP.toRegularFiniteCellClass hBar⟩ := by
intro n P hP
unfold finiteCellCoverage regularCellCoverage
congr 1
apply measure_congr
filter_upwards [hSetEq n P hP] with s hs
change (targetCACE P n ∈ Cfin.set n s) =
(targetCACE P n ∈ regularCellSet Creg n P
(hP.toRegularFiniteCellClass hBar) s)
rw [hs]
have hlengthEq : ∀ n P, (hP : FiniteCellClass P N k c epsilon n) →
finiteCellExpectedLength Cfin n P =
regularCellExpectedLength Creg n
⟨P, hP.toRegularFiniteCellClass hBar⟩ := by
intro n P hP
unfold finiteCellExpectedLength regularCellExpectedLength
apply integral_congr_ae
filter_upwards [hSetEq n P hP] with s hs
rw [hs]
let covF : ℕ → ℝ := fun n =>
coverageInfOrOne fun P : {P : TransportedArray 𝒳 //
FiniteCellClass P N k c epsilon n} =>
finiteCellCoverage Cfin n P
let covR : ℕ → ℝ := fun n =>
coverageInfOrOne fun P : {P : TransportedArray 𝒳 //
RegularFiniteCellClass P N k c epsilonBar 1 1 n} =>
regularCellCoverage Creg n P
have hcoverageRange : ∀ n P,
(hP : RegularFiniteCellClass P N k c epsilonBar 1 1 n) →
0 ≤ regularCellCoverage Creg n ⟨P, hP⟩ ∧
regularCellCoverage Creg n ⟨P, hP⟩ ≤ 1 := by
intro n P hP
letI : IsProbabilityMeasure (sourceObsLaw P n) :=
hP.1.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (targetXLaw P n) :=
hP.1.twoSampleArray.2.2.1 n
letI : IsProbabilityMeasure (twoSampleLaw P N n) := by
unfold twoSampleLaw
infer_instance
exact ⟨ENNReal.toReal_nonneg, measureReal_le_one⟩
have hcovRows : ∀ n, covR n ≤ covF n := by
intro n
obtain ⟨P0, hP0⟩ := hClass n
letI : Nonempty {P : TransportedArray 𝒳 //
FiniteCellClass P N k c epsilon n} := ⟨⟨P0, hP0⟩⟩
letI : Nonempty {P : TransportedArray 𝒳 //
RegularFiniteCellClass P N k c epsilonBar 1 1 n} :=
⟨⟨P0, hP0.toRegularFiniteCellClass hBar⟩⟩
change (coverageInfOrOne fun P : {P : TransportedArray 𝒳 //
RegularFiniteCellClass P N k c epsilonBar 1 1 n} =>
regularCellCoverage Creg n P) ≤
coverageInfOrOne fun P : {P : TransportedArray 𝒳 //
FiniteCellClass P N k c epsilon n} => finiteCellCoverage Cfin n P
rw [coverageInfOrOne_of_nonempty_local,
coverageInfOrOne_of_nonempty_local]
apply le_ciInf
intro P
have hbdd' : BddBelow (Set.range fun Q :
{Q : TransportedArray 𝒳 //
RegularFiniteCellClass Q N k c epsilonBar 1 1 n} =>
regularCellCoverage Creg n Q) :=
⟨0, by
rintro y ⟨Q, rfl⟩
exact (hcoverageRange n Q Q.2).1⟩
exact (ciInf_le hbdd'
⟨P, P.2.toRegularFiniteCellClass hBar⟩).trans_eq (by
exact (hcoverageEq n P P.2).symm)
have hcovRnonneg : ∀ n, 0 ≤ covR n := by
intro n
obtain ⟨P0, hP0⟩ := hClass n
letI : Nonempty {P : TransportedArray 𝒳 //
RegularFiniteCellClass P N k c epsilonBar 1 1 n} :=
⟨⟨P0, hP0.toRegularFiniteCellClass hBar⟩⟩
change 0 ≤ coverageInfOrOne fun P : {P : TransportedArray 𝒳 //
RegularFiniteCellClass P N k c epsilonBar 1 1 n} =>
regularCellCoverage Creg n P
rw [coverageInfOrOne_of_nonempty_local]
exact le_ciInf fun P => (hcoverageRange n P P.2).1
have hcovFone : ∀ n, covF n ≤ 1 := by
intro n
obtain ⟨P0, hP0⟩ := hClass n
letI : Nonempty {P : TransportedArray 𝒳 //
FiniteCellClass P N k c epsilon n} := ⟨⟨P0, hP0⟩⟩
change (coverageInfOrOne fun P : {P : TransportedArray 𝒳 //
FiniteCellClass P N k c epsilon n} => finiteCellCoverage Cfin n P) ≤ 1
rw [coverageInfOrOne_of_nonempty_local]
have hbdd : BddBelow (Set.range fun P :
{P : TransportedArray 𝒳 //
FiniteCellClass P N k c epsilon n} =>
finiteCellCoverage Cfin n P) :=
⟨0, by
rintro y ⟨P, rfl⟩
change 0 ≤ finiteCellCoverage Cfin n (P : TransportedArray 𝒳)
rw [hcoverageEq n P P.2]
exact (hcoverageRange n P
(P.2.toRegularFiniteCellClass hBar)).1⟩
exact (ciInf_le hbdd ⟨P0, hP0⟩).trans
((hcoverageEq n P0 hP0).le.trans
(hcoverageRange n P0
(hP0.toRegularFiniteCellClass hBar)).2)
have hcovRbounded : IsBoundedUnder (· ≥ ·) atTop covR := by
change ∃ b, ∀ᶠ n in atTop, b ≤ covR n
exact ⟨0, Filter.Eventually.of_forall hcovRnonneg⟩
have hcovFcobounded : IsCoboundedUnder (· ≥ ·) atTop covF := by
change ∃ b, ∀ a, (∀ᶠ n in atTop, a ≤ covF n) → a ≤ b
refine ⟨1, fun a ha => ?_⟩
have haone := (ha.and (Filter.Eventually.of_forall hcovFone)).exists.choose_spec
exact haone.1.trans haone.2
have hfiniteHonest : FiniteCellHonest N k c epsilon alpha Cfin := by
refine ⟨hHonest.1, hHonest.2.1, hHonest.2.2.trans ?_⟩
exact Filter.liminf_le_liminf
(Filter.Eventually.of_forall hcovRows) hcovRbounded hcovFcobounded
refine ⟨hfiniteHonest, ?_⟩
intro t0
let rowF : ℕ → ℝ := fun n =>
⨆ P : {P : TransportedArray 𝒳 //
FiniteCellClass P N k c epsilon n ∧
t0 ≤ effectiveStrength P n},
finiteCellExpectedLength Cfin n P
let rowR : ℕ → ℝ := fun n =>
⨆ P : {P : TransportedArray 𝒳 //
RegularFiniteCellClass P N k c epsilonBar 1 1 n ∧
t0 ≤ effectiveStrength P n},
regularCellExpectedLength Creg n ⟨P.1, P.2.1⟩
have hrowRnonneg : ∀ n, 0 ≤ rowR n := by
intro n
cases isEmpty_or_nonempty
{P : TransportedArray 𝒳 //
RegularFiniteCellClass P N k c epsilonBar 1 1 n ∧
t0 ≤ effectiveStrength P n} with
| inl h =>
letI := h
simp [rowR]
| inr h =>
letI := h
let P := Classical.choice h
have hbdd : BddAbove (Set.range fun Q :
{Q : TransportedArray 𝒳 //
RegularFiniteCellClass Q N k c epsilonBar 1 1 n ∧
t0 ≤ effectiveStrength Q n} =>
regularCellExpectedLength Creg n ⟨Q.1, Q.2.1⟩) :=
⟨2, by
rintro y ⟨Q, rfl⟩
exact regularCellExpectedLength_le_two N k Creg n Q Q.2.1⟩
exact (integral_nonneg fun _ => ENNReal.toReal_nonneg).trans
(le_ciSup hbdd P)
have hrowRisk : ∀ n, rowF n ≤ rowR n := by
intro n
cases isEmpty_or_nonempty
{P : TransportedArray 𝒳 //
FiniteCellClass P N k c epsilon n ∧
t0 ≤ effectiveStrength P n} with
| inl h =>
letI := h
simpa [rowF] using hrowRnonneg n
| inr h =>
letI := h
apply ciSup_le
intro P
have hbdd : BddAbove (Set.range fun Q :
{Q : TransportedArray 𝒳 //
RegularFiniteCellClass Q N k c epsilonBar 1 1 n ∧
t0 ≤ effectiveStrength Q n} =>
regularCellExpectedLength Creg n ⟨Q.1, Q.2.1⟩) :=
⟨2, by
rintro y ⟨Q, rfl⟩
exact regularCellExpectedLength_le_two N k Creg n Q Q.2.1⟩
exact (hlengthEq n P P.2.1).le.trans
(le_ciSup hbdd
⟨P, P.2.1.toRegularFiniteCellClass hBar, P.2.2⟩)
have hrowFnonneg : ∀ n, 0 ≤ rowF n := by
intro n
cases isEmpty_or_nonempty
{P : TransportedArray 𝒳 //
FiniteCellClass P N k c epsilon n ∧
t0 ≤ effectiveStrength P n} with
| inl h =>
letI := h
simp [rowF]
| inr h =>
letI := h
let P := Classical.choice h
have hbdd : BddAbove (Set.range fun Q :
{Q : TransportedArray 𝒳 //
FiniteCellClass Q N k c epsilon n ∧
t0 ≤ effectiveStrength Q n} =>
finiteCellExpectedLength Cfin n Q) :=
⟨2, by
rintro y ⟨Q, rfl⟩
change finiteCellExpectedLength Cfin n
(Q : TransportedArray 𝒳) ≤ 2
rw [hlengthEq n Q Q.2.1]
exact regularCellExpectedLength_le_two N k Creg n Q
(Q.2.1.toRegularFiniteCellClass hBar)⟩
exact (integral_nonneg fun _ => ENNReal.toReal_nonneg).trans
(le_ciSup hbdd P)
have hrowRtwo : ∀ n, rowR n ≤ 2 := by
intro n
cases isEmpty_or_nonempty
{P : TransportedArray 𝒳 //
RegularFiniteCellClass P N k c epsilonBar 1 1 n ∧
t0 ≤ effectiveStrength P n} with
| inl h =>
letI := h
simp [rowR]
| inr h =>
letI := h
apply ciSup_le
intro P
exact regularCellExpectedLength_le_two N k Creg n P P.2.1
have hFcob : IsCoboundedUnder (· ≤ ·) atTop rowF := by
change ∃ b, ∀ a, (∀ᶠ n in atTop, rowF n ≤ a) → b ≤ a
refine ⟨0, fun a ha => ?_⟩
have h := ((Filter.Eventually.of_forall hrowFnonneg).and ha).exists.choose_spec
exact h.1.trans h.2
have hRbdd : IsBoundedUnder (· ≤ ·) atTop rowR := by
change ∃ b, ∀ᶠ n in atTop, rowR n ≤ b
exact ⟨2, Filter.Eventually.of_forall hrowRtwo⟩
exact Filter.limsup_le_limsup
(Filter.Eventually.of_forall hrowRisk) hFcob hRbdd
Helpers.FrontierOrder 9 declarations
theorem coverageInfOrOne_of_nonempty
On an inhabited row, coverageInfOrOne is the ordinary infimum.
Formal statement
hypotheses
ι :
Sort*
Nonempty ι
f :
ι → ℝ
conclusion
coverageInfOrOne f = ⨅ i, f i
Proof (Lean source)
lemma coverageInfOrOne_of_nonempty {ι : Sort*} [Nonempty ι] (f : ι → ℝ) :
coverageInfOrOne f = ⨅ i, f i := by
exact coverageInfOrOne_of_nonempty f
theorem coverageInfOrOne_of_isEmpty
On an empty row, coverageInfOrOne has its declared vacuous value.
Formal statement
Proof (Lean source)
lemma coverageInfOrOne_of_isEmpty {ι : Sort*} [IsEmpty ι] (f : ι → ℝ) :
coverageInfOrOne f = 1 := by
exact coverageInfOrOne_of_isEmpty f
theorem oracleCoverageClassInf_le_canonical
The full representative-fiber coverage infimum is bounded by evaluation at the canonical representative of any particular class member.
Formal statement
hypotheses
conclusion
oracleCoverageClassInf N k c epsilon C n ≤ oracleCoverage C P n
Proof (Lean source)
lemma oracleCoverageClassInf_le_canonical
(C : OracleProcedure 𝒳 N k c epsilon) (P : TransportedArray 𝒳) (n : ℕ)
(hP : TransportedIVClass P N k c epsilon n) :
oracleCoverageClassInf N k c epsilon C n ≤ oracleCoverage C P n := by
letI : Nonempty
{Q : TransportedArray 𝒳 //
TransportedIVClass Q N k c epsilon n} :=
⟨⟨P, hP⟩⟩
have hbounded :
BddBelow (range (fun Q :
{Q : TransportedArray 𝒳 //
TransportedIVClass Q N k c epsilon n} =>
oracleCoverage C Q n)) :=
⟨0, by
rintro y ⟨Q, rfl⟩
exact ENNReal.toReal_nonneg⟩
rw [oracleCoverageClassInf, coverageInfOrOne_of_nonempty]
exact ciInf_le hbounded ⟨P, hP⟩
theorem oracleCoverageClassInf_nonneg
Every rowwise full-fiber coverage infimum is nonnegative.
Formal statement
hypotheses
C :
OracleProcedure 𝒳 N k c epsilon
n :
ℕ
h :
∃ P : TransportedArray 𝒳, TransportedIVClass P N k c epsilon n
conclusion
0 ≤ oracleCoverageClassInf N k c epsilon C n
Proof (Lean source)
lemma oracleCoverageClassInf_nonneg
(C : OracleProcedure 𝒳 N k c epsilon) (n : ℕ)
(h : ∃ P : TransportedArray 𝒳,
TransportedIVClass P N k c epsilon n) :
0 ≤ oracleCoverageClassInf N k c epsilon C n := by
let P := h.choose
letI : Nonempty
{Q : TransportedArray 𝒳 //
TransportedIVClass Q N k c epsilon n} :=
⟨⟨P, h.choose_spec⟩⟩
rw [oracleCoverageClassInf, coverageInfOrOne_of_nonempty]
exact le_ciInf (fun q => ENNReal.toReal_nonneg)
theorem oracleExpectedLength_le_frontierRiskRow
The full representative-fiber risk row dominates canonical evaluation at every member above the strength threshold.
Formal statement
hypotheses
C :
OracleProcedure 𝒳 N k c epsilon
P :
n :
ℕ
t0 :
ℝ
hP :
TransportedIVClass P N k c epsilon n
ht :
t0 ≤ effectiveStrength P n
conclusion
oracleExpectedLength C P n ≤ frontierRiskRow N k c epsilon C t0 n
Proof (Lean source)
lemma oracleExpectedLength_le_frontierRiskRow
(C : OracleProcedure 𝒳 N k c epsilon) (P : TransportedArray 𝒳) (n : ℕ) (t0 : ℝ)
(hP : TransportedIVClass P N k c epsilon n)
(ht : t0 ≤ effectiveStrength P n) :
oracleExpectedLength C P n ≤ frontierRiskRow N k c epsilon C t0 n := by
have hsetLength_le_two : ∀ A : Set ℝ, setLength A ≤ 2 := by
intro A
unfold setLength parameterSpace
calc
(volume (A ∩ Icc (-1 : ℝ) 1)).toReal ≤
(volume (Icc (-1 : ℝ) 1)).toReal :=
ENNReal.toReal_mono (by simp [Real.volume_Icc])
(measure_mono Set.inter_subset_right)
_ = 2 := by
rw [Real.volume_Icc, ENNReal.toReal_ofReal (by norm_num)]
norm_num
have hbounded :
BddAbove (range (fun Q :
{Q : TransportedArray 𝒳 //
TransportedIVClass Q N k c epsilon n ∧
t0 ≤ effectiveStrength Q n} =>
oracleExpectedLength C Q n)) :=
⟨2, by
rintro y ⟨Q, rfl⟩
letI : IsProbabilityMeasure (sourceObsLaw Q.1 n) :=
Q.property.1.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (targetXLaw Q.1 n) :=
Q.property.1.twoSampleArray.2.2.1 n
letI : IsProbabilityMeasure (twoSampleLaw Q.1 N n) := by
unfold twoSampleLaw
infer_instance
exact (integral_mono_of_nonneg
(Filter.Eventually.of_forall fun _ => ENNReal.toReal_nonneg)
(integrable_const 2)
(Filter.Eventually.of_forall fun s =>
hsetLength_le_two
(oracleSet C Q.1 n s))).trans_eq (by simp)⟩
unfold frontierRiskRow
exact le_ciSup hbounded ⟨P, hP, ht⟩
theorem oracleExpectedLength_le_finiteCellOracleRiskRow
Canonical evaluation at a finite-cell row is bounded by the full representative-fiber finite-cell risk row.
Formal statement
hypotheses
C :
OracleProcedure 𝒳 N k c epsilon
P :
n :
ℕ
t0 :
ℝ
hP :
FiniteCellClass P N k c epsilon n
ht :
t0 ≤ effectiveStrength P n
conclusion
oracleExpectedLength C P n ≤ finiteCellOracleRiskRow N k c epsilon C t0 n
Proof (Lean source)
lemma oracleExpectedLength_le_finiteCellOracleRiskRow
(C : OracleProcedure 𝒳 N k c epsilon) (P : TransportedArray 𝒳) (n : ℕ) (t0 : ℝ)
(hP : FiniteCellClass P N k c epsilon n)
(ht : t0 ≤ effectiveStrength P n) :
oracleExpectedLength C P n ≤
finiteCellOracleRiskRow N k c epsilon C t0 n := by
have hsetLength_le_two : ∀ A : Set ℝ, setLength A ≤ 2 := by
intro A
unfold setLength parameterSpace
calc
(volume (A ∩ Icc (-1 : ℝ) 1)).toReal ≤
(volume (Icc (-1 : ℝ) 1)).toReal :=
ENNReal.toReal_mono (by simp [Real.volume_Icc])
(measure_mono Set.inter_subset_right)
_ = 2 := by
rw [Real.volume_Icc, ENNReal.toReal_ofReal (by norm_num)]
norm_num
have hbounded :
BddAbove (range (fun Q :
{Q : TransportedArray 𝒳 //
FiniteCellClass Q N k c epsilon n ∧
t0 ≤ effectiveStrength Q n} =>
oracleExpectedLength C Q n)) :=
⟨2, by
rintro y ⟨Q, rfl⟩
letI : IsProbabilityMeasure (sourceObsLaw Q.1 n) :=
Q.property.1.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (targetXLaw Q.1 n) :=
Q.property.1.twoSampleArray.2.2.1 n
letI : IsProbabilityMeasure (twoSampleLaw Q.1 N n) := by
unfold twoSampleLaw
infer_instance
exact (integral_mono_of_nonneg
(Filter.Eventually.of_forall fun _ => ENNReal.toReal_nonneg)
(integrable_const 2)
(Filter.Eventually.of_forall fun s =>
hsetLength_le_two
(oracleSet C Q.1 n s))).trans_eq (by simp)⟩
unfold finiteCellOracleRiskRow
exact le_ciSup hbounded ⟨P, hP, ht⟩
theorem finiteCellOracleRiskRow_le_two
Full-fiber finite-cell oracle risk rows retain the diameter-two bound.
Formal statement
hypotheses
conclusion
finiteCellOracleRiskRow N k c epsilon C t0 n ≤ 2
Proof (Lean source)
lemma finiteCellOracleRiskRow_le_two
(C : OracleProcedure 𝒳 N k c epsilon) (n : ℕ) (t0 : ℝ) :
finiteCellOracleRiskRow N k c epsilon C t0 n ≤ 2 := by
cases isEmpty_or_nonempty
{P : TransportedArray 𝒳 //
FiniteCellClass P N k c epsilon n ∧
t0 ≤ effectiveStrength P n} with
| inl h =>
letI := h
simp [finiteCellOracleRiskRow]
| inr h =>
letI := h
unfold finiteCellOracleRiskRow
refine ciSup_le (fun q => ?_)
letI : IsProbabilityMeasure (sourceObsLaw q.1 n) :=
q.property.1.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (targetXLaw q.1 n) :=
q.property.1.twoSampleArray.2.2.1 n
letI : IsProbabilityMeasure (twoSampleLaw q.1 N n) := by
unfold twoSampleLaw
infer_instance
have hsetLength_le_two : ∀ A : Set ℝ, setLength A ≤ 2 := by
intro A
unfold setLength parameterSpace
calc
(volume (A ∩ Icc (-1 : ℝ) 1)).toReal ≤
(volume (Icc (-1 : ℝ) 1)).toReal :=
ENNReal.toReal_mono (by simp [Real.volume_Icc])
(measure_mono Set.inter_subset_right)
_ = 2 := by
rw [Real.volume_Icc, ENNReal.toReal_ofReal (by norm_num)]
norm_num
exact (integral_mono_of_nonneg
(Filter.Eventually.of_forall fun _ => ENNReal.toReal_nonneg)
(integrable_const 2)
(Filter.Eventually.of_forall fun s =>
hsetLength_le_two
(oracleSet C q.1 n s))).trans_eq (by simp)
theorem fixedGeometry
A globally honest procedure restricts to every fixed geometry because the geometry's declared weight is itself a member of each row's full version fiber.
Formal statement
hypotheses
C :
OracleProcedure 𝒳 N k c epsilon
hC :
OracleHonest N k c epsilon alpha C
g :
Geometry 𝒳
t0 :
ℝ
hc :
0 < c
hepsilon :
0 < epsilon ∧ epsilon < 1 / 2
ht0 :
0 < t0
hg :
AdmissibleGeometry g k epsilon
hkPos :
∀ n, 0 < k n
conclusion
∃ D : OracleProcedure 𝒳 N k c epsilon,
FixedGeometryOracleHonest N k c epsilon alpha ⟨g, hg⟩ D ∧
fixedGeometryRisk N k c epsilon g D t0 ≤ frontierRiskTotal N k c epsilon C t0
Proof (Lean source)
lemma OracleHonest.fixedGeometry
(C : OracleProcedure 𝒳 N k c epsilon)
(hC : OracleHonest N k c epsilon alpha C)
(g : Geometry 𝒳) (t0 : ℝ)
(hc : 0 < c) (hepsilon : 0 < epsilon ∧ epsilon < 1 / 2) (ht0 : 0 < t0)
(hg : AdmissibleGeometry g k epsilon)
(hN : Tendsto (fun n : ℕ => (N n : ℝ) / (n : ℝ)) atTop (𝓝 c))
(hkPos : ∀ n, 0 < k n)
(hkInf : Tendsto (fun n : ℕ => (k n : ℝ)) atTop atTop)
(hkRoot : Tendsto (fun n : ℕ => (k n : ℝ) / sqrt n) atTop (𝓝 0)) :
∃ D : OracleProcedure 𝒳 N k c epsilon,
FixedGeometryOracleHonest N k c epsilon alpha ⟨g, hg⟩ D ∧
fixedGeometryRisk N k c epsilon g D t0 ≤
frontierRiskTotal N k c epsilon C t0 := by
classical
let D : OracleProcedure 𝒳 N k c epsilon := C
have hgeometryWeightInput_eq (n : ℕ) :
(geometryWeightInput g n : 𝒳 → ℝ) = g.weight n := by
simp [geometryWeightInput, fun x => (hg.2.2.2.2.2.1 n x).1]
let geometryVersion (n : ℕ) (P : TransportedArray 𝒳)
(hP : fixedGeometrySlice P g N k c epsilon n) :
TransportWeightVersion P n :=
⟨geometryWeightInput g n,
by simpa [hgeometryWeightInput_eq n] using g.weight_measurable n,
by simpa only [hgeometryWeightInput_eq n] using hP.2.2.2.1.symm⟩
have hset_eq (n : ℕ) (P : TransportedArray 𝒳)
(hP : fixedGeometrySlice P g N k c epsilon n)
(s : TwoSample 𝒳 n (N n)) :
fixedGeometryOracleSet D g n s =
oracleSetAtWeight C P n (geometryVersion n P hP) s := by
simp only [D, fixedGeometryOracleSet, oracleSetAtWeight,
geometryVersion, hP.2.2.2.2]
have hcoverage_eq (n : ℕ) (P : TransportedArray 𝒳)
(hP : fixedGeometrySlice P g N k c epsilon n) :
fixedGeometryOracleCoverage D g P n =
oracleCoverage C P n := by
unfold fixedGeometryOracleCoverage oracleCoverage
congr 1
apply measure_congr
filter_upwards [
C.weightAEInvariant n P hP.1 (geometryVersion n P hP)
(canonicalTransportWeightVersion P n)
] with s hs
simp only [hset_eq n P hP, oracleSet, oracleSetAtWeight]
exact congrArg (fun A : Set ℝ => targetCACE P n ∈ A) hs
have hexpectedLength_eq (n : ℕ) (P : TransportedArray 𝒳)
(hP : fixedGeometrySlice P g N k c epsilon n) :
fixedGeometryOracleExpectedLength D g P n =
oracleExpectedLength C P n := by
unfold fixedGeometryOracleExpectedLength oracleExpectedLength
apply integral_congr_ae
filter_upwards [
C.weightAEInvariant n P hP.1 (geometryVersion n P hP)
(canonicalTransportWeightVersion P n)
] with s hs
simp only [hset_eq n P hP, oracleSet, oracleSetAtWeight]
exact congrArg setLength hs
have hcoverage_nonneg (E : OracleProcedure 𝒳 N k c epsilon)
(P : TransportedArray 𝒳) (n : ℕ) :
0 ≤ oracleCoverage E P n := by
exact ENNReal.toReal_nonneg
have hcoverage_le_one (E : OracleProcedure 𝒳 N k c epsilon)
(P : TransportedArray 𝒳) (n : ℕ)
(hP : TransportedIVClass P N k c epsilon n) :
oracleCoverage E P n ≤ 1 := by
letI : IsProbabilityMeasure (sourceObsLaw P n) :=
hP.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (targetXLaw P n) :=
hP.twoSampleArray.2.2.1 n
letI : IsProbabilityMeasure (twoSampleLaw P N n) := by
unfold twoSampleLaw
infer_instance
exact measureReal_le_one
have hsetLength_nonneg : ∀ A : Set ℝ, 0 ≤ setLength A :=
fun _ => ENNReal.toReal_nonneg
have hsetLength_le_two : ∀ A : Set ℝ, setLength A ≤ 2 := by
intro A
unfold setLength parameterSpace
calc
(volume (A ∩ Icc (-1 : ℝ) 1)).toReal ≤
(volume (Icc (-1 : ℝ) 1)).toReal :=
ENNReal.toReal_mono (by simp [Real.volume_Icc])
(measure_mono Set.inter_subset_right)
_ = 2 := by
rw [Real.volume_Icc, ENNReal.toReal_ofReal (by norm_num)]
norm_num
have hexpectedLength_le_two (E : OracleProcedure 𝒳 N k c epsilon)
(P : TransportedArray 𝒳) (n : ℕ)
(hP : TransportedIVClass P N k c epsilon n) :
oracleExpectedLength E P n ≤ 2 := by
letI : IsProbabilityMeasure (sourceObsLaw P n) :=
hP.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (targetXLaw P n) :=
hP.twoSampleArray.2.2.1 n
letI : IsProbabilityMeasure (twoSampleLaw P N n) := by
unfold twoSampleLaw
infer_instance
exact (integral_mono_of_nonneg
(Filter.Eventually.of_forall fun _ => ENNReal.toReal_nonneg)
(integrable_const 2)
(Filter.Eventually.of_forall fun s =>
hsetLength_le_two (oracleSet E P n s))).trans_eq (by simp)
have hinhabited :
∀ᶠ n in atTop, ∃ P : TransportedArray 𝒳,
fixedGeometrySlice P g N k c epsilon n ∧
t0 ≤ effectiveStrength P n :=
fixedGeometrySlice_eventually_inhabited g N k c epsilon t0 hc
hepsilon ht0 hg hN hkPos hkInf hkRoot
have hcoverage_rows :
∀ᶠ n in atTop,
oracleCoverageClassInf N k c epsilon C n ≤
coverageInfOrOne (fun P : {P : TransportedArray 𝒳 //
fixedGeometrySlice P g N k c epsilon n} =>
fixedGeometryOracleCoverage D g P n) := by
filter_upwards [hinhabited] with n hn
obtain ⟨P₀, hP₀, _ht₀⟩ := hn
letI : Nonempty {P : TransportedArray 𝒳 //
fixedGeometrySlice P g N k c epsilon n} :=
⟨⟨P₀, hP₀⟩⟩
letI : Nonempty {P : TransportedArray 𝒳 //
TransportedIVClass P N k c epsilon n} :=
⟨⟨P₀, hP₀.1⟩⟩
rw [coverageInfOrOne_of_nonempty]
refine le_ciInf (fun P => ?_)
have hglobal_bdd :
BddBelow (range (fun Q :
{Q : TransportedArray 𝒳 //
TransportedIVClass Q N k c epsilon n} =>
oracleCoverage C Q n)) :=
⟨0, by
rintro y ⟨Q, rfl⟩
exact hcoverage_nonneg C Q n⟩
rw [oracleCoverageClassInf, coverageInfOrOne_of_nonempty]
exact (ciInf_le hglobal_bdd
⟨P, P.property.1⟩).trans_eq
(hcoverage_eq n P P.property).symm
have hglobal_coverage_nonneg :
∀ᶠ n in atTop,
0 ≤ oracleCoverageClassInf N k c epsilon C n := by
filter_upwards [hinhabited] with n hn
obtain ⟨P₀, hP₀, _ht₀⟩ := hn
letI : Nonempty
{P : TransportedArray 𝒳 //
TransportedIVClass P N k c epsilon n} :=
⟨⟨P₀, hP₀.1⟩⟩
rw [oracleCoverageClassInf, coverageInfOrOne_of_nonempty]
exact le_ciInf (fun Q => hcoverage_nonneg C Q n)
have hslice_coverage_le_one :
∀ᶠ n in atTop,
coverageInfOrOne (fun P : {P : TransportedArray 𝒳 //
fixedGeometrySlice P g N k c epsilon n} =>
fixedGeometryOracleCoverage D g P n) ≤ 1 := by
filter_upwards [hinhabited] with n hn
obtain ⟨P₀, hP₀, _ht₀⟩ := hn
letI : Nonempty {P : TransportedArray 𝒳 //
fixedGeometrySlice P g N k c epsilon n} :=
⟨⟨P₀, hP₀⟩⟩
rw [coverageInfOrOne_of_nonempty]
have hslice_bdd :
BddBelow (range (fun P :
{P : TransportedArray 𝒳 //
fixedGeometrySlice P g N k c epsilon n} =>
fixedGeometryOracleCoverage D g P n)) :=
⟨0, by
rintro y ⟨P, rfl⟩
change 0 ≤ fixedGeometryOracleCoverage D g P n
rw [hcoverage_eq n P P.property]
exact hcoverage_nonneg C P n⟩
exact (ciInf_le hslice_bdd ⟨P₀, hP₀⟩).trans
((hcoverage_eq n P₀ hP₀).le.trans
(hcoverage_le_one C P₀ n hP₀.1))
have hglobal_coverage_bounded :
IsBoundedUnder (· ≥ ·) atTop
(oracleCoverageClassInf N k c epsilon C) := by
change ∃ b, ∀ᶠ n in atTop, b ≤ _
exact ⟨0, hglobal_coverage_nonneg⟩
have hslice_coverage_cobounded :
IsCoboundedUnder (· ≥ ·) atTop
(fun n => coverageInfOrOne (fun P : {P : TransportedArray 𝒳 //
fixedGeometrySlice P g N k c epsilon n} =>
fixedGeometryOracleCoverage D g P n)) := by
change ∃ b, ∀ a, (∀ᶠ n in atTop, a ≤ _) → a ≤ b
refine ⟨1, fun a ha => ?_⟩
have ha_one :=
(ha.and hslice_coverage_le_one).exists.choose_spec
exact ha_one.1.trans ha_one.2
have hD_honest :
FixedGeometryOracleHonest N k c epsilon alpha ⟨g, hg⟩ D := by
refine ⟨hC.1, hC.2.1, hC.2.2.trans ?_⟩
exact Filter.liminf_le_liminf hcoverage_rows
hglobal_coverage_bounded hslice_coverage_cobounded
have hfixed_risk_rows_nonneg : ∀ n,
0 ≤ (⨆ P : {P : TransportedArray 𝒳 //
fixedGeometrySlice P g N k c epsilon n ∧
t0 ≤ effectiveStrength P n},
fixedGeometryOracleExpectedLength D g P n) := by
intro n
cases isEmpty_or_nonempty
{P : TransportedArray 𝒳 //
fixedGeometrySlice P g N k c epsilon n ∧
t0 ≤ effectiveStrength P n} with
| inl h =>
letI := h
calc
0 ≤ (0 : ℝ) := le_rfl
_ = (⨆ P : {P : TransportedArray 𝒳 //
fixedGeometrySlice P g N k c epsilon n ∧
t0 ≤ effectiveStrength P n},
fixedGeometryOracleExpectedLength D g P n) :=
((iSup_of_empty' _).trans Real.sSup_empty).symm
| inr h =>
let P := Classical.choice h
refine le_trans ?_ (le_ciSup ?_ P)
· exact integral_nonneg (fun s =>
hsetLength_nonneg (fixedGeometryOracleSet D g n s))
· exact ⟨2, by
rintro y ⟨Q, rfl⟩
change fixedGeometryOracleExpectedLength D g Q n ≤ 2
rw [hexpectedLength_eq n Q Q.property.1]
exact hexpectedLength_le_two C Q n Q.property.1.1⟩
have hglobal_risk_rows_le_two : ∀ n,
frontierRiskRow N k c epsilon C t0 n ≤ 2 := by
intro n
cases isEmpty_or_nonempty
{P : TransportedArray 𝒳 //
TransportedIVClass P N k c epsilon n ∧
t0 ≤ effectiveStrength P n} with
| inl h =>
letI := h
unfold frontierRiskRow
calc
(⨆ P : {P : TransportedArray 𝒳 //
TransportedIVClass P N k c epsilon n ∧
t0 ≤ effectiveStrength P n},
oracleExpectedLength C P n) =
0 := (iSup_of_empty' _).trans Real.sSup_empty
_ ≤ 2 := by norm_num
| inr h =>
letI := h
unfold frontierRiskRow
refine ciSup_le (fun q =>
hexpectedLength_le_two C q n q.property.1)
have hrisk_rows :
∀ᶠ n in atTop,
(⨆ P : {P : TransportedArray 𝒳 //
fixedGeometrySlice P g N k c epsilon n ∧
t0 ≤ effectiveStrength P n},
-- … truncated; follow the source link for the rest …
Consequently the conditional minimax value is bounded by the risk of each globally honest procedure, without identifying a.e.-equal weight versions pointwise.
Formal statement
hypotheses
g :
Geometry 𝒳
C :
OracleProcedure 𝒳 N k c epsilon
hC :
OracleHonest N k c epsilon alpha C
t0 :
ℝ
hc :
0 < c
hepsilon :
0 < epsilon ∧ epsilon < 1 / 2
ht0 :
0 < t0
hg :
AdmissibleGeometry g k epsilon
hkPos :
∀ n, 0 < k n
conclusion
fixedGeometryValueTotal N k c epsilon alpha ⟨g, hg⟩ t0
≤ frontierRiskTotal N k c epsilon C t0
Proof (Lean source)
lemma fixedGeometryValue_le_frontierRisk
(g : Geometry 𝒳) (C : OracleProcedure 𝒳 N k c epsilon)
(hC : OracleHonest N k c epsilon alpha C) (t0 : ℝ)
(hc : 0 < c) (hepsilon : 0 < epsilon ∧ epsilon < 1 / 2) (ht0 : 0 < t0)
(hg : AdmissibleGeometry g k epsilon)
(hN : Tendsto (fun n : ℕ => (N n : ℝ) / (n : ℝ)) atTop (𝓝 c))
(hkPos : ∀ n, 0 < k n)
(hkInf : Tendsto (fun n : ℕ => (k n : ℝ)) atTop atTop)
(hkRoot : Tendsto (fun n : ℕ => (k n : ℝ) / sqrt n) atTop (𝓝 0)) :
fixedGeometryValueTotal N k c epsilon alpha ⟨g, hg⟩ t0 ≤
frontierRiskTotal N k c epsilon C t0 := by
classical
obtain ⟨D, hD, hDC⟩ :=
OracleHonest.fixedGeometry (N := N) (k := k) (c := c)
(epsilon := epsilon) (alpha := alpha) C hC g t0
hc hepsilon ht0 hg hN hkPos hkInf hkRoot
have hsetLength_nonneg : ∀ A : Set ℝ, 0 ≤ setLength A :=
fun _ => ENNReal.toReal_nonneg
have hsetLength_le_two : ∀ A : Set ℝ, setLength A ≤ 2 := by
intro A
unfold setLength parameterSpace
calc
(volume (A ∩ Icc (-1 : ℝ) 1)).toReal ≤
(volume (Icc (-1 : ℝ) 1)).toReal :=
ENNReal.toReal_mono (by simp [Real.volume_Icc])
(measure_mono Set.inter_subset_right)
_ = 2 := by
rw [Real.volume_Icc, ENNReal.toReal_ofReal (by norm_num)]
norm_num
have hrisk_nonneg : ∀ E : OracleProcedure 𝒳 N k c epsilon,
0 ≤ fixedGeometryRisk N k c epsilon g E t0 := by
intro E
letI : (atTop : Filter ℕ).NeBot := inferInstance
have hrow_nonneg : ∀ n,
0 ≤ (⨆ P : {P : TransportedArray 𝒳 //
fixedGeometrySlice P g N k c epsilon n ∧
t0 ≤ effectiveStrength P n},
fixedGeometryOracleExpectedLength E g P n) := by
intro n
cases isEmpty_or_nonempty
{P : TransportedArray 𝒳 //
fixedGeometrySlice P g N k c epsilon n ∧
t0 ≤ effectiveStrength P n} with
| inl h =>
simp
| inr h =>
let P := Classical.choice h
refine le_trans ?_ (le_ciSup ?_ P)
· exact integral_nonneg (fun s =>
hsetLength_nonneg (fixedGeometryOracleSet E g n s))
· exact ⟨2, by
rintro y ⟨Q, rfl⟩
change fixedGeometryOracleExpectedLength E g Q n ≤ 2
letI : IsProbabilityMeasure
(sourceObsLaw (Q : TransportedArray 𝒳) n) :=
Q.property.1.1.twoSampleArray.2.1 n
letI : IsProbabilityMeasure
(targetXLaw (Q : TransportedArray 𝒳) n) :=
Q.property.1.1.twoSampleArray.2.2.1 n
letI : IsProbabilityMeasure
(twoSampleLaw (Q : TransportedArray 𝒳) N n) := by
unfold twoSampleLaw
infer_instance
exact (integral_mono_of_nonneg
(Filter.Eventually.of_forall fun _ => ENNReal.toReal_nonneg)
(integrable_const 2)
(Filter.Eventually.of_forall fun s =>
hsetLength_le_two
(fixedGeometryOracleSet E g n s))).trans_eq (by simp)⟩
have hrow_le_two : ∀ n,
(⨆ P : {P : TransportedArray 𝒳 //
fixedGeometrySlice P g N k c epsilon n ∧
t0 ≤ effectiveStrength P n},
fixedGeometryOracleExpectedLength E g P n) ≤ 2 := by
intro n
cases isEmpty_or_nonempty
{P : TransportedArray 𝒳 //
fixedGeometrySlice P g N k c epsilon n ∧
t0 ≤ effectiveStrength P n} with
| inl h =>
simp
| inr h =>
refine ciSup_le (fun P => ?_)
letI : IsProbabilityMeasure
(sourceObsLaw (P : TransportedArray 𝒳) n) :=
P.property.1.1.twoSampleArray.2.1 n
letI : IsProbabilityMeasure
(targetXLaw (P : TransportedArray 𝒳) n) :=
P.property.1.1.twoSampleArray.2.2.1 n
letI : IsProbabilityMeasure
(twoSampleLaw (P : TransportedArray 𝒳) N n) := by
unfold twoSampleLaw
infer_instance
exact (integral_mono_of_nonneg
(Filter.Eventually.of_forall fun _ => ENNReal.toReal_nonneg)
(integrable_const 2)
(Filter.Eventually.of_forall fun s =>
hsetLength_le_two
(fixedGeometryOracleSet E g n s))).trans_eq (by simp)
have hzero_cobounded :
IsCoboundedUnder (· ≤ ·) atTop (fun _ : ℕ => (0 : ℝ)) := by
change ∃ b, ∀ a, (∀ᶠ _n : ℕ in atTop, (0 : ℝ) ≤ a) → b ≤ a
exact ⟨0, fun a ha => ha.exists.choose_spec⟩
have hrow_bounded :
IsBoundedUnder (· ≤ ·) atTop
(fun n => ⨆ P : {P : TransportedArray 𝒳 //
fixedGeometrySlice P g N k c epsilon n ∧
t0 ≤ effectiveStrength P n},
fixedGeometryOracleExpectedLength E g P n) := by
change ∃ b, ∀ᶠ n in atTop, _ ≤ b
exact ⟨2, Filter.Eventually.of_forall hrow_le_two⟩
change 0 ≤ limsup _ atTop
rw [← Filter.limsup_const (f := (atTop : Filter ℕ)) (0 : ℝ)]
exact Filter.limsup_le_limsup
(Filter.Eventually.of_forall hrow_nonneg)
hzero_cobounded hrow_bounded
have hvalues_bdd :
BddBelow (range (fun E :
{E : OracleProcedure 𝒳 N k c epsilon //
FixedGeometryOracleHonest N k c epsilon alpha ⟨g, hg⟩ E} =>
fixedGeometryRisk N k c epsilon g E.1 t0)) :=
⟨0, by
rintro y ⟨E, rfl⟩
exact hrisk_nonneg E⟩
exact (ciInf_le hvalues_bdd ⟨D, hD⟩).trans hDC
Helpers.InversionRisk 9 declarations
def affineInversionSet
The model-free confidence set obtained by inverting one affine inequality.
Definition (Lean source)
noncomputable abbrev affineInversionSet (A B r : ℝ) : Set ℝ :=
affineInversionSet parameterSpace A B r
theorem inversionHandle_eq_affineInversionSet
Score inversion is an instance of model-free affine inversion.
Formal statement
hypotheses
conclusion
inversionHandle weight e n L sample
= affineInversionSet (scoreOutcomeMean weight e n sample) (scoreReceiptMean weight e n sample) (L * sqrt (empiricalKish weight n sample / n))
Proof (Lean source)
lemma inversionHandle_eq_affineInversionSet
{𝒳 : Type*} [MeasurableSpace 𝒳]
(weight e : 𝒳 → ℝ) (n : ℕ) (L : ℝ) (sample : SourceSample 𝒳 n) :
inversionHandle weight e n L sample =
affineInversionSet (scoreOutcomeMean weight e n sample)
(scoreReceiptMean weight e n sample)
(L * sqrt (empiricalKish weight n sample / n)) := by
rfl
theorem regularCellInversion_eq_affineInversionSet
On the nondegenerate target-sample branch, regular-cell inversion is model-free affine inversion.
Formal statement
hypotheses
conclusion
regularCellInversion q e L source target
= affineInversionSet (crossAverage q source target (fun o => oracleInstrumentScore e o * o.2.2.2)) (crossAverage q source target (fun o => oracleInstrumentScore e o * boolReal o.2.2.1)) (L * sqrt ((1 + collisionScale q target) / n))
Proof (Lean source)
lemma regularCellInversion_eq_affineInversionSet
{𝒳 : Type*} [MeasurableSpace 𝒳]
(q e : 𝒳 → ℝ) {n N : ℕ} (L : ℝ)
(source : SourceSample 𝒳 n) (target : TargetSample 𝒳 N)
(hN : 2 ≤ N) :
regularCellInversion q e L source target =
affineInversionSet
(crossAverage q source target
(fun o => oracleInstrumentScore e o * o.2.2.2))
(crossAverage q source target
(fun o => oracleInstrumentScore e o * boolReal o.2.2.1))
(L * sqrt ((1 + collisionScale q target) / n)) := by
simp [regularCellInversion, affineInversionSet,
affineInversionSet, Nat.not_lt.mpr hN]
theorem regularCellInversion_eq_parameterSpace
On the degenerate target-sample branch, regular-cell inversion returns the whole parameter space.
Formal statement
hypotheses
conclusion
regularCellInversion q e L source target = parameterSpace
Proof (Lean source)
lemma regularCellInversion_eq_parameterSpace
{𝒳 : Type*} [MeasurableSpace 𝒳]
(q e : 𝒳 → ℝ) {n N : ℕ} (L : ℝ)
(source : SourceSample 𝒳 n) (target : TargetSample 𝒳 N)
(hN : N < 2) :
regularCellInversion q e L source target = parameterSpace := by
simp [regularCellInversion, hN]
theorem affineInversionSet_length_le
Affine inversion has length at most twice its radius divided by its nonzero slope, as well as at most the diameter of the parameter space.
Formal statement
Proof (Lean source)
lemma affineInversionSet_length_le (A B r : ℝ) (hB : B ≠ 0) (hr : 0 ≤ r) :
setLength (affineInversionSet A B r) ≤ min 2 (2 * r / |B|) := by
change restrictedSetVolume parameterSpace
(affineInversionSet parameterSpace A B r) ≤
min 2 (2 * r / |B|)
have hvol : (volume parameterSpace).toReal = 2 := by
simp [parameterSpace, Real.volume_Icc, ENNReal.toReal_ofReal]
norm_num
have h := affineInversionSet_restrictedVolume_le
parameterSpace (by simp [parameterSpace, Real.volume_Icc])
A B r hB hr
rw [hvol] at h
exact h
theorem affineInversionSet_length_le_two
Affine inversion is always bounded by the diameter of the parameter space.
Formal statement
Proof (Lean source)
lemma affineInversionSet_length_le_two (A B r : ℝ) :
setLength (affineInversionSet A B r) ≤ 2 := by
change restrictedSetVolume parameterSpace
(affineInversionSet parameterSpace A B r) ≤ 2
have hvol : (volume parameterSpace).toReal = 2 := by
simp [parameterSpace, Real.volume_Icc, ENNReal.toReal_ofReal]
norm_num
have h := affineInversionSet_restrictedVolume_le_region
parameterSpace (by simp [parameterSpace, Real.volume_Icc]) A B r
rw [hvol] at h
exact h
theorem expectedLength_affineInversion_le
Expected affine-inversion length is controlled by the mean radius and the probability that the random slope is less than half its positive mean target.
Formal statement
hypotheses
Omega :
Type*
MeasurableSpace Omega
Q :
Measure Omega
A B K :
Omega → ℝ
n :
ℕ
L mu Kbar q :
ℝ
hL :
0 ≤ L
hmu :
0 < mu
hn :
0 < n
hK :
∀ w, 0 ≤ K w
hKint :
Integrable K Q
hKbar :
(∫ w, K w ∂Q) ≤ Kbar
hbad :
(Q {w | mu / 2 < |B w - mu|}).toReal ≤ q
Proof (Lean source)
lemma expectedLength_affineInversion_le
{Omega : Type*} [MeasurableSpace Omega]
(Q : Measure Omega) [IsProbabilityMeasure Q]
(A B K : Omega → ℝ) (n : ℕ) (L mu Kbar q : ℝ)
(hL : 0 ≤ L) (hmu : 0 < mu) (hn : 0 < n)
(hK : ∀ w, 0 ≤ K w)
(hKint : Integrable K Q)
(hKbar : (∫ w, K w ∂Q) ≤ Kbar)
(hbad : (Q {w | mu / 2 < |B w - mu|}).toReal ≤ q) :
(∫ w, setLength (affineInversionSet (A w) (B w)
(L * sqrt (K w / n))) ∂Q) ≤
4 * L * sqrt (Kbar / n) / mu + 2 * q := by
change (∫ w, restrictedSetVolume parameterSpace
(affineInversionSet parameterSpace (A w) (B w)
(L * sqrt (K w / n))) ∂Q) ≤ _
have hvol : (volume parameterSpace).toReal = 2 := by
simp [parameterSpace, Real.volume_Icc, ENNReal.toReal_ofReal]
norm_num
have h := expectedRestrictedVolume_affineInversion_le
Q parameterSpace (by simp [parameterSpace, Real.volume_Icc])
A B K n L mu Kbar q hL hmu hn hK hKint hKbar hbad
rw [hvol] at h
exact h
theorem expectedLength_affineInversion_frontier_le_inflated
Frontier conversion when the mean radius proxy is inflated by at most a factor two. The resulting inverse-root constant is exactly 4 * sqrt 2 * L.
Formal statement
hypotheses
Omega :
Type*
MeasurableSpace Omega
Q :
Measure Omega
A B K :
Omega → ℝ
n :
ℕ
L mu Kbar q kappa Y t :
ℝ
hL :
0 ≤ L
hmu :
0 < mu
hn :
0 < n
hK :
∀ w, 0 ≤ K w
hKint :
Integrable K Q
hKbar :
(∫ w, K w ∂Q) ≤ Kbar
hbad :
(Q {w | mu / 2 < |B w - mu|}).toReal ≤ q
hkappa :
0 < kappa
hY :
0 ≤ Y
hKbar_le :
Kbar ≤ 2 * kappa
hq :
q ≤ Y / (2 * t)
ht :
t = (n : ℝ) * mu ^ 2 / kappa
Proof (Lean source)
lemma expectedLength_affineInversion_frontier_le_inflated
{Omega : Type*} [MeasurableSpace Omega]
(Q : Measure Omega) [IsProbabilityMeasure Q]
(A B K : Omega → ℝ) (n : ℕ)
(L mu Kbar q kappa Y t : ℝ)
(hL : 0 ≤ L) (hmu : 0 < mu) (hn : 0 < n)
(hK : ∀ w, 0 ≤ K w)
(hKint : Integrable K Q)
(hKbar : (∫ w, K w ∂Q) ≤ Kbar)
(hbad : (Q {w | mu / 2 < |B w - mu|}).toReal ≤ q)
(hkappa : 0 < kappa) (hY : 0 ≤ Y)
(hKbar_le : Kbar ≤ 2 * kappa)
(hq : q ≤ Y / (2 * t))
(ht : t = (n : ℝ) * mu ^ 2 / kappa) :
(∫ w, setLength (affineInversionSet (A w) (B w)
(L * sqrt (K w / n))) ∂Q) ≤
max 2 (4 * sqrt 2 * L + Y) *
min 1 (t ^ (-1 / 2 : ℝ)) := by
have htpos : 0 < t := by rw [ht]; positivity
have hbadContribution : 2 * q ≤ Y / t := by
calc
2 * q ≤ 2 * (Y / (2 * t)) :=
mul_le_mul_of_nonneg_left hq (by norm_num)
_ = Y / t := by field_simp [htpos.ne']
have hvol : (volume parameterSpace).toReal = 2 := by
simp [parameterSpace, Real.volume_Icc, ENNReal.toReal_ofReal]
norm_num
have hbadContribution' : (volume parameterSpace).toReal * q ≤ Y / t := by
rw [hvol]
exact hbadContribution
change (∫ w, restrictedSetVolume parameterSpace
(affineInversionSet parameterSpace (A w) (B w)
(L * sqrt (K w / n))) ∂Q) ≤ _
have h := expectedRestrictedVolume_affineInversion_frontier_le
Q parameterSpace (by simp [parameterSpace, Real.volume_Icc])
A B K n L mu Kbar q kappa 2 Y t hL hmu hn hK hKint hKbar
hbad hkappa (by norm_num) hY hKbar_le hbadContribution' ht
rw [hvol] at h
exact h
theorem expectedLength_affineInversion_frontier_le_uninflated
Frontier conversion without radius-proxy inflation. The resulting inverse-root constant is exactly 4 * L.
Formal statement
hypotheses
Omega :
Type*
MeasurableSpace Omega
Q :
Measure Omega
A B K :
Omega → ℝ
n :
ℕ
L mu Kbar q kappa Y t :
ℝ
hL :
0 ≤ L
hmu :
0 < mu
hn :
0 < n
hK :
∀ w, 0 ≤ K w
hKint :
Integrable K Q
hKbar :
(∫ w, K w ∂Q) ≤ Kbar
hbad :
(Q {w | mu / 2 < |B w - mu|}).toReal ≤ q
hkappa :
0 < kappa
hY :
0 ≤ Y
hKbar_le :
Kbar ≤ kappa
hq :
q ≤ Y / (2 * t)
ht :
t = (n : ℝ) * mu ^ 2 / kappa
Proof (Lean source)
lemma expectedLength_affineInversion_frontier_le_uninflated
{Omega : Type*} [MeasurableSpace Omega]
(Q : Measure Omega) [IsProbabilityMeasure Q]
(A B K : Omega → ℝ) (n : ℕ)
(L mu Kbar q kappa Y t : ℝ)
(hL : 0 ≤ L) (hmu : 0 < mu) (hn : 0 < n)
(hK : ∀ w, 0 ≤ K w)
(hKint : Integrable K Q)
(hKbar : (∫ w, K w ∂Q) ≤ Kbar)
(hbad : (Q {w | mu / 2 < |B w - mu|}).toReal ≤ q)
(hkappa : 0 < kappa) (hY : 0 ≤ Y)
(hKbar_le : Kbar ≤ kappa)
(hq : q ≤ Y / (2 * t))
(ht : t = (n : ℝ) * mu ^ 2 / kappa) :
(∫ w, setLength (affineInversionSet (A w) (B w)
(L * sqrt (K w / n))) ∂Q) ≤
max 2 (4 * L + Y) * min 1 (t ^ (-1 / 2 : ℝ)) := by
have htpos : 0 < t := by rw [ht]; positivity
have hbadContribution : 2 * q ≤ Y / t := by
calc
2 * q ≤ 2 * (Y / (2 * t)) :=
mul_le_mul_of_nonneg_left hq (by norm_num)
_ = Y / t := by field_simp [htpos.ne']
have hKbar_le' : Kbar ≤ 1 * kappa := by simpa using hKbar_le
have hvol : (volume parameterSpace).toReal = 2 := by
simp [parameterSpace, Real.volume_Icc, ENNReal.toReal_ofReal]
norm_num
have hbadContribution' : (volume parameterSpace).toReal * q ≤ Y / t := by
rw [hvol]
exact hbadContribution
change (∫ w, restrictedSetVolume parameterSpace
(affineInversionSet parameterSpace (A w) (B w)
(L * sqrt (K w / n))) ∂Q) ≤ _
have h := expectedRestrictedVolume_affineInversion_frontier_le
Q parameterSpace (by simp [parameterSpace, Real.volume_Icc])
A B K n L mu Kbar q kappa 1 Y t hL hmu hn hK hKint hKbar
hbad hkappa (by norm_num) hY hKbar_le' hbadContribution' ht
rw [hvol] at h
simpa using h
Helpers.RateAlgebra 1 declarations
Converts an inverse-root plus inverse-strength bound to the compact min(1,t⁻¹/²) frontier form.
Formal statement
Proof (Lean source)
lemma inverse_strength_to_frontier
{A B t : ℝ} (hA : 0 ≤ A) (hB : 0 ≤ B) (ht : 0 < t) :
min 2 (A / sqrt t + B / t) ≤
max 2 (A + B) * min 1 (t ^ (-1 / 2 : ℝ)) := by
clear hA
exact inverseStrength_to_frontier hB ht
Helpers.RegularCellRisk_Part1 24 declarations Statistics and deterministic constants
Statistics and deterministic constants
def regularCellVarianceConstant
The constant in the regular-cell variance calculation.
Definition (Lean source)
noncomputable def regularCellVarianceConstant (epsilon c : ℝ) : ℝ :=
8 * (epsilon⁻¹ ^ 2 + c⁻¹)
def targetEmpiricalMass
Empirical target mass of one ambient point.
Definition (Lean source)
noncomputable abbrev targetEmpiricalMass {N : ℕ}
(target : TargetSample 𝒳 N) (x : 𝒳) : ℝ :=
empiricalMass target x
def regularCellScore
The affine source score at a candidate effect.
Definition (Lean source)
noncomputable def regularCellScore (e : 𝒳 → ℝ) (theta : ℝ)
(o : SourceObs 𝒳) : ℝ :=
oracleInstrumentScore e o * (o.2.2.2 - theta * boolReal o.2.2.1)
def regularCellScoreMean
The conditional score mean represented by the model's contrast versions.
Definition (Lean source)
noncomputable def regularCellScoreMean (P : TransportedArray 𝒳)
(n : ℕ) (theta : ℝ) (x : 𝒳) : ℝ :=
P.deltaY n x - theta * P.deltaD n x
def regularCellOutcomeMoment
Cross-averaged outcome score.
Definition (Lean source)
noncomputable def regularCellOutcomeMoment (q e : 𝒳 → ℝ) {n N : ℕ}
(source : SourceSample 𝒳 n) (target : TargetSample 𝒳 N) : ℝ :=
crossAverage q source target
(fun o => oracleInstrumentScore e o * o.2.2.2)
def regularCellReceiptMoment
Cross-averaged receipt score.
Definition (Lean source)
noncomputable def regularCellReceiptMoment (q e : 𝒳 → ℝ) {n N : ℕ}
(source : SourceSample 𝒳 n) (target : TargetSample 𝒳 N) : ℝ :=
crossAverage q source target
(fun o => oracleInstrumentScore e o * boolReal o.2.2.1)
def regularCellContrastMoment
Cross-averaged affine score.
Definition (Lean source)
noncomputable def regularCellContrastMoment (q e : 𝒳 → ℝ) (theta : ℝ)
{n N : ℕ} (source : SourceSample 𝒳 n)
(target : TargetSample 𝒳 N) : ℝ :=
regularCellOutcomeMoment q e source target -
theta * regularCellReceiptMoment q e source target
def regularCellKhat
The collision proxy used by the feasible regular-cell rule.
Definition (Lean source)
noncomputable def regularCellKhat (q : 𝒳 → ℝ) {N : ℕ}
(target : TargetSample 𝒳 N) : ℝ :=
1 + collisionScale q target
theorem cellVectorExtension_apply_cell
The zero-extended cell-value function returns a cell's assigned value at every point in that cell.
Formal statement
hypotheses
conclusion
cellVectorExtension design v (design.cell i) = v i
Proof (Lean source)
lemma cellVectorExtension_apply_cell
{m : ℕ} (design : RegularCellDesign 𝒳 m)
(v : Fin m → ℝ) (i : Fin m) :
cellVectorExtension design v (design.cell i) = v i := by
classical
unfold cellVectorExtension
split
· congr 1
exact design.cell.injective (Classical.choose_spec ‹∃ j, design.cell j = design.cell i›)
· exact (‹¬∃ j, design.cell j = design.cell i› ⟨i, rfl⟩).elim
theorem measurable_cellVectorExtension
Extending a finite vector of cell values by zero outside the cells defines a measurable covariate function.
Formal statement
hypotheses
conclusion
Measurable (cellVectorExtension design v)
Proof (Lean source)
lemma measurable_cellVectorExtension
{m : ℕ} (design : RegularCellDesign 𝒳 m)
(v : Fin m → ℝ) :
Measurable (cellVectorExtension design v) := by
classical
have hrepr : cellVectorExtension design v = fun x =>
∑ i : Fin m, if x = design.cell i then v i else 0 := by
funext x
by_cases hx : ∃ i, design.cell i = x
· let i := Classical.choose hx
have hi : design.cell i = x := Classical.choose_spec hx
rw [show cellVectorExtension design v x = v i by
rw [← hi]
exact cellVectorExtension_apply_cell design v i]
rw [Finset.sum_eq_single i]
· simp [hi]
· intro j _ hji
have hne : x ≠ design.cell j := by
intro h
exact hji (design.cell.injective (hi.trans h)).symm
simp [hne]
· simp
· unfold cellVectorExtension
rw [dif_neg hx]
symm
apply Finset.sum_eq_zero
intro i _
rw [if_neg]
intro hxi
exact hx ⟨i, hxi.symm⟩
rw [hrepr]
apply measurable_sum
intro i _
apply Measurable.ite
· simpa [eq_comm] using design.measurableCell i
· exact measurable_const
· exact measurable_const
theorem regularCellInversion_measurableGraph_on_cells
Cell-scoped graph measurability of regular-cell inversion.
Formal statement
hypotheses
N k :
ℕ → ℕ
L :
ℝ
n :
ℕ
cell :
Fin (k n) ↪ 𝒳
hcell :
∀ i, MeasurableSet {cell i}
q e :
𝒳 → ℝ
hq :
he :
conclusion
MeasurableSet {p : TwoSample 𝒳 n (N n) × ℝ | ((∀ i, (p.1.1 i).1 ∈ range cell) ∧ (∀ j, p.1.2 j ∈ range cell)) ∧ p.2 ∈ regularCellInversion q e L p.1.1 p.1.2}
Proof (Lean source)
lemma regularCellInversion_measurableGraph_on_cells
(N k : ℕ → ℕ) (L : ℝ) (n : ℕ)
(cell : Fin (k n) ↪ 𝒳) (hcell : ∀ i, MeasurableSet {cell i})
(q e : 𝒳 → ℝ) (hq : Measurable q) (he : Measurable e) :
MeasurableSet {p : TwoSample 𝒳 n (N n) × ℝ |
((∀ i, (p.1.1 i).1 ∈ range cell) ∧
(∀ j, p.1.2 j ∈ range cell)) ∧
p.2 ∈ regularCellInversion q e L p.1.1 p.1.2} := by
classical
let sameCell : 𝒳 → 𝒳 → Prop := fun x y =>
∃ i : Fin (k n), x = cell i ∧ y = cell i
have hsameCell : MeasurableSet {z : 𝒳 × 𝒳 | sameCell z.1 z.2} := by
rw [show {z : 𝒳 × 𝒳 | sameCell z.1 z.2} =
⋃ i : Fin (k n), {z | z.1 = cell i ∧ z.2 = cell i} by
ext z
simp [sameCell]]
apply MeasurableSet.iUnion
intro i
exact ((hcell i).preimage measurable_fst).inter
((hcell i).preimage measurable_snd)
let kernel : 𝒳 → 𝒳 → ℝ := fun x y =>
if sameCell x y then 1 / q x else 0
have hkernel : Measurable (uncurry kernel) := by
exact Measurable.ite hsameCell
(measurable_const.div (hq.comp measurable_fst)) measurable_const
have hInstrument : Measurable (oracleInstrumentScore e) := by
unfold oracleInstrumentScore
have hz : Measurable fun o : SourceObs 𝒳 => o.2.1 := by fun_prop
have he' : Measurable fun o : SourceObs 𝒳 => e o.1 :=
he.comp measurable_fst
exact Measurable.ite (hz (MeasurableSet.singleton true))
(measurable_const.div he')
(measurable_const.neg.div (measurable_const.sub he'))
have hBool : Measurable (fun o : SourceObs 𝒳 => boolReal o.2.2.1) := by
unfold boolReal
have hd : Measurable fun o : SourceObs 𝒳 => o.2.2.1 := by fun_prop
exact Measurable.ite (hd (MeasurableSet.singleton true))
measurable_const measurable_const
let Gy : SourceObs 𝒳 → ℝ := fun o =>
oracleInstrumentScore e o * o.2.2.2
let Gd : SourceObs 𝒳 → ℝ := fun o =>
oracleInstrumentScore e o * boolReal o.2.2.1
have hGy : Measurable Gy := hInstrument.mul (by fun_prop)
have hGd : Measurable Gd := hInstrument.mul hBool
let moment : (SourceSample 𝒳 n × TargetSample 𝒳 (N n)) →
(SourceObs 𝒳 → ℝ) → ℝ := fun s G =>
(N n : ℝ)⁻¹ * ∑ j,
(n : ℝ)⁻¹ / q (s.2 j) *
∑ i, if sameCell (s.1 i).1 (s.2 j) then G (s.1 i) else 0
have hmoment (G : SourceObs 𝒳 → ℝ) (hG : Measurable G) :
Measurable (fun s => moment s G) := by
unfold moment
apply measurable_const.mul
apply Finset.measurable_sum
intro j hj
have htj : Measurable (fun s :
SourceSample 𝒳 n × TargetSample 𝒳 (N n) => s.2 j) :=
(measurable_pi_apply j).comp measurable_snd
apply (measurable_const.div (hq.comp htj)).mul
apply Finset.measurable_sum
intro i hi
have hsi : Measurable (fun s :
SourceSample 𝒳 n × TargetSample 𝒳 (N n) => s.1 i) :=
(measurable_pi_apply i).comp measurable_fst
have hsix : Measurable (fun s :
SourceSample 𝒳 n × TargetSample 𝒳 (N n) => (s.1 i).1) :=
measurable_fst.comp hsi
apply Measurable.ite
· exact hsameCell.preimage (hsix.prodMk htj)
· exact hG.comp hsi
· exact measurable_const
let scale : TargetSample 𝒳 (N n) → ℝ := fun target =>
((N n : ℝ) * (N n - 1 : ℕ))⁻¹ *
∑ j, ∑ l, if j ≠ l then kernel (target j) (target l) else 0
have hscale : Measurable scale := by
unfold scale
apply measurable_const.mul
apply Finset.measurable_sum
intro j hj
apply Finset.measurable_sum
intro l hl
split_ifs
· have hp : Measurable (fun target : TargetSample 𝒳 (N n) =>
(target j, target l)) :=
(measurable_pi_apply j).prodMk (measurable_pi_apply l)
simpa [uncurry] using hkernel.fun_comp hp
· exact measurable_const
have hrange : MeasurableSet (range cell) := by
rw [show range cell = ⋃ i, {cell i} by
ext x
simp]
exact MeasurableSet.iUnion hcell
let support : Set (TwoSample 𝒳 n (N n) × ℝ) :=
{p | (∀ i, (p.1.1 i).1 ∈ range cell) ∧
(∀ j, p.1.2 j ∈ range cell)}
have hsupport : MeasurableSet support := by
rw [show support =
(⋂ i, {p | (p.1.1 i).1 ∈ range cell}) ∩
⋂ j, {p | p.1.2 j ∈ range cell} by
ext p
simp [support]]
exact (MeasurableSet.iInter fun i =>
hrange.preimage
(measurable_fst.comp
((measurable_pi_apply i).comp
(measurable_fst.comp measurable_fst)))).inter
(MeasurableSet.iInter fun j =>
hrange.preimage
((measurable_pi_apply j).comp
(measurable_snd.comp measurable_fst)))
have hmoment_eq (p : TwoSample 𝒳 n (N n) × ℝ) (hp : p ∈ support)
(G : SourceObs 𝒳 → ℝ) :
crossAverage q p.1.1 p.1.2 G = moment p.1 G := by
unfold crossAverage crossAverage
cellMoment moment
congr 1
apply Finset.sum_congr rfl
intro j hj
congr 1
apply Finset.sum_congr rfl
intro i hi
have hiRange := hp.1 i
have hjRange := hp.2 j
simp only [Set.mem_range] at hiRange hjRange
rcases hiRange with ⟨a, ha⟩
rcases hjRange with ⟨b, hb⟩
by_cases hEq : (p.1.1 i).1 = p.1.2 j
· have hs : sameCell (p.1.1 i).1 (p.1.2 j) :=
⟨b, hEq.trans hb.symm, hb.symm⟩
have hs' : sameCell (p.1.2 j) (p.1.2 j) :=
⟨b, hb.symm, hb.symm⟩
simp [hEq, hs, hs']
· have hnot : ¬ sameCell (p.1.1 i).1 (p.1.2 j) := by
rintro ⟨a, ha', hb'⟩
exact hEq (ha'.trans hb'.symm)
simp [hEq, hnot]
have hscale_eq (p : TwoSample 𝒳 n (N n) × ℝ) (hp : p ∈ support) :
collisionScale q p.1.2 = scale p.1.2 := by
unfold collisionScale collisionScale scale
congr 1
apply Finset.sum_congr rfl
intro j hj
apply Finset.sum_congr rfl
intro l hl
by_cases hjl : j = l
· simp [hjl]
· unfold collisionKernel kernel
have hjRange := hp.2 j
have hlRange := hp.2 l
simp only [Set.mem_range] at hjRange hlRange
rcases hjRange with ⟨a, ha⟩
rcases hlRange with ⟨b, hb⟩
by_cases hEq : p.1.2 j = p.1.2 l
· have hs : sameCell (p.1.2 j) (p.1.2 l) :=
⟨b, hEq.trans hb.symm, hb.symm⟩
have hs' : sameCell (p.1.2 l) (p.1.2 l) :=
⟨b, hb.symm, hb.symm⟩
simp [hjl, hEq, hs, hs']
· have hnot : ¬ sameCell (p.1.2 j) (p.1.2 l) := by
rintro ⟨a, ha', hb'⟩
exact hEq (ha'.trans hb'.symm)
simp [hEq, hnot]
by_cases hsmall : N n < 2
· have hparam : MeasurableSet parameterSpace := by
simp [parameterSpace]
change MeasurableSet (support ∩
{p : TwoSample 𝒳 n (N n) × ℝ | p.2 ∈
regularCellInversion q e L p.1.1 p.1.2})
simp only [regularCellInversion, hsmall, ↓reduceIte]
exact hsupport.inter (hparam.preimage measurable_snd)
· have htheta : MeasurableSet {p : TwoSample 𝒳 n (N n) × ℝ |
p.2 ∈ parameterSpace} := by
exact (by simp [parameterSpace] :
MeasurableSet parameterSpace).preimage measurable_snd
have hineq : MeasurableSet {p : TwoSample 𝒳 n (N n) × ℝ |
|moment p.1 Gy - p.2 * moment p.1 Gd| ≤
L * sqrt ((1 + scale p.1.2) / n)} := by
exact measurableSet_le
(((hmoment Gy hGy).comp measurable_fst).sub
(measurable_snd.mul
((hmoment Gd hGd).comp measurable_fst))).abs
(measurable_const.mul
(Real.continuous_sqrt.measurable.comp
((measurable_const.add (hscale.comp (by fun_prop))).div_const n)))
change MeasurableSet (support ∩
{p : TwoSample 𝒳 n (N n) × ℝ |
p.2 ∈ regularCellInversion q e L p.1.1 p.1.2})
rw [show support ∩ {p : TwoSample 𝒳 n (N n) × ℝ |
p.2 ∈ regularCellInversion q e L p.1.1 p.1.2} =
support ∩ ({p | p.2 ∈ parameterSpace} ∩
{p | |moment p.1 Gy - p.2 * moment p.1 Gd| ≤
L * sqrt ((1 + scale p.1.2) / n)}) by
ext p
by_cases hp : p ∈ support
· simp only [Set.mem_setOf_eq, hp, true_and, Set.mem_inter_iff]
simp only [regularCellInversion, hsmall, ↓reduceIte,
Set.mem_setOf_eq]
rw [hmoment_eq p hp Gy, hmoment_eq p hp Gd, hscale_eq p hp]
· simp [hp]]
exact hsupport.inter (htheta.inter hineq)
def regularCellProcedure
The regular-cell procedure, which receives cell masses and propensity but not the transport weight.
Definition (Lean source)
noncomputable def regularCellProcedure (N k : ℕ → ℕ) (L : ℝ) :
RegularCellProcedure 𝒳 N k where
set n input :=
regularCellInversion
(cellVectorExtension input.design input.q)
(cellVectorExtension input.design input.e)
L input.sample.1 input.sample.2
subset n input := by
intro theta htheta
by_cases hsmall : N n < 2
· simpa [regularCellInversion, hsmall] using htheta
· simp only [regularCellInversion, hsmall, ↓reduceIte] at htheta
exact htheta.1
measurableGraph n design q e :=
regularCellInversion_measurableGraph_on_cells N k L n
design.cell design.measurableCell
(cellVectorExtension design q) (cellVectorExtension design e)
(measurable_cellVectorExtension design q)
(measurable_cellVectorExtension design e)
theorem regularCellProcedure_set_eq_ambient_ae
On a regular finite-cell law, the bundled finite-vector input agrees almost surely with the ambient mass-and-propensity inversion.
Formal statement
hypotheses
N k :
ℕ → ℕ
c epsilon cminus cplus :
ℝ
L :
ℝ
n :
ℕ
P :
hP :
RegularFiniteCellClass P N k c epsilon cminus cplus n
conclusion
∀ᵐ s ∂twoSampleLaw P N n,
(regularCellProcedure (𝒳 := 𝒳) N k L).set n (regularCellInputOfClass P hP s)
= regularCellInversion (sourceCellMass P n) (P.propensity n) L s.1 s.2
Proof (Lean source)
lemma regularCellProcedure_set_eq_ambient_ae
{N k : ℕ → ℕ} {c epsilon cminus cplus : ℝ}
(L : ℝ) (n : ℕ) (P : TransportedArray 𝒳)
(hP : RegularFiniteCellClass P N k c epsilon cminus cplus n) :
∀ᵐ s ∂twoSampleLaw P N n,
(regularCellProcedure (𝒳 := 𝒳) N k L).set n
(regularCellInputOfClass P hP s) =
regularCellInversion (sourceCellMass P n) (P.propensity n)
L s.1 s.2 := by
classical
let design := regularCellDesignOfClass P hP
have hrange : sourceXLaw P n (range design.cell) = 1 :=
(Classical.choose_spec hP.2.2.2.2.2).2.1
have hrangeMeas : MeasurableSet (range design.cell) := by
rw [show range design.cell = ⋃ i, {design.cell i} by
ext x
simp]
exact MeasurableSet.iUnion design.measurableCell
letI : IsProbabilityMeasure (sourceObsLaw P n) :=
hP.1.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (targetXLaw P n) :=
hP.1.twoSampleArray.2.2.1 n
letI : IsProbabilityMeasure (sourceXLaw P n) := by
unfold sourceXLaw
exact Measure.isProbabilityMeasure_map measurable_fst.aemeasurable
letI : IsProbabilityMeasure (twoSampleLaw P N n) := by
unfold twoSampleLaw
infer_instance
have hsourceX : ∀ᵐ x ∂sourceXLaw P n, x ∈ range design.cell :=
(mem_ae_iff_prob_eq_one hrangeMeas).2 hrange
have hsourceObs :
∀ᵐ o ∂sourceObsLaw P n, o.1 ∈ range design.cell := by
unfold sourceXLaw at hsourceX
exact ae_of_ae_map measurable_fst.aemeasurable hsourceX
have hsource :
∀ᵐ source ∂Measure.pi (fun _ : Fin n => sourceObsLaw P n),
∀ i, (source i).1 ∈ range design.cell := by
apply Measure.ae_pi_le_pi
exact Filter.eventually_pi fun _ => hsourceObs
have hrangeCompl : sourceXLaw P n (range design.cell)ᶜ = 0 := by
rw [measure_compl hrangeMeas (measure_ne_top _ _), hrange, measure_univ]
simp
have htargetX :
∀ᵐ x ∂targetXLaw P n, x ∈ range design.cell := by
have hzero := hP.1.transportDomination hrangeCompl
change range design.cell ∈ ae (targetXLaw P n)
rw [mem_ae_iff]
exact hzero
have htarget :
∀ᵐ target ∂Measure.pi (fun _ : Fin (N n) => targetXLaw P n),
∀ j, target j ∈ range design.cell := by
apply Measure.ae_pi_le_pi
exact Filter.eventually_pi fun _ => htargetX
let SG : Set (SourceSample 𝒳 n) :=
{source | ∀ i, (source i).1 ∈ range design.cell}
let TG : Set (TargetSample 𝒳 (N n)) :=
{target | ∀ j, target j ∈ range design.cell}
have hSGmeas : MeasurableSet SG := by
rw [show SG = ⋂ i, {source | (source i).1 ∈ range design.cell} by
ext source
simp [SG]]
exact MeasurableSet.iInter fun i => hrangeMeas.preimage
(measurable_fst.comp (measurable_pi_apply i))
have hTGmeas : MeasurableSet TG := by
rw [show TG = ⋂ j, {target | target j ∈ range design.cell} by
ext target
simp [TG]]
exact MeasurableSet.iInter fun j =>
hrangeMeas.preimage (measurable_pi_apply j)
have hprod : twoSampleLaw P N n (SG ×ˢ TG) = 1 := by
unfold twoSampleLaw
rw [Measure.prod_prod,
(mem_ae_iff_prob_eq_one hSGmeas).1 hsource,
(mem_ae_iff_prob_eq_one hTGmeas).1 htarget]
simp
have hprodAE : ∀ᵐ s ∂twoSampleLaw P N n, s ∈ SG ×ˢ TG :=
(mem_ae_iff_prob_eq_one (hSGmeas.prod hTGmeas)).2 hprod
filter_upwards [hprodAE] with s hs
have hq : ∀ j,
cellVectorExtension design
(fun i => sourceCellMass P n (design.cell i)) (s.2 j) =
sourceCellMass P n (s.2 j) := by
intro j
obtain ⟨i, hi⟩ := hs.2 j
rw [← hi, cellVectorExtension_apply_cell]
have he : ∀ i,
cellVectorExtension design
(fun j => P.propensity n (design.cell j)) (s.1 i).1 =
P.propensity n (s.1 i).1 := by
intro i
obtain ⟨j, hj⟩ := hs.1 i
rw [← hj, cellVectorExtension_apply_cell]
unfold regularCellInputOfClass regularCellProcedure
change regularCellInversion
(cellVectorExtension design
(fun i => sourceCellMass P n (design.cell i)))
(cellVectorExtension design
(fun i => P.propensity n (design.cell i))) L s.1 s.2 = _
unfold regularCellInversion crossAverage collisionScale
crossAverage cellMoment
collisionScale collisionKernel
oracleInstrumentScore
simp_rw [hq, he]
theorem measure_eq_fin_sum_smul_dirac_of_range
A probability distribution supported on finitely many measurable cells is the sum of point masses at those cells, each weighted by its cell probability.
Formal statement
hypotheses
conclusion
μ = ∑ i, μ {cell i} • Measure.dirac (cell i)
Proof (Lean source)
lemma measure_eq_fin_sum_smul_dirac_of_range
(μ : Measure 𝒳) [IsProbabilityMeasure μ] {m : ℕ}
(cell : Fin m ↪ 𝒳) (hcell : ∀ i, MeasurableSet {cell i})
(hrange : μ (range cell) = 1) :
μ = ∑ i, μ {cell i} • Measure.dirac (cell i) := by
exact measure_eq_fin_sum_smul_dirac_of_range
μ cell hcell (by simpa only [measure_univ] using hrange)
theorem property_at_of_ae_of_singleton_pos
A property that holds almost everywhere also holds at any point assigned strictly positive probability.
Formal statement
hypotheses
conclusion
p x
Proof (Lean source)
lemma property_at_of_ae_of_singleton_pos
(μ : Measure 𝒳) (p : 𝒳 → Prop) {x : 𝒳}
(hp : ∀ᵐ y ∂μ, p y) (hx : 0 < (μ {x}).toReal) :
p x := by
exact property_at_of_ae_of_singleton_pos
μ p hp (by
intro hzero
simp [hzero] at hx)
theorem integrable_of_finite_support
Every strongly measurable real-valued function is integrable under a probability distribution supported on finitely many measurable cells.
Formal statement
hypotheses
μ :
m :
ℕ
cell :
Fin m ↪ 𝒳
hcell :
∀ i, MeasurableSet {cell i}
hrange :
μ (range cell) = 1
f :
𝒳 → ℝ
hf :
conclusion
Integrable f μ
Proof (Lean source)
lemma integrable_of_finite_support
(μ : Measure 𝒳) [IsProbabilityMeasure μ] {m : ℕ}
(cell : Fin m ↪ 𝒳) (hcell : ∀ i, MeasurableSet {cell i})
(hrange : μ (range cell) = 1) (f : 𝒳 → ℝ)
(hf : StronglyMeasurable f) :
Integrable f μ := by
exact integrable_of_finite_atomic_support
μ cell hcell (by simpa only [measure_univ] using hrange) f hf
theorem regularCell_transportWeight_and_kish
On the class-supplied finite support the RN weight is the target/source point-mass ratio, and Kish dispersion is the corresponding finite sum.
Formal statement
hypotheses
P :
N k :
ℕ → ℕ
c epsilon cminus cplus :
ℝ
n :
ℕ
hP :
RegularFiniteCellClass P N k c epsilon cminus cplus n
conclusion
∃ cell : Fin (k n) ↪ 𝒳,
sourceXLaw P n (range cell) = 1 ∧
(∀ i, transportWeight P n (cell i) = (targetXLaw P n {cell i}).toReal / sourceCellMass P n (cell i)) ∧
kishDispersion P n
= ∑ i, (targetXLaw P n {cell i}).toReal ^ 2 / sourceCellMass P n (cell i)
Proof (Lean source)
lemma regularCell_transportWeight_and_kish
(P : TransportedArray 𝒳) (N k : ℕ → ℕ)
(c epsilon cminus cplus : ℝ) (n : ℕ)
(hP : RegularFiniteCellClass P N k c epsilon cminus cplus n) :
∃ cell : Fin (k n) ↪ 𝒳,
sourceXLaw P n (range cell) = 1 ∧
(∀ i, transportWeight P n (cell i) =
(targetXLaw P n {cell i}).toReal /
sourceCellMass P n (cell i)) ∧
kishDispersion P n =
∑ i, (targetXLaw P n {cell i}).toReal ^ 2 /
sourceCellMass P n (cell i) := by
classical
rcases hP with
⟨hIV, hk, hcminus, hcminus_one, hcplus, cell, hcell, hrange, hmass⟩
letI : IsProbabilityMeasure (sourceObsLaw P n) :=
hIV.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (sourceXLaw P n) := by
unfold sourceXLaw
exact Measure.isProbabilityMeasure_map measurable_fst.aemeasurable
letI : IsProbabilityMeasure (targetXLaw P n) :=
hIV.twoSampleArray.2.2.1 n
have hsourcePos (i : Fin (k n)) :
0 < sourceCellMass P n (cell i) := by
exact lt_of_lt_of_le
(div_pos hcminus (by exact_mod_cast hk))
(hmass i).1
have hrnStrong :
StronglyMeasurable
(fun x => ((targetXLaw P n).rnDeriv (sourceXLaw P n) x).toReal) :=
(Measure.measurable_rnDeriv (targetXLaw P n) (sourceXLaw P n)).ennreal_toReal
|>.stronglyMeasurable
have hratio (i : Fin (k n)) :
transportWeight P n (cell i) =
(targetXLaw P n {cell i}).toReal /
sourceCellMass P n (cell i) := by
have hRN :=
Measure.setIntegral_toReal_rnDeriv'
hIV.transportDomination (hcell i)
rw [integral_singleton' hrnStrong] at hRN
apply (eq_div_iff (hsourcePos i).ne').2
simpa [transportWeight, sourceCellMass, measureReal_def, mul_comm] using hRN
refine ⟨cell, hrange, hratio, ?_⟩
have hmeasure :
sourceXLaw P n =
∑ i, sourceXLaw P n {cell i} • Measure.dirac (cell i) :=
measure_eq_fin_sum_smul_dirac_of_range
(sourceXLaw P n) cell hcell hrange
have hweightStrong :
StronglyMeasurable (fun x => (transportWeight P n x) ^ 2) := by
exact
((Measure.measurable_rnDeriv (targetXLaw P n) (sourceXLaw P n)).ennreal_toReal
|>.pow_const 2).stronglyMeasurable
rw [kishDispersion, hmeasure,
integral_finset_sum_measure]
· apply Finset.sum_congr rfl
intro i hi
rw [integral_smul_measure,
integral_dirac' _ _ hweightStrong]
simp only [smul_eq_mul]
change
sourceCellMass P n (cell i) * transportWeight P n (cell i) ^ 2 =
(targetXLaw P n {cell i}).toReal ^ 2 /
sourceCellMass P n (cell i)
rw [hratio i]
field_simp [(hsourcePos i).ne']
· intro i hi
exact
(integrable_dirac' hweightStrong (by simp)).smul_measure
(measure_ne_top (sourceXLaw P n) {cell i})
theorem abs_regularCellScore_le
The inverse-propensity affine score has the paper's 2 / epsilon envelope.
Formal statement
hypotheses
e :
𝒳 → ℝ
epsilon theta :
ℝ
o :
hepsilon :
0 < epsilon
hoverlap :
epsilon ≤ e o.1 ∧ e o.1 ≤ 1 - epsilon
htheta :
theta ∈ parameterSpace
hy :
o.2.2.2 ∈ Icc (0 : ℝ) 1
conclusion
|regularCellScore e theta o| ≤ 2 / epsilon
Proof (Lean source)
lemma abs_regularCellScore_le
(e : 𝒳 → ℝ) (epsilon theta : ℝ) (o : SourceObs 𝒳)
(hepsilon : 0 < epsilon)
(hoverlap : epsilon ≤ e o.1 ∧ e o.1 ≤ 1 - epsilon)
(htheta : theta ∈ parameterSpace)
(hy : o.2.2.2 ∈ Icc (0 : ℝ) 1) :
|regularCellScore e theta o| ≤ 2 / epsilon := by
unfold parameterSpace at htheta
have hthetaAbs : |theta| ≤ 1 := (abs_le).2 htheta
have hyAbs : |o.2.2.2| ≤ 1 := by
rw [abs_le]
constructor <;> linarith [hy.1, hy.2]
have hdAbs : |boolReal o.2.2.1| ≤ 1 := by
cases o.2.2.1 <;> simp [boolReal]
have hres : |o.2.2.2 - theta * boolReal o.2.2.1| ≤ 2 := by
calc
_ ≤ |o.2.2.2| + |theta * boolReal o.2.2.1| := abs_sub _ _
_ = |o.2.2.2| + |theta| * |boolReal o.2.2.1| := by rw [abs_mul]
_ ≤ 1 + 1 * 1 := by gcongr
_ = 2 := by norm_num
have hscore : |oracleInstrumentScore e o| ≤ 1 / epsilon := by
unfold oracleInstrumentScore
split
· rw [abs_div, abs_one,
abs_of_pos (lt_of_lt_of_le hepsilon hoverlap.1)]
exact one_div_le_one_div_of_le hepsilon hoverlap.1
· have hden : 0 < 1 - e o.1 := by linarith
rw [abs_div, abs_neg, abs_one, abs_of_pos hden]
exact one_div_le_one_div_of_le hepsilon (by linarith)
rw [regularCellScore, abs_mul]
calc
|oracleInstrumentScore e o| *
|o.2.2.2 - theta * boolReal o.2.2.1| ≤
(1 / epsilon) * 2 :=
mul_le_mul hscore hres (abs_nonneg _) (by positivity)
_ = 2 / epsilon := by ring
theorem regularCell_score_mean_properties_for_witness
On every realized cell, the score conditional mean is DeltaY - theta * DeltaD; it is bounded by two, and at the target CACE its target-mass average is zero.
Formal statement
hypotheses
P :
N k :
ℕ → ℕ
c epsilon cminus cplus :
ℝ
n :
ℕ
hAssignmentContrastIntegrable :
hReceiptContrastIntegrable :
hP :
RegularFiniteCellClass P N k c epsilon cminus cplus n
cell :
Fin (k n) ↪ 𝒳
hcell :
∀ i, MeasurableSet {cell i}
hrange :
sourceXLaw P n (range cell) = 1
hmass :
∀ i,
cminus / (k n : ℝ) ≤ (sourceXLaw P n {cell i}).toReal ∧
(sourceXLaw P n {cell i}).toReal ≤ cplus / (k n : ℝ)
conclusion
(∀ theta ∈ parameterSpace, ∀ i, (∫ o in {o | o.1 = cell i}, regularCellScore (P.propensity n) theta o ∂sourceObsLaw P n) / sourceCellMass P n (cell i) = regularCellScoreMean P n theta (cell i)) ∧
= 0
Proof (Lean source)
lemma regularCell_score_mean_properties_for_witness
(P : TransportedArray 𝒳) (N k : ℕ → ℕ)
(c epsilon cminus cplus : ℝ) (n : ℕ)
(hAssignmentContrastIntegrable :
Integrable (P.assignmentContrast n true) (sourceXLaw P n))
(hReceiptContrastIntegrable :
Integrable (P.receiptContrast n true) (sourceXLaw P n))
(hP : RegularFiniteCellClass P N k c epsilon cminus cplus n)
(cell : Fin (k n) ↪ 𝒳)
(hcell : ∀ i, MeasurableSet {cell i})
(hrange : sourceXLaw P n (range cell) = 1)
(hmass : ∀ i,
cminus / (k n : ℝ) ≤ (sourceXLaw P n {cell i}).toReal ∧
(sourceXLaw P n {cell i}).toReal ≤ cplus / (k n : ℝ)) :
(∀ theta ∈ parameterSpace, ∀ i,
(∫ o in {o | o.1 = cell i},
regularCellScore (P.propensity n) theta o ∂sourceObsLaw P n) /
sourceCellMass P n (cell i) =
regularCellScoreMean P n theta (cell i)) ∧
(∀ theta ∈ parameterSpace, ∀ i,
|regularCellScoreMean P n theta (cell i)| ≤ 2) ∧
(∑ i, (targetXLaw P n {cell i}).toReal *
regularCellScoreMean P n (targetCACE P n) (cell i)) = 0 := by
classical
have hcompact :=
compact_causal_range P N k c epsilon n
hAssignmentContrastIntegrable hReceiptContrastIntegrable hP.1
rcases hcompact with
⟨hDeltaYEq, hDeltaDEq, hOutcomeEq, hFirstEq, hRatio, hTheta⟩
rcases hP with
⟨hIV, hk, hcminus, hcminus_one, hcplus, _⟩
letI : IsProbabilityMeasure (sourceObsLaw P n) :=
hIV.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (sourceXLaw P n) := by
unfold sourceXLaw
exact Measure.isProbabilityMeasure_map measurable_fst.aemeasurable
letI : IsProbabilityMeasure (targetXLaw P n) :=
hIV.twoSampleArray.2.2.1 n
have hsourcePos (i : Fin (k n)) :
0 < sourceCellMass P n (cell i) := by
exact lt_of_lt_of_le
(div_pos hcminus (by exact_mod_cast hk))
(hmass i).1
have hOverlapObs : ∀ᵐ o ∂sourceObsLaw P n,
epsilon ≤ P.propensity n o.1 ∧
P.propensity n o.1 ≤ 1 - epsilon := by
have hx := hIV.instrumentOverlap.2.2
unfold sourceXLaw at hx
exact ae_of_ae_map measurable_fst.aemeasurable hx
have hInstrumentMeasurable : Measurable (instrumentScore P n) := by
have hz : Measurable fun o : SourceObs 𝒳 => o.2.1 := by
fun_prop
have he : Measurable fun o : SourceObs 𝒳 => P.propensity n o.1 :=
(P.propensity_measurable n).comp measurable_fst
unfold instrumentScore
exact Measurable.ite (hz (MeasurableSet.singleton true))
(measurable_const.div he)
(measurable_const.neg.div (measurable_const.sub he))
have hInstrumentBound : ∀ᵐ o ∂sourceObsLaw P n,
|instrumentScore P n o| ≤ 1 / epsilon := by
filter_upwards [hOverlapObs] with o ho
rcases o with ⟨x, z, d, y⟩
cases z
· simp only [instrumentScore, Bool.false_eq_true, ↓reduceIte, abs_neg,
abs_div, abs_one]
rw [abs_of_pos (sub_pos.mpr (lt_of_le_of_lt ho.2
(by linarith [hIV.instrumentOverlap.1])))]
exact one_div_le_one_div_of_le hIV.instrumentOverlap.1
(by linarith [ho.2])
· simp only [instrumentScore, ↓reduceIte, abs_div, abs_one]
rw [abs_of_pos (lt_of_lt_of_le hIV.instrumentOverlap.1 ho.1)]
exact one_div_le_one_div_of_le hIV.instrumentOverlap.1 ho.1
have hBoolMeasurable :
Measurable (fun o : SourceObs 𝒳 => boolReal o.2.2.1) := by
unfold boolReal
have hd : Measurable fun o : SourceObs 𝒳 => o.2.2.1 := by
fun_prop
exact Measurable.ite (hd (MeasurableSet.singleton true))
measurable_const measurable_const
have hOutcomeScoreIntegrable :
Integrable (fun o =>
instrumentScore P n o * o.2.2.2) (sourceObsLaw P n) := by
refine Integrable.of_bound
(hInstrumentMeasurable.mul (by fun_prop)).aestronglyMeasurable
(1 / epsilon) ?_
filter_upwards [hInstrumentBound, (sourceObservationFacts_of_class P N k c epsilon n hIV).2.2.1]
with o hs hy
rw [Real.norm_eq_abs, abs_mul, abs_of_nonneg hy.1]
exact (mul_le_mul hs hy.2 hy.1
(one_div_nonneg.mpr hIV.instrumentOverlap.1.le)).trans_eq (mul_one _)
have hReceiptScoreIntegrable :
Integrable (fun o =>
instrumentScore P n o * boolReal o.2.2.1)
(sourceObsLaw P n) := by
refine Integrable.of_bound
(hInstrumentMeasurable.mul hBoolMeasurable).aestronglyMeasurable
(1 / epsilon) ?_
filter_upwards [hInstrumentBound] with o hs
have hd : |boolReal o.2.2.1| ≤ 1 := by
cases o.2.2.1 <;> simp [boolReal]
rw [Real.norm_eq_abs, abs_mul]
exact (mul_le_mul hs hd (abs_nonneg _)
(one_div_nonneg.mpr hIV.instrumentOverlap.1.le)).trans_eq (mul_one _)
have hmean (theta : ℝ) (htheta : theta ∈ parameterSpace)
(i : Fin (k n)) :
(∫ o in {o | o.1 = cell i},
regularCellScore (P.propensity n) theta o ∂sourceObsLaw P n) /
sourceCellMass P n (cell i) =
regularCellScoreMean P n theta (cell i) := by
have hOutcomeSet :=
(sourceObservationFacts_of_class P N k c epsilon n hIV).2.2.2.2.2.1
({cell i} : Set 𝒳) (hcell i)
have hReceiptSet :=
(sourceObservationFacts_of_class P N k c epsilon n hIV).2.2.2.2.2.2.2.1
({cell i} : Set 𝒳) (hcell i)
have hscore :
(∫ o in {o | o.1 = cell i},
regularCellScore (P.propensity n) theta o ∂sourceObsLaw P n) =
(∫ o in {o | o.1 ∈ ({cell i} : Set 𝒳)},
instrumentScore P n o * o.2.2.2 ∂sourceObsLaw P n) -
theta * (∫ o in {o | o.1 ∈ ({cell i} : Set 𝒳)},
instrumentScore P n o * boolReal o.2.2.1
∂sourceObsLaw P n) := by
change
(∫ o in {o | o.1 ∈ ({cell i} : Set 𝒳)},
instrumentScore P n o *
(o.2.2.2 - theta * boolReal o.2.2.1)
∂sourceObsLaw P n) = _
rw [show (fun o : SourceObs 𝒳 =>
instrumentScore P n o *
(o.2.2.2 - theta * boolReal o.2.2.1)) =
(fun o => instrumentScore P n o * o.2.2.2 -
theta * (instrumentScore P n o * boolReal o.2.2.1)) by
funext o
ring]
rw [integral_sub hOutcomeScoreIntegrable.integrableOn
(hReceiptScoreIntegrable.const_mul theta).integrableOn,
integral_const_mul]
rw [hscore, hOutcomeSet, hReceiptSet,
integral_singleton' (P.deltaY_measurable n).stronglyMeasurable,
integral_singleton' (P.deltaD_measurable n).stronglyMeasurable]
change
(sourceCellMass P n (cell i) * P.deltaY n (cell i) -
theta * (sourceCellMass P n (cell i) * P.deltaD n (cell i))) /
sourceCellMass P n (cell i) =
P.deltaY n (cell i) - theta * P.deltaD n (cell i)
field_simp [(hsourcePos i).ne']
have hbound (theta : ℝ) (htheta : theta ∈ parameterSpace)
(i : Fin (k n)) :
|regularCellScoreMean P n theta (cell i)| ≤ 2 := by
have hy := property_at_of_ae_of_singleton_pos
(sourceXLaw P n) (fun x => P.deltaY n x ∈ Icc (-1 : ℝ) 1)
(sourceObservationFacts_of_class P N k c epsilon n hIV).2.2.2.2.2.2.1 (hsourcePos i)
have hd := property_at_of_ae_of_singleton_pos
(sourceXLaw P n) (fun x => P.deltaD n x ∈ Icc (0 : ℝ) 1)
(sourceObservationFacts_of_class P N k c epsilon n hIV).2.2.2.2.2.2.2.2 (hsourcePos i)
have hthetaAbs : |theta| ≤ 1 := by
exact (abs_le).2 htheta
have hyAbs : |P.deltaY n (cell i)| ≤ 1 :=
(abs_le).2 hy
have hdAbs : |P.deltaD n (cell i)| ≤ 1 := by
rw [abs_of_nonneg hd.1]
exact hd.2
unfold regularCellScoreMean
calc
|P.deltaY n (cell i) - theta * P.deltaD n (cell i)| ≤
|P.deltaY n (cell i)| +
|theta| * |P.deltaD n (cell i)| := by
simpa [abs_mul] using
(abs_sub (P.deltaY n (cell i))
(theta * P.deltaD n (cell i)))
_ ≤ 1 + 1 * 1 := by gcongr
_ = 2 := by norm_num
refine ⟨hmean, hbound, ?_⟩
have hrangeMeas : MeasurableSet (range cell) := by
rw [show range cell = ⋃ i, {cell i} by
ext x
simp]
exact MeasurableSet.iUnion hcell
have hsourceCompl : sourceXLaw P n (range cell)ᶜ = 0 := by
rw [measure_compl hrangeMeas (measure_ne_top _ _), hrange, measure_univ]
simp
have htargetCompl : targetXLaw P n (range cell)ᶜ = 0 :=
hIV.transportDomination hsourceCompl
have htargetRange : targetXLaw P n (range cell) = 1 := by
calc
targetXLaw P n (range cell) =
targetXLaw P n univ :=
measure_of_measure_compl_eq_zero htargetCompl
_ = 1 := measure_univ
have htargetMeasure :
targetXLaw P n =
∑ i, targetXLaw P n {cell i} • Measure.dirac (cell i) :=
measure_eq_fin_sum_smul_dirac_of_range
(targetXLaw P n) cell hcell htargetRange
have hscoreMeanStrong :
StronglyMeasurable
(regularCellScoreMean P n (targetCACE P n)) :=
((P.deltaY_measurable n).sub
(measurable_const.mul (P.deltaD_measurable n))).stronglyMeasurable
have hsum :
(∑ i, (targetXLaw P n {cell i}).toReal *
regularCellScoreMean P n (targetCACE P n) (cell i)) =
∫ x, regularCellScoreMean P n (targetCACE P n) x
∂targetXLaw P n := by
conv_rhs => rw [htargetMeasure]
rw [integral_finset_sum_measure]
· apply Finset.sum_congr rfl
intro i hi
rw [integral_smul_measure,
integral_dirac' _ _ hscoreMeanStrong]
simp
· intro i hi
exact
(integrable_dirac' hscoreMeanStrong (by simp)).smul_measure
(measure_ne_top (targetXLaw P n) {cell i})
rw [hsum]
have hDeltaYTargetAE :
∀ᵐ x ∂targetXLaw P n, |P.deltaY n x| ≤ 1 := by
apply hIV.transportDomination.ae_le
filter_upwards [(sourceObservationFacts_of_class P N k c epsilon n hIV).2.2.2.2.2.2.1] with x hx
exact (abs_le).2 hx
have hDeltaDTargetAE :
∀ᵐ x ∂targetXLaw P n, |P.deltaD n x| ≤ 1 := by
apply hIV.transportDomination.ae_le
filter_upwards [(sourceObservationFacts_of_class P N k c epsilon n hIV).2.2.2.2.2.2.2.2] with x hx
rw [abs_of_nonneg hx.1]
exact hx.2
have hDeltaYTargetInt :
Integrable (P.deltaY n) (targetXLaw P n) :=
Integrable.of_bound (P.deltaY_measurable n).aestronglyMeasurable
1 (hDeltaYTargetAE.mono fun x hx => by simpa [Real.norm_eq_abs] using hx)
have hDeltaDTargetInt :
Integrable (P.deltaD n) (targetXLaw P n) :=
Integrable.of_bound (P.deltaD_measurable n).aestronglyMeasurable
1 (hDeltaDTargetAE.mono fun x hx => by simpa [Real.norm_eq_abs] using hx)
have hOutcomeTarget :
(∫ x, P.deltaY n x ∂targetXLaw P n) =
transportedOutcomeITT P n := by
simpa [transportedOutcomeITT, transportWeight, smul_eq_mul] using
(integral_rnDeriv_smul
(μ := targetXLaw P n) (ν := sourceXLaw P n)
(f := P.deltaY n) hIV.transportDomination).symm
have hFirstTarget :
(∫ x, P.deltaD n x ∂targetXLaw P n) =
transportedFirstStage P n := by
rw [transportedFirstStage_eq_weighted_deltaD P k epsilon n
-- … truncated; follow the source link for the rest …
theorem regularCell_score_mean_properties
On every realized cell, the score conditional mean is DeltaY - theta * DeltaD; it is bounded by two, and at the target CACE its target-mass average is zero.
Formal statement
hypotheses
P :
N k :
ℕ → ℕ
c epsilon cminus cplus :
ℝ
n :
ℕ
hAssignmentContrastIntegrable :
hReceiptContrastIntegrable :
hP :
RegularFiniteCellClass P N k c epsilon cminus cplus n
conclusion
∃ cell : Fin (k n) ↪ 𝒳,
(∀ theta ∈ parameterSpace, ∀ i, (∫ o in {o | o.1 = cell i}, regularCellScore (P.propensity n) theta o ∂sourceObsLaw P n) / sourceCellMass P n (cell i) = regularCellScoreMean P n theta (cell i)) ∧
= 0
Proof (Lean source)
lemma regularCell_score_mean_properties
(P : TransportedArray 𝒳) (N k : ℕ → ℕ)
(c epsilon cminus cplus : ℝ) (n : ℕ)
(hAssignmentContrastIntegrable :
Integrable (P.assignmentContrast n true) (sourceXLaw P n))
(hReceiptContrastIntegrable :
Integrable (P.receiptContrast n true) (sourceXLaw P n))
(hP : RegularFiniteCellClass P N k c epsilon cminus cplus n) :
∃ cell : Fin (k n) ↪ 𝒳,
(∀ theta ∈ parameterSpace, ∀ i,
(∫ o in {o | o.1 = cell i},
regularCellScore (P.propensity n) theta o ∂sourceObsLaw P n) /
sourceCellMass P n (cell i) =
regularCellScoreMean P n theta (cell i)) ∧
(∀ theta ∈ parameterSpace, ∀ i,
|regularCellScoreMean P n theta (cell i)| ≤ 2) ∧
(∑ i, (targetXLaw P n {cell i}).toReal *
regularCellScoreMean P n (targetCACE P n) (cell i)) = 0 := by
rcases hP with
⟨hIV, hk, hcminus, hcminus_one, hcplus,
cell, hcell, hrange, hmass⟩
refine ⟨cell, regularCell_score_mean_properties_for_witness
P N k c epsilon cminus cplus n
hAssignmentContrastIntegrable hReceiptContrastIntegrable
⟨hIV, hk, hcminus, hcminus_one, hcplus,
cell, hcell, hrange, hmass⟩
cell hcell hrange hmass⟩
theorem crossAverage_eq_empirical
The cross average equals the source-sample average of the score weighted by each source observation's empirical target-cell mass divided by its cell mass.
Formal statement
hypotheses
conclusion
crossAverage q source target G
= (n : ℝ)⁻¹ * ∑ i, (targetEmpiricalMass target (source i).1 / q (source i).1) * G (source i)
Proof (Lean source)
lemma crossAverage_eq_empirical
(q : 𝒳 → ℝ) {n N : ℕ}
(source : SourceSample 𝒳 n) (target : TargetSample 𝒳 N)
(G : SourceObs 𝒳 → ℝ) :
crossAverage q source target G =
(n : ℝ)⁻¹ * ∑ i,
(targetEmpiricalMass target (source i).1 / q (source i).1) *
G (source i) := by
classical
unfold crossAverage targetEmpiricalMass crossAverage
cellMoment empiricalMass
simp_rw [Finset.mul_sum]
rw [Finset.sum_comm]
apply Finset.sum_congr rfl
intro i hi
ring_nf
rw [Finset.mul_sum, Finset.sum_mul, Finset.sum_mul]
apply Finset.sum_congr rfl
intro j hj
by_cases hji : target j = (source i).1
· rw [hji]
simp only [ite_true]
ring
· have hij : (source i).1 ≠ target j := by simpa [eq_comm] using hji
simp only [if_neg hji, if_neg hij]
ring
theorem regularCell_crossAverage_identity
The difference of the two cross averages is the source average weighted by empirical target masses.
Formal statement
hypotheses
conclusion
regularCellContrastMoment q e theta source target
= (n : ℝ)⁻¹ * ∑ i, (targetEmpiricalMass target (source i).1 / q (source i).1) * regularCellScore e theta (source i)
Proof (Lean source)
lemma regularCell_crossAverage_identity
(q e : 𝒳 → ℝ) (theta : ℝ) {n N : ℕ}
(source : SourceSample 𝒳 n) (target : TargetSample 𝒳 N) :
regularCellContrastMoment q e theta source target =
(n : ℝ)⁻¹ * ∑ i,
(targetEmpiricalMass target (source i).1 / q (source i).1) *
regularCellScore e theta (source i) := by
rw [regularCellContrastMoment, regularCellOutcomeMoment,
regularCellReceiptMoment, crossAverage_eq_empirical,
crossAverage_eq_empirical]
simp only [regularCellScore]
simp_rw [Finset.mul_sum]
rw [← Finset.sum_sub_distrib]
apply Finset.sum_congr rfl
intro i hi
ring
theorem iid_average_integral
The expectation of an average of independent draws equals the population expectation of the summand.
Formal statement
hypotheses
Proof (Lean source)
lemma iid_average_integral
{Ω : Type*} [MeasurableSpace Ω] (μ : Measure Ω)
[IsProbabilityMeasure μ] (m : ℕ) (hm : 0 < m)
(F : Ω → ℝ) (hF : MemLp F 2 μ) :
(∫ sample : Fin m → Ω,
(m : ℝ)⁻¹ * ∑ i, F (sample i)
∂Measure.pi (fun _ : Fin m => μ)) =
∫ o, F o ∂μ := by
exact iid_average_integral μ m hm F
(hF.integrable (by norm_num))
theorem iid_average_variance
The variance of an average of positive-number independent draws equals the summand variance divided by the sample size.
Formal statement
hypotheses
Proof (Lean source)
lemma iid_average_variance
{Ω : Type*} [MeasurableSpace Ω] (μ : Measure Ω)
[IsProbabilityMeasure μ] (m : ℕ) (hm : 0 < m)
(F : Ω → ℝ) (hF : MemLp F 2 μ) :
variance
(fun sample : Fin m → Ω =>
(m : ℝ)⁻¹ * ∑ i, F (sample i))
(Measure.pi (fun _ : Fin m => μ)) =
(m : ℝ)⁻¹ * variance F μ := by
exact iid_average_variance μ m F hF
Helpers.RegularCellRisk_Part2 4 declarations Statistics and deterministic constants
Statistics and deterministic constants
theorem weighted_iid_average_mean_variance
For the source-sample average weighted by target cell masses, this result gives its mean, an upper bound on its variance, and square integrability. The mean is the target-mass-weighted sum of the cell means.
Formal statement
hypotheses
P :
n :
ℕ
hn :
0 < n
m :
ℕ
cell :
Fin m ↪ 𝒳
hcell :
∀ i, MeasurableSet {cell i}
hrange :
sourceXLaw P n (range cell) = 1
targetMass :
𝒳 → ℝ
G :
SourceObs 𝒳 → ℝ
a :
𝒳 → ℝ
C :
ℝ
hC :
0 ≤ C
hG :
MemLp G 2 (sourceObsLaw P n)
hGbound :
∀ᵐ o ∂sourceObsLaw P n, |G o| ≤ C
hqpos :
∀ i, 0 < sourceCellMass P n (cell i)
hmean :
conclusion
let F := fun o : SourceObs 𝒳
=> (targetMass o.1 / sourceCellMass P n o.1) * G o (∫ sample : SourceSample 𝒳 n, (n : ℝ)⁻¹ * ∑ i, F (sample i) ∂Measure.pi (fun _ : Fin n => sourceObsLaw P n)) = ∑ i, targetMass (cell i) * a (cell i) ∧
variance (fun sample : SourceSample 𝒳 n => (n : ℝ)⁻¹ * ∑ i, F (sample i)) (Measure.pi (fun _ : Fin n => sourceObsLaw P n))
≤ C ^ 2 / n * ∑ i, targetMass (cell i) ^ 2 / sourceCellMass P n (cell i) ∧
MemLp (fun sample : SourceSample 𝒳 n => (n : ℝ)⁻¹ * ∑ i, F (sample i)) 2 (Measure.pi (fun _ : Fin n => sourceObsLaw P n))
Proof (Lean source)
lemma weighted_iid_average_mean_variance
(P : TransportedArray 𝒳) (n : ℕ) (hn : 0 < n)
[IsProbabilityMeasure (sourceObsLaw P n)]
{m : ℕ} (cell : Fin m ↪ 𝒳)
(hcell : ∀ i, MeasurableSet {cell i})
(hrange : sourceXLaw P n (range cell) = 1)
(targetMass : 𝒳 → ℝ) (G : SourceObs 𝒳 → ℝ)
(a : 𝒳 → ℝ) (C : ℝ)
(hC : 0 ≤ C)
(hG : MemLp G 2 (sourceObsLaw P n))
(hGbound : ∀ᵐ o ∂sourceObsLaw P n, |G o| ≤ C)
(hqpos : ∀ i, 0 < sourceCellMass P n (cell i))
(hmean : ∀ i,
(∫ o in {o | o.1 = cell i}, G o ∂sourceObsLaw P n) /
sourceCellMass P n (cell i) = a (cell i)) :
let F := fun o : SourceObs 𝒳 =>
(targetMass o.1 / sourceCellMass P n o.1) * G o
(∫ sample : SourceSample 𝒳 n,
(n : ℝ)⁻¹ * ∑ i, F (sample i)
∂Measure.pi (fun _ : Fin n => sourceObsLaw P n)) =
∑ i, targetMass (cell i) * a (cell i) ∧
variance
(fun sample : SourceSample 𝒳 n =>
(n : ℝ)⁻¹ * ∑ i, F (sample i))
(Measure.pi (fun _ : Fin n => sourceObsLaw P n)) ≤
C ^ 2 / n * ∑ i,
targetMass (cell i) ^ 2 / sourceCellMass P n (cell i) ∧
MemLp
(fun sample : SourceSample 𝒳 n =>
(n : ℝ)⁻¹ * ∑ i, F (sample i)) 2
(Measure.pi (fun _ : Fin n => sourceObsLaw P n)) := by
classical
letI : IsProbabilityMeasure (sourceXLaw P n) := by
unfold sourceXLaw
exact Measure.isProbabilityMeasure_map measurable_fst.aemeasurable
let μ := sourceObsLaw P n
let q := sourceCellMass P n
let coeff : Fin m → ℝ := fun i => targetMass (cell i) / q (cell i)
let S : Fin m → Set (SourceObs 𝒳) := fun i => {o | o.1 = cell i}
let F : SourceObs 𝒳 → ℝ := fun o => (targetMass o.1 / q o.1) * G o
let B : SourceObs 𝒳 → ℝ := fun o =>
∑ i, coeff i * (S i).indicator G o
have hS (i : Fin m) : MeasurableSet (S i) := by
exact hcell i |>.preimage measurable_fst
have hBmem : MemLp B 2 μ := by
dsimp only [B]
apply memLp_finset_sum
intro i hi
exact (hG.indicator (hS i)).const_mul (coeff i)
have hrangeAE : ∀ᵐ o ∂μ, o.1 ∈ range cell := by
have hx : ∀ᵐ x ∂sourceXLaw P n, x ∈ range cell := by
change range cell ∈ ae (sourceXLaw P n)
rw [mem_ae_iff]
have hrangeMeas : MeasurableSet (range cell) := by
rw [show range cell = ⋃ i, {cell i} by
ext x
simp]
exact MeasurableSet.iUnion hcell
rw [measure_compl hrangeMeas (measure_ne_top _ _), hrange,
measure_univ]
simp
unfold sourceXLaw at hx
exact ae_of_ae_map measurable_fst.aemeasurable hx
have hFB : F =ᵐ[μ] B := by
filter_upwards [hrangeAE] with o ho
obtain ⟨i, hi⟩ := ho
dsimp only [F, B, coeff, S]
rw [← hi]
rw [Finset.sum_eq_single i]
· rw [Set.indicator_of_mem (show
o ∈ {z : SourceObs 𝒳 | z.1 = cell i} by exact hi.symm)]
· intro j hj hji
simp only [Set.indicator_apply, Set.mem_setOf_eq]
have hne : cell i ≠ cell j :=
fun h => hji (cell.injective h.symm)
have hneO : o.1 ≠ cell j := fun h => hne (hi.trans h)
simp [hneO]
· simp
have hFmem : MemLp F 2 μ := by
refine ⟨hBmem.1.congr hFB.symm, ?_⟩
rw [eLpNorm_congr_ae hFB]
exact hBmem.2
have hBsq (o : SourceObs 𝒳) :
B o ^ 2 =
∑ i, coeff i ^ 2 * (S i).indicator (fun z => G z ^ 2) o := by
by_cases ho : o.1 ∈ range cell
· obtain ⟨i, hi⟩ := ho
have hoi : o.1 = cell i := hi.symm
dsimp only [B, coeff, S]
rw [Finset.sum_eq_single i]
· rw [Finset.sum_eq_single i]
· simp only [Set.indicator_of_mem, Set.mem_setOf_eq, hoi]
ring
· intro j hj hji
simp only [Set.indicator_apply, Set.mem_setOf_eq]
have hne : o.1 ≠ cell j := fun h =>
hji (cell.injective (hoi.symm.trans h).symm)
simp [hne]
· simp
· intro j hj hji
simp only [Set.indicator_apply, Set.mem_setOf_eq]
have hne : o.1 ≠ cell j := fun h =>
hji (cell.injective (hoi.symm.trans h).symm)
simp [hne]
· simp
· have hnot (i : Fin m) : o ∉ S i := by
intro hi
exact ho ⟨i, hi.symm⟩
simp [B, hnot]
have hsetMeasure (i : Fin m) :
(μ (S i)).toReal = q (cell i) := by
change
(sourceObsLaw P n {o | o.1 = cell i}).toReal =
(sourceXLaw P n {cell i}).toReal
unfold sourceXLaw
rw [Measure.map_apply measurable_fst (hcell i)]
rfl
have hsetSquare (i : Fin m) :
(∫ o in S i, G o ^ 2 ∂μ) ≤ C ^ 2 * q (cell i) := by
calc
(∫ o in S i, G o ^ 2 ∂μ) ≤ ∫ _o in S i, C ^ 2 ∂μ := by
apply integral_mono_ae
· exact hG.integrable_sq.integrableOn
· exact integrableOn_const
· filter_upwards [ae_restrict_of_ae hGbound] with o ho
have habs := (abs_le.mp ho)
nlinarith [sq_nonneg (G o)]
_ = C ^ 2 * q (cell i) := by
rw [setIntegral_const]
change (μ (S i)).toReal * C ^ 2 = C ^ 2 * q (cell i)
rw [hsetMeasure]
ring
have hBsqInt :
(∫ o, B o ^ 2 ∂μ) ≤
C ^ 2 * ∑ i, targetMass (cell i) ^ 2 / q (cell i) := by
simp_rw [hBsq]
rw [integral_finset_sum]
· calc
∑ i, ∫ o, coeff i ^ 2 *
(S i).indicator (fun z => G z ^ 2) o ∂μ
= ∑ i, coeff i ^ 2 * ∫ o in S i, G o ^ 2 ∂μ := by
apply Finset.sum_congr rfl
intro i hi
rw [integral_const_mul, integral_indicator (hS i)]
_ ≤ ∑ i, coeff i ^ 2 * (C ^ 2 * q (cell i)) := by
gcongr with i
exact hsetSquare i
_ = C ^ 2 * ∑ i,
targetMass (cell i) ^ 2 / q (cell i) := by
rw [Finset.mul_sum]
apply Finset.sum_congr rfl
intro i hi
dsimp only [coeff]
field_simp [(hqpos i).ne']
<;> ring
· intro i hi
exact
((hG.integrable_sq.indicator (hS i)).const_mul (coeff i ^ 2))
have hFmean :
(∫ o, F o ∂μ) = ∑ i, targetMass (cell i) * a (cell i) := by
rw [integral_congr_ae hFB]
dsimp only [B]
rw [integral_finset_sum]
· calc
∑ i, ∫ o, coeff i * (S i).indicator G o ∂μ =
∑ i, coeff i * ∫ o in S i, G o ∂μ := by
apply Finset.sum_congr rfl
intro i hi
rw [integral_const_mul, integral_indicator (hS i)]
_ = ∑ i, targetMass (cell i) * a (cell i) := by
apply Finset.sum_congr rfl
intro i hi
have hm := hmean i
dsimp only [coeff, q, S] at hm ⊢
rw [div_eq_iff (hqpos i).ne'] at hm
change targetMass (cell i) / sourceCellMass P n (cell i) *
(∫ o in {o | o.1 = cell i}, G o ∂sourceObsLaw P n) =
targetMass (cell i) * a (cell i)
rw [hm]
field_simp [(hqpos i).ne']
· intro i hi
exact ((hG.integrable (by norm_num)).indicator (hS i)).const_mul (coeff i)
refine ⟨?_, ?_, ?_⟩
· simpa only [F] using
(iid_average_integral μ n hn F hFmem).trans hFmean
· rw [iid_average_variance μ n hn F hFmem]
calc
(n : ℝ)⁻¹ * variance F μ ≤
(n : ℝ)⁻¹ * ∫ o, F o ^ 2 ∂μ := by
gcongr
exact variance_le_expectation_sq hFmem.aestronglyMeasurable
_ = (n : ℝ)⁻¹ * ∫ o, B o ^ 2 ∂μ := by
congr 1
apply integral_congr_ae
filter_upwards [hFB] with o ho
rw [ho]
_ ≤ (n : ℝ)⁻¹ *
(C ^ 2 * ∑ i, targetMass (cell i) ^ 2 / q (cell i)) := by
gcongr
_ = C ^ 2 / n *
∑ i, targetMass (cell i) ^ 2 / q (cell i) := by ring
· apply MemLp.const_mul
apply memLp_finset_sum
intro i hi
exact hFmem.comp_measurePreserving
(measurePreserving_eval (fun _ : Fin n => μ) i)
theorem integral_targetEmpiricalMass_sq
The second moment of a cell's empirical target mass equals the squared population cell mass plus its binomial sampling correction.
Formal statement
hypotheses
conclusion
= μ.real {a} ^ 2 + (m : ℝ)⁻¹ * (μ.real {a} - μ.real {a} ^ 2)
Proof (Lean source)
lemma integral_targetEmpiricalMass_sq
(μ : Measure 𝒳) [IsProbabilityMeasure μ] {m : ℕ} (hm : 0 < m)
(a : 𝒳) (ha : MeasurableSet {a}) :
(∫ target : Fin m → 𝒳, targetEmpiricalMass target a ^ 2
∂Measure.pi (fun _ : Fin m => μ)) =
μ.real {a} ^ 2 + (m : ℝ)⁻¹ * (μ.real {a} - μ.real {a} ^ 2) := by
exact integral_empiricalMass_sq μ hm a ha
theorem regularCell_source_conditional_mean_variance_for_witness
Conditional on a fixed target sample, the source mean and variance have the paper's finite-cell expressions.
Formal statement
hypotheses
P :
N k :
ℕ → ℕ
c epsilon cminus cplus theta :
ℝ
n :
ℕ
target :
TargetSample 𝒳 (N n)
hn :
0 < n
hIV :
TransportedIVClass P N k c epsilon n
hk :
0 < k n
hcm :
0 < cminus
hcmOne :
cminus ≤ 1
hcp :
1 ≤ cplus
cell :
Fin (k n) ↪ 𝒳
hcell :
∀ i, MeasurableSet {cell i}
hrange :
sourceXLaw P n (range cell) = 1
hmass :
∀ i,
cminus / (k n : ℝ) ≤ (sourceXLaw P n {cell i}).toReal ∧
(sourceXLaw P n {cell i}).toReal ≤ cplus / (k n : ℝ)
htheta :
theta ∈ parameterSpace
conclusion
let q := sourceCellMass P n let e := P.propensity n (∫ source, regularCellContrastMoment q e theta source target ∂Measure.pi (fun _ : Fin n => sourceObsLaw P n))
= ∑ i, targetEmpiricalMass target (cell i) * regularCellScoreMean P n theta (cell i) ∧
variance (fun source => regularCellContrastMoment q e theta source target) (Measure.pi (fun _ : Fin n => sourceObsLaw P n))
≤ 4 / (epsilon ^ 2 * n) * ∑ i, targetEmpiricalMass target (cell i) ^ 2 / q (cell i) ∧
MemLp (fun source => regularCellContrastMoment q e theta source target) 2 (Measure.pi (fun _ : Fin n => sourceObsLaw P n)) ∧
(∀ i, (∫ o in {o | o.1 = cell i}, regularCellScore e theta o ∂sourceObsLaw P n) / q (cell i) = regularCellScoreMean P n theta (cell i)) ∧
(∀ᵐ o ∂sourceObsLaw P n, |regularCellScore e theta o| ≤ 2 / epsilon)
Proof (Lean source)
lemma regularCell_source_conditional_mean_variance_for_witness
(P : TransportedArray 𝒳) (N k : ℕ → ℕ)
(c epsilon cminus cplus theta : ℝ) (n : ℕ)
(target : TargetSample 𝒳 (N n))
(hn : 0 < n)
(hIV : TransportedIVClass P N k c epsilon n)
(hk : 0 < k n) (hcm : 0 < cminus)
(hcmOne : cminus ≤ 1) (hcp : 1 ≤ cplus)
(cell : Fin (k n) ↪ 𝒳)
(hcell : ∀ i, MeasurableSet {cell i})
(hrange : sourceXLaw P n (range cell) = 1)
(hmass : ∀ i,
cminus / (k n : ℝ) ≤ (sourceXLaw P n {cell i}).toReal ∧
(sourceXLaw P n {cell i}).toReal ≤ cplus / (k n : ℝ))
(htheta : theta ∈ parameterSpace) :
let q := sourceCellMass P n
let e := P.propensity n
(∫ source, regularCellContrastMoment q e theta source target
∂Measure.pi (fun _ : Fin n => sourceObsLaw P n)) =
∑ i, targetEmpiricalMass target (cell i) *
regularCellScoreMean P n theta (cell i) ∧
variance
(fun source => regularCellContrastMoment q e theta source target)
(Measure.pi (fun _ : Fin n => sourceObsLaw P n)) ≤
4 / (epsilon ^ 2 * n) *
∑ i, targetEmpiricalMass target (cell i) ^ 2 / q (cell i) ∧
MemLp
(fun source => regularCellContrastMoment q e theta source target) 2
(Measure.pi (fun _ : Fin n => sourceObsLaw P n)) ∧
(∀ i,
(∫ o in {o | o.1 = cell i},
regularCellScore e theta o ∂sourceObsLaw P n) / q (cell i) =
regularCellScoreMean P n theta (cell i)) ∧
MemLp (regularCellScore e theta) 2 (sourceObsLaw P n) ∧
(∀ᵐ o ∂sourceObsLaw P n,
|regularCellScore e theta o| ≤ 2 / epsilon) := by
classical
letI : IsProbabilityMeasure (sourceObsLaw P n) :=
hIV.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (sourceXLaw P n) := by
unfold sourceXLaw
exact Measure.isProbabilityMeasure_map measurable_fst.aemeasurable
have hqpos (i : Fin (k n)) :
0 < sourceCellMass P n (cell i) := by
exact lt_of_lt_of_le
(div_pos hcm (by exact_mod_cast hk)) (hmass i).1
have hInstrumentMeasurable : Measurable (instrumentScore P n) := by
have hz : Measurable fun o : SourceObs 𝒳 => o.2.1 := by fun_prop
have he : Measurable fun o : SourceObs 𝒳 => P.propensity n o.1 :=
(P.propensity_measurable n).comp measurable_fst
unfold instrumentScore
exact Measurable.ite (hz (MeasurableSet.singleton true))
(measurable_const.div he)
(measurable_const.neg.div (measurable_const.sub he))
have hInstrumentBound : ∀ᵐ o ∂sourceObsLaw P n,
|instrumentScore P n o| ≤ 1 / epsilon := by
have hOverlapObs : ∀ᵐ o ∂sourceObsLaw P n,
epsilon ≤ P.propensity n o.1 ∧
P.propensity n o.1 ≤ 1 - epsilon := by
have hx := hIV.instrumentOverlap.2.2
unfold sourceXLaw at hx
exact ae_of_ae_map measurable_fst.aemeasurable hx
filter_upwards [hOverlapObs] with o ho
rcases o with ⟨x, z, d, y⟩
cases z
· simp only [instrumentScore, Bool.false_eq_true, ↓reduceIte, abs_neg,
abs_div, abs_one]
rw [abs_of_pos (sub_pos.mpr (lt_of_le_of_lt ho.2
(by linarith [hIV.instrumentOverlap.1])))]
exact one_div_le_one_div_of_le hIV.instrumentOverlap.1
(by linarith [ho.2])
· simp only [instrumentScore, ↓reduceIte, abs_div, abs_one]
rw [abs_of_pos (lt_of_lt_of_le hIV.instrumentOverlap.1 ho.1)]
exact one_div_le_one_div_of_le hIV.instrumentOverlap.1 ho.1
have hBoolMeasurable :
Measurable (fun o : SourceObs 𝒳 => boolReal o.2.2.1) := by
unfold boolReal
have hd : Measurable fun o : SourceObs 𝒳 => o.2.2.1 := by fun_prop
exact Measurable.ite (hd (MeasurableSet.singleton true))
measurable_const measurable_const
have hOutcomeScoreIntegrable :
Integrable (fun o => instrumentScore P n o * o.2.2.2)
(sourceObsLaw P n) := by
refine Integrable.of_bound
(hInstrumentMeasurable.mul (by fun_prop)).aestronglyMeasurable
(1 / epsilon) ?_
filter_upwards [hInstrumentBound, (sourceObservationFacts_of_class P N k c epsilon n hIV).2.2.1]
with o hs hy
rw [Real.norm_eq_abs, abs_mul, abs_of_nonneg hy.1]
exact (mul_le_mul hs hy.2 hy.1
(one_div_nonneg.mpr hIV.instrumentOverlap.1.le)).trans_eq (mul_one _)
have hReceiptScoreIntegrable :
Integrable (fun o =>
instrumentScore P n o * boolReal o.2.2.1)
(sourceObsLaw P n) := by
refine Integrable.of_bound
(hInstrumentMeasurable.mul hBoolMeasurable).aestronglyMeasurable
(1 / epsilon) ?_
filter_upwards [hInstrumentBound] with o hs
have hd : |boolReal o.2.2.1| ≤ 1 := by
cases o.2.2.1 <;> simp [boolReal]
rw [Real.norm_eq_abs, abs_mul]
exact (mul_le_mul hs hd (abs_nonneg _)
(one_div_nonneg.mpr hIV.instrumentOverlap.1.le)).trans_eq (mul_one _)
have hmean (i : Fin (k n)) :
(∫ o in {o | o.1 = cell i},
regularCellScore (P.propensity n) theta o ∂sourceObsLaw P n) /
sourceCellMass P n (cell i) =
regularCellScoreMean P n theta (cell i) := by
have hOutcomeSet :=
(sourceObservationFacts_of_class P N k c epsilon n hIV).2.2.2.2.2.1
({cell i} : Set 𝒳) (hcell i)
have hReceiptSet :=
(sourceObservationFacts_of_class P N k c epsilon n hIV).2.2.2.2.2.2.2.1
({cell i} : Set 𝒳) (hcell i)
have hscore :
(∫ o in {o | o.1 = cell i},
regularCellScore (P.propensity n) theta o ∂sourceObsLaw P n) =
(∫ o in {o | o.1 ∈ ({cell i} : Set 𝒳)},
instrumentScore P n o * o.2.2.2 ∂sourceObsLaw P n) -
theta * (∫ o in {o | o.1 ∈ ({cell i} : Set 𝒳)},
instrumentScore P n o * boolReal o.2.2.1
∂sourceObsLaw P n) := by
change
(∫ o in {o | o.1 ∈ ({cell i} : Set 𝒳)},
instrumentScore P n o *
(o.2.2.2 - theta * boolReal o.2.2.1)
∂sourceObsLaw P n) = _
rw [show (fun o : SourceObs 𝒳 =>
instrumentScore P n o *
(o.2.2.2 - theta * boolReal o.2.2.1)) =
(fun o => instrumentScore P n o * o.2.2.2 -
theta * (instrumentScore P n o * boolReal o.2.2.1)) by
funext o
ring]
rw [integral_sub hOutcomeScoreIntegrable.integrableOn
(hReceiptScoreIntegrable.const_mul theta).integrableOn,
integral_const_mul]
rw [hscore, hOutcomeSet, hReceiptSet,
integral_singleton' (P.deltaY_measurable n).stronglyMeasurable,
integral_singleton' (P.deltaD_measurable n).stronglyMeasurable]
change
(sourceCellMass P n (cell i) * P.deltaY n (cell i) -
theta * (sourceCellMass P n (cell i) * P.deltaD n (cell i))) /
sourceCellMass P n (cell i) =
P.deltaY n (cell i) - theta * P.deltaD n (cell i)
field_simp [(hqpos i).ne']
have hScoreMeas :
Measurable (regularCellScore (P.propensity n) theta) := by
change Measurable (fun o : SourceObs 𝒳 =>
instrumentScore P n o *
(o.2.2.2 - theta * boolReal o.2.2.1))
have hyMeas : Measurable (fun o : SourceObs 𝒳 => o.2.2.2) := by
fun_prop
exact hInstrumentMeasurable.mul
(hyMeas.sub (measurable_const.mul hBoolMeasurable))
have hScoreBound : ∀ᵐ o ∂sourceObsLaw P n,
|regularCellScore (P.propensity n) theta o| ≤ 2 / epsilon := by
have hOverlapObs : ∀ᵐ o ∂sourceObsLaw P n,
epsilon ≤ P.propensity n o.1 ∧
P.propensity n o.1 ≤ 1 - epsilon := by
have hx := hIV.instrumentOverlap.2.2
unfold sourceXLaw at hx
exact ae_of_ae_map measurable_fst.aemeasurable hx
filter_upwards [hOverlapObs, (sourceObservationFacts_of_class P N k c epsilon n hIV).2.2.1] with o he hy
exact abs_regularCellScore_le (P.propensity n) epsilon theta o
hIV.instrumentOverlap.1 he htheta hy
have hScoreMem :
MemLp (regularCellScore (P.propensity n) theta) 2
(sourceObsLaw P n) :=
MemLp.of_bound hScoreMeas.aestronglyMeasurable (2 / epsilon)
(hScoreBound.mono fun o ho => by simpa [Real.norm_eq_abs] using ho)
obtain ⟨hMean, hVar, hSourceMem⟩ :=
weighted_iid_average_mean_variance P n hn cell hcell hrange
(targetEmpiricalMass target)
(regularCellScore (P.propensity n) theta)
(regularCellScoreMean P n theta) (2 / epsilon)
(div_nonneg (by norm_num) hIV.instrumentOverlap.1.le)
hScoreMem hScoreBound hqpos hmean
refine ⟨?_, ?_, ?_, hmean, hScoreMem, hScoreBound⟩
· simpa only [regularCell_crossAverage_identity] using hMean
· simp only [regularCell_crossAverage_identity]
calc
variance
(fun source : SourceSample 𝒳 n =>
(n : ℝ)⁻¹ * ∑ i,
(targetEmpiricalMass target (source i).1 /
sourceCellMass P n (source i).1) *
regularCellScore (P.propensity n) theta (source i))
(Measure.pi (fun _ : Fin n => sourceObsLaw P n)) ≤
(2 / epsilon) ^ 2 / n *
∑ i, targetEmpiricalMass target (cell i) ^ 2 /
sourceCellMass P n (cell i) := hVar
_ = 4 / (epsilon ^ 2 * n) *
∑ i, targetEmpiricalMass target (cell i) ^ 2 /
sourceCellMass P n (cell i) := by
congr 1
field_simp [hIV.instrumentOverlap.1.ne']
<;> ring
· simpa only [regularCell_crossAverage_identity] using hSourceMem
Conditional on a fixed target sample, the source mean and variance have the paper's finite-cell expressions.
Formal statement
hypotheses
P :
N k :
ℕ → ℕ
c epsilon cminus cplus theta :
ℝ
n :
ℕ
target :
TargetSample 𝒳 (N n)
hn :
0 < n
hP :
RegularFiniteCellClass P N k c epsilon cminus cplus n
htheta :
theta ∈ parameterSpace
conclusion
let q := sourceCellMass P n let e := P.propensity n ∃ cell : Fin (k n) ↪ 𝒳, (∫ source, regularCellContrastMoment q e theta source target ∂Measure.pi (fun _ : Fin n => sourceObsLaw P n))
= ∑ i, targetEmpiricalMass target (cell i) * regularCellScoreMean P n theta (cell i) ∧
variance (fun source => regularCellContrastMoment q e theta source target) (Measure.pi (fun _ : Fin n => sourceObsLaw P n))
≤ 4 / (epsilon ^ 2 * n) * ∑ i, targetEmpiricalMass target (cell i) ^ 2 / q (cell i)
Proof (Lean source)
lemma regularCell_source_conditional_mean_variance
(P : TransportedArray 𝒳) (N k : ℕ → ℕ)
(c epsilon cminus cplus theta : ℝ) (n : ℕ)
(target : TargetSample 𝒳 (N n))
(hn : 0 < n)
(hP : RegularFiniteCellClass P N k c epsilon cminus cplus n)
(htheta : theta ∈ parameterSpace) :
let q := sourceCellMass P n
let e := P.propensity n
∃ cell : Fin (k n) ↪ 𝒳,
(∫ source, regularCellContrastMoment q e theta source target
∂Measure.pi (fun _ : Fin n => sourceObsLaw P n)) =
∑ i, targetEmpiricalMass target (cell i) *
regularCellScoreMean P n theta (cell i) ∧
variance
(fun source => regularCellContrastMoment q e theta source target)
(Measure.pi (fun _ : Fin n => sourceObsLaw P n)) ≤
4 / (epsilon ^ 2 * n) *
∑ i, targetEmpiricalMass target (cell i) ^ 2 / q (cell i) := by
rcases hP with
⟨hIV, hk, hcminus, hcminus_one, hcplus, cell, hcell, hrange, hmass⟩
have h := regularCell_source_conditional_mean_variance_for_witness
P N k c epsilon cminus cplus theta n target hn hIV hk hcminus
hcminus_one hcplus cell hcell hrange hmass htheta
exact ⟨cell, h.1, h.2.1⟩
Helpers.RegularCellRisk_Part3 6 declarations Statistics and deterministic constants
Statistics and deterministic constants
theorem regularCell_target_score_mean_variance_for_witness
At the target CACE, randomness of the target empirical score mean contributes at most 4 / N.
Formal statement
hypotheses
P :
N k :
ℕ → ℕ
c epsilon cminus cplus :
ℝ
n :
ℕ
hIV :
TransportedIVClass P N k c epsilon n
hk :
0 < k n
hcminus :
0 < cminus
hcminus_one :
cminus ≤ 1
hcplus :
1 ≤ cplus
cell :
Fin (k n) ↪ 𝒳
hcell :
∀ i, MeasurableSet {cell i}
hrange :
sourceXLaw P n (range cell) = 1
hmass :
∀ i,
cminus / (k n : ℝ) ≤ (sourceXLaw P n {cell i}).toReal ∧
(sourceXLaw P n {cell i}).toReal ≤ cplus / (k n : ℝ)
conclusion
variance (fun target : TargetSample 𝒳 (N n) => ∑ i, targetEmpiricalMass target (cell i) * regularCellScoreMean P n (targetCACE P n) (cell i)) (Measure.pi (fun _ : Fin (N n) => targetXLaw P n))
≤ 4 / (N n : ℝ) ∧
MemLp (fun target : TargetSample 𝒳 (N n) => ∑ i, targetEmpiricalMass target (cell i) * regularCellScoreMean P n (targetCACE P n) (cell i)) 2 (Measure.pi (fun _ : Fin (N n) => targetXLaw P n)) ∧
(∫ target : TargetSample 𝒳 (N n), ∑ i, targetEmpiricalMass target (cell i) * regularCellScoreMean P n (targetCACE P n) (cell i) ∂Measure.pi (fun _ : Fin (N n) => targetXLaw P n))
= 0
Proof (Lean source)
lemma regularCell_target_score_mean_variance_for_witness
(P : TransportedArray 𝒳) (N k : ℕ → ℕ)
(c epsilon cminus cplus : ℝ) (n : ℕ)
(hIV : TransportedIVClass P N k c epsilon n)
(hk : 0 < k n) (hcminus : 0 < cminus)
(hcminus_one : cminus ≤ 1) (hcplus : 1 ≤ cplus)
(cell : Fin (k n) ↪ 𝒳)
(hcell : ∀ i, MeasurableSet {cell i})
(hrange : sourceXLaw P n (range cell) = 1)
(hmass : ∀ i,
cminus / (k n : ℝ) ≤ (sourceXLaw P n {cell i}).toReal ∧
(sourceXLaw P n {cell i}).toReal ≤ cplus / (k n : ℝ)) :
variance
(fun target : TargetSample 𝒳 (N n) =>
∑ i, targetEmpiricalMass target (cell i) *
regularCellScoreMean P n (targetCACE P n) (cell i))
(Measure.pi (fun _ : Fin (N n) => targetXLaw P n)) ≤
4 / (N n : ℝ) ∧
MemLp
(fun target : TargetSample 𝒳 (N n) =>
∑ i, targetEmpiricalMass target (cell i) *
regularCellScoreMean P n (targetCACE P n) (cell i)) 2
(Measure.pi (fun _ : Fin (N n) => targetXLaw P n)) ∧
(∫ target : TargetSample 𝒳 (N n),
∑ i, targetEmpiricalMass target (cell i) *
regularCellScoreMean P n (targetCACE P n) (cell i)
∂Measure.pi (fun _ : Fin (N n) => targetXLaw P n)) = 0 := by
classical
letI : IsProbabilityMeasure (sourceObsLaw P n) :=
hIV.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (sourceXLaw P n) := by
unfold sourceXLaw
exact Measure.isProbabilityMeasure_map measurable_fst.aemeasurable
letI : IsProbabilityMeasure (targetXLaw P n) :=
hIV.twoSampleArray.2.2.1 n
have hAssignmentInt :
Integrable (P.assignmentContrast n true) (sourceXLaw P n) :=
integrable_of_finite_support (sourceXLaw P n) cell hcell hrange
_ (P.assignmentContrast_measurable n true).stronglyMeasurable
have hReceiptInt :
Integrable (P.receiptContrast n true) (sourceXLaw P n) :=
integrable_of_finite_support (sourceXLaw P n) cell hcell hrange
_ (P.receiptContrast_measurable n true).stronglyMeasurable
rcases compact_causal_range P N k c epsilon n
hAssignmentInt hReceiptInt hIV with
⟨hDeltaYEq, hDeltaDEq, hOutcomeEq, hFirstEq, hRatio, hTheta⟩
have hsourcePos (i : Fin (k n)) :
0 < (sourceXLaw P n {cell i}).toReal := by
exact lt_of_lt_of_le
(div_pos hcminus (by exact_mod_cast hk)) (hmass i).1
have hcellBound (i : Fin (k n)) :
|regularCellScoreMean P n (targetCACE P n) (cell i)| ≤ 2 := by
have hy := property_at_of_ae_of_singleton_pos
(sourceXLaw P n) (fun x => P.deltaY n x ∈ Icc (-1 : ℝ) 1)
(sourceObservationFacts_of_class P N k c epsilon n hIV).2.2.2.2.2.2.1 (hsourcePos i)
have hd := property_at_of_ae_of_singleton_pos
(sourceXLaw P n) (fun x => P.deltaD n x ∈ Icc (0 : ℝ) 1)
(sourceObservationFacts_of_class P N k c epsilon n hIV).2.2.2.2.2.2.2.2 (hsourcePos i)
have hthetaAbs : |targetCACE P n| ≤ 1 := (abs_le).2 hTheta
have hyAbs : |P.deltaY n (cell i)| ≤ 1 := (abs_le).2 hy
have hdAbs : |P.deltaD n (cell i)| ≤ 1 := by
rw [abs_of_nonneg hd.1]
exact hd.2
unfold regularCellScoreMean
calc
|P.deltaY n (cell i) -
targetCACE P n * P.deltaD n (cell i)| ≤
|P.deltaY n (cell i)| +
|targetCACE P n| * |P.deltaD n (cell i)| := by
simpa [abs_mul] using
(abs_sub (P.deltaY n (cell i))
(targetCACE P n * P.deltaD n (cell i)))
_ ≤ 1 + 1 * 1 := by gcongr
_ = 2 := by norm_num
let B : 𝒳 → ℝ := fun x =>
if hx : x ∈ range cell
then regularCellScoreMean P n (targetCACE P n) x else 0
have hrangeMeas : MeasurableSet (range cell) := by
rw [show range cell = ⋃ i, {cell i} by
ext x
simp]
exact MeasurableSet.iUnion hcell
have hBMeas : Measurable B := by
dsimp [B]
exact Measurable.ite hrangeMeas
((P.deltaY_measurable n).sub
(measurable_const.mul (P.deltaD_measurable n)))
measurable_const
have hBBound : ∀ x, |B x| ≤ 2 := by
intro x
by_cases hx : x ∈ range cell
· obtain ⟨i, rfl⟩ := hx
simpa [B] using hcellBound i
· simp only [B, dif_neg hx, abs_zero]
norm_num
have hBMem : MemLp B 2 (targetXLaw P n) :=
MemLp.of_bound hBMeas.aestronglyMeasurable 2
(Filter.Eventually.of_forall fun x => by
simpa [Real.norm_eq_abs] using hBBound x)
by_cases hNzero : N n = 0
· rw [hNzero]
refine ⟨?_, ?_, ?_⟩
· simp [targetEmpiricalMass, empiricalMass, variance, evariance]
· simp [targetEmpiricalMass, empiricalMass]
· simp [targetEmpiricalMass, empiricalMass]
have hNpos : 0 < N n := Nat.pos_of_ne_zero hNzero
have hstat :
(fun target : TargetSample 𝒳 (N n) =>
∑ i, targetEmpiricalMass target (cell i) *
regularCellScoreMean P n (targetCACE P n) (cell i)) =
(fun target => (N n : ℝ)⁻¹ * ∑ j, B (target j)) := by
funext target
unfold targetEmpiricalMass empiricalMass
calc
_ =
∑ i, ∑ j, (N n : ℝ)⁻¹ *
(if target j = cell i then 1 else 0) *
regularCellScoreMean P n (targetCACE P n) (cell i) := by
apply Finset.sum_congr rfl
intro i hi
rw [Finset.mul_sum, Finset.sum_mul]
_ = ∑ j, ∑ i, (N n : ℝ)⁻¹ *
(if target j = cell i then 1 else 0) *
regularCellScoreMean P n (targetCACE P n) (cell i) := by
rw [Finset.sum_comm]
_ = ∑ j, (N n : ℝ)⁻¹ * B (target j) := by
apply Finset.sum_congr rfl
intro j hj
simp_rw [mul_assoc]
rw [← Finset.mul_sum]
congr 1
by_cases hx : target j ∈ range cell
· obtain ⟨i, hi⟩ := hx
rw [← hi]
rw [Finset.sum_eq_single i]
· simp [B]
· intro i' hi' hne
have htargetne : target j ≠ cell i' := by
rw [← hi]
exact fun h => hne (cell.injective h.symm)
have hcellne : cell i ≠ cell i' :=
fun h => hne (cell.injective h).symm
rw [if_neg hcellne]
simp
· simp
· have hne : ∀ i, target j ≠ cell i := by
intro i hEq
exact hx ⟨i, hEq.symm⟩
have hne' : ∀ i, cell i ≠ target j :=
fun i hEq => hne i hEq.symm
simp [B, hx, hne, hne']
_ = (N n : ℝ)⁻¹ * ∑ j, B (target j) := by
rw [Finset.mul_sum]
have hAvgMem :
MemLp (fun target : TargetSample 𝒳 (N n) =>
(N n : ℝ)⁻¹ * ∑ j, B (target j)) 2
(Measure.pi (fun _ : Fin (N n) => targetXLaw P n)) := by
apply MemLp.const_mul
apply memLp_finset_sum
intro j hj
exact hBMem.comp_measurePreserving
(measurePreserving_eval (fun _ : Fin (N n) => targetXLaw P n) j)
refine ⟨?_, ?_, ?_⟩
· rw [hstat, iid_average_variance
(targetXLaw P n) (N n) hNpos B hBMem]
calc
(N n : ℝ)⁻¹ * variance B (targetXLaw P n) ≤
(N n : ℝ)⁻¹ *
∫ x, B x ^ 2 ∂targetXLaw P n := by
gcongr
exact variance_le_expectation_sq hBMeas.aestronglyMeasurable
_ ≤ (N n : ℝ)⁻¹ * 4 := by
gcongr
calc
(∫ x, B x ^ 2 ∂targetXLaw P n) ≤ ∫ _x, (4 : ℝ)
∂targetXLaw P n := by
apply integral_mono_of_nonneg
· exact Filter.Eventually.of_forall fun x => sq_nonneg (B x)
· exact integrable_const 4
· exact Filter.Eventually.of_forall fun x => by
have hb := (abs_le).1 (hBBound x)
nlinarith
_ = 4 := by simp
_ = 4 / (N n : ℝ) := by ring
· rw [hstat]
exact hAvgMem
· rw [hstat, iid_average_integral
(targetXLaw P n) (N n) hNpos B hBMem]
have hrangeCompl : sourceXLaw P n (range cell)ᶜ = 0 := by
rw [measure_compl hrangeMeas (measure_ne_top _ _), hrange, measure_univ]
simp
have htargetRange :
∀ᵐ x ∂targetXLaw P n, x ∈ range cell := by
have hc := hIV.transportDomination hrangeCompl
change range cell ∈ ae (targetXLaw P n)
rw [mem_ae_iff]
exact hc
have hBScore :
B =ᵐ[targetXLaw P n]
regularCellScoreMean P n (targetCACE P n) := by
filter_upwards [htargetRange] with x hx
dsimp only [B]
rw [dif_pos hx]
rw [integral_congr_ae hBScore]
have hDeltaYTargetInt :
Integrable (P.deltaY n) (targetXLaw P n) := by
refine Integrable.of_bound
(P.deltaY_measurable n).aestronglyMeasurable 1 ?_
apply hIV.transportDomination.ae_le
filter_upwards [(sourceObservationFacts_of_class P N k c epsilon n hIV).2.2.2.2.2.2.1] with x hx
simpa [Real.norm_eq_abs] using (show |P.deltaY n x| ≤ 1 from
(abs_le).2 hx)
have hDeltaDTargetInt :
Integrable (P.deltaD n) (targetXLaw P n) := by
refine Integrable.of_bound
(P.deltaD_measurable n).aestronglyMeasurable 1 ?_
apply hIV.transportDomination.ae_le
filter_upwards [(sourceObservationFacts_of_class P N k c epsilon n hIV).2.2.2.2.2.2.2.2] with x hx
rw [Real.norm_eq_abs, abs_of_nonneg hx.1]
exact hx.2
have hOutcomeTarget :
(∫ x, P.deltaY n x ∂targetXLaw P n) =
transportedOutcomeITT P n := by
simpa [transportedOutcomeITT, transportWeight, smul_eq_mul] using
(integral_rnDeriv_smul
(μ := targetXLaw P n) (ν := sourceXLaw P n)
(f := P.deltaY n) hIV.transportDomination).symm
have hFirstTarget :
(∫ x, P.deltaD n x ∂targetXLaw P n) =
transportedFirstStage P n := by
rw [transportedFirstStage_eq_weighted_deltaD P k epsilon n
(sourceObservationFacts_of_class P N k c epsilon n hIV)
hIV.instrumentOverlap hIV.weightEnvelope]
simpa [transportWeight, smul_eq_mul] using
(integral_rnDeriv_smul
(μ := targetXLaw P n) (ν := sourceXLaw P n)
(f := P.deltaD n) hIV.transportDomination).symm
simp only [regularCellScoreMean]
rw [integral_sub hDeltaYTargetInt
(hDeltaDTargetInt.const_mul (targetCACE P n)),
integral_const_mul, hOutcomeTarget, hFirstTarget]
have hfirstPos : 0 < transportedFirstStage P n := by
rw [hFirstEq]
exact hIV.targetComplierPositivity
rw [div_eq_iff hfirstPos.ne'] at hRatio
linarith
theorem regularCell_target_score_mean_variance
At the target CACE, randomness of the target empirical score mean contributes at most 4 / N.
Formal statement
hypotheses
P :
N k :
ℕ → ℕ
c epsilon cminus cplus :
ℝ
n :
ℕ
hP :
RegularFiniteCellClass P N k c epsilon cminus cplus n
conclusion
∃ cell : Fin (k n) ↪ 𝒳,
variance (fun target : TargetSample 𝒳 (N n) => ∑ i, targetEmpiricalMass target (cell i) * regularCellScoreMean P n (targetCACE P n) (cell i)) (Measure.pi (fun _ : Fin (N n) => targetXLaw P n))
≤ 4 / (N n : ℝ)
Proof (Lean source)
lemma regularCell_target_score_mean_variance
(P : TransportedArray 𝒳) (N k : ℕ → ℕ)
(c epsilon cminus cplus : ℝ) (n : ℕ)
(hP : RegularFiniteCellClass P N k c epsilon cminus cplus n) :
∃ cell : Fin (k n) ↪ 𝒳,
variance
(fun target : TargetSample 𝒳 (N n) =>
∑ i, targetEmpiricalMass target (cell i) *
regularCellScoreMean P n (targetCACE P n) (cell i))
(Measure.pi (fun _ : Fin (N n) => targetXLaw P n)) ≤
4 / (N n : ℝ) := by
rcases hP with
⟨hIV, hk, hcminus, hcminus_one, hcplus, cell, hcell, hrange, hmass⟩
exact ⟨cell, (regularCell_target_score_mean_variance_for_witness
P N k c epsilon cminus cplus n hIV hk hcminus hcminus_one hcplus
cell hcell hrange hmass).1⟩
theorem regularCell_multinomial_second_moment_for_witness
Exact multinomial second moment and its regular-cell upper bound.
Formal statement
hypotheses
P :
N k :
ℕ → ℕ
c epsilon cminus cplus :
ℝ
n :
ℕ
hNpos :
0 < N n
hIV :
TransportedIVClass P N k c epsilon n
hk :
0 < k n
hcminus :
0 < cminus
hcminus_one :
cminus ≤ 1
hcplus :
1 ≤ cplus
cell :
Fin (k n) ↪ 𝒳
hcell :
∀ i, MeasurableSet {cell i}
hrange :
sourceXLaw P n (range cell) = 1
hmass :
∀ i,
cminus / (k n : ℝ) ≤ (sourceXLaw P n {cell i}).toReal ∧
(sourceXLaw P n {cell i}).toReal ≤ cplus / (k n : ℝ)
conclusion
(∫ target : TargetSample 𝒳 (N n), ∑ i, targetEmpiricalMass target (cell i) ^ 2 / sourceCellMass P n (cell i) ∂Measure.pi (fun _ : Fin (N n) => targetXLaw P n))
= kishDispersion P n
+ (N n : ℝ)⁻¹ * ∑ i, ((targetXLaw P n {cell i}).toReal - (targetXLaw P n {cell i}).toReal ^ 2) / sourceCellMass P n (cell i) ∧
(∫ target : TargetSample 𝒳 (N n), ∑ i, targetEmpiricalMass target (cell i) ^ 2 / sourceCellMass P n (cell i) ∂Measure.pi (fun _ : Fin (N n) => targetXLaw P n))
≤ kishDispersion P n + (k n : ℝ) / (cminus * (N n : ℝ)) ∧
Integrable (fun target : TargetSample 𝒳 (N n) => ∑ i, targetEmpiricalMass target (cell i) ^ 2 / sourceCellMass P n (cell i)) (Measure.pi (fun _ : Fin (N n) => targetXLaw P n))
Proof (Lean source)
lemma regularCell_multinomial_second_moment_for_witness
(P : TransportedArray 𝒳) (N k : ℕ → ℕ)
(c epsilon cminus cplus : ℝ) (n : ℕ)
(hNpos : 0 < N n)
(hIV : TransportedIVClass P N k c epsilon n)
(hk : 0 < k n) (hcminus : 0 < cminus)
(hcminus_one : cminus ≤ 1) (hcplus : 1 ≤ cplus)
(cell : Fin (k n) ↪ 𝒳)
(hcell : ∀ i, MeasurableSet {cell i})
(hrange : sourceXLaw P n (range cell) = 1)
(hmass : ∀ i,
cminus / (k n : ℝ) ≤ (sourceXLaw P n {cell i}).toReal ∧
(sourceXLaw P n {cell i}).toReal ≤ cplus / (k n : ℝ)) :
(∫ target : TargetSample 𝒳 (N n),
∑ i, targetEmpiricalMass target (cell i) ^ 2 /
sourceCellMass P n (cell i)
∂Measure.pi (fun _ : Fin (N n) => targetXLaw P n)) =
kishDispersion P n +
(N n : ℝ)⁻¹ *
∑ i, ((targetXLaw P n {cell i}).toReal -
(targetXLaw P n {cell i}).toReal ^ 2) /
sourceCellMass P n (cell i) ∧
(∫ target : TargetSample 𝒳 (N n),
∑ i, targetEmpiricalMass target (cell i) ^ 2 /
sourceCellMass P n (cell i)
∂Measure.pi (fun _ : Fin (N n) => targetXLaw P n)) ≤
kishDispersion P n +
(k n : ℝ) / (cminus * (N n : ℝ)) ∧
Integrable
(fun target : TargetSample 𝒳 (N n) =>
∑ i, targetEmpiricalMass target (cell i) ^ 2 /
sourceCellMass P n (cell i))
(Measure.pi (fun _ : Fin (N n) => targetXLaw P n)) := by
classical
letI : IsProbabilityMeasure (sourceObsLaw P n) :=
hIV.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (sourceXLaw P n) := by
unfold sourceXLaw
exact Measure.isProbabilityMeasure_map measurable_fst.aemeasurable
letI : IsProbabilityMeasure (targetXLaw P n) :=
hIV.twoSampleArray.2.2.1 n
let μTarget :=
Measure.pi (fun _ : Fin (N n) => targetXLaw P n)
have hsourcePos (i : Fin (k n)) :
0 < sourceCellMass P n (cell i) := by
exact lt_of_lt_of_le
(div_pos hcminus (by exact_mod_cast hk)) (hmass i).1
have hphatMem (i : Fin (k n)) :
MemLp (fun target : TargetSample 𝒳 (N n) =>
targetEmpiricalMass target (cell i)) 2 μTarget := by
unfold targetEmpiricalMass empiricalMass
apply MemLp.const_mul
apply memLp_finset_sum
intro j hj
refine MemLp.of_bound ?_ 1 ?_
· have hjmeas : Measurable fun target :
TargetSample 𝒳 (N n) => target j :=
measurable_pi_apply j
exact
(Measurable.ite (hjmeas (hcell i))
measurable_const measurable_const).aestronglyMeasurable
· filter_upwards with target
split_ifs <;> simp
have hintegrable (i : Fin (k n)) :
Integrable (fun target : TargetSample 𝒳 (N n) =>
targetEmpiricalMass target (cell i) ^ 2 /
sourceCellMass P n (cell i)) μTarget := by
have hsquare := (hphatMem i).integrable_sq
have hmul := hsquare.const_mul
(sourceCellMass P n (cell i))⁻¹
simpa [div_eq_mul_inv, mul_comm] using hmul
have hmoment :
(∫ target : TargetSample 𝒳 (N n),
∑ i, targetEmpiricalMass target (cell i) ^ 2 /
sourceCellMass P n (cell i) ∂μTarget) =
∑ i, ((targetXLaw P n {cell i}).toReal ^ 2 +
(N n : ℝ)⁻¹ *
((targetXLaw P n {cell i}).toReal -
(targetXLaw P n {cell i}).toReal ^ 2)) /
sourceCellMass P n (cell i) := by
rw [integral_finset_sum _ (fun i _ => hintegrable i)]
apply Finset.sum_congr rfl
intro i hi
rw [integral_div,
integral_targetEmpiricalMass_sq (targetXLaw P n) hNpos
(cell i) (hcell i)]
simp [measureReal_def]
have hrnStrong :
StronglyMeasurable
(fun x => ((targetXLaw P n).rnDeriv (sourceXLaw P n) x).toReal) :=
(Measure.measurable_rnDeriv (targetXLaw P n) (sourceXLaw P n)).ennreal_toReal
|>.stronglyMeasurable
have hratio (i : Fin (k n)) :
transportWeight P n (cell i) =
(targetXLaw P n {cell i}).toReal /
sourceCellMass P n (cell i) := by
have hRN :=
Measure.setIntegral_toReal_rnDeriv'
hIV.transportDomination (hcell i)
rw [integral_singleton' hrnStrong] at hRN
apply (eq_div_iff (hsourcePos i).ne').2
simpa [transportWeight, sourceCellMass, measureReal_def, mul_comm] using hRN
have hmeasure :
sourceXLaw P n =
∑ i, sourceXLaw P n {cell i} • Measure.dirac (cell i) :=
measure_eq_fin_sum_smul_dirac_of_range
(sourceXLaw P n) cell hcell hrange
have hweightStrong :
StronglyMeasurable (fun x => (transportWeight P n x) ^ 2) :=
((Measure.measurable_rnDeriv (targetXLaw P n) (sourceXLaw P n)).ennreal_toReal
|>.pow_const 2).stronglyMeasurable
have hkish :
kishDispersion P n =
∑ i, (targetXLaw P n {cell i}).toReal ^ 2 /
sourceCellMass P n (cell i) := by
rw [kishDispersion, hmeasure, integral_finset_sum_measure]
· apply Finset.sum_congr rfl
intro i hi
rw [integral_smul_measure, integral_dirac' _ _ hweightStrong]
simp only [smul_eq_mul]
change
sourceCellMass P n (cell i) * transportWeight P n (cell i) ^ 2 =
(targetXLaw P n {cell i}).toReal ^ 2 /
sourceCellMass P n (cell i)
rw [hratio i]
field_simp [(hsourcePos i).ne']
· intro i hi
exact
(integrable_dirac' hweightStrong (by simp)).smul_measure
(measure_ne_top (sourceXLaw P n) {cell i})
have hsplit :
(∑ i, ((targetXLaw P n {cell i}).toReal ^ 2 +
(N n : ℝ)⁻¹ *
((targetXLaw P n {cell i}).toReal -
(targetXLaw P n {cell i}).toReal ^ 2)) /
sourceCellMass P n (cell i)) =
(∑ i, (targetXLaw P n {cell i}).toReal ^ 2 /
sourceCellMass P n (cell i)) +
(N n : ℝ)⁻¹ *
∑ i, ((targetXLaw P n {cell i}).toReal -
(targetXLaw P n {cell i}).toReal ^ 2) /
sourceCellMass P n (cell i) := by
rw [Finset.mul_sum, ← Finset.sum_add_distrib]
apply Finset.sum_congr rfl
intro i hi
field_simp [(hsourcePos i).ne']
refine ⟨?_, ?_, ?_⟩
· rw [hmoment, hsplit, ← hkish]
· rw [hmoment, hsplit, ← hkish]
have hrangeMeas : MeasurableSet (range cell) := by
rw [show range cell = ⋃ i, {cell i} by
ext x
simp]
exact MeasurableSet.iUnion hcell
have hsourceCompl : sourceXLaw P n (range cell)ᶜ = 0 := by
rw [measure_compl hrangeMeas (measure_ne_top _ _), hrange, measure_univ]
simp
have htargetCompl : targetXLaw P n (range cell)ᶜ = 0 :=
hIV.transportDomination hsourceCompl
have htargetRange : targetXLaw P n (range cell) = 1 :=
(measure_of_measure_compl_eq_zero htargetCompl).trans measure_univ
have hpsum :
∑ i, (targetXLaw P n {cell i}).toReal = 1 := by
have htargetMeasure :=
measure_eq_fin_sum_smul_dirac_of_range
(targetXLaw P n) cell hcell htargetRange
have hu := congrArg (fun μ : Measure 𝒳 => μ univ)
htargetMeasure
simp only [measure_univ, Measure.finset_sum_apply,
Measure.smul_apply, Measure.dirac_apply' _ MeasurableSet.univ,
Set.mem_univ, if_true, Pi.one_apply, mul_one] at hu
have hu' : (1 : ENNReal) =
∑ i, targetXLaw P n {cell i} := by
simpa [Set.indicator_of_mem, smul_eq_mul] using hu
have huReal := congrArg toReal hu'
rw [ENNReal.toReal_sum
(fun _ _ => measure_ne_top (targetXLaw P n) _)] at huReal
simpa using huReal.symm
have hcorr :
∑ i, ((targetXLaw P n {cell i}).toReal -
(targetXLaw P n {cell i}).toReal ^ 2) /
sourceCellMass P n (cell i) ≤
(k n : ℝ) / cminus := by
calc
_ ≤ ∑ i, (targetXLaw P n {cell i}).toReal *
((k n : ℝ) / cminus) := by
apply Finset.sum_le_sum
intro i hi
have hp0 : 0 ≤ (targetXLaw P n {cell i}).toReal :=
ENNReal.toReal_nonneg
have hp1 : (targetXLaw P n {cell i}).toReal ≤ 1 :=
measureReal_le_one
have hnum :
(targetXLaw P n {cell i}).toReal -
(targetXLaw P n {cell i}).toReal ^ 2 ≤
(targetXLaw P n {cell i}).toReal := by
nlinarith [sq_nonneg (targetXLaw P n {cell i}).toReal]
rw [div_le_iff₀ (hsourcePos i)]
have hkreal : 0 < (k n : ℝ) := by exact_mod_cast hk
have hq := (hmass i).1
have hkq :
cminus ≤ (k n : ℝ) *
sourceCellMass P n (cell i) := by
change cminus ≤ (k n : ℝ) *
(sourceXLaw P n {cell i}).toReal
rw [div_le_iff₀ hkreal] at hq
nlinarith
have hone :
1 ≤ ((k n : ℝ) / cminus) *
sourceCellMass P n (cell i) := by
calc
1 ≤ ((k n : ℝ) *
sourceCellMass P n (cell i)) / cminus := by
exact (le_div_iff₀ hcminus).2 (by simpa using hkq)
_ = ((k n : ℝ) / cminus) *
sourceCellMass P n (cell i) := by ring
calc
(targetXLaw P n {cell i}).toReal -
(targetXLaw P n {cell i}).toReal ^ 2 ≤
(targetXLaw P n {cell i}).toReal := hnum
_ ≤ (targetXLaw P n {cell i}).toReal *
(((k n : ℝ) / cminus) *
sourceCellMass P n (cell i)) :=
by simpa using mul_le_mul_of_nonneg_left hone hp0
_ = (targetXLaw P n {cell i}).toReal *
((k n : ℝ) / cminus) *
sourceCellMass P n (cell i) := by ring
_ = (k n : ℝ) / cminus := by rw [← Finset.sum_mul, hpsum, one_mul]
calc
kishDispersion P n + (N n : ℝ)⁻¹ *
∑ i, ((targetXLaw P n {cell i}).toReal -
(targetXLaw P n {cell i}).toReal ^ 2) /
sourceCellMass P n (cell i) ≤
kishDispersion P n + (N n : ℝ)⁻¹ *
((k n : ℝ) / cminus) := by
gcongr
_ = kishDispersion P n +
(k n : ℝ) / (cminus * (N n : ℝ)) := by
field_simp [hcminus.ne', (Nat.cast_pos.mpr hNpos).ne']
· exact integrable_finset_sum _ (fun i _ => hintegrable i)
theorem regularCell_multinomial_second_moment
Exact multinomial second moment and its regular-cell upper bound.
Formal statement
hypotheses
P :
N k :
ℕ → ℕ
c epsilon cminus cplus :
ℝ
n :
ℕ
hNpos :
0 < N n
hP :
RegularFiniteCellClass P N k c epsilon cminus cplus n
conclusion
∃ cell : Fin (k n) ↪ 𝒳,
(∫ target : TargetSample 𝒳 (N n), ∑ i, targetEmpiricalMass target (cell i) ^ 2 / sourceCellMass P n (cell i) ∂Measure.pi (fun _ : Fin (N n) => targetXLaw P n))
= kishDispersion P n
+ (N n : ℝ)⁻¹ * ∑ i, ((targetXLaw P n {cell i}).toReal - (targetXLaw P n {cell i}).toReal ^ 2) / sourceCellMass P n (cell i) ∧
(∫ target : TargetSample 𝒳 (N n), ∑ i, targetEmpiricalMass target (cell i) ^ 2 / sourceCellMass P n (cell i) ∂Measure.pi (fun _ : Fin (N n) => targetXLaw P n))
≤ kishDispersion P n + (k n : ℝ) / (cminus * (N n : ℝ))
Proof (Lean source)
lemma regularCell_multinomial_second_moment
(P : TransportedArray 𝒳) (N k : ℕ → ℕ)
(c epsilon cminus cplus : ℝ) (n : ℕ)
(hNpos : 0 < N n)
(hP : RegularFiniteCellClass P N k c epsilon cminus cplus n) :
∃ cell : Fin (k n) ↪ 𝒳,
(∫ target : TargetSample 𝒳 (N n),
∑ i, targetEmpiricalMass target (cell i) ^ 2 /
sourceCellMass P n (cell i)
∂Measure.pi (fun _ : Fin (N n) => targetXLaw P n)) =
kishDispersion P n +
(N n : ℝ)⁻¹ *
∑ i, ((targetXLaw P n {cell i}).toReal -
(targetXLaw P n {cell i}).toReal ^ 2) /
sourceCellMass P n (cell i) ∧
(∫ target : TargetSample 𝒳 (N n),
∑ i, targetEmpiricalMass target (cell i) ^ 2 /
sourceCellMass P n (cell i)
∂Measure.pi (fun _ : Fin (N n) => targetXLaw P n)) ≤
kishDispersion P n +
(k n : ℝ) / (cminus * (N n : ℝ)) := by
rcases hP with
⟨hIV, hk, hcminus, hcminus_one, hcplus, cell, hcell, hrange, hmass⟩
have h := regularCell_multinomial_second_moment_for_witness
P N k c epsilon cminus cplus n hNpos hIV hk hcminus hcminus_one
hcplus cell hcell hrange hmass
exact ⟨cell, h.1, h.2.1⟩
theorem variance_prod_eq_integral_variance_add
The variance of a statistic based on two independent samples equals its average conditional variance given the second sample plus the variance of its conditional mean.
Formal statement
Proof (Lean source)
lemma variance_prod_eq_integral_variance_add
{Ω T : Type*} [MeasurableSpace Ω] [MeasurableSpace T]
(μ : Measure Ω) (ν : Measure T)
[IsProbabilityMeasure μ] [IsProbabilityMeasure ν]
(F : Ω × T → ℝ) (m : T → ℝ)
(hF : MemLp F 2 (μ.prod ν))
(hsection : ∀ t, MemLp (fun s => F (s, t)) 2 μ)
(hm : MemLp m 2 ν)
(hmean : ∀ t, (∫ s, F (s, t) ∂μ) = m t) :
variance F (μ.prod ν) =
(∫ t, variance (fun s => F (s, t)) μ ∂ν) +
variance m ν := by
exact variance_prod_eq_integral_variance_add
μ ν F m hF (Filter.Eventually.of_forall hsection) hm
(Filter.Eventually.of_forall hmean)
theorem weighted_cross_aestronglyMeasurable
The weighted cross-sample average is almost-everywhere strongly measurable under the joint two-sample distribution.
Formal statement
hypotheses
P :
N :
ℕ → ℕ
n :
m :
ℕ
cell :
Fin m ↪ 𝒳
hcell :
∀ i, MeasurableSet {cell i}
hrange :
sourceXLaw P n (range cell) = 1
G :
SourceObs 𝒳 → ℝ
hG :
conclusion
AEStronglyMeasurable (fun s : TwoSample 𝒳 n (N n) => (n : ℝ)⁻¹ * ∑ r, (targetEmpiricalMass s.2 (s.1 r).1 / sourceCellMass P n (s.1 r).1) * G (s.1 r)) (twoSampleLaw P N n)
Proof (Lean source)
lemma weighted_cross_aestronglyMeasurable
(P : TransportedArray 𝒳) (N : ℕ → ℕ) (n : ℕ)
[IsProbabilityMeasure (sourceObsLaw P n)]
[IsProbabilityMeasure (targetXLaw P n)]
{m : ℕ} (cell : Fin m ↪ 𝒳)
(hcell : ∀ i, MeasurableSet {cell i})
(hrange : sourceXLaw P n (range cell) = 1)
(G : SourceObs 𝒳 → ℝ) (hG : Measurable G) :
AEStronglyMeasurable
(fun s : TwoSample 𝒳 n (N n) =>
(n : ℝ)⁻¹ * ∑ r,
(targetEmpiricalMass s.2 (s.1 r).1 /
sourceCellMass P n (s.1 r).1) * G (s.1 r))
(twoSampleLaw P N n) := by
classical
let μS := Measure.pi (fun _ : Fin n => sourceObsLaw P n)
let μT := Measure.pi (fun _ : Fin (N n) => targetXLaw P n)
let q := sourceCellMass P n
let S : Fin m → Set (SourceObs 𝒳) := fun i => {o | o.1 = cell i}
let B : TwoSample 𝒳 n (N n) → ℝ := fun s =>
(n : ℝ)⁻¹ * ∑ r, ∑ i,
(targetEmpiricalMass s.2 (cell i) / q (cell i)) *
(S i).indicator G (s.1 r)
have hS (i : Fin m) : MeasurableSet (S i) :=
(hcell i).preimage measurable_fst
have hphat (i : Fin m) :
Measurable (fun target : TargetSample 𝒳 (N n) =>
targetEmpiricalMass target (cell i)) := by
unfold targetEmpiricalMass empiricalMass
apply measurable_const.mul
refine Finset.measurable_fun_sum univ fun j hj => ?_
have hjm : Measurable fun target : TargetSample 𝒳 (N n) => target j :=
measurable_pi_apply j
exact Measurable.ite (hjm (hcell i)) measurable_const measurable_const
have hB : Measurable B := by
dsimp only [B]
apply measurable_const.mul
refine Finset.measurable_fun_sum univ fun r hr => ?_
refine Finset.measurable_fun_sum univ fun i hi => ?_
have ht : Measurable (fun s : TwoSample 𝒳 n (N n) =>
targetEmpiricalMass s.2 (cell i) / q (cell i)) :=
((hphat i).comp measurable_snd).div measurable_const
have heval : Measurable (fun s : TwoSample 𝒳 n (N n) => s.1 r) :=
(measurable_pi_apply r).comp measurable_fst
exact ht.mul ((hG.indicator (hS i)).comp heval)
letI : IsProbabilityMeasure (sourceXLaw P n) := by
unfold sourceXLaw
exact Measure.isProbabilityMeasure_map measurable_fst.aemeasurable
have hRangeObs : ∀ᵐ o ∂sourceObsLaw P n, o.1 ∈ range cell := by
have hx : ∀ᵐ x ∂sourceXLaw P n, x ∈ range cell := by
change range cell ∈ ae (sourceXLaw P n)
rw [mem_ae_iff]
have hm : MeasurableSet (range cell) := by
rw [show range cell = ⋃ i, {cell i} by
ext x
simp]
exact MeasurableSet.iUnion hcell
rw [measure_compl hm (measure_ne_top _ _), hrange, measure_univ]
simp
unfold sourceXLaw at hx
exact ae_of_ae_map measurable_fst.aemeasurable hx
have hRangeSample :
∀ᵐ source ∂μS, ∀ r, (source r).1 ∈ range cell := by
rw [ae_all_iff]
intro r
have hx : ∀ᵐ o ∂Measure.map (fun source : SourceSample 𝒳 n => source r) μS,
o.1 ∈ range cell := by
rw [(measurePreserving_eval
(fun _ : Fin n => sourceObsLaw P n) r).map_eq]
exact hRangeObs
exact ae_of_ae_map (measurable_pi_apply r).aemeasurable hx
have hEqSource : ∀ᵐ source ∂μS, ∀ target,
(n : ℝ)⁻¹ * ∑ r,
(targetEmpiricalMass target (source r).1 / q (source r).1) *
G (source r) =
B (source, target) := by
filter_upwards [hRangeSample] with source hs
intro target
dsimp only [B]
congr 1
apply Finset.sum_congr rfl
intro r hr
obtain ⟨i, hi⟩ := hs r
rw [← hi]
rw [Finset.sum_eq_single i]
· rw [Set.indicator_of_mem (show source r ∈ S i by
change (source r).1 = cell i
exact hi.symm)]
· intro j hj hji
simp only [Set.indicator_apply]
have hnot : source r ∉ S j := by
intro hmem
exact hji (cell.injective (show cell j = cell i by
exact hmem.symm.trans hi.symm))
simp [hnot]
· simp
have hEqProd :
(fun s : TwoSample 𝒳 n (N n) =>
(n : ℝ)⁻¹ * ∑ r,
(targetEmpiricalMass s.2 (s.1 r).1 / q (s.1 r).1) *
G (s.1 r)) =ᵐ[μS.prod μT] B := by
have hx : ∀ᵐ source ∂Measure.map
(fun s : TwoSample 𝒳 n (N n) => s.1) (μS.prod μT),
∀ target, (n : ℝ)⁻¹ * ∑ r,
(targetEmpiricalMass target (source r).1 / q (source r).1) *
G (source r) = B (source, target) := by
rw [(show MeasurePreserving
(fun s : TwoSample 𝒳 n (N n) => s.1) (μS.prod μT) μS from
measurePreserving_fst).map_eq]
exact hEqSource
filter_upwards
[ae_of_ae_map measurable_fst.aemeasurable hx] with s hs
exact hs s.2
unfold twoSampleLaw
exact hB.aestronglyMeasurable.congr hEqProd.symm
Helpers.RegularCellRisk_Part4 3 declarations Statistics and deterministic constants
Statistics and deterministic constants
theorem one_le_regularCell_kish
For a transported-IV distribution supported on the regular cells, the Kish dispersion of the transport weights is at least one.
Formal statement
hypotheses
P :
N k :
ℕ → ℕ
c epsilon :
ℝ
n :
ℕ
hIV :
TransportedIVClass P N k c epsilon n
m :
ℕ
cell :
Fin m ↪ 𝒳
hcell :
∀ i, MeasurableSet {cell i}
hrange :
sourceXLaw P n (range cell) = 1
conclusion
1 ≤ kishDispersion P n
Proof (Lean source)
lemma one_le_regularCell_kish
(P : TransportedArray 𝒳) (N k : ℕ → ℕ)
(c epsilon : ℝ) (n : ℕ)
(hIV : TransportedIVClass P N k c epsilon n)
{m : ℕ} (cell : Fin m ↪ 𝒳)
(hcell : ∀ i, MeasurableSet {cell i})
(hrange : sourceXLaw P n (range cell) = 1) :
1 ≤ kishDispersion P n := by
letI : IsProbabilityMeasure (sourceObsLaw P n) :=
hIV.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (sourceXLaw P n) := by
unfold sourceXLaw
exact Measure.isProbabilityMeasure_map measurable_fst.aemeasurable
letI : IsProbabilityMeasure (targetXLaw P n) :=
hIV.twoSampleArray.2.2.1 n
have hwStrong : StronglyMeasurable (transportWeight P n) := by
exact
(Measure.measurable_rnDeriv (targetXLaw P n)
(sourceXLaw P n)).ennreal_toReal.stronglyMeasurable
have hwSqInt :
Integrable (fun x => transportWeight P n x ^ 2) (sourceXLaw P n) :=
integrable_of_finite_support (sourceXLaw P n) cell hcell hrange _
(hwStrong.pow 2)
have hwMem : MemLp (transportWeight P n) 2 (sourceXLaw P n) :=
(memLp_two_iff_integrable_sq hwStrong.aestronglyMeasurable).2 hwSqInt
have hwMean :
(∫ x, transportWeight P n x ∂sourceXLaw P n) = 1 := by
simpa [transportWeight, measureReal_def] using
Measure.integral_toReal_rnDeriv hIV.transportDomination
have hv := variance_nonneg (transportWeight P n) (sourceXLaw P n)
rw [variance_eq_sub hwMem, hwMean] at hv
simpa [kishDispersion] using hv
theorem weighted_two_sample_variance
The weighted cross-sample average has variance bounded by its source-sampling contribution plus its target-sampling contribution, and its expectation equals the stated target mean.
Formal statement
hypotheses
P :
N :
ℕ → ℕ
n :
ℕ
hn :
0 < n
m :
ℕ
cell :
Fin m ↪ 𝒳
hcell :
∀ i, MeasurableSet {cell i}
hrange :
sourceXLaw P n (range cell) = 1
G :
SourceObs 𝒳 → ℝ
a :
𝒳 → ℝ
C V R mean :
ℝ
hC :
0 ≤ C
hGmeas :
hGmem :
MemLp G 2 (sourceObsLaw P n)
hGbound :
∀ᵐ o ∂sourceObsLaw P n, |G o| ≤ C
hqpos :
∀ i, 0 < sourceCellMass P n (cell i)
hcellMean :
hTargetMem :
MemLp (fun target : TargetSample 𝒳 (N n) => ∑ i, targetEmpiricalMass target (cell i) * a (cell i)) 2 (Measure.pi (fun _ : Fin (N n) => targetXLaw P n))
hTargetVar :
variance (fun target : TargetSample 𝒳 (N n) => ∑ i, targetEmpiricalMass target (cell i) * a (cell i)) (Measure.pi (fun _ : Fin (N n) => targetXLaw P n))
≤ V
hTargetMean :
(∫ target : TargetSample 𝒳 (N n), ∑ i, targetEmpiricalMass target (cell i) * a (cell i) ∂Measure.pi (fun _ : Fin (N n) => targetXLaw P n))
= mean
hMassInt :
Integrable (fun target : TargetSample 𝒳 (N n) => ∑ i, targetEmpiricalMass target (cell i) ^ 2 / sourceCellMass P n (cell i)) (Measure.pi (fun _ : Fin (N n) => targetXLaw P n))
hMassBound :
(∫ target : TargetSample 𝒳 (N n), ∑ i, targetEmpiricalMass target (cell i) ^ 2 / sourceCellMass P n (cell i) ∂Measure.pi (fun _ : Fin (N n) => targetXLaw P n))
≤ kishDispersion P n + R
conclusion
let F := fun s : TwoSample 𝒳 n (N n)
=> (n : ℝ)⁻¹ * ∑ r, (targetEmpiricalMass s.2 (s.1 r).1 / sourceCellMass P n (s.1 r).1) * G (s.1 r) variance F (twoSampleLaw P N n) ≤ C ^ 2 / n * (kishDispersion P n + R)
+ V ∧
(∫ s, F s ∂twoSampleLaw P N n) = mean
Proof (Lean source)
lemma weighted_two_sample_variance
(P : TransportedArray 𝒳) (N : ℕ → ℕ) (n : ℕ)
(hn : 0 < n)
[IsProbabilityMeasure (sourceObsLaw P n)]
[IsProbabilityMeasure (targetXLaw P n)]
{m : ℕ} (cell : Fin m ↪ 𝒳)
(hcell : ∀ i, MeasurableSet {cell i})
(hrange : sourceXLaw P n (range cell) = 1)
(G : SourceObs 𝒳 → ℝ) (a : 𝒳 → ℝ) (C V R mean : ℝ)
(hC : 0 ≤ C)
(hGmeas : Measurable G)
(hGmem : MemLp G 2 (sourceObsLaw P n))
(hGbound : ∀ᵐ o ∂sourceObsLaw P n, |G o| ≤ C)
(hqpos : ∀ i, 0 < sourceCellMass P n (cell i))
(hcellMean : ∀ i,
(∫ o in {o | o.1 = cell i}, G o ∂sourceObsLaw P n) /
sourceCellMass P n (cell i) = a (cell i))
(hTargetMem : MemLp
(fun target : TargetSample 𝒳 (N n) =>
∑ i, targetEmpiricalMass target (cell i) * a (cell i)) 2
(Measure.pi (fun _ : Fin (N n) => targetXLaw P n)))
(hTargetVar :
variance
(fun target : TargetSample 𝒳 (N n) =>
∑ i, targetEmpiricalMass target (cell i) * a (cell i))
(Measure.pi (fun _ : Fin (N n) => targetXLaw P n)) ≤ V)
(hTargetMean :
(∫ target : TargetSample 𝒳 (N n),
∑ i, targetEmpiricalMass target (cell i) * a (cell i)
∂Measure.pi (fun _ : Fin (N n) => targetXLaw P n)) = mean)
(hMassInt : Integrable
(fun target : TargetSample 𝒳 (N n) =>
∑ i, targetEmpiricalMass target (cell i) ^ 2 /
sourceCellMass P n (cell i))
(Measure.pi (fun _ : Fin (N n) => targetXLaw P n)))
(hMassBound :
(∫ target : TargetSample 𝒳 (N n),
∑ i, targetEmpiricalMass target (cell i) ^ 2 /
sourceCellMass P n (cell i)
∂Measure.pi (fun _ : Fin (N n) => targetXLaw P n)) ≤
kishDispersion P n + R) :
let F := fun s : TwoSample 𝒳 n (N n) =>
(n : ℝ)⁻¹ * ∑ r,
(targetEmpiricalMass s.2 (s.1 r).1 /
sourceCellMass P n (s.1 r).1) * G (s.1 r)
variance F (twoSampleLaw P N n) ≤
C ^ 2 / n * (kishDispersion P n + R) + V ∧
(∫ s, F s ∂twoSampleLaw P N n) = mean := by
classical
let μS := Measure.pi (fun _ : Fin n => sourceObsLaw P n)
let μT := Measure.pi (fun _ : Fin (N n) => targetXLaw P n)
let mass : TargetSample 𝒳 (N n) → ℝ := fun target =>
∑ i, targetEmpiricalMass target (cell i) ^ 2 /
sourceCellMass P n (cell i)
let M : TargetSample 𝒳 (N n) → ℝ := fun target =>
∑ i, targetEmpiricalMass target (cell i) * a (cell i)
let F : TwoSample 𝒳 n (N n) → ℝ := fun s =>
(n : ℝ)⁻¹ * ∑ r,
(targetEmpiricalMass s.2 (s.1 r).1 /
sourceCellMass P n (s.1 r).1) * G (s.1 r)
have hcond (target : TargetSample 𝒳 (N n)) :=
weighted_iid_average_mean_variance P n hn cell hcell hrange
(targetEmpiricalMass target) G a C hC hGmem hGbound hqpos hcellMean
have hFae : AEStronglyMeasurable F (μS.prod μT) := by
dsimp only [F]
simpa only [twoSampleLaw] using
weighted_cross_aestronglyMeasurable P N n cell hcell hrange G hGmeas
have hFSqAE :
AEStronglyMeasurable (fun z => F z ^ 2) (μS.prod μT) :=
hFae.pow 2
have hInnerAE :
AEStronglyMeasurable
(fun target => ∫ source, F (source, target) ^ 2 ∂μS) μT :=
hFSqAE.prod_swap.integral_prod_right'
have hInnerEq (target : TargetSample 𝒳 (N n)) :
(∫ source, F (source, target) ^ 2 ∂μS) =
variance (fun source => F (source, target)) μS + M target ^ 2 := by
have hv := variance_eq_sub (hcond target).2.2
have hm := (hcond target).1
have hv' :
variance (fun source => F (source, target)) μS =
(∫ source, F (source, target) ^ 2 ∂μS) - M target ^ 2 := by
simpa only [F, M, μS, Pi.pow_apply, hm] using hv
linarith [hv']
have hInnerBound (target : TargetSample 𝒳 (N n)) :
(∫ source, F (source, target) ^ 2 ∂μS) ≤
(C ^ 2 / n) * mass target + M target ^ 2 := by
rw [hInnerEq]
exact add_le_add (hcond target).2.1 le_rfl
have hDomInt :
Integrable (fun target => (C ^ 2 / n) * mass target + M target ^ 2)
μT := by
exact (hMassInt.const_mul (C ^ 2 / n)).add hTargetMem.integrable_sq
have hInnerInt :
Integrable (fun target => ∫ source, F (source, target) ^ 2 ∂μS)
μT := by
apply Integrable.mono' hDomInt hInnerAE
filter_upwards with target
rw [Real.norm_eq_abs, abs_of_nonneg]
· exact hInnerBound target
· exact integral_nonneg fun _ => sq_nonneg _
have hFSqInt : Integrable (fun z => F z ^ 2) (μS.prod μT) := by
apply (integrable_prod_iff' hFSqAE).2
constructor
· exact Filter.Eventually.of_forall fun target =>
(hcond target).2.2.integrable_sq
· have heq :
(fun target => ∫ source, ‖F (source, target) ^ 2‖ ∂μS) =
fun target => ∫ source, F (source, target) ^ 2 ∂μS := by
funext target
apply integral_congr_ae
filter_upwards with source
rw [Real.norm_eq_abs, abs_of_nonneg (sq_nonneg _)]
rw [heq]
exact hInnerInt
have hFmem : MemLp F 2 (μS.prod μT) :=
(memLp_two_iff_integrable_sq hFae).2 hFSqInt
have hVarFunInt :
Integrable (fun target => variance (fun source => F (source, target)) μS)
μT := by
have heq : (fun target =>
variance (fun source => F (source, target)) μS) =
fun target => (∫ source, F (source, target) ^ 2 ∂μS) - M target ^ 2 := by
funext target
rw [hInnerEq]
ring
rw [heq]
exact hInnerInt.sub hTargetMem.integrable_sq
have hVarIntegral :
(∫ target, variance (fun source => F (source, target)) μS ∂μT) ≤
(C ^ 2 / n) *
(kishDispersion P n + R) := by
calc
_ ≤ ∫ target, (C ^ 2 / n) * mass target ∂μT := by
apply integral_mono hVarFunInt (hMassInt.const_mul _)
intro target
simpa only [F, mass, μS] using (hcond target).2.1
_ = (C ^ 2 / n) * ∫ target, mass target ∂μT := by
rw [integral_const_mul]
_ ≤ (C ^ 2 / n) * (kishDispersion P n + R) := by
gcongr
have hTotal :=
variance_prod_eq_integral_variance_add μS μT F M hFmem
(fun target => (hcond target).2.2) hTargetMem
(fun target => (hcond target).1)
constructor
· unfold twoSampleLaw
rw [hTotal]
exact add_le_add hVarIntegral hTargetVar
· unfold twoSampleLaw
rw [integral_prod_symm _ (hFmem.integrable (by norm_num))]
calc
(∫ target, ∫ source, F (source, target) ∂μS ∂μT) =
∫ target, M target ∂μT := by
apply integral_congr_ae
filter_upwards with target
simpa only [F, M, μS] using (hcond target).1
_ = mean := by
simpa only [M, μT] using hTargetMean
theorem target_receipt_mean_variance_for_witness
For the witness construction, the empirical target receipt contrast has the stated mean and variance bound determined by the target cell distribution.
Formal statement
hypotheses
P :
N k :
ℕ → ℕ
c epsilon cminus cplus :
ℝ
n :
ℕ
hNpos :
0 < N n
hIV :
TransportedIVClass P N k c epsilon n
hk :
0 < k n
hcm :
0 < cminus
cell :
Fin (k n) ↪ 𝒳
hcell :
∀ i, MeasurableSet {cell i}
hrange :
sourceXLaw P n (range cell) = 1
hmass :
∀ i,
cminus / (k n : ℝ) ≤ (sourceXLaw P n {cell i}).toReal ∧
(sourceXLaw P n {cell i}).toReal ≤ cplus / (k n : ℝ)
conclusion
let M := fun target : TargetSample 𝒳 (N n)
=> ∑ i, targetEmpiricalMass target (cell i) * P.deltaD n (cell i) variance M (Measure.pi (fun _ : Fin (N n) => targetXLaw P n)) ≤ 1 / (N n : ℝ) ∧
(∫ target, M target ∂Measure.pi (fun _ : Fin (N n) => targetXLaw P n))
= transportedFirstStage P n
Proof (Lean source)
lemma target_receipt_mean_variance_for_witness
(P : TransportedArray 𝒳) (N k : ℕ → ℕ)
(c epsilon cminus cplus : ℝ) (n : ℕ) (hNpos : 0 < N n)
(hIV : TransportedIVClass P N k c epsilon n)
(hk : 0 < k n) (hcm : 0 < cminus)
(cell : Fin (k n) ↪ 𝒳)
(hcell : ∀ i, MeasurableSet {cell i})
(hrange : sourceXLaw P n (range cell) = 1)
(hmass : ∀ i,
cminus / (k n : ℝ) ≤ (sourceXLaw P n {cell i}).toReal ∧
(sourceXLaw P n {cell i}).toReal ≤ cplus / (k n : ℝ)) :
let M := fun target : TargetSample 𝒳 (N n) =>
∑ i, targetEmpiricalMass target (cell i) * P.deltaD n (cell i)
variance M (Measure.pi (fun _ : Fin (N n) => targetXLaw P n)) ≤
1 / (N n : ℝ) ∧
MemLp M 2 (Measure.pi (fun _ : Fin (N n) => targetXLaw P n)) ∧
(∫ target, M target
∂Measure.pi (fun _ : Fin (N n) => targetXLaw P n)) =
transportedFirstStage P n := by
classical
letI : IsProbabilityMeasure (sourceObsLaw P n) :=
hIV.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (sourceXLaw P n) := by
unfold sourceXLaw
exact Measure.isProbabilityMeasure_map measurable_fst.aemeasurable
letI : IsProbabilityMeasure (targetXLaw P n) :=
hIV.twoSampleArray.2.2.1 n
have hsourcePos (i : Fin (k n)) :
0 < (sourceXLaw P n {cell i}).toReal :=
lt_of_lt_of_le (div_pos hcm (by exact_mod_cast hk)) (hmass i).1
have hcellBound (i : Fin (k n)) :
|P.deltaD n (cell i)| ≤ 1 := by
have hd := property_at_of_ae_of_singleton_pos
(sourceXLaw P n) (fun x => P.deltaD n x ∈ Icc (0 : ℝ) 1)
(sourceObservationFacts_of_class P N k c epsilon n hIV).2.2.2.2.2.2.2.2 (hsourcePos i)
rw [abs_of_nonneg hd.1]
exact hd.2
let B : 𝒳 → ℝ := fun x =>
if x ∈ range cell then P.deltaD n x else 0
have hrangeMeas : MeasurableSet (range cell) := by
rw [show range cell = ⋃ i, {cell i} by
ext x
simp]
exact MeasurableSet.iUnion hcell
have hBMeas : Measurable B := by
dsimp only [B]
exact Measurable.ite hrangeMeas (P.deltaD_measurable n) measurable_const
have hBBound : ∀ x, |B x| ≤ 1 := by
intro x
by_cases hx : x ∈ range cell
· obtain ⟨i, rfl⟩ := hx
simpa [B] using hcellBound i
· have hzero : B x = 0 := by
dsimp only [B]
rw [if_neg hx]
rw [hzero]
norm_num
have hBMem : MemLp B 2 (targetXLaw P n) :=
MemLp.of_bound hBMeas.aestronglyMeasurable 1
(Filter.Eventually.of_forall fun x => by
simpa [Real.norm_eq_abs] using hBBound x)
have hstat :
(fun target : TargetSample 𝒳 (N n) =>
∑ i, targetEmpiricalMass target (cell i) * P.deltaD n (cell i)) =
(fun target => (N n : ℝ)⁻¹ * ∑ j, B (target j)) := by
funext target
unfold targetEmpiricalMass empiricalMass
calc
_ = ∑ i, ∑ j, (N n : ℝ)⁻¹ *
(if target j = cell i then 1 else 0) * P.deltaD n (cell i) := by
apply Finset.sum_congr rfl
intro i hi
rw [Finset.mul_sum, Finset.sum_mul]
_ = ∑ j, ∑ i, (N n : ℝ)⁻¹ *
(if target j = cell i then 1 else 0) * P.deltaD n (cell i) := by
rw [Finset.sum_comm]
_ = ∑ j, (N n : ℝ)⁻¹ * B (target j) := by
apply Finset.sum_congr rfl
intro j hj
simp_rw [mul_assoc]
rw [← Finset.mul_sum]
congr 1
by_cases hx : target j ∈ range cell
· obtain ⟨i, hi⟩ := hx
rw [← hi, Finset.sum_eq_single i]
· simp [B]
· intro i' hi' hne
have htargetne : target j ≠ cell i' := by
rw [← hi]
exact fun h => hne (cell.injective h.symm)
have hcellne : cell i ≠ cell i' :=
fun h => hne (cell.injective h).symm
rw [if_neg hcellne]
simp
· simp
· have hne : ∀ i, target j ≠ cell i := by
intro i hEq
exact hx ⟨i, hEq.symm⟩
have hne' : ∀ i, cell i ≠ target j :=
fun i hEq => hne i hEq.symm
simp [B, hx, hne, hne']
_ = (N n : ℝ)⁻¹ * ∑ j, B (target j) := by
rw [Finset.mul_sum]
have hAvgMem :
MemLp (fun target : TargetSample 𝒳 (N n) =>
(N n : ℝ)⁻¹ * ∑ j, B (target j)) 2
(Measure.pi (fun _ : Fin (N n) => targetXLaw P n)) := by
apply MemLp.const_mul
apply memLp_finset_sum
intro j hj
exact hBMem.comp_measurePreserving
(measurePreserving_eval (fun _ : Fin (N n) => targetXLaw P n) j)
have hrangeCompl : sourceXLaw P n (range cell)ᶜ = 0 := by
rw [measure_compl hrangeMeas (measure_ne_top _ _), hrange, measure_univ]
simp
have htargetRange :
∀ᵐ x ∂targetXLaw P n, x ∈ range cell := by
have hc := hIV.transportDomination hrangeCompl
change range cell ∈ ae (targetXLaw P n)
rw [mem_ae_iff]
exact hc
have hBDelta : B =ᵐ[targetXLaw P n] P.deltaD n := by
filter_upwards [htargetRange] with x hx
dsimp only [B]
rw [if_pos hx]
have hDeltaTargetInt :
Integrable (P.deltaD n) (targetXLaw P n) := by
refine Integrable.of_bound (P.deltaD_measurable n).aestronglyMeasurable 1 ?_
apply hIV.transportDomination.ae_le
filter_upwards [(sourceObservationFacts_of_class P N k c epsilon n hIV).2.2.2.2.2.2.2.2] with x hx
rw [Real.norm_eq_abs, abs_of_nonneg hx.1]
exact hx.2
have hFirstTarget :
(∫ x, P.deltaD n x ∂targetXLaw P n) =
transportedFirstStage P n := by
rw [transportedFirstStage_eq_weighted_deltaD P k epsilon n
(sourceObservationFacts_of_class P N k c epsilon n hIV)
hIV.instrumentOverlap hIV.weightEnvelope]
simpa [transportWeight, smul_eq_mul] using
(integral_rnDeriv_smul
(μ := targetXLaw P n) (ν := sourceXLaw P n)
(f := P.deltaD n) hIV.transportDomination).symm
refine ⟨?_, ?_, ?_⟩
· rw [hstat, iid_average_variance
(targetXLaw P n) (N n) hNpos B hBMem]
calc
(N n : ℝ)⁻¹ * variance B (targetXLaw P n) ≤
(N n : ℝ)⁻¹ * ∫ x, B x ^ 2 ∂targetXLaw P n := by
gcongr
exact variance_le_expectation_sq hBMeas.aestronglyMeasurable
_ ≤ (N n : ℝ)⁻¹ * 1 := by
gcongr
calc
(∫ x, B x ^ 2 ∂targetXLaw P n) ≤ ∫ _x, (1 : ℝ)
∂targetXLaw P n := by
apply integral_mono_of_nonneg
· exact Filter.Eventually.of_forall fun x => sq_nonneg (B x)
· exact integrable_const 1
· exact Filter.Eventually.of_forall fun x => by
have hb := (abs_le).1 (hBBound x)
nlinarith
_ = 1 := by simp
_ = 1 / (N n : ℝ) := by ring
· rw [hstat]
exact hAvgMem
· rw [hstat, iid_average_integral
(targetXLaw P n) (N n) hNpos B hBMem]
rw [integral_congr_ae hBDelta, hFirstTarget]
Helpers.RegularCellRisk_Part5_Part1 2 declarations Statistics and deterministic constants
Statistics and deterministic constants
theorem regularCell_eventually_uniform_moment_variance
Eventually and uniformly over the regular class, both cross moments obey the exact paper variance constant; the receipt moment is centered at the transported first stage.
Formal statement
hypotheses
N k :
ℕ → ℕ
c epsilon cminus cplus :
ℝ
hc :
0 < c
hepsilon :
0 < epsilon ∧ epsilon < 1 / 2
hcminus :
0 < cminus ∧ cminus ≤ 1
hcplus :
1 ≤ cplus
hkPos :
∀ n, 0 < k n
conclusion
∀ᶠ n in atTop,
∀ P : TransportedArray 𝒳,
RegularFiniteCellClass P N k c epsilon cminus cplus n → variance (fun s : TwoSample 𝒳 n (N n) => regularCellContrastMoment (sourceCellMass P n) (P.propensity n) (targetCACE P n) s.1 s.2) (twoSampleLaw P N n)
≤ regularCellVarianceConstant epsilon c * kishDispersion P n / n ∧
variance (fun s : TwoSample 𝒳 n (N n) => regularCellReceiptMoment (sourceCellMass P n) (P.propensity n) s.1 s.2) (twoSampleLaw P N n)
≤ regularCellVarianceConstant epsilon c * kishDispersion P n / n ∧
(∫ s : TwoSample 𝒳 n (N n), regularCellReceiptMoment (sourceCellMass P n) (P.propensity n) s.1 s.2 ∂twoSampleLaw P N n)
= transportedFirstStage P n
Proof (Lean source)
lemma regularCell_eventually_uniform_moment_variance
(N k : ℕ → ℕ) (c epsilon cminus cplus : ℝ)
(hc : 0 < c)
(hepsilon : 0 < epsilon ∧ epsilon < 1 / 2)
(hcminus : 0 < cminus ∧ cminus ≤ 1)
(hcplus : 1 ≤ cplus)
(hN : Tendsto (fun n : ℕ => (N n : ℝ) / (n : ℝ)) atTop (𝓝 c))
(hkPos : ∀ n, 0 < k n)
(hkRoot : Tendsto (fun n : ℕ => (k n : ℝ) / sqrt n)
atTop (𝓝 0)) :
∀ᶠ n in atTop, ∀ P : TransportedArray 𝒳,
RegularFiniteCellClass P N k c epsilon cminus cplus n →
variance
(fun s : TwoSample 𝒳 n (N n) =>
regularCellContrastMoment (sourceCellMass P n)
(P.propensity n) (targetCACE P n) s.1 s.2)
(twoSampleLaw P N n) ≤
regularCellVarianceConstant epsilon c * kishDispersion P n / n ∧
variance
(fun s : TwoSample 𝒳 n (N n) =>
regularCellReceiptMoment (sourceCellMass P n)
(P.propensity n) s.1 s.2)
(twoSampleLaw P N n) ≤
regularCellVarianceConstant epsilon c * kishDispersion P n / n ∧
(∫ s : TwoSample 𝒳 n (N n),
regularCellReceiptMoment (sourceCellMass P n)
(P.propensity n) s.1 s.2 ∂twoSampleLaw P N n) =
transportedFirstStage P n := by
have hsqrtTop :
Tendsto (fun n : ℕ => sqrt (n : ℝ)) atTop atTop :=
Real.tendsto_sqrt_atTop.comp tendsto_natCast_atTop_atTop
have hinvSqrt :
Tendsto (fun n : ℕ => (sqrt (n : ℝ))⁻¹) atTop (𝓝 0) :=
tendsto_inv_atTop_zero.comp hsqrtTop
have hkn : Tendsto (fun n : ℕ => (k n : ℝ) / (n : ℝ))
atTop (𝓝 0) := by
have heq : (fun n : ℕ => (k n : ℝ) / (n : ℝ)) =
fun n => ((k n : ℝ) / sqrt n) * (sqrt n)⁻¹ := by
funext n
by_cases hn : n = 0
· simp [hn]
· have hs : sqrt (n : ℝ) ≠ 0 := by positivity
rw [div_eq_mul_inv, div_eq_mul_inv]
field_simp [hs, Real.sq_sqrt (Nat.cast_nonneg n)]
rw [Real.sq_sqrt (Nat.cast_nonneg n)]
rw [heq]
simpa using hkRoot.mul hinvSqrt
have hNratio :
∀ᶠ n : ℕ in atTop, c / 2 < (N n : ℝ) / (n : ℝ) :=
(tendsto_order.1 hN).1 _ (by linarith)
have hnpos : ∀ᶠ n : ℕ in atTop, 0 < n :=
eventually_atTop.2 ⟨1, fun n hn => Nat.zero_lt_of_lt hn⟩
have hkOverN :
Tendsto (fun n : ℕ => (k n : ℝ) / (N n : ℝ)) atTop (𝓝 0) := by
have hquot := hkn.div hN hc.ne'
have hquot' :
Tendsto
((fun n : ℕ => (k n : ℝ) / (n : ℝ)) /
fun n => (N n : ℝ) / (n : ℝ)) atTop (𝓝 0) := by
simpa [hc.ne'] using hquot
apply hquot'.congr'
filter_upwards [hnpos, hNratio] with n hn hratio
have hnreal : 0 < (n : ℝ) := by exact_mod_cast hn
have hNreal : 0 < (N n : ℝ) := by
have hp : 0 < (N n : ℝ) / (n : ℝ) :=
lt_trans (half_pos hc) hratio
rcases (div_pos_iff.mp hp) with h | h
· exact h.1
· exact (not_lt_of_ge hnreal.le h.2).elim
change
((k n : ℝ) / (n : ℝ)) / ((N n : ℝ) / (n : ℝ)) =
(k n : ℝ) / (N n : ℝ)
field_simp [hnreal.ne', hNreal.ne']
have hkSmall :
∀ᶠ n : ℕ in atTop, (k n : ℝ) / (N n : ℝ) < cminus :=
(tendsto_order.1 hkOverN).2 _ hcminus.1
filter_upwards [hnpos, hNratio, hkSmall] with n hn hratio hkSmallN
intro P hP
classical
rcases hP with
⟨hIV, hk, hcm, hcmOne, hcp, cell, hcell, hrange, hmass⟩
letI : IsProbabilityMeasure (sourceObsLaw P n) :=
hIV.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (sourceXLaw P n) := by
unfold sourceXLaw
exact Measure.isProbabilityMeasure_map measurable_fst.aemeasurable
letI : IsProbabilityMeasure (targetXLaw P n) :=
hIV.twoSampleArray.2.2.1 n
have hnreal : 0 < (n : ℝ) := by exact_mod_cast hn
have hNreal : 0 < (N n : ℝ) := by
have hp : 0 < (N n : ℝ) / (n : ℝ) :=
lt_trans (half_pos hc) hratio
rcases (div_pos_iff.mp hp) with h | h
· exact h.1
· exact (not_lt_of_ge hnreal.le h.2).elim
have hNpos : 0 < N n := by exact_mod_cast hNreal
have hqpos (i : Fin (k n)) :
0 < sourceCellMass P n (cell i) :=
lt_of_lt_of_le (div_pos hcm (by exact_mod_cast hk)) (hmass i).1
have hRleOne :
(k n : ℝ) / (cminus * (N n : ℝ)) ≤ 1 := by
have hkdiv :
(k n : ℝ) / (N n : ℝ) < cminus := hkSmallN
rw [div_le_iff₀ (mul_pos hcm hNreal)]
rw [div_lt_iff₀ hNreal] at hkdiv
nlinarith
have hfourN : 4 / (N n : ℝ) ≤ 8 / (c * (n : ℝ)) := by
have hlower : c / 2 * (n : ℝ) < (N n : ℝ) := by
exact (lt_div_iff₀ hnreal).1 hratio
apply (div_le_div_iff₀ hNreal (mul_pos hc hnreal)).2
nlinarith
have hAssignmentInt :
Integrable (P.assignmentContrast n true) (sourceXLaw P n) :=
integrable_of_finite_support (sourceXLaw P n) cell hcell hrange _
(P.assignmentContrast_measurable n true).stronglyMeasurable
have hReceiptInt :
Integrable (P.receiptContrast n true) (sourceXLaw P n) :=
integrable_of_finite_support (sourceXLaw P n) cell hcell hrange _
(P.receiptContrast_measurable n true).stronglyMeasurable
have hTheta : targetCACE P n ∈ parameterSpace := by
rcases compact_causal_range P N k c epsilon n
hAssignmentInt hReceiptInt hIV with ⟨_, _, _, _, _, htheta⟩
exact htheta
let i0 : Fin (k n) := ⟨0, hk⟩
let target0 : TargetSample 𝒳 (N n) := fun _ => cell i0
have hScoreCore :=
regularCell_source_conditional_mean_variance_for_witness
P N k c epsilon cminus cplus (targetCACE P n) n target0 hn
hIV hk hcm hcmOne hcp cell hcell hrange hmass hTheta
have hScoreMean := hScoreCore.2.2.2.1
have hScoreMem := hScoreCore.2.2.2.2.1
have hScoreBound := hScoreCore.2.2.2.2.2
have hInstrumentMeasurable : Measurable (instrumentScore P n) := by
have hz : Measurable fun o : SourceObs 𝒳 => o.2.1 := by fun_prop
have he : Measurable fun o : SourceObs 𝒳 => P.propensity n o.1 :=
(P.propensity_measurable n).comp measurable_fst
unfold instrumentScore
exact Measurable.ite (hz (MeasurableSet.singleton true))
(measurable_const.div he)
(measurable_const.neg.div (measurable_const.sub he))
have hBoolMeasurable :
Measurable (fun o : SourceObs 𝒳 => boolReal o.2.2.1) := by
unfold boolReal
have hd : Measurable fun o : SourceObs 𝒳 => o.2.2.1 := by fun_prop
exact Measurable.ite (hd (MeasurableSet.singleton true))
measurable_const measurable_const
have hScoreMeas :
Measurable (regularCellScore (P.propensity n) (targetCACE P n)) := by
change Measurable (fun o : SourceObs 𝒳 =>
instrumentScore P n o *
(o.2.2.2 - targetCACE P n * boolReal o.2.2.1))
have hy : Measurable fun o : SourceObs 𝒳 => o.2.2.2 := by fun_prop
exact hInstrumentMeasurable.mul
(hy.sub (measurable_const.mul hBoolMeasurable))
have hTargetScore :=
regularCell_target_score_mean_variance_for_witness
P N k c epsilon cminus cplus n hIV hk hcm hcmOne hcp
cell hcell hrange hmass
have hMulti :=
regularCell_multinomial_second_moment_for_witness
P N k c epsilon cminus cplus n hNpos hIV hk hcm hcmOne hcp
cell hcell hrange hmass
have hContrastRaw :=
weighted_two_sample_variance P N n hn cell hcell hrange
(regularCellScore (P.propensity n) (targetCACE P n))
(regularCellScoreMean P n (targetCACE P n))
(2 / epsilon) (4 / (N n : ℝ))
((k n : ℝ) / (cminus * (N n : ℝ))) 0
(div_nonneg (by norm_num) hepsilon.1.le) hScoreMeas hScoreMem
hScoreBound hqpos hScoreMean hTargetScore.2.1 hTargetScore.1
hTargetScore.2.2 hMulti.2.2 hMulti.2.1
have hkish : 1 ≤ kishDispersion P n :=
one_le_regularCell_kish P N k c epsilon n hIV cell hcell hrange
have hRleKish :
(k n : ℝ) / (cminus * (N n : ℝ)) ≤ kishDispersion P n :=
hRleOne.trans hkish
have hContrast :
variance
(fun s : TwoSample 𝒳 n (N n) =>
regularCellContrastMoment (sourceCellMass P n)
(P.propensity n) (targetCACE P n) s.1 s.2)
(twoSampleLaw P N n) ≤
regularCellVarianceConstant epsilon c * kishDispersion P n / n := by
have hraw : variance
(fun s : TwoSample 𝒳 n (N n) =>
regularCellContrastMoment (sourceCellMass P n)
(P.propensity n) (targetCACE P n) s.1 s.2)
(twoSampleLaw P N n) ≤
(2 / epsilon) ^ 2 / n *
(kishDispersion P n +
(k n : ℝ) / (cminus * (N n : ℝ))) +
4 / (N n : ℝ) := by
simpa only [regularCell_crossAverage_identity] using hContrastRaw.1
have hcond :
(2 / epsilon) ^ 2 / n *
(kishDispersion P n +
(k n : ℝ) / (cminus * (N n : ℝ))) ≤
8 * epsilon⁻¹ ^ 2 * kishDispersion P n / n := by
have hsum : kishDispersion P n +
(k n : ℝ) / (cminus * (N n : ℝ)) ≤
2 * kishDispersion P n := by linarith
calc
_ ≤ (2 / epsilon) ^ 2 / n * (2 * kishDispersion P n) := by
gcongr
_ = 8 * epsilon⁻¹ ^ 2 * kishDispersion P n / n := by
field_simp [hepsilon.1.ne', hnreal.ne']
<;> ring
have htarget :
4 / (N n : ℝ) ≤
8 * c⁻¹ * kishDispersion P n / n := by
calc
4 / (N n : ℝ) ≤ 8 / (c * (n : ℝ)) := hfourN
_ ≤ 8 * c⁻¹ * kishDispersion P n / n := by
field_simp [hc.ne', hnreal.ne']
nlinarith
calc
_ ≤ (2 / epsilon) ^ 2 / n *
(kishDispersion P n +
(k n : ℝ) / (cminus * (N n : ℝ))) +
4 / (N n : ℝ) := hraw
_ ≤ 8 * epsilon⁻¹ ^ 2 * kishDispersion P n / n +
8 * c⁻¹ * kishDispersion P n / n :=
add_le_add hcond htarget
_ = regularCellVarianceConstant epsilon c *
kishDispersion P n / n := by
unfold regularCellVarianceConstant
ring
let Gd : SourceObs 𝒳 → ℝ := fun o =>
oracleInstrumentScore (P.propensity n) o * boolReal o.2.2.1
have hGdMeas : Measurable Gd := by
change Measurable (fun o : SourceObs 𝒳 =>
instrumentScore P n o * boolReal o.2.2.1)
exact hInstrumentMeasurable.mul hBoolMeasurable
have hInstrumentBound : ∀ᵐ o ∂sourceObsLaw P n,
|instrumentScore P n o| ≤ 1 / epsilon := by
have hoverlap : ∀ᵐ o ∂sourceObsLaw P n,
epsilon ≤ P.propensity n o.1 ∧
P.propensity n o.1 ≤ 1 - epsilon := by
have hx := hIV.instrumentOverlap.2.2
unfold sourceXLaw at hx
exact ae_of_ae_map measurable_fst.aemeasurable hx
filter_upwards [hoverlap] with o ho
rcases o with ⟨x, z, d, y⟩
cases z
· simp only [instrumentScore, Bool.false_eq_true, ↓reduceIte, abs_neg,
abs_div, abs_one]
rw [abs_of_pos (sub_pos.mpr (lt_of_le_of_lt ho.2
-- … truncated; follow the source link for the rest …
theorem regularCell_cross_memLp_for_witness
Under the regular finite-cell witness conditions, the weighted cross-sample average of a bounded measurable source score is square-integrable.
Formal statement
hypotheses
P :
N k :
ℕ → ℕ
c epsilon :
ℝ
n :
ℕ
hn :
0 < n
hNpos :
0 < N n
hIV :
TransportedIVClass P N k c epsilon n
m :
ℕ
hm :
0 < m
cminus :
ℝ
hcminus :
0 < cminus
cell :
Fin m ↪ 𝒳
hcell :
∀ i, MeasurableSet {cell i}
hrange :
sourceXLaw P n (range cell) = 1
hmass :
∀ i : Fin m, cminus / (m : ℝ) ≤ sourceCellMass P n (cell i)
G :
SourceObs 𝒳 → ℝ
C :
ℝ
hC :
0 ≤ C
hGmeas :
hGbound :
∀ᵐ o ∂sourceObsLaw P n, |G o| ≤ C
conclusion
MemLp (fun s : TwoSample 𝒳 n (N n) => (n : ℝ)⁻¹ * ∑ r, (targetEmpiricalMass s.2 (s.1 r).1 / sourceCellMass P n (s.1 r).1) * G (s.1 r)) 2 (twoSampleLaw P N n)
Proof (Lean source)
lemma regularCell_cross_memLp_for_witness
(P : TransportedArray 𝒳) (N k : ℕ → ℕ)
(c epsilon : ℝ) (n : ℕ)
(hn : 0 < n) (hNpos : 0 < N n)
(hIV : TransportedIVClass P N k c epsilon n)
{m : ℕ} (hm : 0 < m) (cminus : ℝ) (hcminus : 0 < cminus)
(cell : Fin m ↪ 𝒳)
(hcell : ∀ i, MeasurableSet {cell i})
(hrange : sourceXLaw P n (range cell) = 1)
(hmass : ∀ i : Fin m,
cminus / (m : ℝ) ≤ sourceCellMass P n (cell i))
(G : SourceObs 𝒳 → ℝ) (C : ℝ) (hC : 0 ≤ C)
(hGmeas : Measurable G)
(hGbound : ∀ᵐ o ∂sourceObsLaw P n, |G o| ≤ C) :
MemLp
(fun s : TwoSample 𝒳 n (N n) =>
(n : ℝ)⁻¹ * ∑ r,
(targetEmpiricalMass s.2 (s.1 r).1 /
sourceCellMass P n (s.1 r).1) * G (s.1 r)) 2
(twoSampleLaw P N n) := by
classical
letI : IsProbabilityMeasure (sourceObsLaw P n) :=
hIV.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (sourceXLaw P n) := by
unfold sourceXLaw
exact Measure.isProbabilityMeasure_map measurable_fst.aemeasurable
letI : IsProbabilityMeasure (targetXLaw P n) :=
hIV.twoSampleArray.2.2.1 n
letI : IsProbabilityMeasure (twoSampleLaw P N n) := by
unfold twoSampleLaw
infer_instance
let μS := Measure.pi (fun _ : Fin n => sourceObsLaw P n)
let μT := Measure.pi (fun _ : Fin (N n) => targetXLaw P n)
have hAE :=
weighted_cross_aestronglyMeasurable P N n cell hcell hrange G hGmeas
have hrangeMeas : MeasurableSet (range cell) := by
rw [show range cell = ⋃ i, {cell i} by
ext x
simp]
exact MeasurableSet.iUnion hcell
have hRangeObs : ∀ᵐ o ∂sourceObsLaw P n, o.1 ∈ range cell := by
have hx : ∀ᵐ x ∂sourceXLaw P n, x ∈ range cell := by
change range cell ∈ ae (sourceXLaw P n)
rw [mem_ae_iff]
rw [measure_compl hrangeMeas (measure_ne_top _ _), hrange, measure_univ]
simp
unfold sourceXLaw at hx
exact ae_of_ae_map measurable_fst.aemeasurable hx
have hGoodObs : ∀ᵐ o ∂sourceObsLaw P n,
o.1 ∈ range cell ∧ |G o| ≤ C :=
hRangeObs.and hGbound
have hGoodSource :
∀ᵐ source ∂μS, ∀ r, (source r).1 ∈ range cell ∧
|G (source r)| ≤ C := by
rw [ae_all_iff]
intro r
have hx : ∀ᵐ o ∂Measure.map
(fun source : SourceSample 𝒳 n => source r) μS,
o.1 ∈ range cell ∧ |G o| ≤ C := by
rw [(measurePreserving_eval
(fun _ : Fin n => sourceObsLaw P n) r).map_eq]
exact hGoodObs
exact ae_of_ae_map (measurable_pi_apply r).aemeasurable hx
have hGoodProd :
∀ᵐ s ∂μS.prod μT, ∀ r, (s.1 r).1 ∈ range cell ∧
|G (s.1 r)| ≤ C := by
have hx : ∀ᵐ source ∂Measure.map
(fun s : TwoSample 𝒳 n (N n) => s.1) (μS.prod μT),
∀ r, (source r).1 ∈ range cell ∧ |G (source r)| ≤ C := by
rw [(show MeasurePreserving
(fun s : TwoSample 𝒳 n (N n) => s.1) (μS.prod μT) μS from
measurePreserving_fst).map_eq]
exact hGoodSource
exact ae_of_ae_map measurable_fst.aemeasurable hx
refine MemLp.of_bound hAE (C * (m : ℝ) / cminus) ?_
unfold twoSampleLaw at hGoodProd ⊢
filter_upwards [hGoodProd] with s hs
rw [Real.norm_eq_abs]
have hmreal : 0 < (m : ℝ) := by exact_mod_cast hm
have hnreal : 0 < (n : ℝ) := by exact_mod_cast hn
have hcmreal : 0 < cminus / (m : ℝ) := div_pos hcminus hmreal
calc
|(n : ℝ)⁻¹ * ∑ r,
(targetEmpiricalMass s.2 (s.1 r).1 /
sourceCellMass P n (s.1 r).1) * G (s.1 r)| ≤
(n : ℝ)⁻¹ * ∑ _r : Fin n, (m : ℝ) / cminus * C := by
rw [abs_mul, abs_of_pos (inv_pos.mpr hnreal)]
gcongr
apply (Finset.abs_sum_le_sum_abs _ _).trans
apply Finset.sum_le_sum
intro r hr
obtain ⟨i, hi⟩ := hs r |>.1
have hq : cminus / (m : ℝ) ≤
sourceCellMass P n ((s.1 r).1) := by
rw [← hi]
exact hmass i
have hqpos : 0 < sourceCellMass P n ((s.1 r).1) :=
hcmreal.trans_le hq
rw [abs_mul, abs_div, abs_of_pos hqpos]
calc
|targetEmpiricalMass s.2 (s.1 r).1| /
sourceCellMass P n (s.1 r).1 * |G (s.1 r)| ≤
((m : ℝ) / cminus) * C := by
apply mul_le_mul
· calc
_ ≤ 1 / sourceCellMass P n (s.1 r).1 := by
gcongr
exact abs_targetEmpiricalMass_le_one hNpos _ _
_ ≤ (m : ℝ) / cminus := by
calc
1 / sourceCellMass P n (s.1 r).1 ≤
1 / (cminus / (m : ℝ)) :=
one_div_le_one_div_of_le hcmreal hq
_ = (m : ℝ) / cminus := by
field_simp [hcminus.ne', hmreal.ne']
· exact (hs r).2
· exact abs_nonneg _
· positivity
_ = (m : ℝ) / cminus * C := rfl
_ = C * (m : ℝ) / cminus := by
simp [hnreal.ne']
ring
Helpers.RegularCellRisk_Part5_Part2 3 declarations R1.8--R1.9: collision scale
R1.8--R1.9: collision scale
theorem collisionScale_memLp_for_witness
Under the regular finite-cell witness conditions, the target-sample collision-scale statistic is square-integrable.
Formal statement
hypotheses
P :
N k :
ℕ → ℕ
c epsilon :
ℝ
n :
ℕ
hNpos :
0 < N n
hIV :
TransportedIVClass P N k c epsilon n
m :
ℕ
hm :
0 < m
cminus :
ℝ
hcminus :
0 < cminus
cell :
Fin m ↪ 𝒳
hcell :
∀ i, MeasurableSet {cell i}
hrange :
sourceXLaw P n (range cell) = 1
hmass :
∀ i : Fin m, cminus / (m : ℝ) ≤ sourceCellMass P n (cell i)
conclusion
Proof (Lean source)
lemma collisionScale_memLp_for_witness
(P : TransportedArray 𝒳) (N k : ℕ → ℕ)
(c epsilon : ℝ) (n : ℕ)
(hNpos : 0 < N n)
(hIV : TransportedIVClass P N k c epsilon n)
{m : ℕ} (hm : 0 < m) (cminus : ℝ) (hcminus : 0 < cminus)
(cell : Fin m ↪ 𝒳)
(hcell : ∀ i, MeasurableSet {cell i})
(hrange : sourceXLaw P n (range cell) = 1)
(hmass : ∀ i : Fin m,
cminus / (m : ℝ) ≤ sourceCellMass P n (cell i)) :
MemLp (collisionScale (sourceCellMass P n)) 2
(Measure.pi (fun _ : Fin (N n) => targetXLaw P n)) := by
classical
letI : IsProbabilityMeasure (sourceObsLaw P n) :=
hIV.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (sourceXLaw P n) := by
unfold sourceXLaw
exact Measure.isProbabilityMeasure_map measurable_fst.aemeasurable
letI : IsProbabilityMeasure (targetXLaw P n) :=
hIV.twoSampleArray.2.2.1 n
let μT := Measure.pi (fun _ : Fin (N n) => targetXLaw P n)
let q := sourceCellMass P n
let kernel : 𝒳 → 𝒳 → ℝ := fun x y =>
∑ i : Fin m, if x = cell i ∧ y = cell i then 1 / q (cell i) else 0
let scale : TargetSample 𝒳 (N n) → ℝ := fun target =>
((N n : ℝ) * (N n - 1 : ℕ))⁻¹ *
∑ j, ∑ l, if j ≠ l then kernel (target j) (target l) else 0
have hkernel : Measurable (uncurry kernel) := by
dsimp only [kernel]
apply Finset.measurable_sum
intro i hi
apply Measurable.ite
· exact (((hcell i).preimage measurable_fst).inter
((hcell i).preimage measurable_snd))
· exact measurable_const
· exact measurable_const
have hscale : Measurable scale := by
dsimp only [scale]
apply measurable_const.mul
apply Finset.measurable_sum
intro j hj
apply Finset.measurable_sum
intro l hl
split_ifs
· have hp : Measurable (fun target : TargetSample 𝒳 (N n) =>
(target j, target l)) :=
(measurable_pi_apply j).prodMk (measurable_pi_apply l)
simpa [uncurry] using hkernel.fun_comp hp
· exact measurable_const
have hmreal : 0 < (m : ℝ) := by exact_mod_cast hm
have hcmreal : 0 < cminus / (m : ℝ) := div_pos hcminus hmreal
have hqpos (i : Fin m) : 0 < q (cell i) :=
hcmreal.trans_le (hmass i)
have hqinv (i : Fin m) : 1 / q (cell i) ≤ (m : ℝ) / cminus := by
calc
1 / q (cell i) ≤ 1 / (cminus / (m : ℝ)) :=
one_div_le_one_div_of_le hcmreal (hmass i)
_ = (m : ℝ) / cminus := by
field_simp [hcminus.ne', hmreal.ne']
have hkernel_nonneg (x y : 𝒳) : 0 ≤ kernel x y := by
dsimp only [kernel]
apply Finset.sum_nonneg
intro i hi
split_ifs
· exact one_div_nonneg.mpr (hqpos i).le
· exact le_rfl
have hkernel_bound (x y : 𝒳) :
|kernel x y| ≤ (m : ℝ) / cminus := by
rw [abs_of_nonneg (hkernel_nonneg x y)]
by_cases hxy : ∃ i : Fin m, x = cell i ∧ y = cell i
· obtain ⟨i, hxi, hyi⟩ := hxy
dsimp only [kernel]
rw [Finset.sum_eq_single i]
· simpa [hxi, hyi] using hqinv i
· intro i' hi' hne
have hxne : x ≠ cell i' := by
rw [hxi]
exact fun h => hne (cell.injective h).symm
simp [hxne]
· simp
· dsimp only [kernel]
simp only [not_exists, not_and] at hxy
have hzero :
(∑ i : Fin m,
if x = cell i ∧ y = cell i then 1 / q (cell i) else 0) = 0 := by
apply Finset.sum_eq_zero
intro i hi
rw [if_neg]
intro h
exact (hxy i h.1) h.2
rw [hzero]
exact div_nonneg hmreal.le hcminus.le
have hscale_bound (target : TargetSample 𝒳 (N n)) :
|scale target| ≤ 2 * ((m : ℝ) / cminus) := by
by_cases hNtwo : 2 ≤ N n
· have hNreal : 0 < (N n : ℝ) := by positivity
have hNm1real : 0 < ((N n - 1 : ℕ) : ℝ) := by
exact_mod_cast Nat.sub_pos_of_lt hNtwo
let S : ℝ := ∑ j, ∑ l,
if j ≠ l then kernel (target j) (target l) else 0
have hsum_nonneg : 0 ≤ S := by
dsimp only [S]
apply Finset.sum_nonneg
intro j hj
apply Finset.sum_nonneg
intro l hl
split_ifs
· exact hkernel_nonneg _ _
· exact le_rfl
have hsum_le :
S ≤ (N n : ℝ) ^ 2 * ((m : ℝ) / cminus) := by
dsimp only [S]
calc
_ ≤ ∑ _j : Fin (N n), ∑ _l : Fin (N n),
((m : ℝ) / cminus) := by
apply Finset.sum_le_sum
intro j hj
apply Finset.sum_le_sum
intro l hl
by_cases hjl : j ≠ l
· rw [if_pos hjl]
exact (le_abs_self _).trans (hkernel_bound _ _)
· rw [if_neg hjl]
exact (div_nonneg hmreal.le hcminus.le)
_ = (N n : ℝ) ^ 2 * ((m : ℝ) / cminus) := by
simp
ring
change
|((N n : ℝ) * (N n - 1 : ℕ))⁻¹ * S| ≤
2 * ((m : ℝ) / cminus)
rw [abs_mul, abs_of_pos (inv_pos.mpr (mul_pos hNreal hNm1real)),
abs_of_nonneg hsum_nonneg]
calc
((N n : ℝ) * (N n - 1 : ℕ))⁻¹ * S ≤
((N n : ℝ) * (N n - 1 : ℕ))⁻¹ *
((N n : ℝ) ^ 2 * ((m : ℝ) / cminus)) :=
mul_le_mul_of_nonneg_left hsum_le
(inv_nonneg.mpr (mul_pos hNreal hNm1real).le)
_ ≤ 2 * ((m : ℝ) / cminus) := by
have hNm1 : (N n : ℝ) ≤ 2 * ((N n - 1 : ℕ) : ℝ) := by
exact_mod_cast (show N n ≤ 2 * (N n - 1) by omega)
have hratio :
(N n : ℝ) / ((N n - 1 : ℕ) : ℝ) ≤ 2 :=
(div_le_iff₀ hNm1real).2 hNm1
calc
((N n : ℝ) * (N n - 1 : ℕ))⁻¹ *
((N n : ℝ) ^ 2 * ((m : ℝ) / cminus)) =
((N n : ℝ) / (N n - 1 : ℕ)) *
((m : ℝ) / cminus) := by
field_simp [hNreal.ne', hNm1real.ne']
<;> ring
_ ≤ 2 * ((m : ℝ) / cminus) :=
mul_le_mul_of_nonneg_right hratio
(div_nonneg hmreal.le hcminus.le)
· have hNone : N n = 1 := by omega
simp [scale, hNone]
positivity
have hscaleMem : MemLp scale 2 μT :=
MemLp.of_bound hscale.aestronglyMeasurable
(2 * ((m : ℝ) / cminus))
(Filter.Eventually.of_forall fun target => by
simpa [Real.norm_eq_abs] using hscale_bound target)
have hrangeMeas : MeasurableSet (range cell) := by
rw [show range cell = ⋃ i, {cell i} by
ext x
simp]
exact MeasurableSet.iUnion hcell
have hrangeCompl : sourceXLaw P n (range cell)ᶜ = 0 := by
rw [measure_compl hrangeMeas (measure_ne_top _ _), hrange, measure_univ]
simp
have htargetRange : ∀ᵐ x ∂targetXLaw P n, x ∈ range cell := by
have hc := hIV.transportDomination hrangeCompl
change range cell ∈ ae (targetXLaw P n)
rw [mem_ae_iff]
exact hc
have htargetSample :
∀ᵐ target ∂μT, ∀ j, target j ∈ range cell := by
rw [ae_all_iff]
intro j
have hx : ∀ᵐ x ∂Measure.map
(fun target : TargetSample 𝒳 (N n) => target j) μT,
x ∈ range cell := by
rw [(measurePreserving_eval
(fun _ : Fin (N n) => targetXLaw P n) j).map_eq]
exact htargetRange
exact ae_of_ae_map (measurable_pi_apply j).aemeasurable hx
have hkernel_eq (x y : 𝒳)
(hx : x ∈ range cell) (hy : y ∈ range cell) :
collisionKernel q x y = kernel x y := by
obtain ⟨i, hi⟩ := hx
obtain ⟨j, hj⟩ := hy
by_cases hxy : x = y
· have hij : i = j := by
apply cell.injective
exact hi.trans (hxy.trans hj.symm)
subst j
have hcoll : collisionKernel q x y = 1 / q x := by
simp [collisionKernel, collisionKernel, hxy]
rw [hcoll]
dsimp only [kernel]
rw [Finset.sum_eq_single i]
· rw [if_pos ⟨hi.symm, hj.symm⟩, hi]
· intro i' hi' hne
have hxne : x ≠ cell i' := by
intro h
exact hne (cell.injective (hi.trans h)).symm
simp [hxne]
· simp
· unfold collisionKernel collisionKernel kernel
rw [if_neg hxy]
symm
apply Finset.sum_eq_zero
intro i' hi'
have hnot : ¬(x = cell i' ∧ y = cell i') := by
rintro ⟨hxi, hyi⟩
exact hxy (hxi.trans hyi.symm)
simp [hnot]
have heq : scale =ᵐ[μT] collisionScale q := by
filter_upwards [htargetSample] with target ht
unfold scale collisionScale collisionScale
congr 1
apply Finset.sum_congr rfl
intro j hj
apply Finset.sum_congr rfl
intro l hl
split_ifs
· exact hkernel_eq _ _ (ht j) (ht l) |>.symm
· rfl
exact (memLp_congr_ae heq).1 hscaleMem
theorem regularCell_moments_memLp
The two cross moments and the collision proxy are square-integrable under the sampling law. The paper applies Chebyshev's inequality to each of these three statistics (writeup.tex:918-924 and 931-934) and so presupposes exactly this; it is pure regularity, carrying no rate or constant. Each holds because on a RegularFiniteCellClass member every ingredient is bounded: the affine score by 2 / epsilon, and the cell weights by k n / cminus on the class's finite support.
Formal statement
hypotheses
P :
N k :
ℕ → ℕ
c epsilon cminus cplus theta :
ℝ
n :
ℕ
hn :
0 < n
hNpos :
0 < N n
hP :
RegularFiniteCellClass P N k c epsilon cminus cplus n
htheta :
theta ∈ parameterSpace
conclusion
MemLp (fun s : TwoSample 𝒳 n (N n) => regularCellContrastMoment (sourceCellMass P n) (P.propensity n) theta s.1 s.2) 2 (twoSampleLaw P N n) ∧
MemLp (fun s : TwoSample 𝒳 n (N n) => regularCellReceiptMoment (sourceCellMass P n) (P.propensity n) s.1 s.2) 2 (twoSampleLaw P N n) ∧
Proof (Lean source)
lemma regularCell_moments_memLp
(P : TransportedArray 𝒳) (N k : ℕ → ℕ)
(c epsilon cminus cplus theta : ℝ) (n : ℕ)
(hn : 0 < n) (hNpos : 0 < N n)
(hP : RegularFiniteCellClass P N k c epsilon cminus cplus n)
(htheta : theta ∈ parameterSpace) :
MemLp
(fun s : TwoSample 𝒳 n (N n) =>
regularCellContrastMoment (sourceCellMass P n)
(P.propensity n) theta s.1 s.2) 2
(twoSampleLaw P N n) ∧
MemLp
(fun s : TwoSample 𝒳 n (N n) =>
regularCellReceiptMoment (sourceCellMass P n)
(P.propensity n) s.1 s.2) 2
(twoSampleLaw P N n) ∧
MemLp (collisionScale (sourceCellMass P n)) 2
(Measure.pi (fun _ : Fin (N n) => targetXLaw P n)) := by
classical
rcases hP with
⟨hIV, hk, hcm, hcmOne, hcp, cell, hcell, hrange, hmass⟩
have hInstrumentMeasurable : Measurable (instrumentScore P n) := by
have hz : Measurable fun o : SourceObs 𝒳 => o.2.1 := by fun_prop
have he : Measurable fun o : SourceObs 𝒳 => P.propensity n o.1 :=
(P.propensity_measurable n).comp measurable_fst
unfold instrumentScore
exact Measurable.ite (hz (MeasurableSet.singleton true))
(measurable_const.div he)
(measurable_const.neg.div (measurable_const.sub he))
have hBoolMeasurable :
Measurable (fun o : SourceObs 𝒳 => boolReal o.2.2.1) := by
unfold boolReal
have hd : Measurable fun o : SourceObs 𝒳 => o.2.2.1 := by fun_prop
exact Measurable.ite (hd (MeasurableSet.singleton true))
measurable_const measurable_const
have hScoreMeas :
Measurable (regularCellScore (P.propensity n) theta) := by
change Measurable (fun o : SourceObs 𝒳 =>
instrumentScore P n o * (o.2.2.2 - theta * boolReal o.2.2.1))
have hy : Measurable fun o : SourceObs 𝒳 => o.2.2.2 := by fun_prop
exact hInstrumentMeasurable.mul
(hy.sub (measurable_const.mul hBoolMeasurable))
have hScoreBound : ∀ᵐ o ∂sourceObsLaw P n,
|regularCellScore (P.propensity n) theta o| ≤ 2 / epsilon := by
have hoverlap : ∀ᵐ o ∂sourceObsLaw P n,
epsilon ≤ P.propensity n o.1 ∧
P.propensity n o.1 ≤ 1 - epsilon := by
have hx := hIV.instrumentOverlap.2.2
unfold sourceXLaw at hx
exact ae_of_ae_map measurable_fst.aemeasurable hx
filter_upwards [hoverlap, (sourceObservationFacts_of_class P N k c epsilon n hIV).2.2.1] with o hoverlap hy
exact abs_regularCellScore_le (P.propensity n) epsilon theta o
hIV.instrumentOverlap.1 hoverlap htheta hy
let Gd : SourceObs 𝒳 → ℝ := fun o =>
oracleInstrumentScore (P.propensity n) o * boolReal o.2.2.1
have hGdMeas : Measurable Gd :=
hInstrumentMeasurable.mul hBoolMeasurable
have hGdBound : ∀ᵐ o ∂sourceObsLaw P n, |Gd o| ≤ 1 / epsilon := by
have hInstrumentBound : ∀ᵐ o ∂sourceObsLaw P n,
|instrumentScore P n o| ≤ 1 / epsilon := by
have hoverlap : ∀ᵐ o ∂sourceObsLaw P n,
epsilon ≤ P.propensity n o.1 ∧
P.propensity n o.1 ≤ 1 - epsilon := by
have hx := hIV.instrumentOverlap.2.2
unfold sourceXLaw at hx
exact ae_of_ae_map measurable_fst.aemeasurable hx
filter_upwards [hoverlap] with o ho
rcases o with ⟨x, z, d, y⟩
cases z
· simp only [instrumentScore, Bool.false_eq_true, ↓reduceIte, abs_neg,
abs_div, abs_one]
rw [abs_of_pos (sub_pos.mpr (lt_of_le_of_lt ho.2
(by linarith [hIV.instrumentOverlap.1])))]
exact one_div_le_one_div_of_le hIV.instrumentOverlap.1
(by linarith [ho.2])
· simp only [instrumentScore, ↓reduceIte, abs_div, abs_one]
rw [abs_of_pos (lt_of_lt_of_le hIV.instrumentOverlap.1 ho.1)]
exact one_div_le_one_div_of_le hIV.instrumentOverlap.1 ho.1
filter_upwards [hInstrumentBound] with o ho
have hd : |boolReal o.2.2.1| ≤ 1 := by
cases o.2.2.1 <;> simp [boolReal]
change |instrumentScore P n o * boolReal o.2.2.1| ≤ 1 / epsilon
rw [abs_mul]
exact (mul_le_mul ho hd (abs_nonneg _)
(one_div_nonneg.mpr hIV.instrumentOverlap.1.le)).trans_eq (mul_one _)
have hContrast := regularCell_cross_memLp_for_witness
P N k c epsilon n hn hNpos hIV hk cminus hcm cell hcell hrange
(fun i => (hmass i).1)
(regularCellScore (P.propensity n) theta) (2 / epsilon)
(div_nonneg (by norm_num) hIV.instrumentOverlap.1.le)
hScoreMeas hScoreBound
have hReceipt := regularCell_cross_memLp_for_witness
P N k c epsilon n hn hNpos hIV hk cminus hcm cell hcell hrange
(fun i => (hmass i).1) Gd (1 / epsilon)
(one_div_nonneg.mpr hIV.instrumentOverlap.1.le) hGdMeas hGdBound
refine ⟨?_, ?_, ?_⟩
· simpa only [regularCell_crossAverage_identity] using hContrast
· simpa only [regularCellReceiptMoment, crossAverage_eq_empirical, Gd] using
hReceipt
· exact collisionScale_memLp_for_witness P N k c epsilon n hNpos hIV hk
cminus hcm cell hcell hrange (fun i => (hmass i).1)
theorem variance_offDiag_kernel_le
A bounded off-diagonal kernel average from an independent sample has variance at most 32 times the squared kernel bound divided by the sample size.
Formal statement
hypotheses
X :
Type*
μ :
N :
ℕ
kernel :
X → X → ℝ
hkernel :
Measurable (uncurry kernel)
M :
ℝ
_hM :
0 ≤ M
hbound :
∀ x y, |kernel x y| ≤ M
Proof (Lean source)
lemma variance_offDiag_kernel_le
{X : Type*} [MeasurableSpace X] (μ : Measure X)
[IsProbabilityMeasure μ] (N : ℕ)
(kernel : X → X → ℝ) (hkernel : Measurable (uncurry kernel))
(M : ℝ) (_hM : 0 ≤ M)
(hbound : ∀ x y, |kernel x y| ≤ M) :
variance
(fun target : Fin N → X =>
((N : ℝ) * (N - 1 : ℕ))⁻¹ *
∑ i, ∑ j, if i ≠ j then kernel (target i) (target j) else 0)
(Measure.pi (fun _ : Fin N => μ)) ≤
32 * M ^ 2 / (N : ℝ) := by
exact variance_offDiag_kernel_le
μ N kernel hkernel M hbound
Helpers.RegularCellRisk_Part5_Part3 3 declarations R1.9b: measurability of the known design input, and exact centering
R1.9b: measurability of the known design input, and exact centering
theorem regularCell_collision_moments
The collision component is unbiased, is bounded on the realized target support, and has the paper's variance bound.
Formal statement
hypotheses
P :
N k :
ℕ → ℕ
c epsilon cminus cplus :
ℝ
n :
ℕ
hNtwo :
2 ≤ N n
hP :
RegularFiniteCellClass P N k c epsilon cminus cplus n
conclusion
(∫ target : TargetSample 𝒳 (N n), collisionScale (sourceCellMass P n) target ∂Measure.pi (fun _ : Fin (N n) => targetXLaw P n))
= kishDispersion P n ∧
(∀ᵐ z : 𝒳 × 𝒳 ∂(targetXLaw P n).prod (targetXLaw P n), |collisionKernel (sourceCellMass P n) z.1 z.2| ≤ (k n : ℝ) / cminus) ∧
≤ 32 * (k n : ℝ) ^ 2 / (cminus ^ 2 * (N n : ℝ))
Proof (Lean source)
lemma regularCell_collision_moments
(P : TransportedArray 𝒳) (N k : ℕ → ℕ)
(c epsilon cminus cplus : ℝ) (n : ℕ)
(hNtwo : 2 ≤ N n)
(hP : RegularFiniteCellClass P N k c epsilon cminus cplus n) :
(∫ target : TargetSample 𝒳 (N n),
collisionScale (sourceCellMass P n) target
∂Measure.pi (fun _ : Fin (N n) => targetXLaw P n)) =
kishDispersion P n ∧
(∀ᵐ z : 𝒳 × 𝒳 ∂(targetXLaw P n).prod (targetXLaw P n),
|collisionKernel (sourceCellMass P n) z.1 z.2| ≤
(k n : ℝ) / cminus) ∧
variance
(collisionScale (sourceCellMass P n))
(Measure.pi (fun _ : Fin (N n) => targetXLaw P n)) ≤
32 * (k n : ℝ) ^ 2 / (cminus ^ 2 * (N n : ℝ)) := by
classical
rcases hP with
⟨hIV, hk, hcm, hcmOne, hcp, cell, hcell, hrange, hmass⟩
letI : IsProbabilityMeasure (sourceObsLaw P n) :=
hIV.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (sourceXLaw P n) := by
unfold sourceXLaw
exact Measure.isProbabilityMeasure_map measurable_fst.aemeasurable
letI : IsProbabilityMeasure (targetXLaw P n) :=
hIV.twoSampleArray.2.2.1 n
let μT := targetXLaw P n
let μN := Measure.pi (fun _ : Fin (N n) => μT)
let q := sourceCellMass P n
let kernel : 𝒳 → 𝒳 → ℝ := fun x y =>
∑ i : Fin (k n),
if x = cell i ∧ y = cell i then 1 / q (cell i) else 0
let scale : TargetSample 𝒳 (N n) → ℝ := fun target =>
((N n : ℝ) * (N n - 1 : ℕ))⁻¹ *
∑ j, ∑ l, if j ≠ l then kernel (target j) (target l) else 0
have hkernel : Measurable (uncurry kernel) := by
dsimp only [kernel]
apply measurable_sum
intro i hi
apply Measurable.ite
· exact (((hcell i).preimage measurable_fst).inter
((hcell i).preimage measurable_snd))
· exact measurable_const
· exact measurable_const
have hkreal : 0 < (k n : ℝ) := by exact_mod_cast hk
have hqpos (i : Fin (k n)) : 0 < q (cell i) :=
(div_pos hcm hkreal).trans_le (hmass i).1
have hqinv (i : Fin (k n)) :
1 / q (cell i) ≤ (k n : ℝ) / cminus := by
calc
1 / q (cell i) ≤ 1 / (cminus / (k n : ℝ)) :=
one_div_le_one_div_of_le (div_pos hcm hkreal) (hmass i).1
_ = (k n : ℝ) / cminus := by
field_simp [hcm.ne', hkreal.ne']
have hkernel_nonneg (x y : 𝒳) : 0 ≤ kernel x y := by
dsimp only [kernel]
apply sum_nonneg
intro i hi
split_ifs
· exact one_div_nonneg.mpr (hqpos i).le
· exact le_rfl
have hkernel_bound (x y : 𝒳) :
|kernel x y| ≤ (k n : ℝ) / cminus := by
rw [abs_of_nonneg (hkernel_nonneg x y)]
by_cases hxy : ∃ i : Fin (k n), x = cell i ∧ y = cell i
· obtain ⟨i, hxi, hyi⟩ := hxy
dsimp only [kernel]
rw [Finset.sum_eq_single i]
· simpa [hxi, hyi] using hqinv i
· intro i' hi' hne
have hxne : x ≠ cell i' := by
rw [hxi]
exact fun h => hne (cell.injective h).symm
simp [hxne]
· simp
· have hzero : kernel x y = 0 := by
dsimp only [kernel]
apply Finset.sum_eq_zero
intro i hi
rw [if_neg]
exact fun h => hxy ⟨i, h⟩
rw [hzero]
exact div_nonneg hkreal.le hcm.le
have hrangeMeas : MeasurableSet (range cell) := by
rw [show range cell = ⋃ i, {cell i} by
ext x
simp]
exact MeasurableSet.iUnion hcell
have hrangeCompl : sourceXLaw P n (range cell)ᶜ = 0 := by
rw [measure_compl hrangeMeas (measure_ne_top _ _), hrange, measure_univ]
simp
have htargetRange : ∀ᵐ x ∂μT, x ∈ range cell := by
have hc := hIV.transportDomination hrangeCompl
change range cell ∈ ae μT
rw [mem_ae_iff]
exact hc
have hkernel_eq (x y : 𝒳)
(hx : x ∈ range cell) (hy : y ∈ range cell) :
collisionKernel q x y = kernel x y := by
obtain ⟨i, hi⟩ := hx
obtain ⟨j, hj⟩ := hy
by_cases hxy : x = y
· have hij : i = j := by
apply cell.injective
exact hi.trans (hxy.trans hj.symm)
subst j
have hcoll : collisionKernel q x y = 1 / q x := by
simp [collisionKernel, collisionKernel, hxy]
rw [hcoll]
dsimp only [kernel]
rw [Finset.sum_eq_single i]
· rw [if_pos ⟨hi.symm, hj.symm⟩, hi]
· intro i' hi' hne
have hxne : x ≠ cell i' := by
intro h
exact hne (cell.injective (hi.trans h)).symm
simp [hxne]
· simp
· unfold collisionKernel collisionKernel kernel
rw [if_neg hxy]
symm
apply Finset.sum_eq_zero
intro i' hi'
rw [if_neg]
rintro ⟨hxi, hyi⟩
exact hxy (hxi.trans hyi.symm)
have htargetSample :
∀ᵐ target ∂μN, ∀ j, target j ∈ range cell := by
rw [ae_all_iff]
intro j
have hx : ∀ᵐ x ∂Measure.map
(fun target : TargetSample 𝒳 (N n) => target j) μN,
x ∈ range cell := by
rw [(measurePreserving_eval
(fun _ : Fin (N n) => μT) j).map_eq]
exact htargetRange
exact ae_of_ae_map (measurable_pi_apply j).aemeasurable hx
have hscale_eq : scale =ᵐ[μN] collisionScale q := by
filter_upwards [htargetSample] with target ht
unfold scale collisionScale collisionScale
congr 1
apply Finset.sum_congr rfl
intro j hj
apply Finset.sum_congr rfl
intro l hl
split_ifs
· exact hkernel_eq _ _ (ht j) (ht l) |>.symm
· rfl
have hprodSupport :
∀ᵐ z : 𝒳 × 𝒳 ∂μT.prod μT,
z.1 ∈ range cell ∧ z.2 ∈ range cell := by
have hfst : ∀ᵐ z : 𝒳 × 𝒳 ∂μT.prod μT,
z.1 ∈ range cell := by
have hm : ∀ᵐ x ∂Measure.map fst (μT.prod μT),
x ∈ range cell := by
simpa [MeasurePreserving.map_eq
(measurePreserving_fst (μ := μT) (ν := μT))] using htargetRange
exact (ae_map_iff
(measurePreserving_fst (μ := μT) (ν := μT)).aemeasurable
hrangeMeas).mp hm
have hsnd : ∀ᵐ z : 𝒳 × 𝒳 ∂μT.prod μT,
z.2 ∈ range cell := by
have hm : ∀ᵐ x ∂Measure.map snd (μT.prod μT),
x ∈ range cell := by
simpa [MeasurePreserving.map_eq
(measurePreserving_snd (μ := μT) (ν := μT))] using htargetRange
exact (ae_map_iff
(measurePreserving_snd (μ := μT) (ν := μT)).aemeasurable
hrangeMeas).mp hm
exact hfst.and hsnd
have horiginalBound :
∀ᵐ z : 𝒳 × 𝒳 ∂μT.prod μT,
|collisionKernel q z.1 z.2| ≤ (k n : ℝ) / cminus := by
filter_upwards [hprodSupport] with z hz
rw [hkernel_eq z.1 z.2 hz.1 hz.2]
exact hkernel_bound _ _
have hpairIntegral :
(∫ z : 𝒳 × 𝒳, kernel z.1 z.2 ∂μT.prod μT) =
∑ i : Fin (k n),
(μT {cell i}).toReal ^ 2 / q (cell i) := by
have htermInt (i : Fin (k n)) :
Integrable
(fun z : 𝒳 × 𝒳 =>
if z.1 = cell i ∧ z.2 = cell i then 1 / q (cell i) else 0)
(μT.prod μT) := by
apply Integrable.of_bound
(Measurable.ite
(((hcell i).preimage measurable_fst).inter
((hcell i).preimage measurable_snd))
measurable_const measurable_const).aestronglyMeasurable
|1 / q (cell i)|
filter_upwards with z
split_ifs with hsplit <;> simp [hsplit]
rw [show (fun z : 𝒳 × 𝒳 => kernel z.1 z.2) =
fun z => ∑ i : Fin (k n),
if z.1 = cell i ∧ z.2 = cell i then 1 / q (cell i) else 0 by
rfl]
rw [integral_finset_sum _ (fun i _ => htermInt i)]
apply Finset.sum_congr rfl
intro i hi
let f : 𝒳 → ℝ := fun x => if x = cell i then 1 / q (cell i) else 0
let g : 𝒳 → ℝ := fun y => if y = cell i then 1 else 0
have hf : Integrable f μT := by
apply Integrable.of_bound
(Measurable.ite (hcell i) measurable_const measurable_const
|>.aestronglyMeasurable) |1 / q (cell i)|
filter_upwards with x
split_ifs with hsplit <;> simp [hsplit]
have hg : Integrable g μT := by
apply Integrable.of_bound
(Measurable.ite (hcell i) measurable_const measurable_const
|>.aestronglyMeasurable) 1
filter_upwards with x
split_ifs with hsplit <;> simp [hsplit]
have hfg :
(fun z : 𝒳 × 𝒳 =>
if z.1 = cell i ∧ z.2 = cell i then 1 / q (cell i) else 0) =
fun z => f z.1 * g z.2 := by
funext z
by_cases hx : z.1 = cell i <;> by_cases hy : z.2 = cell i
<;> simp [f, g, hx, hy]
rw [hfg, integral_prod_mul f g]
have hfint : (∫ x, f x ∂μT) =
(μT {cell i}).toReal * (1 / q (cell i)) := by
rw [show f = indicator {cell i} (fun _ => 1 / q (cell i)) by
funext x
dsimp only [f]
rw [Set.indicator_apply]
by_cases hx : x = cell i
· rw [if_pos hx, if_pos]
exact hx
· rw [if_neg hx, if_neg]
exact fun h => hx h]
rw [integral_indicator (hcell i)]
simp [measureReal_def]
have hgint : (∫ x, g x ∂μT) = (μT {cell i}).toReal := by
rw [show g = indicator {cell i} (fun _ => (1 : ℝ)) by
funext x
dsimp only [g]
rw [Set.indicator_apply]
by_cases hx : x = cell i
· rw [if_pos hx, if_pos]
exact hx
· rw [if_neg hx, if_neg]
exact fun h => hx h]
rw [integral_indicator (hcell i)]
simp [measureReal_def]
rw [hfint, hgint]
-- … truncated; follow the source link for the rest …
theorem regularCell_Khat_lower_tail
The lower-tail failure probability for Khat has the two paper branches.
Formal statement
hypotheses
P :
N k :
ℕ → ℕ
c epsilon cminus cplus :
ℝ
n :
ℕ
hNtwo :
2 ≤ N n
hP :
RegularFiniteCellClass P N k c epsilon cminus cplus n
conclusion
(kishDispersion P n ≤ 2 → ∀ target : TargetSample 𝒳 (N n), kishDispersion P n / 2 ≤ regularCellKhat (sourceCellMass P n) target) ∧
(2 < kishDispersion P n → (Measure.pi (fun _ : Fin (N n) => targetXLaw P n) {target | regularCellKhat (sourceCellMass P n) target < kishDispersion P n / 2}).toReal ≤ 128 * (k n : ℝ) ^ 2 / (cminus ^ 2 * (N n : ℝ) * kishDispersion P n ^ 2))
Proof (Lean source)
lemma regularCell_Khat_lower_tail
(P : TransportedArray 𝒳) (N k : ℕ → ℕ)
(c epsilon cminus cplus : ℝ) (n : ℕ)
(hNtwo : 2 ≤ N n)
(hP : RegularFiniteCellClass P N k c epsilon cminus cplus n) :
(kishDispersion P n ≤ 2 →
∀ target : TargetSample 𝒳 (N n),
kishDispersion P n / 2 ≤
regularCellKhat (sourceCellMass P n) target) ∧
(2 < kishDispersion P n →
(Measure.pi (fun _ : Fin (N n) => targetXLaw P n)
{target | regularCellKhat (sourceCellMass P n) target <
kishDispersion P n / 2}).toReal ≤
128 * (k n : ℝ) ^ 2 /
(cminus ^ 2 * (N n : ℝ) * kishDispersion P n ^ 2)) := by
classical
rcases hP with
⟨hIV, hk, hcm, hcmOne, hcp, cell, hcell, hrange, hmass⟩
letI : IsProbabilityMeasure (sourceObsLaw P n) :=
hIV.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (sourceXLaw P n) := by
unfold sourceXLaw
exact Measure.isProbabilityMeasure_map measurable_fst.aemeasurable
letI : IsProbabilityMeasure (targetXLaw P n) :=
hIV.twoSampleArray.2.2.1 n
let μN := Measure.pi (fun _ : Fin (N n) => targetXLaw P n)
let q := sourceCellMass P n
have hscale_nonneg (target : TargetSample 𝒳 (N n)) :
0 ≤ collisionScale q target := by
unfold collisionScale collisionScale
apply mul_nonneg (inv_nonneg.mpr (mul_nonneg (by positivity) (by positivity)))
apply Finset.sum_nonneg
intro j hj
apply Finset.sum_nonneg
intro l hl
split_ifs
· unfold Causalean.Stat.collisionKernel
split_ifs
· exact one_div_nonneg.mpr measureReal_nonneg
· exact le_rfl
· exact le_rfl
have hPfull : RegularFiniteCellClass P N k c epsilon cminus cplus n :=
⟨hIV, hk, hcm, hcmOne, hcp, cell, hcell, hrange, hmass⟩
have hcoll := regularCell_collision_moments
P N k c epsilon cminus cplus n hNtwo hPfull
have hmem : MemLp (collisionScale q) 2 μN :=
collisionScale_memLp_for_witness P N k c epsilon n
(lt_of_lt_of_le (by omega) hNtwo) hIV hk cminus hcm cell hcell
hrange (fun i => (hmass i).1)
constructor
· intro hkish target
unfold regularCellKhat
linarith [hscale_nonneg target]
· intro hkish
have hkappa : 0 < kishDispersion P n / 2 := by linarith
have hsubset :
{target : TargetSample 𝒳 (N n) |
regularCellKhat q target < kishDispersion P n / 2} ⊆
{target |
kishDispersion P n / 2 ≤
|collisionScale q target -
∫ t, collisionScale q t ∂μN|} := by
intro target ht
rw [hcoll.1]
unfold regularCellKhat at ht
change 1 + collisionScale (𝒳 := 𝒳) q target <
kishDispersion P n / 2 at ht
have hneg :
collisionScale (𝒳 := 𝒳) q target - kishDispersion P n < 0 := by
linarith
have habs :
kishDispersion P n / 2 ≤
|collisionScale q target - kishDispersion P n| := by
rw [abs_of_neg hneg]
linarith
exact habs
have hcheb :=
meas_ge_le_variance_div_sq hmem hkappa
have hvar := hcoll.2.2
calc
(μN {target | regularCellKhat q target <
kishDispersion P n / 2}).toReal ≤
(μN {target | kishDispersion P n / 2 ≤
|collisionScale q target -
∫ t, collisionScale q t ∂μN|}).toReal :=
measureReal_mono hsubset
_ ≤ (ofReal
(variance (collisionScale q) μN /
(kishDispersion P n / 2) ^ 2)).toReal :=
ENNReal.toReal_mono ENNReal.ofReal_ne_top hcheb
_ = variance (collisionScale q) μN /
(kishDispersion P n / 2) ^ 2 := by
rw [ENNReal.toReal_ofReal]
exact div_nonneg (variance_nonneg _ _) (sq_nonneg _)
_ ≤ (32 * (k n : ℝ) ^ 2 /
(cminus ^ 2 * (N n : ℝ))) /
(kishDispersion P n / 2) ^ 2 := by
gcongr
_ = 128 * (k n : ℝ) ^ 2 /
(cminus ^ 2 * (N n : ℝ) *
kishDispersion P n ^ 2) := by
have hNreal : 0 < (N n : ℝ) := by positivity
have hkappa0 : kishDispersion P n ≠ 0 := by linarith
field_simp [hcm.ne', hNreal.ne', hkappa0]
ring
theorem regularCell_Khat_lower_tail_uniform
Uniformly, the probability that the collision proxy undershoots half the true dispersion vanishes.
Formal statement
hypotheses
N k :
ℕ → ℕ
c epsilon cminus cplus :
ℝ
hc :
0 < c
hcminus :
0 < cminus ∧ cminus ≤ 1
conclusion
Tendsto (fun n => ⨆ P : {P : TransportedArray 𝒳 // RegularFiniteCellClass P N k c epsilon cminus cplus n}, (Measure.pi (fun _ : Fin (N n) => targetXLaw P.1 n) {target | regularCellKhat (sourceCellMass P.1 n) target < kishDispersion P.1 n / 2}).toReal) atTop (𝓝 0)
Proof (Lean source)
lemma regularCell_Khat_lower_tail_uniform
(N k : ℕ → ℕ) (c epsilon cminus cplus : ℝ)
(hc : 0 < c) (hcminus : 0 < cminus ∧ cminus ≤ 1)
(hN : Tendsto (fun n : ℕ => (N n : ℝ) / (n : ℝ)) atTop (𝓝 c))
(hkRoot : Tendsto (fun n : ℕ => (k n : ℝ) / sqrt n)
atTop (𝓝 0)) :
Tendsto
(fun n => ⨆ P : {P : TransportedArray 𝒳 //
RegularFiniteCellClass P N k c epsilon cminus cplus n},
(Measure.pi (fun _ : Fin (N n) => targetXLaw P.1 n)
{target | regularCellKhat (sourceCellMass P.1 n) target <
kishDispersion P.1 n / 2}).toReal)
atTop (𝓝 0) := by
classical
let R : ℕ → ℝ := fun n =>
128 * (k n : ℝ) ^ 2 / (cminus ^ 2 * (N n : ℝ))
have hnpos : ∀ᶠ n : ℕ in atTop, 0 < n :=
eventually_atTop.2 ⟨1, fun n hn => Nat.zero_lt_of_lt hn⟩
have hNratio :
∀ᶠ n : ℕ in atTop, c / 2 < (N n : ℝ) / (n : ℝ) :=
(tendsto_order.1 hN).1 _ (by linarith)
obtain ⟨m : ℕ, hm : 2 / c < m⟩ := exists_nat_gt (2 / c)
have hm_event : ∀ᶠ n : ℕ in atTop, m ≤ n :=
eventually_atTop.2 ⟨m, fun _ hn => hn⟩
have hNtwo : ∀ᶠ n : ℕ in atTop, 2 ≤ N n := by
filter_upwards [hnpos, hNratio, hm_event] with n hn hratio hmn
have hnreal : 0 < (n : ℝ) := by exact_mod_cast hn
have hmnreal : (m : ℝ) ≤ n := by exact_mod_cast hmn
have hc_n : 2 < c * (n : ℝ) := by
have hcm : 2 < (m : ℝ) * c := (div_lt_iff₀ hc).mp hm
have hmcn : (m : ℝ) * c ≤ (n : ℝ) * c :=
mul_le_mul_of_nonneg_right hmnreal hc.le
nlinarith
have hNreal : c * (n : ℝ) / 2 < (N n : ℝ) := by
apply (lt_div_iff₀ hnreal).mp at hratio
nlinarith
exact_mod_cast (show (1 : ℝ) < N n by linarith)
have hkSq :
Tendsto (fun n : ℕ => ((k n : ℝ) / sqrt n) ^ 2)
atTop (𝓝 0) := by
simpa using hkRoot.pow 2
have hquot :
Tendsto
(fun n : ℕ =>
(((k n : ℝ) / sqrt n) ^ 2) /
((N n : ℝ) / (n : ℝ))) atTop (𝓝 0) := by
have hdiv := hkSq.div hN hc.ne'
rw [zero_div] at hdiv
exact hdiv
have hk2N :
Tendsto (fun n : ℕ => (k n : ℝ) ^ 2 / (N n : ℝ))
atTop (𝓝 0) := by
apply hquot.congr'
filter_upwards [hnpos, hNratio] with n hn hratio
have hnreal : 0 < (n : ℝ) := by exact_mod_cast hn
have hNreal : 0 < (N n : ℝ) := by
have : 0 < (N n : ℝ) / (n : ℝ) :=
lt_trans (half_pos hc) hratio
rcases div_pos_iff.mp this with h | h
· exact h.1
· exact (not_lt_of_ge hnreal.le h.2).elim
have hsqrt : sqrt (n : ℝ) ≠ 0 := by positivity
rw [div_pow]
rw [Real.sq_sqrt hnreal.le]
field_simp [hnreal.ne', hNreal.ne', hsqrt]
have hR : Tendsto R atTop (𝓝 0) := by
have hconst :
Tendsto (fun _ : ℕ => (128 : ℝ)) atTop (𝓝 128) :=
tendsto_const_nhds
have ht := (hconst.mul hk2N).div_const (cminus ^ 2)
have ht' :
Tendsto
(fun n : ℕ => 128 * ((k n : ℝ) ^ 2 / (N n : ℝ)) /
cminus ^ 2) atTop (𝓝 0) := by
simpa using ht
apply ht'.congr'
exact Filter.Eventually.of_forall fun n => by
unfold R
simp only [div_eq_mul_inv]
ring
refine squeeze_zero' (g := R) ?_ ?_ hR
· exact Filter.Eventually.of_forall fun n =>
Real.iSup_nonneg fun P => measureReal_nonneg
· filter_upwards [hNtwo] with n hn
cases isEmpty_or_nonempty {P : TransportedArray 𝒳 //
RegularFiniteCellClass P N k c epsilon cminus cplus n}
· simp
unfold R
positivity
apply ciSup_le
intro P
by_cases hsmall : kishDispersion P.1 n ≤ 2
· have hdet := (regularCell_Khat_lower_tail
P.1 N k c epsilon cminus cplus n hn P.2).1 hsmall
have hempty :
{target : TargetSample 𝒳 (N n) |
regularCellKhat (sourceCellMass P.1 n) target <
kishDispersion P.1 n / 2} =
(∅ : Set (TargetSample 𝒳 (N n))) := by
ext target
simp only [Set.mem_setOf_eq, Set.mem_empty_iff_false, iff_false]
exact not_lt_of_ge (hdet target)
rw [hempty]
simp
unfold R
have hNpos : 0 < (N n : ℝ) := by
exact_mod_cast (lt_of_lt_of_le (by omega : 0 < 2) hn)
positivity
· have hlarge : 2 < kishDispersion P.1 n := lt_of_not_ge hsmall
have htail := (regularCell_Khat_lower_tail
P.1 N k c epsilon cminus cplus n hn P.2).2 hlarge
calc
_ ≤ 128 * (k n : ℝ) ^ 2 /
(cminus ^ 2 * (N n : ℝ) * kishDispersion P.1 n ^ 2) := htail
_ ≤ R n := by
unfold R
have hNpos : 0 < (N n : ℝ) := by
exact_mod_cast (lt_of_lt_of_le (by omega : 0 < 2) hn)
have hden : 0 < cminus ^ 2 * (N n : ℝ) :=
mul_pos (sq_pos_of_pos hcminus.1) hNpos
have hkappaSq : 1 ≤ kishDispersion P.1 n ^ 2 := by nlinarith
apply div_le_div_of_nonneg_left (by positivity) hden
nlinarith
Helpers.RegularCellRisk_Part5_Part4 4 declarations R1.10: honesty in the exact liminf form
R1.10: honesty in the exact liminf form
theorem regularCell_sourceCellMass_measurable
On a regular finite-cell class member the known source-cell mass function is measurable. It vanishes off the class's finite cell support, because that support carries full source mass, and on the support it is constant on each of the class's measurable atoms; so it is a finite sum of scaled indicators.
Formal statement
hypotheses
P :
N k :
ℕ → ℕ
c epsilon cminus cplus :
ℝ
n :
ℕ
hP :
RegularFiniteCellClass P N k c epsilon cminus cplus n
conclusion
Measurable (sourceCellMass P n)
Proof (Lean source)
lemma regularCell_sourceCellMass_measurable
(P : TransportedArray 𝒳) (N k : ℕ → ℕ)
(c epsilon cminus cplus : ℝ) (n : ℕ)
(hP : RegularFiniteCellClass P N k c epsilon cminus cplus n) :
Measurable (sourceCellMass P n) := by
classical
rcases hP with
⟨hIV, hk, hcminus, hcminusOne, hcplus, cell, hcell, hrange, hmass⟩
letI : IsProbabilityMeasure (sourceObsLaw P n) :=
hIV.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (sourceXLaw P n) := by
unfold sourceXLaw
exact Measure.isProbabilityMeasure_map measurable_fst.aemeasurable
have hrangeMeas : MeasurableSet (range cell) := by
rw [show range cell = ⋃ i, {cell i} by
ext x
simp]
exact MeasurableSet.iUnion hcell
have hcompl : sourceXLaw P n (range cell)ᶜ = 0 := by
rw [measure_compl hrangeMeas (measure_ne_top _ _), hrange, measure_univ]
simp
have hoff (x : 𝒳) (hx : x ∉ range cell) :
sourceCellMass P n x = 0 := by
unfold sourceCellMass
have hsub : ({x} : Set 𝒳) ⊆ (range cell)ᶜ := by
intro y hy
rw [Set.mem_singleton_iff] at hy
subst y
exact hx
have hzero : sourceXLaw P n {x} = 0 :=
nonpos_iff_eq_zero.mp ((measure_mono hsub).trans_eq hcompl)
simp [hzero]
rw [show sourceCellMass P n =
fun x => ∑ i : Fin (k n),
if x = cell i then sourceCellMass P n (cell i) else 0 by
funext x
by_cases hx : x ∈ range cell
· rcases hx with ⟨i, rfl⟩
simp
· rw [hoff x hx]
symm
apply Finset.sum_eq_zero
intro i hi
simp [show x ≠ cell i by
intro h
exact hx ⟨i, h.symm⟩]]
apply measurable_sum
intro i hi
exact Measurable.ite (hcell i) measurable_const measurable_const
theorem integral_targetEmpiricalMass
The expected empirical target mass of a measurable cell equals its population target probability.
Formal statement
hypotheses
conclusion
= μ.real {a}
Proof (Lean source)
lemma integral_targetEmpiricalMass
(μ : Measure 𝒳) [IsProbabilityMeasure μ] {m : ℕ} (hm : 0 < m)
(a : 𝒳) (ha : MeasurableSet {a}) :
(∫ target : Fin m → 𝒳, targetEmpiricalMass target a
∂Measure.pi (fun _ : Fin m => μ)) =
μ.real {a} := by
exact integral_empiricalMass μ hm a ha
theorem regularCell_contrastMoment_mean_zero
At the target CACE the two-sample contrast moment is exactly centered: its mean under the sampling law is zero. This is the identification identity theta_T = mu_{Y,n} / mu_n transported to the level of the cross-averaged moments, and it is what makes the paper's Chebyshev step at writeup.tex:925-930 a bound on a centered moment.
Formal statement
hypotheses
P :
N k :
ℕ → ℕ
c epsilon cminus cplus :
ℝ
n :
ℕ
hn :
0 < n
hNpos :
0 < N n
hP :
RegularFiniteCellClass P N k c epsilon cminus cplus n
conclusion
(∫ s : TwoSample 𝒳 n (N n), regularCellContrastMoment (sourceCellMass P n) (P.propensity n) (targetCACE P n) s.1 s.2 ∂twoSampleLaw P N n)
= 0
Proof (Lean source)
lemma regularCell_contrastMoment_mean_zero
(P : TransportedArray 𝒳) (N k : ℕ → ℕ)
(c epsilon cminus cplus : ℝ) (n : ℕ)
(hn : 0 < n) (hNpos : 0 < N n)
(hP : RegularFiniteCellClass P N k c epsilon cminus cplus n) :
(∫ s : TwoSample 𝒳 n (N n),
regularCellContrastMoment (sourceCellMass P n) (P.propensity n)
(targetCACE P n) s.1 s.2 ∂twoSampleLaw P N n) = 0 := by
classical
rcases hP with
⟨hIV, hk, hcm, hcmOne, hcp, cell, hcell, hrange, hmass⟩
letI : IsProbabilityMeasure (sourceObsLaw P n) :=
hIV.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (sourceXLaw P n) := by
unfold sourceXLaw
exact Measure.isProbabilityMeasure_map measurable_fst.aemeasurable
letI : IsProbabilityMeasure (targetXLaw P n) :=
hIV.twoSampleArray.2.2.1 n
letI : IsProbabilityMeasure (twoSampleLaw P N n) := by
unfold twoSampleLaw
infer_instance
have hAssignmentInt :
Integrable (P.assignmentContrast n true) (sourceXLaw P n) :=
integrable_of_finite_support (sourceXLaw P n) cell hcell hrange _
(P.assignmentContrast_measurable n true).stronglyMeasurable
have hReceiptInt :
Integrable (P.receiptContrast n true) (sourceXLaw P n) :=
integrable_of_finite_support (sourceXLaw P n) cell hcell hrange _
(P.receiptContrast_measurable n true).stronglyMeasurable
have hTheta : targetCACE P n ∈ parameterSpace := by
rcases compact_causal_range P N k c epsilon n
hAssignmentInt hReceiptInt hIV with ⟨_, _, _, _, _, htheta⟩
exact htheta
have hPfull :
RegularFiniteCellClass P N k c epsilon cminus cplus n :=
⟨hIV, hk, hcm, hcmOne, hcp, cell, hcell, hrange, hmass⟩
have hcond (target : TargetSample 𝒳 (N n)) :=
regularCell_source_conditional_mean_variance_for_witness
P N k c epsilon cminus cplus (targetCACE P n) n target hn
hIV hk hcm hcmOne hcp cell hcell hrange hmass hTheta
have hscore :=
regularCell_score_mean_properties_for_witness
P N k c epsilon cminus cplus n hAssignmentInt hReceiptInt hPfull
cell hcell hrange hmass
have hmem :=
(regularCell_moments_memLp
P N k c epsilon cminus cplus (targetCACE P n) n
hn hNpos hPfull hTheta).1
let μT := Measure.pi (fun _ : Fin (N n) => targetXLaw P n)
have hphatMem (i : Fin (k n)) :
MemLp (fun target : TargetSample 𝒳 (N n) =>
targetEmpiricalMass target (cell i)) 2 μT := by
unfold targetEmpiricalMass empiricalMass
apply MemLp.const_mul
apply memLp_finset_sum
intro j hj
refine MemLp.of_bound ?_ 1 ?_
· have hjmeas : Measurable fun target :
TargetSample 𝒳 (N n) => target j :=
measurable_pi_apply j
exact
(Measurable.ite (hjmeas (hcell i))
measurable_const measurable_const).aestronglyMeasurable
· filter_upwards with target
split_ifs <;> simp
unfold twoSampleLaw
rw [integral_prod_symm _ (hmem.integrable (by norm_num))]
calc
(∫ target, ∫ source,
regularCellContrastMoment (sourceCellMass P n) (P.propensity n)
(targetCACE P n) source target
∂Measure.pi (fun _ : Fin n => sourceObsLaw P n)
∂Measure.pi (fun _ : Fin (N n) => targetXLaw P n)) =
∫ target, ∑ i, targetEmpiricalMass target (cell i) *
regularCellScoreMean P n (targetCACE P n) (cell i) ∂μT := by
apply integral_congr_ae
filter_upwards with target
simpa only [μT] using (hcond target).1
_ = ∑ i, (targetXLaw P n {cell i}).toReal *
regularCellScoreMean P n (targetCACE P n) (cell i) := by
rw [integral_finset_sum]
· apply Finset.sum_congr rfl
intro i hi
rw [integral_mul_const,
integral_targetEmpiricalMass
(targetXLaw P n) hNpos (cell i) (hcell i)]
simp only [Measure.real]
· intro i hi
exact (hphatMem i).integrable (by norm_num) |>.mul_const _
_ = 0 := hscore.2.2
theorem regularCellProcedure_coverage_liminf
The regular-cell inversion procedure is uniformly asymptotically honest. The conclusion is deliberately the coverage component of RegularCellHonest, so the theorem assembly can use it directly.
Formal statement
hypotheses
N k :
ℕ → ℕ
c epsilon alpha cminus cplus L :
ℝ
hc :
0 < c
hepsilon :
0 < epsilon ∧ epsilon < 1 / 2
halpha :
0 < alpha ∧ alpha < 1
hcminus :
0 < cminus ∧ cminus ≤ 1
hcplus :
1 ≤ cplus
hkPos :
∀ n, 0 < k n
hCarrier :
∀ n, ∃ cell : Fin (k n) ↪ 𝒳, ∀ i, MeasurableSet {cell i}
hL :
sqrt (2 * regularCellVarianceConstant epsilon c / alpha) ≤ L
conclusion
1 - alpha
≤ liminf (fun n => ⨅ P : {P : TransportedArray 𝒳 // RegularFiniteCellClass P N k c epsilon cminus cplus n}, regularCellCoverage (regularCellProcedure (𝒳 := 𝒳) N k L) n P) atTop
Proof (Lean source)
lemma regularCellProcedure_coverage_liminf
(N k : ℕ → ℕ) (c epsilon alpha cminus cplus L : ℝ)
(hc : 0 < c)
(hepsilon : 0 < epsilon ∧ epsilon < 1 / 2)
(halpha : 0 < alpha ∧ alpha < 1)
(hcminus : 0 < cminus ∧ cminus ≤ 1)
(hcplus : 1 ≤ cplus)
(hN : Tendsto (fun n : ℕ => (N n : ℝ) / (n : ℝ)) atTop (𝓝 c))
(hkPos : ∀ n, 0 < k n)
(hkInf : Tendsto (fun n : ℕ => (k n : ℝ)) atTop atTop)
(hkRoot : Tendsto (fun n : ℕ => (k n : ℝ) / sqrt n)
atTop (𝓝 0))
(hCarrier : ∀ n, ∃ cell : Fin (k n) ↪ 𝒳,
∀ i, MeasurableSet {cell i})
(hL : sqrt
(2 * regularCellVarianceConstant epsilon c / alpha) ≤ L) :
1 - alpha ≤ liminf
(fun n => ⨅ P : {P : TransportedArray 𝒳 //
RegularFiniteCellClass P N k c epsilon cminus cplus n},
regularCellCoverage (regularCellProcedure (𝒳 := 𝒳) N k L) n P)
atTop := by
classical
let δ : ℕ → ℝ := fun n =>
⨆ P : {P : TransportedArray 𝒳 //
RegularFiniteCellClass P N k c epsilon cminus cplus n},
(Measure.pi (fun _ : Fin (N n) => targetXLaw P.1 n)
{target | regularCellKhat (sourceCellMass P.1 n) target <
kishDispersion P.1 n / 2}).toReal
have hδ : Tendsto δ atTop (𝓝 0) :=
regularCell_Khat_lower_tail_uniform
N k c epsilon cminus cplus hc hcminus hN hkRoot
have hnpos : ∀ᶠ n : ℕ in atTop, 0 < n :=
eventually_atTop.2 ⟨1, fun n hn => Nat.zero_lt_of_lt hn⟩
have hNratio :
∀ᶠ n : ℕ in atTop, c / 2 < (N n : ℝ) / (n : ℝ) :=
(tendsto_order.1 hN).1 _ (by linarith)
obtain ⟨m : ℕ, hm : 2 / c < m⟩ := exists_nat_gt (2 / c)
have hm_event : ∀ᶠ n : ℕ in atTop, m ≤ n :=
eventually_atTop.2 ⟨m, fun _ hn => hn⟩
have hNtwo : ∀ᶠ n : ℕ in atTop, 2 ≤ N n := by
filter_upwards [hnpos, hNratio, hm_event] with n hn hratio hmn
have hnreal : 0 < (n : ℝ) := by exact_mod_cast hn
have hmnreal : (m : ℝ) ≤ n := by exact_mod_cast hmn
have hc_n : 2 < c * (n : ℝ) := by
have hcm : 2 < (m : ℝ) * c := (div_lt_iff₀ hc).mp hm
have hmcn : (m : ℝ) * c ≤ (n : ℝ) * c :=
mul_le_mul_of_nonneg_right hmnreal hc.le
nlinarith
have hNreal : c * (n : ℝ) / 2 < (N n : ℝ) := by
apply (lt_div_iff₀ hnreal).mp at hratio
nlinarith
exact_mod_cast (show (1 : ℝ) < N n by linarith)
have hvariance :=
regularCell_eventually_uniform_moment_variance
(𝒳 := 𝒳)
N k c epsilon cminus cplus hc hepsilon hcminus hcplus
hN hkPos hkRoot
have hrows :
∀ᶠ n in atTop,
1 - alpha - δ n ≤
⨅ P : {P : TransportedArray 𝒳 //
RegularFiniteCellClass P N k c epsilon cminus cplus n},
regularCellCoverage
(regularCellProcedure (𝒳 := 𝒳) N k L) n P := by
filter_upwards [hnpos, hNtwo, hvariance] with n hn hN2 hvar
obtain ⟨P₀, hP₀⟩ :=
regularFiniteCellClass_inhabited
N k c epsilon cminus cplus hCarrier hc hepsilon hcminus hcplus
hN hkPos hkInf hkRoot n
letI : Nonempty {P : TransportedArray 𝒳 //
RegularFiniteCellClass P N k c epsilon cminus cplus n} :=
⟨⟨P₀, hP₀⟩⟩
apply le_ciInf
intro Psub
let P := Psub.1
have hP : RegularFiniteCellClass P N k c epsilon cminus cplus n :=
Psub.2
have hIV := hP.1
have hk := hP.2.1
have hcm := hP.2.2.1
have hcmOne := hP.2.2.2.1
have hcp := hP.2.2.2.2.1
let hw := hP.2.2.2.2.2
let cell := choose hw
have hcell := (Classical.choose_spec hw).1
have hrange := (Classical.choose_spec hw).2.1
have hmass := (Classical.choose_spec hw).2.2
have hPfull := hP
let design := regularCellDesignOfClass P hPfull
letI : IsProbabilityMeasure (sourceObsLaw P n) :=
hIV.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (sourceXLaw P n) := by
unfold sourceXLaw
exact Measure.isProbabilityMeasure_map measurable_fst.aemeasurable
letI : IsProbabilityMeasure (targetXLaw P n) :=
hIV.twoSampleArray.2.2.1 n
letI : IsProbabilityMeasure (twoSampleLaw P N n) := by
unfold twoSampleLaw
infer_instance
let q := sourceCellMass P n
let e := P.propensity n
let theta := targetCACE P n
let R : Set (TwoSample 𝒳 n (N n)) :=
{s | (∀ i, (s.1 i).1 ∈ range cell) ∧
(∀ j, s.2 j ∈ range cell)}
let A : Set (TwoSample 𝒳 n (N n)) :=
{s | theta ∈
regularCellSet (regularCellProcedure (𝒳 := 𝒳) N k L) n P hPfull s}
let S := R ∩ A
have hq : Measurable q := by
exact regularCell_sourceCellMass_measurable
P N k c epsilon cminus cplus n hPfull
have hgraph :=
(regularCellProcedure (𝒳 := 𝒳) N k L).measurableGraph
n design
(fun i => sourceCellMass P n (design.cell i))
(fun i => P.propensity n (design.cell i))
have hSmeas : MeasurableSet S := by
have hp :
Measurable fun s : TwoSample 𝒳 n (N n) => (s, theta) :=
measurable_id.prodMk measurable_const
exact hgraph.preimage hp
have hrangeMeas : MeasurableSet (range cell) := by
rw [show range cell = ⋃ i, {cell i} by
ext x
simp]
exact MeasurableSet.iUnion hcell
have hsourceCompl : sourceXLaw P n (range cell)ᶜ = 0 := by
rw [measure_compl hrangeMeas (measure_ne_top _ _), hrange, measure_univ]
simp
have htargetCompl : targetXLaw P n (range cell)ᶜ = 0 :=
hIV.transportDomination hsourceCompl
have htargetRange : targetXLaw P n (range cell) = 1 := by
exact (prob_compl_eq_zero_iff hrangeMeas).1 htargetCompl
let SG : Set (SourceSample 𝒳 n) :=
{source | ∀ i, (source i).1 ∈ range cell}
let TG : Set (TargetSample 𝒳 (N n)) :=
{target | ∀ j, target j ∈ range cell}
have hSGmeas : MeasurableSet SG := by
rw [show SG = ⋂ i,
{source | (source i).1 ∈ range cell} by
ext source
simp [SG]]
exact MeasurableSet.iInter fun i =>
hrangeMeas.preimage
(measurable_fst.comp (measurable_pi_apply i))
have hTGmeas : MeasurableSet TG := by
rw [show TG = ⋂ j,
{target | target j ∈ range cell} by
ext target
simp [TG]]
exact MeasurableSet.iInter fun j =>
hrangeMeas.preimage (measurable_pi_apply j)
have hsourceXAE :
∀ᵐ x ∂sourceXLaw P n, x ∈ range cell :=
(mem_ae_iff_prob_eq_one hrangeMeas).2 hrange
have hsourceObsAE :
∀ᵐ o ∂sourceObsLaw P n, o.1 ∈ range cell := by
unfold sourceXLaw at hsourceXAE
exact ae_of_ae_map measurable_fst.aemeasurable hsourceXAE
have hSGAE :
∀ᵐ source ∂Measure.pi (fun _ : Fin n => sourceObsLaw P n),
source ∈ SG := by
apply Measure.ae_pi_le_pi
exact Filter.eventually_pi fun i => hsourceObsAE
have htargetXAE :
∀ᵐ x ∂targetXLaw P n, x ∈ range cell :=
(mem_ae_iff_prob_eq_one hrangeMeas).2 htargetRange
have hTGAE :
∀ᵐ target ∂Measure.pi (fun _ : Fin (N n) => targetXLaw P n),
target ∈ TG := by
apply Measure.ae_pi_le_pi
exact Filter.eventually_pi fun j => htargetXAE
have hSGone :
Measure.pi (fun _ : Fin n => sourceObsLaw P n) SG = 1 :=
(mem_ae_iff_prob_eq_one hSGmeas).1 hSGAE
have hTGone :
Measure.pi (fun _ : Fin (N n) => targetXLaw P n) TG = 1 :=
(mem_ae_iff_prob_eq_one hTGmeas).1 hTGAE
have hReq : R = SG ×ˢ TG := by
ext s
simp [R, SG, TG]
have hRone : twoSampleLaw P N n R = 1 := by
rw [hReq]
unfold twoSampleLaw
rw [Measure.prod_prod, hSGone, hTGone]
simp
have hRmeas : MeasurableSet R := by
rw [hReq]
exact hSGmeas.prod hTGmeas
have hRcompl : twoSampleLaw P N n Rᶜ = 0 :=
(prob_compl_eq_zero_iff hRmeas).2 hRone
have hAssignmentInt :
Integrable (P.assignmentContrast n true) (sourceXLaw P n) :=
integrable_of_finite_support (sourceXLaw P n) cell hcell hrange _
(P.assignmentContrast_measurable n true).stronglyMeasurable
have hReceiptInt :
Integrable (P.receiptContrast n true) (sourceXLaw P n) :=
integrable_of_finite_support (sourceXLaw P n) cell hcell hrange _
(P.receiptContrast_measurable n true).stronglyMeasurable
have hTheta : theta ∈ parameterSpace := by
rcases compact_causal_range P N k c epsilon n
hAssignmentInt hReceiptInt hIV with ⟨_, _, _, _, _, htheta⟩
exact htheta
let Z : TwoSample 𝒳 n (N n) → ℝ := fun s =>
regularCellContrastMoment q e theta s.1 s.2
let KB : Set (TwoSample 𝒳 n (N n)) :=
{s | regularCellKhat q s.2 < kishDispersion P n / 2}
let threshold : ℝ :=
L * sqrt (kishDispersion P n / (2 * n))
let ZB : Set (TwoSample 𝒳 n (N n)) :=
{s | threshold < |Z s|}
have hkappa :=
one_le_regularCell_kish
P N k c epsilon n hIV cell hcell hrange
have hkappaPos : 0 < kishDispersion P n :=
lt_of_lt_of_le zero_lt_one hkappa
have hnreal : 0 < (n : ℝ) := by exact_mod_cast hn
have hBpos : 0 < regularCellVarianceConstant epsilon c := by
unfold regularCellVarianceConstant
have heps : 0 < epsilon := hepsilon.1
positivity
have hargPos :
0 < 2 * regularCellVarianceConstant epsilon c / alpha := by
exact div_pos (mul_pos (by norm_num) hBpos) halpha.1
have hLpos : 0 < L :=
lt_of_lt_of_le (Real.sqrt_pos.2 hargPos) hL
have hthresholdPos : 0 < threshold := by
unfold threshold
apply mul_pos hLpos
apply Real.sqrt_pos.2
positivity
have hmean :
(∫ s : TwoSample 𝒳 n (N n), Z s ∂twoSampleLaw P N n) = 0 := by
simpa [Z, q, e, theta] using
regularCell_contrastMoment_mean_zero
P N k c epsilon cminus cplus n hn
(lt_of_lt_of_le (by omega) hN2) hPfull
have hmem : MemLp Z 2 (twoSampleLaw P N n) := by
simpa [Z, q, e, theta] using
(regularCell_moments_memLp
P N k c epsilon cminus cplus (targetCACE P n) n hn
(lt_of_lt_of_le (by omega) hN2) hPfull hTheta).1
have hZsubset :
ZB ⊆
{s | threshold ≤
|Z s - ∫ t, Z t ∂twoSampleLaw P N n|} := by
-- … truncated; follow the source link for the rest …
Helpers.RegularCellRisk_Part5_Part5 2 declarations
theorem regularCell_firstStage_bad_probability
The empirical first stage misses half its positive mean with probability at most 4 B / t, eventually and uniformly.
Formal statement
hypotheses
N k :
ℕ → ℕ
c epsilon cminus cplus :
ℝ
hc :
0 < c
hepsilon :
0 < epsilon ∧ epsilon < 1 / 2
hcminus :
0 < cminus ∧ cminus ≤ 1
hcplus :
1 ≤ cplus
hkPos :
∀ n, 0 < k n
conclusion
∀ᶠ n in atTop,
∀ P : TransportedArray 𝒳,
RegularFiniteCellClass P N k c epsilon cminus cplus n → (twoSampleLaw P N n {s | transportedFirstStage P n / 2 < |regularCellReceiptMoment (sourceCellMass P n) (P.propensity n) s.1 s.2 - transportedFirstStage P n|}).toReal
≤ 4 * regularCellVarianceConstant epsilon c / effectiveStrength P n
Proof (Lean source)
lemma regularCell_firstStage_bad_probability
(N k : ℕ → ℕ) (c epsilon cminus cplus : ℝ)
(hc : 0 < c)
(hepsilon : 0 < epsilon ∧ epsilon < 1 / 2)
(hcminus : 0 < cminus ∧ cminus ≤ 1)
(hcplus : 1 ≤ cplus)
(hN : Tendsto (fun n : ℕ => (N n : ℝ) / (n : ℝ)) atTop (𝓝 c))
(hkPos : ∀ n, 0 < k n)
(hkRoot : Tendsto (fun n : ℕ => (k n : ℝ) / sqrt n)
atTop (𝓝 0)) :
∀ᶠ n in atTop, ∀ P : TransportedArray 𝒳,
RegularFiniteCellClass P N k c epsilon cminus cplus n →
(twoSampleLaw P N n
{s | transportedFirstStage P n / 2 <
|regularCellReceiptMoment (sourceCellMass P n)
(P.propensity n) s.1 s.2 -
transportedFirstStage P n|}).toReal ≤
4 * regularCellVarianceConstant epsilon c /
effectiveStrength P n := by
have hvar := regularCell_eventually_uniform_moment_variance
(𝒳 := 𝒳) N k c epsilon cminus cplus hc hepsilon hcminus hcplus
hN hkPos hkRoot
have hnpos : ∀ᶠ n : ℕ in atTop, 0 < n :=
eventually_atTop.2 ⟨1, fun n hn => Nat.zero_lt_of_lt hn⟩
have hNratio :
∀ᶠ n : ℕ in atTop, c / 2 < (N n : ℝ) / (n : ℝ) :=
(tendsto_order.1 hN).1 _ (by linarith)
filter_upwards [hvar, hnpos, hNratio] with n hvarn hn hratio
intro P hP
classical
rcases hP with
⟨hIV, hk, hcm, hcmOne, hcp, cell, hcell, hrange, hmass⟩
have hPfull : RegularFiniteCellClass P N k c epsilon cminus cplus n :=
⟨hIV, hk, hcm, hcmOne, hcp, cell, hcell, hrange, hmass⟩
letI : IsProbabilityMeasure (sourceObsLaw P n) :=
hIV.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (sourceXLaw P n) := by
unfold sourceXLaw
exact Measure.isProbabilityMeasure_map measurable_fst.aemeasurable
letI : IsProbabilityMeasure (targetXLaw P n) :=
hIV.twoSampleArray.2.2.1 n
letI : IsProbabilityMeasure (twoSampleLaw P N n) := by
unfold twoSampleLaw
infer_instance
have hAssignmentInt :
Integrable (P.assignmentContrast n true) (sourceXLaw P n) :=
integrable_of_finite_support (sourceXLaw P n) cell hcell hrange _
(P.assignmentContrast_measurable n true).stronglyMeasurable
have hReceiptInt :
Integrable (P.receiptContrast n true) (sourceXLaw P n) :=
integrable_of_finite_support (sourceXLaw P n) cell hcell hrange _
(P.receiptContrast_measurable n true).stronglyMeasurable
have hcompact := compact_causal_range P N k c epsilon n
hAssignmentInt hReceiptInt hIV
have hfirstEq :
transportedFirstStage P n = targetComplierShare P n :=
hcompact.2.2.2.1
have htheta : targetCACE P n ∈ parameterSpace :=
hcompact.2.2.2.2.2
have hfirst : 0 < transportedFirstStage P n := by
rw [hfirstEq]
exact hIV.targetComplierPositivity
have hmoment := hvarn P hPfull
have hNpos : 0 < N n := by
have hnreal : 0 < (n : ℝ) := by exact_mod_cast hn
have : 0 < (N n : ℝ) / (n : ℝ) :=
lt_trans (half_pos hc) hratio
have hNreal : 0 < (N n : ℝ) := by
rcases div_pos_iff.mp this with h | h
· exact h.1
· exact (not_lt_of_ge hnreal.le h.2).elim
exact_mod_cast hNreal
have hmem := (regularCell_moments_memLp
P N k c epsilon cminus cplus (targetCACE P n) n hn hNpos
hPfull htheta).2.1
have hcheb := meas_ge_le_variance_div_sq hmem (half_pos hfirst)
have hset :
{s : TwoSample 𝒳 n (N n) |
transportedFirstStage P n / 2 <
|regularCellReceiptMoment (sourceCellMass P n)
(P.propensity n) s.1 s.2 -
transportedFirstStage P n|} ⊆
{s |
transportedFirstStage P n / 2 ≤
|regularCellReceiptMoment (sourceCellMass P n)
(P.propensity n) s.1 s.2 -
∫ t, regularCellReceiptMoment (sourceCellMass P n)
(P.propensity n) t.1 t.2 ∂twoSampleLaw P N n|} := by
intro s hs
rw [hmoment.2.2]
change transportedFirstStage P n / 2 <
|regularCellReceiptMoment (sourceCellMass P n)
(P.propensity n) s.1 s.2 - transportedFirstStage P n| at hs
exact hs.le
calc
_ ≤ (twoSampleLaw P N n
{s | transportedFirstStage P n / 2 ≤
|regularCellReceiptMoment (sourceCellMass P n)
(P.propensity n) s.1 s.2 -
∫ t, regularCellReceiptMoment (sourceCellMass P n)
(P.propensity n) t.1 t.2 ∂twoSampleLaw P N n|}).toReal :=
measureReal_mono hset
_ ≤ (ofReal
(variance
(fun s : TwoSample 𝒳 n (N n) =>
regularCellReceiptMoment (sourceCellMass P n)
(P.propensity n) s.1 s.2)
(twoSampleLaw P N n) /
(transportedFirstStage P n / 2) ^ 2)).toReal :=
ENNReal.toReal_mono ENNReal.ofReal_ne_top hcheb
_ = variance
(fun s : TwoSample 𝒳 n (N n) =>
regularCellReceiptMoment (sourceCellMass P n)
(P.propensity n) s.1 s.2)
(twoSampleLaw P N n) /
(transportedFirstStage P n / 2) ^ 2 := by
rw [ENNReal.toReal_ofReal]
exact div_nonneg (variance_nonneg _ _) (sq_nonneg _)
_ ≤ (regularCellVarianceConstant epsilon c *
kishDispersion P n / n) /
(transportedFirstStage P n / 2) ^ 2 := by
gcongr
exact hmoment.2.1
_ = 4 * regularCellVarianceConstant epsilon c /
effectiveStrength P n := by
have hnreal : 0 < (n : ℝ) := by exact_mod_cast hn
have hkappa := one_le_regularCell_kish
P N k c epsilon n hIV cell hcell hrange
have hkappaPos : 0 < kishDispersion P n :=
lt_of_lt_of_le zero_lt_one hkappa
unfold effectiveStrength
field_simp [hnreal.ne', hfirst.ne', hkappaPos.ne']
ring
theorem regularCell_Khat_mean
Khat has mean 1 + kappa, bounded by 2 kappa; the latter uses the probability-density lower bound kappa ≥ 1.
Formal statement
hypotheses
P :
N k :
ℕ → ℕ
c epsilon cminus cplus :
ℝ
n :
ℕ
hNtwo :
2 ≤ N n
hP :
RegularFiniteCellClass P N k c epsilon cminus cplus n
conclusion
(∫ target : TargetSample 𝒳 (N n), regularCellKhat (sourceCellMass P n) target ∂Measure.pi (fun _ : Fin (N n) => targetXLaw P n))
= 1 + kishDispersion P n ∧
1 + kishDispersion P n ≤ 2 * kishDispersion P n
Proof (Lean source)
lemma regularCell_Khat_mean
(P : TransportedArray 𝒳) (N k : ℕ → ℕ)
(c epsilon cminus cplus : ℝ) (n : ℕ)
(hNtwo : 2 ≤ N n)
(hP : RegularFiniteCellClass P N k c epsilon cminus cplus n) :
(∫ target : TargetSample 𝒳 (N n),
regularCellKhat (sourceCellMass P n) target
∂Measure.pi (fun _ : Fin (N n) => targetXLaw P n)) =
1 + kishDispersion P n ∧
1 + kishDispersion P n ≤ 2 * kishDispersion P n := by
classical
have hcoll := regularCell_collision_moments
P N k c epsilon cminus cplus n hNtwo hP
rcases hP with
⟨hIV, hk, hcm, hcmOne, hcp, cell, hcell, hrange, hmass⟩
letI : IsProbabilityMeasure (sourceObsLaw P n) :=
hIV.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (sourceXLaw P n) := by
unfold sourceXLaw
exact Measure.isProbabilityMeasure_map measurable_fst.aemeasurable
letI : IsProbabilityMeasure (targetXLaw P n) :=
hIV.twoSampleArray.2.2.1 n
have hmem := collisionScale_memLp_for_witness
P N k c epsilon n (lt_of_lt_of_le (by omega) hNtwo)
hIV hk cminus hcm cell hcell hrange (fun i => (hmass i).1)
constructor
· unfold regularCellKhat
rw [integral_add]
· simp [hcoll.1]
· exact integrable_const 1
· exact hmem.integrable (by norm_num)
· have hkappa := one_le_regularCell_kish
P N k c epsilon n hIV cell hcell hrange
linarith
Helpers.ScoreInversion 14 declarations
def oracleInstrumentScore
Instrument multiplier formed from the declared propensity input.
Definition (Lean source)
noncomputable def oracleInstrumentScore (e : 𝒳 → ℝ)
(o : SourceObs 𝒳) : ℝ :=
if o.2.1 then 1 / e o.1 else -1 / (1 - e o.1)
def scoreOutcomeMean
Empirical transported outcome score.
Definition (Lean source)
noncomputable def scoreOutcomeMean (weight e : 𝒳 → ℝ) (n : ℕ)
(sample : SourceSample 𝒳 n) : ℝ :=
(n : ℝ)⁻¹ * ∑ i, weight (sample i).1 *
oracleInstrumentScore e (sample i) * (sample i).2.2.2
def scoreReceiptMean
Empirical transported receipt score.
Definition (Lean source)
noncomputable def scoreReceiptMean (weight e : 𝒳 → ℝ) (n : ℕ)
(sample : SourceSample 𝒳 n) : ℝ :=
(n : ℝ)⁻¹ * ∑ i, weight (sample i).1 *
oracleInstrumentScore e (sample i) * boolReal (sample i).2.2.1
def empiricalKish
Empirical second moment of the oracle transport weight.
Definition (Lean source)
noncomputable abbrev empiricalKish (weight : 𝒳 → ℝ) (n : ℕ)
(sample : SourceSample 𝒳 n) : ℝ :=
empiricalKishDispersion
(fun o : SourceObs 𝒳 => weight o.1) n sample
Inversion of the affine transported score over Theta.
Definition (Lean source)
noncomputable def inversionHandle (weight e : 𝒳 → ℝ) (n : ℕ) (L : ℝ)
(sample : SourceSample 𝒳 n) : Set ℝ :=
{theta | theta ∈ parameterSpace ∧
|scoreOutcomeMean weight e n sample -
theta * scoreReceiptMean weight e n sample| ≤
L * sqrt (empiricalKish weight n sample / n)}
theorem inversionHandle_congr_of_sample
The inversion handle only inspects the weight at realized source-sample covariates.
Formal statement
hypotheses
weight weight' e :
𝒳 → ℝ
n :
ℕ
L :
ℝ
sample :
SourceSample 𝒳 n
h :
∀ i, weight (sample i).1 = weight' (sample i).1
conclusion
inversionHandle weight e n L sample = inversionHandle weight' e n L sample
Proof (Lean source)
lemma inversionHandle_congr_of_sample
{weight weight' e : 𝒳 → ℝ} {n : ℕ} {L : ℝ}
{sample : SourceSample 𝒳 n}
(h : ∀ i, weight (sample i).1 = weight' (sample i).1) :
inversionHandle weight e n L sample =
inversionHandle weight' e n L sample := by
unfold inversionHandle scoreOutcomeMean scoreReceiptMean empiricalKish
empiricalKishDispersion
simp_rw [h]
theorem sourceSample_weight_version_ae_eq
Every density-ratio version agrees with the canonical version simultaneously at the finitely many realized source covariates.
Formal statement
hypotheses
conclusion
(fun s : SourceSample 𝒳 n => fun i => w.1.1 (s i).1)
=ᵐ[ Measure.pi (fun _ : Fin n => sourceObsLaw P n)] (fun s : SourceSample 𝒳 n => fun i => transportWeight P n (s i).1)
Proof (Lean source)
lemma sourceSample_weight_version_ae_eq
(P : TransportedArray 𝒳) (n : ℕ) (w : TransportWeightVersion P n)
[SigmaFinite (sourceObsLaw P n)] :
(fun s : SourceSample 𝒳 n => fun i => w.1.1 (s i).1) =ᵐ[
Measure.pi (fun _ : Fin n => sourceObsLaw P n)]
(fun s : SourceSample 𝒳 n =>
fun i => transportWeight P n (s i).1) := by
have hwObs :
(fun o : SourceObs 𝒳 => w.1.1 o.1) =ᵐ[sourceObsLaw P n]
(fun o : SourceObs 𝒳 => transportWeight P n o.1) := by
simpa only [sourceXLaw, Function.comp_def] using
(MeasureTheory.ae_eq_comp measurable_fst.aemeasurable w.property.2)
exact Measure.ae_eq_pi (fun _ => hwObs)
theorem inversionHandle_weight_version_ae_eq
Score inversion is unchanged almost surely throughout a density-ratio version fiber.
Formal statement
hypotheses
conclusion
(fun s : SourceSample 𝒳 n => inversionHandle w.1.1 (P.propensity n) n L s)
=ᵐ[ Measure.pi (fun _ : Fin n => sourceObsLaw P n)] (fun s : SourceSample 𝒳 n => inversionHandle (transportWeight P n) (P.propensity n) n L s)
Proof (Lean source)
lemma inversionHandle_weight_version_ae_eq
(P : TransportedArray 𝒳) (n : ℕ) (w : TransportWeightVersion P n)
(L : ℝ) [SigmaFinite (sourceObsLaw P n)] :
(fun s : SourceSample 𝒳 n =>
inversionHandle w.1.1 (P.propensity n) n L s) =ᵐ[
Measure.pi (fun _ : Fin n => sourceObsLaw P n)]
(fun s : SourceSample 𝒳 n =>
inversionHandle (transportWeight P n) (P.propensity n) n L s) := by
filter_upwards [sourceSample_weight_version_ae_eq P n w] with s hs
exact inversionHandle_congr_of_sample (fun i => congrFun hs i)
theorem scoreProcedure_set_weight_version_ae_eq
A score-inversion procedure gives the same random set almost surely throughout a model's density-ratio version fiber.
Formal statement
hypotheses
N k :
ℕ → ℕ
c epsilon L :
ℝ
C :
OracleProcedure 𝒳 N k c epsilon
hC :
∀ n x, C.set n x = inversionHandle x.2.1.1 x.2.2 n L x.1.1
P :
n :
ℕ
hP :
TransportedIVClass P N k c epsilon n
w :
conclusion
Proof (Lean source)
lemma scoreProcedure_set_weight_version_ae_eq
(N k : ℕ → ℕ) (c epsilon L : ℝ)
(C : OracleProcedure 𝒳 N k c epsilon)
(hC : ∀ n x, C.set n x =
inversionHandle x.2.1.1 x.2.2 n L x.1.1)
(P : TransportedArray 𝒳) (n : ℕ)
(hP : TransportedIVClass P N k c epsilon n)
(w : TransportWeightVersion P n) :
(oracleSetAtWeight C P n w) =ᵐ[twoSampleLaw P N n]
(oracleSet C P n) := by
letI : IsProbabilityMeasure (sourceObsLaw P n) :=
hP.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (targetXLaw P n) :=
hP.twoSampleArray.2.2.1 n
letI : IsProbabilityMeasure
(Measure.pi (fun _ : Fin n => sourceObsLaw P n)) := by
infer_instance
letI : IsProbabilityMeasure
(Measure.pi (fun _ : Fin (N n) => targetXLaw P n)) := by
infer_instance
have hs := inversionHandle_weight_version_ae_eq P n w L
have hs' :
(fun z : TwoSample 𝒳 n (N n) =>
inversionHandle w.1.1 (P.propensity n) n L z.1) =ᵐ[
twoSampleLaw P N n]
(fun z : TwoSample 𝒳 n (N n) =>
inversionHandle (transportWeight P n) (P.propensity n) n L z.1) := by
let μS := Measure.pi (fun _ : Fin n => sourceObsLaw P n)
let μT := Measure.pi (fun _ : Fin (N n) => targetXLaw P n)
have hmap : Measure.map fst (μS.prod μT) = μS := by
simp [μS, μT]
have hsmap :
(fun s : SourceSample 𝒳 n =>
inversionHandle w.1.1 (P.propensity n) n L s) =ᵐ[
Measure.map fst (μS.prod μT)]
(fun s : SourceSample 𝒳 n =>
inversionHandle (transportWeight P n) (P.propensity n) n L s) := by
rw [hmap]
exact hs
simpa only [twoSampleLaw, μS, μT, Function.comp_def] using
(MeasureTheory.ae_eq_comp measurable_fst.aemeasurable hsmap)
filter_upwards [hs'] with z hz
simpa only [oracleSetAtWeight, oracleSet, hC, transportWeightInput] using hz
theorem scoreProcedure_coverage_weight_version_eq
For a score-inversion procedure, coverage is unchanged when the canonical transport weight is replaced by any admissible version.
Formal statement
hypotheses
N k :
ℕ → ℕ
c epsilon L :
ℝ
C :
OracleProcedure 𝒳 N k c epsilon
hC :
∀ n x, C.set n x = inversionHandle x.2.1.1 x.2.2 n L x.1.1
P :
n :
ℕ
hP :
TransportedIVClass P N k c epsilon n
w :
conclusion
oracleCoverageAtWeight C P n w = oracleCoverage C P n
Proof (Lean source)
lemma scoreProcedure_coverage_weight_version_eq
(N k : ℕ → ℕ) (c epsilon L : ℝ)
(C : OracleProcedure 𝒳 N k c epsilon)
(hC : ∀ n x, C.set n x =
inversionHandle x.2.1.1 x.2.2 n L x.1.1)
(P : TransportedArray 𝒳) (n : ℕ)
(hP : TransportedIVClass P N k c epsilon n)
(w : TransportWeightVersion P n) :
oracleCoverageAtWeight C P n w = oracleCoverage C P n := by
unfold oracleCoverageAtWeight oracleCoverage
congr 1
apply measure_congr
filter_upwards [
scoreProcedure_set_weight_version_ae_eq N k c epsilon L C hC P n hP w
] with s hs
exact congrArg (fun A : Set ℝ => targetCACE P n ∈ A) hs
theorem scoreProcedure_expectedLength_weight_version_eq
For a score-inversion procedure, expected confidence-set length is unchanged when the canonical transport weight is replaced by any admissible version.
Formal statement
hypotheses
N k :
ℕ → ℕ
c epsilon L :
ℝ
C :
OracleProcedure 𝒳 N k c epsilon
hC :
∀ n x, C.set n x = inversionHandle x.2.1.1 x.2.2 n L x.1.1
P :
n :
ℕ
hP :
TransportedIVClass P N k c epsilon n
w :
conclusion
oracleExpectedLengthAtWeight C P n w = oracleExpectedLength C P n
Proof (Lean source)
lemma scoreProcedure_expectedLength_weight_version_eq
(N k : ℕ → ℕ) (c epsilon L : ℝ)
(C : OracleProcedure 𝒳 N k c epsilon)
(hC : ∀ n x, C.set n x =
inversionHandle x.2.1.1 x.2.2 n L x.1.1)
(P : TransportedArray 𝒳) (n : ℕ)
(hP : TransportedIVClass P N k c epsilon n)
(w : TransportWeightVersion P n) :
oracleExpectedLengthAtWeight C P n w =
oracleExpectedLength C P n := by
unfold oracleExpectedLengthAtWeight oracleExpectedLength
apply integral_congr_ae
filter_upwards [
scoreProcedure_set_weight_version_ae_eq N k c epsilon L C hC P n hP w
] with s hs
rw [hs]
theorem scoreProcedure_oracleCoverageClassInf_eq
For score inversion, taking the infimum over the full version fiber is exactly the canonical coverage infimum.
Formal statement
hypotheses
N k :
ℕ → ℕ
c epsilon L :
ℝ
C :
OracleProcedure 𝒳 N k c epsilon
hC :
∀ n x, C.set n x = inversionHandle x.2.1.1 x.2.2 n L x.1.1
n :
ℕ
hrow :
Nonempty {P : TransportedArray 𝒳 // TransportedIVClass P N k c epsilon n}
conclusion
oracleCoverageClassInf N k c epsilon C n
Proof (Lean source)
lemma scoreProcedure_oracleCoverageClassInf_eq
(N k : ℕ → ℕ) (c epsilon L : ℝ)
(C : OracleProcedure 𝒳 N k c epsilon)
(hC : ∀ n x, C.set n x =
inversionHandle x.2.1.1 x.2.2 n L x.1.1)
(n : ℕ)
(hrow : Nonempty {P : TransportedArray 𝒳 //
TransportedIVClass P N k c epsilon n}) :
oracleCoverageClassInf N k c epsilon C n =
(⨅ P : {P : TransportedArray 𝒳 //
TransportedIVClass P N k c epsilon n},
oracleCoverage C P n) := by
letI := hrow
simp only [oracleCoverageClassInf, coverageInfOrOne]
exact coverageInfOrOne_of_nonempty _
theorem scoreProcedure_frontierRiskRow_eq
For score inversion, the full representative-fiber risk row equals the canonical row used by the existing score calculation.
Formal statement
hypotheses
N k :
ℕ → ℕ
c epsilon L t0 :
ℝ
C :
OracleProcedure 𝒳 N k c epsilon
hC :
∀ n x, C.set n x = inversionHandle x.2.1.1 x.2.2 n L x.1.1
n :
ℕ
conclusion
frontierRiskRow N k c epsilon C t0 n
= (⨆ P : {P : TransportedArray 𝒳 // TransportedIVClass P N k c epsilon n ∧ t0 ≤ effectiveStrength P n}, oracleExpectedLength C P n)
Proof (Lean source)
lemma scoreProcedure_frontierRiskRow_eq
(N k : ℕ → ℕ) (c epsilon L t0 : ℝ)
(C : OracleProcedure 𝒳 N k c epsilon)
(hC : ∀ n x, C.set n x =
inversionHandle x.2.1.1 x.2.2 n L x.1.1)
(n : ℕ) :
frontierRiskRow N k c epsilon C t0 n =
(⨆ P : {P : TransportedArray 𝒳 //
TransportedIVClass P N k c epsilon n ∧
t0 ≤ effectiveStrength P n},
oracleExpectedLength C P n) := by
rfl
theorem inversionHandle_length_le
Affine inversion has length at most the interval diameter and, away from a zero empirical first stage, at most twice radius over slope.
Formal statement
hypotheses
weight e :
𝒳 → ℝ
n :
ℕ
L :
ℝ
sample :
SourceSample 𝒳 n
hL :
0 ≤ L
hB :
scoreReceiptMean weight e n sample ≠ 0
conclusion
setLength (inversionHandle weight e n L sample)
Proof (Lean source)
lemma inversionHandle_length_le (weight e : 𝒳 → ℝ) (n : ℕ) (L : ℝ)
(sample : SourceSample 𝒳 n) (hL : 0 ≤ L)
(hB : scoreReceiptMean weight e n sample ≠ 0) :
setLength (inversionHandle weight e n L sample) ≤
min 2 (2 * (L * sqrt (empiricalKish weight n sample / n)) /
|scoreReceiptMean weight e n sample|) := by
let A := scoreOutcomeMean weight e n sample
let B := scoreReceiptMean weight e n sample
let r := L * sqrt (empiricalKish weight n sample / n)
have hr : 0 ≤ r := mul_nonneg hL (Real.sqrt_nonneg _)
have hAbsB : 0 < |B| := abs_pos.mpr hB
apply le_min
· unfold setLength parameterSpace
calc
(volume (inversionHandle weight e n L sample ∩ Icc (-1) 1)).toReal
≤ (volume (Icc (-1 : ℝ) 1)).toReal := ENNReal.toReal_mono
(by simp [Real.volume_Icc]) (measure_mono (Set.inter_subset_right))
_ = 2 := by simp [Real.volume_Icc] <;> norm_num
· have hsub :
inversionHandle weight e n L sample ∩ parameterSpace ⊆
Icc (A / B - r / |B|) (A / B + r / |B|) := by
intro x hx
have hscore := hx.1.2
change |A - x * B| ≤ r at hscore
change A / B - r / |B| ≤ x ∧ x ≤ A / B + r / |B|
have hid : A - x * B = B * (A / B - x) := by
field_simp [B, hB]
rw [hid, abs_mul] at hscore
have hx' : |A / B - x| ≤ r / |B| := by
rw [le_div_iff₀ hAbsB]
simpa [mul_comm] using hscore
rw [abs_le] at hx'
constructor <;> linarith
unfold setLength
calc
(volume (inversionHandle weight e n L sample ∩ parameterSpace)).toReal
≤ (volume (Icc (A / B - r / |B|)
(A / B + r / |B|))).toReal := ENNReal.toReal_mono
(by simp [Real.volume_Icc]) (measure_mono hsub)
_ = 2 * r / |B| := by
simp [Real.volume_Icc, ENNReal.toReal_ofReal,
div_nonneg hr hAbsB.le]
field_simp
ring
Helpers.ScoreRisk 25 declarations prop:compact-causal-range is stated in the source for every P in the class, with no integrability hypothesis, and its own proof (writeup.tex:382) applies the change-of-measure identity to Delta_Y as a BOUNDED measurable
S1.0: the contrast functions are bounded, hence integrable
prop:compact-causal-range is stated in the source for every P in the class,
with no integrability hypothesis, and its own proof (writeup.tex:382) applies
the change-of-measure identity to Delta_Y as a BOUNDED measurable function.
In the Lean encoding that boundedness is not a structure field: the contrast
functions are pinned only through the set-integral identities of
OutcomeTransport and ReceiptTransport. It is nevertheless forced by them,
because those identities equate a set integral of the contrast with a set
integral of an integrand bounded by one.
structure ScoreRiskClassAtoms
Atom-by-atom interface required of an abstract model predicate by the score-risk calculation. Each field is an implication schema, so a fixed geometry slice supplies the same atoms through its first component.
Definition (Lean source)
parameters
fields
fullDataSupport :
∀ n P
if
cls n P
then
FullDataSupport P n
populationPresence :
∀ n P
if
cls n P
then
twoSampleArray :
∀ n P
if
cls n P
then
TwoSampleArray P N c
instrumentOverlap :
∀ n P
if
cls n P
then
InstrumentOverlap P n epsilon
sourceObservation :
∀ n P
if
cls n P
then
ivRandomization :
∀ n P
if
cls n P
then
IVRandomization P n
ivExclusion :
∀ n P
if
cls n P
then
IVExclusion P n
ivMonotonicity :
∀ n P
if
cls n P
then
IVMonotonicity P n
outcomeTransport :
∀ n P
if
cls n P
then
OutcomeTransport P n
receiptTransport :
∀ n P
if
cls n P
then
ReceiptTransport P n
targetComplierPositivity :
∀ n P
if
cls n P
then
transportDomination :
∀ n P
if
cls n P
then
weightEnvelope :
∀ n P
if
cls n P
then
WeightEnvelope P k n
weightSecondMoment :
∀ n P
if
cls n P
then
WeightSecondMoment P k n
degradingArray :
∀ n P
if
cls n P
then
DegradingArray P k
def transportedIVScoreRiskAtoms
The global transported-IV class instantiates the abstract atom interface.
Definition (Lean source)
def transportedIVScoreRiskAtoms (N k : ℕ → ℕ) (c epsilon : ℝ) :
ScoreRiskClassAtoms
(fun n (P : TransportedArray 𝒳) =>
TransportedIVClass P N k c epsilon n) N k c epsilon where
fullDataSupport := fun _ _ h => h.fullDataSupport
populationPresence := fun _ _ h => h.populationPresence
twoSampleArray := fun _ _ h => h.twoSampleArray
instrumentOverlap := fun _ _ h => h.instrumentOverlap
sourceObservation := fun _ _ h => h.sourceObservation
ivRandomization := fun _ _ h => h.ivRandomization
ivExclusion := fun _ _ h => h.ivExclusion
ivMonotonicity := fun _ _ h => h.ivMonotonicity
outcomeTransport := fun _ _ h => h.outcomeTransport
receiptTransport := fun _ _ h => h.receiptTransport
targetComplierPositivity := fun _ _ h => h.targetComplierPositivity
transportDomination := fun _ _ h => h.transportDomination
weightEnvelope := fun _ _ h => h.weightEnvelope
weightSecondMoment := fun _ _ h => h.weightSecondMoment
degradingArray := fun _ _ h => h.degradingArray
def fixedGeometryScoreRiskAtoms
Every fixed-geometry slice instantiates the same abstract atom interface.
Definition (Lean source)
def fixedGeometryScoreRiskAtoms (g : Geometry 𝒳)
(N k : ℕ → ℕ) (c epsilon : ℝ) :
ScoreRiskClassAtoms
(fun n (P : TransportedArray 𝒳) =>
fixedGeometrySlice P g N k c epsilon n) N k c epsilon where
fullDataSupport := fun _ _ h => h.1.fullDataSupport
populationPresence := fun _ _ h => h.1.populationPresence
twoSampleArray := fun _ _ h => h.1.twoSampleArray
instrumentOverlap := fun _ _ h => h.1.instrumentOverlap
sourceObservation := fun _ _ h => h.1.sourceObservation
ivRandomization := fun _ _ h => h.1.ivRandomization
ivExclusion := fun _ _ h => h.1.ivExclusion
ivMonotonicity := fun _ _ h => h.1.ivMonotonicity
outcomeTransport := fun _ _ h => h.1.outcomeTransport
receiptTransport := fun _ _ h => h.1.receiptTransport
targetComplierPositivity := fun _ _ h => h.1.targetComplierPositivity
transportDomination := fun _ _ h => h.1.transportDomination
weightEnvelope := fun _ _ h => h.1.weightEnvelope
weightSecondMoment := fun _ _ h => h.1.weightSecondMoment
degradingArray := fun _ _ h => h.1.degradingArray
def oracleAffineScore
The one-observation affine oracle score.
Definition (Lean source)
noncomputable def oracleAffineScore (weight e : 𝒳 → ℝ) (theta : ℝ)
(o : SourceObs 𝒳) : ℝ :=
weight o.1 * oracleInstrumentScore e o *
(o.2.2.2 - theta * boolReal o.2.2.1)
theorem scoreOutcomeMean_sub_receiptMean
The score average is the difference of the outcome and receipt averages.
Formal statement
hypotheses
conclusion
scoreOutcomeMean weight e n sample - theta * scoreReceiptMean weight e n sample
= (n : ℝ)⁻¹ * ∑ i, oracleAffineScore weight e theta (sample i)
Proof (Lean source)
lemma scoreOutcomeMean_sub_receiptMean
(weight e : 𝒳 → ℝ) (theta : ℝ) (n : ℕ)
(sample : SourceSample 𝒳 n) :
scoreOutcomeMean weight e n sample -
theta * scoreReceiptMean weight e n sample =
(n : ℝ)⁻¹ * ∑ i, oracleAffineScore weight e theta (sample i) := by
simp only [scoreOutcomeMean, scoreReceiptMean, oracleAffineScore]
simp_rw [Finset.mul_sum]
rw [← Finset.sum_sub_distrib]
apply Finset.sum_congr rfl
intro i hi
ring
theorem oracleAffineScore_mean_zero
S1.1: component score means imply the affine score identity, and the identifying ratio centers the score at the target effect.
Formal statement
hypotheses
Q :
weight e :
𝒳 → ℝ
theta muY mu :
ℝ
hY :
Integrable (fun o => weight o.1 * oracleInstrumentScore e o * o.2.2.2) Q
hD :
hYmean :
(∫ o, weight o.1 * oracleInstrumentScore e o * o.2.2.2 ∂Q) = muY
hDmean :
(∫ o, weight o.1 * oracleInstrumentScore e o * boolReal o.2.2.1 ∂Q) = mu
hmu :
mu ≠ 0
hratio :
muY / mu = theta
conclusion
(∫ o, oracleAffineScore weight e theta o ∂Q) = 0
Proof (Lean source)
lemma oracleAffineScore_mean_zero
(Q : Measure (SourceObs 𝒳)) [IsProbabilityMeasure Q]
(weight e : 𝒳 → ℝ) (theta muY mu : ℝ)
(hY : Integrable
(fun o => weight o.1 * oracleInstrumentScore e o * o.2.2.2) Q)
(hD : Integrable
(fun o => weight o.1 * oracleInstrumentScore e o *
boolReal o.2.2.1) Q)
(hYmean : (∫ o, weight o.1 * oracleInstrumentScore e o * o.2.2.2 ∂Q) =
muY)
(hDmean : (∫ o, weight o.1 * oracleInstrumentScore e o *
boolReal o.2.2.1 ∂Q) = mu)
(hmu : mu ≠ 0) (hratio : muY / mu = theta) :
(∫ o, oracleAffineScore weight e theta o ∂Q) = 0 := by
have hpoint :
oracleAffineScore weight e theta =
fun o => weight o.1 * oracleInstrumentScore e o * o.2.2.2 -
theta * (weight o.1 * oracleInstrumentScore e o *
boolReal o.2.2.1) := by
funext o
simp [oracleAffineScore]
ring
rw [hpoint, integral_sub hY (hD.const_mul theta), integral_const_mul,
hYmean, hDmean]
apply (sub_eq_zero.mpr ?_)
rw [← hratio]
field_simp [hmu]
theorem abs_oracleInstrumentScore_residual_le
S1.2: overlap and the outcome/parameter ranges give the exact 2 / epsilon envelope for the oracle affine score without the weight.
Formal statement
hypotheses
e :
𝒳 → ℝ
epsilon theta :
ℝ
o :
hepsilon :
0 < epsilon
hoverlap :
epsilon ≤ e o.1 ∧ e o.1 ≤ 1 - epsilon
htheta :
theta ∈ parameterSpace
hy :
o.2.2.2 ∈ Icc (0 : ℝ) 1
conclusion
|oracleInstrumentScore e o * (o.2.2.2 - theta * boolReal o.2.2.1)| ≤ 2 / epsilon
Proof (Lean source)
lemma abs_oracleInstrumentScore_residual_le
(e : 𝒳 → ℝ) (epsilon theta : ℝ) (o : SourceObs 𝒳)
(hepsilon : 0 < epsilon)
(hoverlap : epsilon ≤ e o.1 ∧ e o.1 ≤ 1 - epsilon)
(htheta : theta ∈ parameterSpace)
(hy : o.2.2.2 ∈ Icc (0 : ℝ) 1) :
|oracleInstrumentScore e o *
(o.2.2.2 - theta * boolReal o.2.2.1)| ≤ 2 / epsilon := by
simpa [regularCellScore] using
abs_regularCellScore_le e epsilon theta o hepsilon hoverlap htheta hy
theorem oracleAffineScore_sq_le
S1.2, squared form: after multiplying by a nonnegative weight, the score square is bounded by 4 w² / epsilon².
Formal statement
hypotheses
weight e :
𝒳 → ℝ
epsilon theta :
ℝ
o :
hepsilon :
0 < epsilon
hoverlap :
epsilon ≤ e o.1 ∧ e o.1 ≤ 1 - epsilon
htheta :
theta ∈ parameterSpace
hy :
o.2.2.2 ∈ Icc (0 : ℝ) 1
conclusion
oracleAffineScore weight e theta o ^ 2 ≤ 4 * weight o.1 ^ 2 / epsilon ^ 2
Proof (Lean source)
lemma oracleAffineScore_sq_le
(weight e : 𝒳 → ℝ) (epsilon theta : ℝ) (o : SourceObs 𝒳)
(hepsilon : 0 < epsilon)
(hoverlap : epsilon ≤ e o.1 ∧ e o.1 ≤ 1 - epsilon)
(htheta : theta ∈ parameterSpace)
(hy : o.2.2.2 ∈ Icc (0 : ℝ) 1) :
oracleAffineScore weight e theta o ^ 2 ≤
4 * weight o.1 ^ 2 / epsilon ^ 2 := by
have hscore := abs_oracleInstrumentScore_residual_le
e epsilon theta o hepsilon hoverlap htheta hy
have hbound : 0 ≤ 2 / epsilon := by positivity
have hsq :
|oracleInstrumentScore e o *
(o.2.2.2 - theta * boolReal o.2.2.1)| ^ 2 ≤
(2 / epsilon) ^ 2 := by
have hfactor :
0 ≤ (2 / epsilon -
|oracleInstrumentScore e o *
(o.2.2.2 - theta * boolReal o.2.2.1)|) *
(2 / epsilon +
|oracleInstrumentScore e o *
(o.2.2.2 - theta * boolReal o.2.2.1)|) :=
mul_nonneg (sub_nonneg.mpr hscore)
(add_nonneg hbound (abs_nonneg _))
nlinarith
rw [sq_abs] at hsq
rw [oracleAffineScore, mul_assoc, mul_pow]
calc
weight o.1 ^ 2 *
(oracleInstrumentScore e o *
(o.2.2.2 - theta * boolReal o.2.2.1)) ^ 2 ≤
weight o.1 ^ 2 * (2 / epsilon) ^ 2 := by
gcongr
_ = 4 * weight o.1 ^ 2 / epsilon ^ 2 := by ring
theorem weight_fourth_le_envelope
S1.3: the paper's weight envelope implies w⁴ ≤ 4 k_n² w² pointwise.
Formal statement
hypotheses
w k :
ℝ
hw :
0 ≤ w
hwk :
w ≤ 2 * k
conclusion
w ^ 4 ≤ 4 * k ^ 2 * w ^ 2
Proof (Lean source)
lemma weight_fourth_le_envelope
(w k : ℝ) (hw : 0 ≤ w) (hwk : w ≤ 2 * k) :
w ^ 4 ≤ 4 * k ^ 2 * w ^ 2 := by
have hk : 0 ≤ k := by linarith
have hfactor :
0 ≤ (2 * k - w) * (2 * k + w) :=
mul_nonneg (sub_nonneg.mpr hwk) (add_nonneg (by positivity) hw)
nlinarith [sq_nonneg w]
theorem iid_empiricalAverage_mean_variance
Exact mean and variance of an i.i.d. empirical average. This is the model-free sampling leaf used for both the score and empirical Kish moments.
Formal statement
hypotheses
Proof (Lean source)
lemma iid_empiricalAverage_mean_variance
{Ω : Type*} [MeasurableSpace Ω] (μ : Measure Ω)
[IsProbabilityMeasure μ] (n : ℕ) (hn : 0 < n)
(F : Ω → ℝ) (hF : MemLp F 2 μ) :
(∫ sample : Fin n → Ω,
(n : ℝ)⁻¹ * ∑ i, F (sample i)
∂Measure.pi (fun _ : Fin n => μ)) = ∫ o, F o ∂μ ∧
variance
(fun sample : Fin n → Ω => (n : ℝ)⁻¹ * ∑ i, F (sample i))
(Measure.pi (fun _ : Fin n => μ)) =
(n : ℝ)⁻¹ * variance F μ := by
exact ⟨iid_average_integral μ n hn F hF,
iid_average_variance μ n hn F hF⟩
theorem empiricalKish_variance_le
S1.3: an i.i.d. empirical Kish average has variance at most 4 k² kappa / n under the paper's fourth-moment envelope.
Formal statement
hypotheses
μ :
w :
𝒳 → ℝ
n :
ℕ
k kappa :
ℝ
hn :
0 < n
hF :
MemLp (fun o => w o.1 ^ 2) 2 μ
hkappa :
(∫ o, w o.1 ^ 2 ∂μ) = kappa
hfourth :
∀ᵐ o ∂μ, w o.1 ^ 4 ≤ 4 * k ^ 2 * w o.1 ^ 2
conclusion
variance (fun sample : SourceSample 𝒳 n => empiricalKish w n sample) (Measure.pi (fun _ : Fin n => μ))
≤ 4 * k ^ 2 * kappa / n
Proof (Lean source)
lemma empiricalKish_variance_le
(μ : Measure (SourceObs 𝒳)) [IsProbabilityMeasure μ]
(w : 𝒳 → ℝ) (n : ℕ) (k kappa : ℝ) (hn : 0 < n)
(hF : MemLp (fun o => w o.1 ^ 2) 2 μ)
(hkappa : (∫ o, w o.1 ^ 2 ∂μ) = kappa)
(hfourth : ∀ᵐ o ∂μ, w o.1 ^ 4 ≤ 4 * k ^ 2 * w o.1 ^ 2) :
variance
(fun sample : SourceSample 𝒳 n => empiricalKish w n sample)
(Measure.pi (fun _ : Fin n => μ)) ≤
4 * k ^ 2 * kappa / n := by
exact empiricalKishDispersion_variance_le μ
(fun o : SourceObs 𝒳 => w o.1) n k kappa hn hF hkappa hfourth
theorem one_le_secondMoment_of_mean_one
S1.4: a probability density of mean one has second moment at least one.
Formal statement
hypotheses
conclusion
1 ≤ ∫ x, w x ^ 2 ∂μ
Proof (Lean source)
lemma one_le_secondMoment_of_mean_one
(μ : Measure 𝒳) [IsProbabilityMeasure μ] (w : 𝒳 → ℝ)
(hw : MemLp w 2 μ) (hmean : (∫ x, w x ∂μ) = 1) :
1 ≤ ∫ x, w x ^ 2 ∂μ := by
exact one_le_secondMoment_of_mean_one μ w hw hmean
theorem probability_abs_sub_mean_gt_le
A reusable Chebyshev leaf with a supplied exact mean and variance bound.
Formal statement
Proof (Lean source)
lemma probability_abs_sub_mean_gt_le
{Ω : Type*} [MeasurableSpace Ω] (Q : Measure Ω)
[IsProbabilityMeasure Q] (F : Ω → ℝ) (mean v a : ℝ)
(hF : MemLp F 2 Q) (ha : 0 < a)
(hmean : (∫ x, F x ∂Q) = mean)
(hvar : variance F Q ≤ v) :
(Q {x | a < |F x - mean|}).toReal ≤ v / a ^ 2 := by
exact probability_abs_sub_mean_gt_le
Q F mean v a hF ha hmean hvar
theorem empiricalKish_lower_tail_le
S1.4: the lower tail of empirical Kish is bounded by the paper's exact 16 k² / (n kappa) expression.
Formal statement
hypotheses
n :
ℕ
Q :
w :
𝒳 → ℝ
k kappa :
ℝ
hn :
0 < n
hkappa :
0 < kappa
hF :
MemLp (empiricalKish w n) 2 Q
hmean :
(∫ sample, empiricalKish w n sample ∂Q) = kappa
hvar :
variance (empiricalKish w n) Q ≤ 4 * k ^ 2 * kappa / n
conclusion
(Q {sample | empiricalKish w n sample < kappa / 2}).toReal
≤ 16 * k ^ 2 / ((n : ℝ) * kappa)
Proof (Lean source)
lemma empiricalKish_lower_tail_le
(n : ℕ) (Q : Measure (SourceSample 𝒳 n)) [IsProbabilityMeasure Q]
(w : 𝒳 → ℝ) (k kappa : ℝ)
(hn : 0 < n) (hkappa : 0 < kappa)
(hF : MemLp (empiricalKish w n) 2 Q)
(hmean : (∫ sample, empiricalKish w n sample ∂Q) = kappa)
(hvar : variance (empiricalKish w n) Q ≤
4 * k ^ 2 * kappa / n) :
(Q {sample | empiricalKish w n sample < kappa / 2}).toReal ≤
16 * k ^ 2 / ((n : ℝ) * kappa) := by
exact empiricalKishDispersion_lower_tail_le n Q
(fun o : SourceObs 𝒳 => w o.1) k kappa hn hkappa hF hmean hvar
theorem abstractClass_coverage_liminf
S1.5 order leaf: pointwise rows with a vanishing uniform error imply the paper's exact liminf honesty statement. Eventual inhabitation is essential only to bound the infimum rows from above; without it an empty real-valued indexed infimum is zero.
Formal statement
hypotheses
cls :
ℕ → TransportedArray 𝒳 → Prop
coverage :
ℕ → TransportedArray 𝒳 → ℝ
alpha :
ℝ
delta :
ℕ → ℝ
hInhab :
∀ᶠ n in atTop, ∃ P, cls n P
hcoverage :
∀ n P
if
cls n P
then
0 ≤ coverage n P ∧ coverage n P ≤ 1
hrow :
∀ n P
if
cls n P
then
1 - alpha - delta n ≤ coverage n P
conclusion
1 - alpha
Proof (Lean source)
lemma abstractClass_coverage_liminf
(cls : ℕ → TransportedArray 𝒳 → Prop)
(coverage : ℕ → TransportedArray 𝒳 → ℝ)
(alpha : ℝ) (delta : ℕ → ℝ)
(hdelta : Tendsto delta atTop (𝓝 0))
(hInhab : ∀ᶠ n in atTop, ∃ P, cls n P)
(hcoverage : ∀ n P, cls n P →
0 ≤ coverage n P ∧ coverage n P ≤ 1)
(hrow : ∀ n P, cls n P →
1 - alpha - delta n ≤ coverage n P) :
1 - alpha ≤ liminf
(fun n => ⨅ P : {P : TransportedArray 𝒳 // cls n P},
coverage n P) atTop := by
exact classCoverage_liminf cls coverage alpha delta hdelta
hInhab hcoverage hrow
S1.6: the receipt mean/variance calculation gives exactly the paper's 4 / (epsilon² t) bad-slope bound.
Formal statement
hypotheses
Ω :
Type*
Q :
B :
Ω → ℝ
n :
ℕ
epsilon mu kappa t :
ℝ
hn :
0 < n
hepsilon :
0 < epsilon
hmu :
0 < mu
hkappa :
0 < kappa
hB :
MemLp B 2 Q
hmean :
(∫ x, B x ∂Q) = mu
hvar :
variance B Q ≤ kappa / (epsilon ^ 2 * n)
ht :
t = (n : ℝ) * mu ^ 2 / kappa
conclusion
(Q {x | mu / 2 < |B x - mu|}).toReal ≤ 4 / (epsilon ^ 2 * t)
Proof (Lean source)
lemma scoreReceiptMean_bad_probability
{Ω : Type*} [MeasurableSpace Ω] (Q : Measure Ω)
[IsProbabilityMeasure Q] (B : Ω → ℝ)
(n : ℕ) (epsilon mu kappa t : ℝ)
(hn : 0 < n) (hepsilon : 0 < epsilon) (hmu : 0 < mu)
(hkappa : 0 < kappa) (hB : MemLp B 2 Q)
(hmean : (∫ x, B x ∂Q) = mu)
(hvar : variance B Q ≤ kappa / (epsilon ^ 2 * n))
(ht : t = (n : ℝ) * mu ^ 2 / kappa) :
(Q {x | mu / 2 < |B x - mu|}).toReal ≤
4 / (epsilon ^ 2 * t) := by
have hcheb := probability_abs_sub_mean_gt_le Q B mu
(kappa / (epsilon ^ 2 * n)) (mu / 2)
hB (half_pos hmu) hmean hvar
calc
_ ≤ (kappa / (epsilon ^ 2 * n)) / (mu / 2) ^ 2 := hcheb
_ = 4 / (epsilon ^ 2 * t) := by
rw [ht]
have hnR : (0 : ℝ) < n := by exact_mod_cast hn
field_simp [hnR.ne', hepsilon.ne', hmu.ne', hkappa.ne']
ring
theorem empiricalKish_mean
S1.7: the empirical Kish statistic has exactly the population second moment as its expectation.
Formal statement
hypotheses
conclusion
= ∫ o, w o.1 ^ 2 ∂μ
Proof (Lean source)
lemma empiricalKish_mean
(μ : Measure (SourceObs 𝒳)) [IsProbabilityMeasure μ]
(w : 𝒳 → ℝ) (n : ℕ) (hn : 0 < n)
(hF : MemLp (fun o => w o.1 ^ 2) 2 μ) :
(∫ sample : SourceSample 𝒳 n, empiricalKish w n sample
∂Measure.pi (fun _ : Fin n => μ)) =
∫ o, w o.1 ^ 2 ∂μ := by
exact empiricalKishDispersion_mean μ
(fun o : SourceObs 𝒳 => w o.1) n hn (hF.integrable (by norm_num))
theorem scoreInversion_expectedLength_frontier_le
S1.8: direct uninflated affine-inversion frontier bound. The statement is law- and class-agnostic, so both oracle model classes feed it the same moment facts.
Formal statement
hypotheses
Ω :
Type*
Q :
A B K :
Ω → ℝ
n :
ℕ
L mu kappa q epsilon t :
ℝ
hL :
0 ≤ L
hmu :
0 < mu
hn :
0 < n
hK :
∀ w, 0 ≤ K w
hKint :
Integrable K Q
hKmean :
(∫ w, K w ∂Q) = kappa
hbad :
(Q {w | mu / 2 < |B w - mu|}).toReal ≤ q
hkappa :
0 < kappa
hepsilon :
0 < epsilon
hq :
q ≤ 4 / (epsilon ^ 2 * t)
ht :
t = (n : ℝ) * mu ^ 2 / kappa
Proof (Lean source)
lemma scoreInversion_expectedLength_frontier_le
{Ω : Type*} [MeasurableSpace Ω]
(Q : Measure Ω) [IsProbabilityMeasure Q]
(A B K : Ω → ℝ) (n : ℕ)
(L mu kappa q epsilon t : ℝ)
(hL : 0 ≤ L) (hmu : 0 < mu) (hn : 0 < n)
(hK : ∀ w, 0 ≤ K w)
(hKint : Integrable K Q)
(hKmean : (∫ w, K w ∂Q) = kappa)
(hbad : (Q {w | mu / 2 < |B w - mu|}).toReal ≤ q)
(hkappa : 0 < kappa) (hepsilon : 0 < epsilon)
(hq : q ≤ 4 / (epsilon ^ 2 * t))
(ht : t = (n : ℝ) * mu ^ 2 / kappa) :
(∫ w, setLength (affineInversionSet (A w) (B w)
(L * sqrt (K w / n))) ∂Q) ≤
max 2 (4 * L + 8 / epsilon ^ 2) *
min 1 (t ^ (-1 / 2 : ℝ)) := by
apply expectedLength_affineInversion_frontier_le_uninflated
Q A B K n L mu kappa q kappa (8 / epsilon ^ 2) t
· exact hL
· exact hmu
· exact hn
· exact hK
· exact hKint
· exact hKmean.le
· exact hbad
· exact hkappa
· positivity
· exact le_rfl
· calc
q ≤ 4 / (epsilon ^ 2 * t) := hq
_ = (8 / epsilon ^ 2) / (2 * t) := by
have htpos : 0 < t := by rw [ht]; positivity
field_simp [hepsilon.ne', htpos.ne']
ring
· exact ht
theorem min_one_inverseSqrt_anti
The inverse-root cap is antitone on positive strengths.
Formal statement
Proof (Lean source)
lemma min_one_inverseSqrt_anti
{t0 t : ℝ} (ht0 : 0 < t0) (htt : t0 ≤ t) :
min 1 (t ^ (-1 / 2 : ℝ)) ≤
min 1 (t0 ^ (-1 / 2 : ℝ)) := by
exact inverseSqrtCap_anti ht0 htt
def abstractClassFrontierRisk
Abstract version of the paper's frontier risk, with the model class represented by a per-index predicate.
Definition (Lean source)
noncomputable abbrev abstractClassFrontierRisk
(cls : ℕ → TransportedArray 𝒳 → Prop)
(strength expectedLength : ℕ → TransportedArray 𝒳 → ℝ)
(t0 : ℝ) : ℝ :=
classFrontierRisk cls strength expectedLength t0
theorem abstractClassFrontierRisk_le
S1.8 order leaf: a pointwise uninflated frontier bound collapses both the class supremum and the asymptotic limsup at the threshold value. The capped min is retained, including both of its branches.
Formal statement
hypotheses
cls :
ℕ → TransportedArray 𝒳 → Prop
strength expectedLength :
ℕ → TransportedArray 𝒳 → ℝ
C0 t0 :
ℝ
hC0 :
0 ≤ C0
ht0 :
0 < t0
hLengthNonneg :
∀ n P
if
cls n P
then
0 ≤ expectedLength n P
hpoint :
∀ n P
if
cls n P
then
expectedLength n P ≤ C0 * min 1 (strength n P ^ (-1 / 2 : ℝ))
conclusion
abstractClassFrontierRisk cls strength expectedLength t0 ≤ C0 * min 1 (t0 ^ (-1 / 2 : ℝ))
Proof (Lean source)
lemma abstractClassFrontierRisk_le
(cls : ℕ → TransportedArray 𝒳 → Prop)
(strength expectedLength : ℕ → TransportedArray 𝒳 → ℝ)
(C0 t0 : ℝ) (hC0 : 0 ≤ C0) (ht0 : 0 < t0)
(hLengthNonneg : ∀ n P, cls n P → 0 ≤ expectedLength n P)
(hpoint : ∀ n P, cls n P →
expectedLength n P ≤
C0 * min 1 (strength n P ^ (-1 / 2 : ℝ))) :
abstractClassFrontierRisk cls strength expectedLength t0 ≤
C0 * min 1 (t0 ^ (-1 / 2 : ℝ)) := by
exact classFrontierRisk_le cls strength expectedLength
C0 t0 hC0 ht0 hLengthNonneg hpoint
A measurable function whose set integrals are dominated by the measure of the set is almost everywhere bounded by one. Truncation to the bounded slices {m < f ≤ m + 1} is what makes this hold without assuming f integrable: on each slice f is bounded, so the set integral is genuine rather than Lean's junk value for a non-integrable integrand.
Formal statement
hypotheses
Ω :
Type*
mu :
f :
Ω → ℝ
hf :
hdom :
∀ A
if
then
|∫ x in A, f x ∂mu| ≤ (mu A).toReal
conclusion
∀ᵐ x ∂mu, |f x| ≤ 1
Proof (Lean source)
lemma abs_le_one_ae_of_setIntegral_le
{Ω : Type*} [MeasurableSpace Ω] (mu : Measure Ω)
[IsFiniteMeasure mu] (f : Ω → ℝ) (hf : Measurable f)
(hdom : ∀ A, MeasurableSet A →
|∫ x in A, f x ∂mu| ≤ (mu A).toReal) :
∀ᵐ x ∂mu, |f x| ≤ 1 := by
exact abs_le_one_ae_of_setIntegral_le_measure
mu f hf hdom
On a global class member both population contrast functions are integrable against the source covariate law. This is what lets compact_causal_range be applied on the global class without carrying its two integrability hypotheses as assumptions; every regular-cell consumer already discharges them from the finite cell support, and this is the corresponding global discharge.
Formal statement
hypotheses
conclusion
Proof (Lean source)
lemma transportedIVClass_contrast_integrable
(P : TransportedArray 𝒳) (N k : ℕ → ℕ) (c epsilon : ℝ) (n : ℕ)
(hP : TransportedIVClass P N k c epsilon n) :
Integrable (P.assignmentContrast n true) (sourceXLaw P n) ∧
Integrable (P.receiptContrast n true) (sourceXLaw P n) := by
have hFullX : Measurable (fullX : FullData 𝒳 → 𝒳) := by
unfold fullX
fun_prop
haveI : IsProbabilityMeasure (P.assignedSourceLaw n) :=
(sourceObservationFacts_of_class P N k c epsilon n hP).1
haveI hPopulationProb :
IsProbabilityMeasure (populationLaw P n true) := by
rw [← (sourceObservationFacts_of_class P N k c epsilon n hP).2.2.2.1]
exact Measure.isProbabilityMeasure_map measurable_fst.aemeasurable
haveI hPopulationXProb :
IsProbabilityMeasure (populationXLaw P n true) :=
Measure.isProbabilityMeasure_map hFullX.aemeasurable
have hSupportSource : ∀ᵐ o ∂populationLaw P n true,
fullY0 o ∈ Icc (0 : ℝ) 1 ∧
fullY1 o ∈ Icc (0 : ℝ) 1 :=
ProbabilityTheory.cond_absolutelyContinuous.ae_le
hP.fullDataSupport.2
have hAssignmentBound :
∀ᵐ o ∂populationLaw P n true,
|P.assignmentOutcome n true o -
P.assignmentOutcome n false o| ≤ 1 := by
filter_upwards [hSupportSource,
hP.ivExclusion true true, hP.ivExclusion true false]
with o ho htrue hfalse
rw [htrue, hfalse]
unfold derivedAssignmentOutcome potentialOutcome potentialReceipt
cases fullD0 o <;> cases fullD1 o <;>
simp only [Bool.false_eq_true, ↓reduceIte]
all_goals
rw [abs_le]
constructor <;> linarith [ho.1.1, ho.1.2, ho.2.1, ho.2.2]
have hReceiptBound :
∀ᵐ o ∂populationLaw P n true,
|boolReal (fullD1 o) - boolReal (fullD0 o)| ≤ 1 := by
filter_upwards with o
cases fullD0 o <;> cases fullD1 o <;> norm_num [boolReal]
have hAssignmentDom : ∀ A, MeasurableSet A →
|∫ x in A, P.assignmentContrast n true x
∂populationXLaw P n true| ≤
(populationXLaw P n true A).toReal := by
intro A hA
rw [← hP.outcomeTransport.1 true A hA]
have hbound := norm_setIntegral_le_of_norm_le_const_ae'
(f := fun o : FullData 𝒳 =>
P.assignmentOutcome n true o -
P.assignmentOutcome n false o)
(s := {o | fullX o ∈ A}) (C := 1)
(measure_lt_top (populationLaw P n true) _) (by
filter_upwards [hAssignmentBound] with o ho
intro _
simpa [Real.norm_eq_abs] using ho)
rw [one_mul] at hbound
rw [populationXLaw, Measure.map_apply hFullX hA, ← measureReal_def]
simp only [Real.norm_eq_abs] at hbound
exact hbound
have hReceiptDom : ∀ A, MeasurableSet A →
|∫ x in A, P.receiptContrast n true x
∂populationXLaw P n true| ≤
(populationXLaw P n true A).toReal := by
intro A hA
rw [← hP.receiptTransport.1 true A hA]
have hbound := norm_setIntegral_le_of_norm_le_const_ae'
(f := fun o : FullData 𝒳 =>
boolReal (fullD1 o) - boolReal (fullD0 o))
(s := {o | fullX o ∈ A}) (C := 1)
(measure_lt_top (populationLaw P n true) _) (by
filter_upwards [hReceiptBound] with o ho
intro _
simpa [Real.norm_eq_abs] using ho)
rw [one_mul] at hbound
rw [populationXLaw, Measure.map_apply hFullX hA, ← measureReal_def]
simp only [Real.norm_eq_abs] at hbound
exact hbound
have hAssignmentAE :
∀ᵐ x ∂populationXLaw P n true,
|P.assignmentContrast n true x| ≤ 1 :=
abs_le_one_ae_of_setIntegral_le
(populationXLaw P n true) (P.assignmentContrast n true)
(P.assignmentContrast_measurable n true) hAssignmentDom
have hReceiptAE :
∀ᵐ x ∂populationXLaw P n true,
|P.receiptContrast n true x| ≤ 1 :=
abs_le_one_ae_of_setIntegral_le
(populationXLaw P n true) (P.receiptContrast n true)
(P.receiptContrast_measurable n true) hReceiptDom
rw [sourceXLaw_eq_populationXLaw_source P n
(sourceObservationFacts_of_class P N k c epsilon n hP)]
constructor
· exact Integrable.of_bound
(P.assignmentContrast_measurable n true).aestronglyMeasurable 1
(by simpa [Real.norm_eq_abs] using hAssignmentAE)
· exact Integrable.of_bound
(P.receiptContrast_measurable n true).aestronglyMeasurable 1
(by simpa [Real.norm_eq_abs] using hReceiptAE)
theorem toTransportedIVClass
The abstract atom interface reconstructs the paper's transported-IV class for each of its members.
Formal statement
hypotheses
cls :
ℕ → TransportedArray 𝒳 → Prop
N k :
ℕ → ℕ
c epsilon :
ℝ
atoms :
ScoreRiskClassAtoms cls N k c epsilon
n :
ℕ
P :
hP :
cls n P
conclusion
TransportedIVClass P N k c epsilon n where fullDataSupport
Proof (Lean source)
lemma ScoreRiskClassAtoms.toTransportedIVClass
{cls : ℕ → TransportedArray 𝒳 → Prop}
{N k : ℕ → ℕ} {c epsilon : ℝ}
(atoms : ScoreRiskClassAtoms cls N k c epsilon)
{n : ℕ} {P : TransportedArray 𝒳} (hP : cls n P) :
TransportedIVClass P N k c epsilon n where
fullDataSupport := atoms.fullDataSupport n P hP
populationPresence := atoms.populationPresence n P hP
twoSampleArray := atoms.twoSampleArray n P hP
instrumentOverlap := atoms.instrumentOverlap n P hP
sourceObservation := atoms.sourceObservation n P hP
ivRandomization := atoms.ivRandomization n P hP
ivExclusion := atoms.ivExclusion n P hP
ivMonotonicity := atoms.ivMonotonicity n P hP
outcomeTransport := atoms.outcomeTransport n P hP
receiptTransport := atoms.receiptTransport n P hP
targetComplierPositivity := atoms.targetComplierPositivity n P hP
transportDomination := atoms.transportDomination n P hP
weightEnvelope := atoms.weightEnvelope n P hP
weightSecondMoment := atoms.weightSecondMoment n P hP
degradingArray := atoms.degradingArray n P hP
Consequently compact_causal_range applies to every member of any abstract score-risk class, without an integrability hypothesis in the class interface.
Formal statement
hypotheses
cls :
ℕ → TransportedArray 𝒳 → Prop
N k :
ℕ → ℕ
c epsilon :
ℝ
atoms :
ScoreRiskClassAtoms cls N k c epsilon
n :
ℕ
P :
hP :
cls n P
conclusion
(P.deltaY n =ᵐ[sourceXLaw P n] P.assignmentContrast n true) ∧
(P.deltaD n =ᵐ[sourceXLaw P n] P.receiptContrast n true) ∧
= ∫ o, (fullY1 o - fullY0 o) * (if fullD1 o = true ∧ fullD0 o = false then 1 else 0) ∂populationLaw P n false ∧
transportedFirstStage P n = targetComplierShare P n ∧
targetCACE P n ∈ parameterSpace
Proof (Lean source)
lemma scoreRiskClass_compact_causal_range
{cls : ℕ → TransportedArray 𝒳 → Prop}
{N k : ℕ → ℕ} {c epsilon : ℝ}
(atoms : ScoreRiskClassAtoms cls N k c epsilon)
{n : ℕ} {P : TransportedArray 𝒳} (hP : cls n P) :
(P.deltaY n =ᵐ[sourceXLaw P n] P.assignmentContrast n true) ∧
(P.deltaD n =ᵐ[sourceXLaw P n] P.receiptContrast n true) ∧
transportedOutcomeITT P n =
∫ o, (fullY1 o - fullY0 o) *
(if fullD1 o = true ∧ fullD0 o = false then 1 else 0)
∂populationLaw P n false ∧
transportedFirstStage P n = targetComplierShare P n ∧
transportedOutcomeITT P n / transportedFirstStage P n =
targetCACE P n ∧
targetCACE P n ∈ parameterSpace := by
have hIV := atoms.toTransportedIVClass hP
obtain ⟨hY, hD⟩ :=
transportedIVClass_contrast_integrable P N k c epsilon n hIV
rcases compact_causal_range P N k c epsilon n hY hD hIV with
⟨_, _, hOutcome, hFirst, hRatio, hRange⟩
exact ⟨Filter.Eventually.of_forall fun _ => rfl,
Filter.Eventually.of_forall fun _ => rfl,
hOutcome, hFirst, hRatio, hRange⟩
Helpers.Witness 17 declarations The following definitions spell out the three independent coins used by the paper.
Measure-theoretic leaves for the witness
The following definitions spell out the three independent coins used by the paper. The first chooses compliance, the second chooses the common receipt type when the unit is not a complier, and the third chooses the binary treated outcome.
def geometryKish
Geometry-side Kish dispersion.
Definition (Lean source)
noncomputable def geometryKish (g : Geometry 𝒳) (n : ℕ) : ℝ :=
∫ x, (g.weight n x) ^ 2 ∂g.sourceX n
def geometryMu
First-stage value displayed by the fixed-strength construction.
Definition (Lean source)
noncomputable def geometryMu (g : Geometry 𝒳) (t0 : ℝ) (n : ℕ) : ℝ :=
sqrt (t0 * geometryKish g n / (n : ℝ))
def geometryCompliance
Compliance probability supplied by the geometry handle.
Definition (Lean source)
noncomputable def geometryCompliance (g : Geometry 𝒳) (t0 : ℝ)
(n : ℕ) : 𝒳 → ℝ :=
fun x => geometryMu g t0 n * g.weight n x / geometryKish g n
def PotentialCoordinates
Potential coordinates (D(0),D(1),Y(0),Y(1)) conditional on X.
Definition (Lean source)
def potentialCoordinates
Collects a unit's two potential treatment receipts and two potential outcomes into its potential-coordinate vector.
Definition (Lean source)
def potentialCoordinates (o : FullData 𝒳) : PotentialCoordinates :=
(fullD0 o, fullD1 o, fullY0 o, fullY1 o)
def geometryPopulationX
Geometry covariate marginal for the selected population.
Definition (Lean source)
def geometryPopulationX (g : Geometry 𝒳) (n : ℕ) (s : Bool) : Measure 𝒳 :=
if s then g.sourceX n else g.targetX n
def geometryWitnessFamily
The named least-favourable continuum attached to a geometry and strength. The compliance probability is clamped only to make the array globally valid; on the eventual geometry range the clamp is inactive.
Definition (Lean source)
noncomputable def geometryWitnessFamily (g : Geometry 𝒳) (t0 h : ℝ) :
TransportedArray 𝒳 :=
witnessArrayH h g.sourceX g.targetX
(fun m x => min 1 (geometryCompliance g t0 m x)) g.propensity
(fun m => measurable_const.min (geometryCompliance_measurable g t0 m))
g.propensity_measurable
theorem geometryWitnessFamily_source_chiSq_bound
The source-observation chi-square divergence between a geometry witness at perturbation level h and its zero-perturbation counterpart is bounded by eight times the squared perturbation, scaled by the geometry's signal and inverse Kish dispersion.
Formal statement
hypotheses
g :
Geometry 𝒳
k :
ℕ → ℕ
epsilon t0 :
ℝ
hepsilon :
0 < epsilon
hg :
AdmissibleGeometry g k epsilon
n :
ℕ
h :
ℝ
hh :
|h| ≤ 1 / 4
hvalid :
∀ x, 0 ≤ geometryCompliance g t0 n x ∧ geometryCompliance g t0 n x ≤ 1
conclusion
chiSqDiv (sourceObsLaw (geometryWitnessFamily g t0 h) n) (sourceObsLaw (geometryWitnessFamily g t0 0) n)
≤ 8 * geometryMu g t0 n ^ 2 * h ^ 2 / geometryKish g n
Proof (Lean source)
lemma geometryWitnessFamily_source_chiSq_bound
(g : Geometry 𝒳) (k : ℕ → ℕ) (epsilon t0 : ℝ)
(hepsilon : 0 < epsilon) (hg : AdmissibleGeometry g k epsilon)
(n : ℕ) (h : ℝ) (hh : |h| ≤ 1 / 4)
(hvalid : ∀ x, 0 ≤ geometryCompliance g t0 n x ∧
geometryCompliance g t0 n x ≤ 1) :
chiSqDiv
(sourceObsLaw (geometryWitnessFamily g t0 h) n)
(sourceObsLaw (geometryWitnessFamily g t0 0) n) ≤
8 * geometryMu g t0 n ^ 2 * h ^ 2 / geometryKish g n := by
let p : ℕ → 𝒳 → ℝ :=
fun m x => min 1 (geometryCompliance g t0 m x)
have hpmeas : ∀ m, Measurable (p m) := fun m =>
measurable_const.min (geometryCompliance_measurable g t0 m)
have hp : ∀ m x, 0 ≤ p m x ∧ p m x ≤ 1 := by
intro m x
have hcomp : 0 ≤ geometryCompliance g t0 m x := by
exact div_nonneg
(mul_nonneg (Real.sqrt_nonneg _) (hg.2.2.2.2.2.1 m x).1)
(geometryKish_pos g k epsilon hg m).le
exact ⟨le_min (by norm_num) hcomp, min_le_left _ _⟩
have he01 : ∀ m x, 0 ≤ g.propensity m x ∧ g.propensity m x ≤ 1 :=
fun m x => ⟨hepsilon.le.trans (hg.2.2.2.2.1 m x).1,
(hg.2.2.2.2.1 m x).2.trans (sub_le_self 1 hepsilon.le)⟩
have hchi :
chiSqDiv
(sourceObsLaw (geometryWitnessFamily g t0 h) n)
(sourceObsLaw (geometryWitnessFamily g t0 0) n) =
∫ x, 8 * g.propensity n x * p n x ^ 2 * h ^ 2 /
(1 + p n x) ∂g.sourceX n := by
change chiSqDiv
(sourceObsLaw (witnessArrayH h g.sourceX g.targetX p g.propensity
hpmeas g.propensity_measurable) n)
(sourceObsLaw (witnessArrayH 0 g.sourceX g.targetX p g.propensity
hpmeas g.propensity_measurable) n) = _
exact witnessArrayH_source_chiSq_eq h g.sourceX g.targetX p g.propensity
hpmeas g.propensity_measurable hg.1 hh hp he01 n
rw [hchi]
let F : 𝒳 → ℝ := fun x =>
8 * g.propensity n x * p n x ^ 2 * h ^ 2 / (1 + p n x)
let G : 𝒳 → ℝ := fun x =>
8 * geometryCompliance g t0 n x ^ 2 * h ^ 2
letI : IsProbabilityMeasure (g.sourceX n) := hg.1 n
have hFmeas : Measurable F := by
dsimp [F]
exact ((((g.propensity_measurable n).const_mul 8).mul
((hpmeas n).pow_const 2)).mul_const (h ^ 2)).div
(measurable_const.add (hpmeas n))
have hGmeas : Measurable G := by
dsimp [G]
exact (((geometryCompliance_measurable g t0 n).pow_const 2).const_mul 8).mul_const
(h ^ 2)
have hFint : Integrable F (g.sourceX n) := by
refine Integrable.of_bound hFmeas.aestronglyMeasurable (8 * h ^ 2) ?_
filter_upwards with x
rw [Real.norm_eq_abs, abs_of_nonneg (by
dsimp [F]
exact div_nonneg
(mul_nonneg
(mul_nonneg
(mul_nonneg (by norm_num) (he01 n x).1)
(sq_nonneg _)) (sq_nonneg _))
(by linarith [(hp n x).1]))]
dsimp [F]
apply (div_le_iff₀ (by linarith [(hp n x).1])).2
have hpSq : p n x ^ 2 ≤ 1 :=
by simpa using
(sq_le_sq₀ (hp n x).1 (by norm_num)).2 (hp n x).2
have hep : g.propensity n x * p n x ^ 2 ≤ 1 := by
calc
g.propensity n x * p n x ^ 2 ≤ 1 * p n x ^ 2 :=
mul_le_mul_of_nonneg_right (he01 n x).2 (sq_nonneg _)
_ ≤ 1 := by simpa using hpSq
have hnum :
8 * g.propensity n x * p n x ^ 2 * h ^ 2 ≤ 8 * h ^ 2 := by
calc
8 * g.propensity n x * p n x ^ 2 * h ^ 2 =
(8 * h ^ 2) * (g.propensity n x * p n x ^ 2) := by ring
_ ≤ (8 * h ^ 2) * 1 :=
mul_le_mul_of_nonneg_left hep (by positivity)
_ = 8 * h ^ 2 := by ring
nlinarith [mul_nonneg (by positivity : 0 ≤ 8 * h ^ 2) (hp n x).1]
have hGint : Integrable G (g.sourceX n) := by
refine Integrable.of_bound hGmeas.aestronglyMeasurable (8 * h ^ 2) ?_
filter_upwards with x
rw [Real.norm_eq_abs, abs_of_nonneg (by dsimp [G]; positivity)]
dsimp [G]
have hcSq : geometryCompliance g t0 n x ^ 2 ≤ 1 :=
by simpa using
(sq_le_sq₀ (hvalid x).1 (by norm_num)).2 (hvalid x).2
exact mul_le_mul_of_nonneg_right
(by simpa using
(mul_le_mul_of_nonneg_left hcSq (by norm_num : (0 : ℝ) ≤ 8)))
(sq_nonneg h)
change ∫ x, F x ∂g.sourceX n ≤ _
calc
∫ x, F x ∂g.sourceX n ≤ ∫ x, G x ∂g.sourceX n := by
apply integral_mono hFint hGint
intro x
have hpn : p n x = geometryCompliance g t0 n x :=
min_eq_right (hvalid x).2
dsimp [F, G]
rw [hpn]
apply (div_le_iff₀ (by linarith [(hvalid x).1])).2
let A : ℝ := 8 * geometryCompliance g t0 n x ^ 2 * h ^ 2
have hA : 0 ≤ A := by dsimp [A]; positivity
have hnum :
8 * g.propensity n x * geometryCompliance g t0 n x ^ 2 * h ^ 2 ≤ A := by
calc
8 * g.propensity n x * geometryCompliance g t0 n x ^ 2 * h ^ 2 =
A * g.propensity n x := by dsimp [A]; ring
_ ≤ A * 1 := mul_le_mul_of_nonneg_left (he01 n x).2 hA
_ = A := by ring
change _ ≤ A * (1 + geometryCompliance g t0 n x)
nlinarith [mul_nonneg hA (hvalid x).1]
_ = 8 * geometryMu g t0 n ^ 2 * h ^ 2 /
geometryKish g n := by
change (∫ x, 8 * geometryCompliance g t0 n x ^ 2 * h ^ 2
∂g.sourceX n) = _
rw [show (fun x => 8 * geometryCompliance g t0 n x ^ 2 * h ^ 2) =
fun x => (8 * h ^ 2) * geometryCompliance g t0 n x ^ 2 by
funext x
ring]
rw [integral_const_mul,
geometryCompliance_sq_integral g k epsilon t0 hg n]
ring
def leastFavorableWitness
The continuum family Q_{n,h}^g. Each population has the same conditional potential-data kernel given X, integrated against g.sourceX in the source and g.targetX in the target. Compliance types have probabilities p_n, (1-p_n)/2, (1-p_n)/2; Y(0)=0; and binary Y(1) has mean 1/2+h for compliers and 1/2 otherwise.
Definition (Lean source)
noncomputable def leastFavorableWitness
(g : Geometry 𝒳) (N k : ℕ → ℕ) (c epsilon alpha t0 rho : ℝ)
(n : ℕ) (h : ℝ) (P : TransportedArray 𝒳) : Prop :=
AdmissibleGeometry g k epsilon ∧
0 < alpha ∧ alpha < 1 ∧ -- @realizes \alpha(noncoverage level in (0,1))
0 < t0 ∧ -- @realizes t_0(positive fixed-strength threshold)
rho = (1 - alpha) / 8 ∧
|h| ≤ min (1 / 4) (rho * t0 ^ (-1 / 2 : ℝ)) ∧
fixedGeometrySlice P g N k c epsilon n ∧
(∃ K : 𝒳 → Measure PotentialCoordinates,
(∀ x, IsProbabilityMeasure (K x)) ∧
(∀ B, MeasurableSet B → Measurable fun x => K x B) ∧
∀ s A B, MeasurableSet A → MeasurableSet B →
populationLaw P n s
{o | fullX o ∈ A ∧ potentialCoordinates o ∈ B} =
∫⁻ x in A, K x B ∂geometryPopulationX g n s) ∧
transportedFirstStage P n = geometryMu g t0 n ∧
-- @realizes \mu_n(fixed-strength witness first stage; range inherited from class membership)
effectiveStrength P n = t0 ∧
targetCACE P n = 1 / 2 + h ∧
(∀ s : Bool, ∀ᵐ o ∂populationLaw P n s,
fullY0 o = 0 ∧ (fullY1 o = 0 ∨ fullY1 o = 1)) ∧
(∀ s : Bool, ∀ A, MeasurableSet A →
(populationLaw P n s
{o | fullX o ∈ A ∧ fullD0 o = false ∧ fullD1 o = true}).toReal =
∫ x in A, geometryCompliance g t0 n x ∂geometryPopulationX g n s) ∧
(∀ s : Bool, ∀ A, MeasurableSet A →
(populationLaw P n s
{o | fullX o ∈ A ∧ fullD0 o = true ∧ fullD1 o = true}).toReal =
∫ x in A, (1 - geometryCompliance g t0 n x) / 2
∂geometryPopulationX g n s) ∧
(∀ s : Bool, ∀ A, MeasurableSet A →
(populationLaw P n s
{o | fullX o ∈ A ∧ fullD0 o = false ∧ fullD1 o = false}).toReal =
∫ x in A, (1 - geometryCompliance g t0 n x) / 2
∂geometryPopulationX g n s) ∧
(∀ s : Bool, ∀ A, MeasurableSet A →
∫ o in {o | fullX o ∈ A ∧ fullD0 o = false ∧ fullD1 o = true},
fullY1 o ∂populationLaw P n s =
∫ x in A, geometryCompliance g t0 n x * (1 / 2 + h)
∂geometryPopulationX g n s) ∧
(∀ s : Bool, ∀ A, MeasurableSet A →
∫ o in {o | fullX o ∈ A ∧ fullD0 o = true ∧ fullD1 o = true},
fullY1 o ∂populationLaw P n s =
∫ x in A, (1 - geometryCompliance g t0 n x) / 4
∂geometryPopulationX g n s) ∧
(∀ s : Bool, ∀ A, MeasurableSet A →
∫ o in {o | fullX o ∈ A ∧ fullD0 o = false ∧ fullD1 o = false},
fullY1 o ∂populationLaw P n s =
∫ x in A, (1 - geometryCompliance g t0 n x) / 4
∂geometryPopulationX g n s)
def geometryHandle
Full fixed-strength geometry handle: admissibility, valid compliance probabilities, exact first-stage/strength identities, a continuum of witness laws, and the chi-square/total-variation comparisons with the center.
Definition (Lean source)
noncomputable def geometryHandle
(g : Geometry 𝒳) (N k : ℕ → ℕ) (c epsilon alpha t0 rho : ℝ)
(n : ℕ) : Prop :=
AdmissibleGeometry g k epsilon ∧
0 < alpha ∧ alpha < 1 ∧
0 < t0 ∧ -- @realizes t_0(positive frontier threshold)
rho = (1 - alpha) / 8 ∧
(∀ x, 0 ≤ geometryCompliance g t0 n x ∧
geometryCompliance g t0 n x ≤ 1) ∧
(∫ x, g.weight n x * geometryCompliance g t0 n x ∂g.sourceX n) =
geometryMu g t0 n ∧
(n : ℝ) * geometryMu g t0 n ^ 2 / geometryKish g n = t0 ∧
∃ Q : ℝ → TransportedArray 𝒳,
(∀ h, |h| ≤ min (1 / 4) (rho * t0 ^ (-1 / 2 : ℝ)) →
leastFavorableWitness g N k c epsilon alpha t0 rho n h (Q h)) ∧
(∀ h, |h| ≤ min (1 / 4) (rho * t0 ^ (-1 / 2 : ℝ)) →
1 + chiSqDiv
(twoSampleLaw (Q h) N n) (twoSampleLaw (Q 0) N n) ≤
exp (8 * t0 * h ^ 2)) ∧
(∀ h, |h| ≤ min (1 / 4) (rho * t0 ^ (-1 / 2 : ℝ)) →
tvDist
(twoSampleLaw (Q h) N n) (twoSampleLaw (Q 0) N n) ≤ 2 * rho)
The continuum construction supplies a member of every sufficiently late fixed-geometry strength slice. In particular, lower bounds use an exhibited law and never a nonemptiness assumption on the model class.
Formal statement
hypotheses
g :
Geometry 𝒳
N k :
ℕ → ℕ
c epsilon t0 :
ℝ
hc :
0 < c
hepsilon :
0 < epsilon ∧ epsilon < 1 / 2
ht0 :
0 < t0
hg :
AdmissibleGeometry g k epsilon
hkPos :
∀ n, 0 < k n
conclusion
∀ᶠ n in atTop,
∃ P : TransportedArray 𝒳,
fixedGeometrySlice P g N k c epsilon n ∧ t0 ≤ effectiveStrength P n
Proof (Lean source)
lemma fixedGeometrySlice_eventually_inhabited
(g : Geometry 𝒳) (N k : ℕ → ℕ) (c epsilon t0 : ℝ)
(hc : 0 < c) (hepsilon : 0 < epsilon ∧ epsilon < 1 / 2)
(ht0 : 0 < t0)
(hg : AdmissibleGeometry g k epsilon)
(hN : Tendsto (fun n : ℕ => (N n : ℝ) / (n : ℝ)) atTop (𝓝 c))
(hkPos : ∀ n, 0 < k n)
(hkInf : Tendsto (fun n : ℕ => (k n : ℝ)) atTop atTop)
(hkRoot : Tendsto (fun n : ℕ => (k n : ℝ) / sqrt n) atTop (𝓝 0)) :
∀ᶠ n in atTop, ∃ P : TransportedArray 𝒳,
fixedGeometrySlice P g N k c epsilon n ∧
t0 ≤ effectiveStrength P n := by
let μS : ℕ → Measure 𝒳 := g.sourceX
let μT : ℕ → Measure 𝒳 := g.targetX
let w : ℕ → 𝒳 → ℝ := g.weight
let p : ℕ → 𝒳 → ℝ := fun m x =>
min 1 (geometryCompliance g t0 m x)
let e : ℕ → 𝒳 → ℝ := g.propensity
have hpmeas : ∀ m, Measurable (p m) := fun m =>
measurable_const.min (geometryCompliance_measurable g t0 m)
have hemeas : ∀ m, Measurable (e m) := g.propensity_measurable
have hwmeas : ∀ m, Measurable (w m) := g.weight_measurable
have hμS : ∀ m, IsProbabilityMeasure (μS m) := hg.1
have hμT : ∀ m, IsProbabilityMeasure (μT m) := hg.2.1
have hp : ∀ m x, 0 ≤ p m x ∧ p m x ≤ 1 := by
intro m x
have hkish := geometryKish_pos g k epsilon hg m
have hcomp : 0 ≤ geometryCompliance g t0 m x := by
exact div_nonneg
(mul_nonneg (Real.sqrt_nonneg _) (hg.2.2.2.2.2.1 m x).1)
hkish.le
exact ⟨le_min (by norm_num) hcomp, min_le_left _ _⟩
have he : ∀ m x, epsilon ≤ e m x ∧ e m x ≤ 1 - epsilon :=
hg.2.2.2.2.1
have hw : ∀ m x, 0 ≤ w m x ∧ w m x ≤ 2 * (k m : ℝ) :=
hg.2.2.2.2.2.1
have hw2 : ∀ m, ∫ x, (w m x) ^ 2 ∂μS m ≤ (k m : ℝ) :=
hg.2.2.2.2.2.2.2.1
have htransport : ∀ m A, MeasurableSet A →
μT m A = ofReal (∫ x in A, w m x ∂μS m) :=
hg.2.2.2.2.2.2.2.2
have hvalid :=
geometryCompliance_eventually_valid g k epsilon t0 ht0 hg hkRoot
have hfirst : Tendsto (fun m => ∫ x, w m x * p m x ∂μS m)
atTop (𝓝 0) := by
apply (geometryMu_tendsto_zero g k epsilon t0 ht0 hg hkRoot).congr'
filter_upwards [hvalid] with m hm
have hpm : p m = geometryCompliance g t0 m := by
funext x
exact min_eq_right (hm x).2
rw [show w m = g.weight m from rfl, hpm,
show μS m = g.sourceX m from rfl]
exact (geometry_firstStage g k epsilon t0 hg m).symm
filter_upwards [hvalid, eventually_atTop.2 ⟨1, fun _ hn => hn⟩] with n hnvalid hn
have hnpos : 0 < n := by omega
have hpm : p n = geometryCompliance g t0 n := by
funext x
exact min_eq_right (hnvalid x).2
have he01 : ∀ m x, 0 ≤ e m x ∧ e m x ≤ 1 := fun m x =>
⟨hepsilon.1.le.trans (he m x).1,
(he m x).2.trans (sub_le_self 1 hepsilon.1.le)⟩
have hwint (m : ℕ) : Integrable (w m) (μS m) := by
letI : IsProbabilityMeasure (μS m) := hμS m
refine Integrable.of_bound (hwmeas m).aestronglyMeasurable
(2 * (k m : ℝ)) ?_
filter_upwards with x
rw [Real.norm_eq_abs, abs_of_nonneg (hw m x).1]
exact (hw m x).2
have hrn (m : ℕ) :
(fun x => ((μT m).rnDeriv (μS m) x).toReal) =ᵐ[μS m] w m := by
letI : IsProbabilityMeasure (μS m) := hμS m
exact transportWeight_eq_of_integral (hwmeas m) (hwint m)
(fun x => (hw m x).1) (htransport m)
have hmuPos : 0 < geometryMu g t0 n := by
rw [geometryMu]
exact Real.sqrt_pos.2 (div_pos
(mul_pos ht0 (geometryKish_pos g k epsilon hg n))
(by exact_mod_cast hnpos))
have hpos : 0 < ∫ x, p n x ∂μT n := by
letI : IsProbabilityMeasure (μS n) := hμS n
rw [target_eq_withDensity_of_integral (hwmeas n) (hwint n)
(Filter.Eventually.of_forall fun x => (hw n x).1) (htransport n)]
change 0 < ∫ x, p n x
∂(μS n).withDensity (fun x => ofReal (w n x))
rw [show (fun x => ofReal (w n x)) =
ofReal ∘ w n by rfl]
rw [integral_withDensity_eq_integral_toReal_smul
(ENNReal.measurable_ofReal.comp (hwmeas n))
(Filter.Eventually.of_forall fun x => ENNReal.ofReal_lt_top) (p n)]
have hint :
(∫ x, ((ofReal ∘ w n) x).toReal • p n x ∂μS n) =
∫ x, w n x * p n x ∂μS n := by
apply integral_congr_ae
filter_upwards with x
simp only [Function.comp_apply, ENNReal.toReal_ofReal (hw n x).1,
smul_eq_mul]
rw [hint, hpm]
rw [show (∫ x, w n x * geometryCompliance g t0 n x ∂μS n) =
geometryMu g t0 n by
simpa [w, μS] using geometry_firstStage g k epsilon t0 hg n]
exact hmuPos
let P := witnessArray μS μT p e hpmeas hemeas
have hIV : TransportedIVClass P N k c epsilon n :=
witnessArray_mem_transportedIVClass μS μT w p e N k c epsilon
hpmeas hemeas hwmeas hμS hμT hp hepsilon he hw hw2 htransport
hc hN hkInf hkRoot hfirst n hpos
have hsource : sourceXLaw P n = g.sourceX n := by
unfold P
rw [witnessArray_sourceXLaw μS μT p e hpmeas hemeas
hμS hμT hp he01 n]
have htarget : targetXLaw P n = g.targetX n := by
unfold P
rw [witnessArray_targetXLaw μS μT p e hpmeas hemeas hμS hμT hp n]
have hweight :
transportWeight P n =ᵐ[sourceXLaw P n] g.weight n := by
rw [hsource]
unfold transportWeight
rw [hsource, htarget]
exact hrn n
have hweightG :
transportWeight P n =ᵐ[g.sourceX n] g.weight n := by
simpa only [hsource] using hweight
have hpropensity : P.propensity n = g.propensity n := by
rfl
have hfirstExact :
transportedFirstStage P n = geometryMu g t0 n := by
letI : IsProbabilityMeasure (μS n) := hμS n
letI : IsProbabilityMeasure (μT n) := hμT n
rw [transportedFirstStage_eq_weighted_deltaD P k epsilon n
(sourceObservationFacts_of_class P N k c epsilon n hIV)
hIV.instrumentOverlap hIV.weightEnvelope]
change (∫ x, transportWeight P n x * p n x ∂sourceXLaw P n) = _
rw [hsource]
calc
(∫ x, transportWeight P n x * p n x ∂g.sourceX n) =
∫ x, w n x * p n x ∂μS n := by
change (∫ x, transportWeight P n x * p n x ∂μS n) = _
apply integral_congr_ae
filter_upwards [hweightG] with x hx
rw [hx]
_ = geometryMu g t0 n := by
rw [hpm]
simpa [w, μS] using geometry_firstStage g k epsilon t0 hg n
have hkishExact :
kishDispersion P n = geometryKish g n := by
unfold kishDispersion geometryKish
rw [hsource]
exact integral_congr_ae (hweightG.fun_comp fun z => z ^ 2)
refine ⟨P, ⟨hIV, hsource, htarget, hweight, hpropensity⟩, ?_⟩
rw [effectiveStrength, hfirstExact, hkishExact,
geometry_strength g k epsilon t0 ht0 hg n hnpos]
theorem geometryWitnessFamily_eventually_leastFavorable
The named geometry family is least favourable at every sufficiently late index throughout the prescribed local range.
Formal statement
hypotheses
g :
Geometry 𝒳
N k :
ℕ → ℕ
c epsilon alpha t0 rho :
ℝ
hc :
0 < c
hepsilon :
0 < epsilon ∧ epsilon < 1 / 2
halpha :
0 < alpha ∧ alpha < 1
ht0 :
0 < t0
hrho :
rho = (1 - alpha) / 8
hg :
AdmissibleGeometry g k epsilon
hkPos :
∀ n, 0 < k n
conclusion
∀ᶠ n in atTop,
∀ h,
|h|
≤ min (1 / 4) (rho * t0 ^ (-1 / 2 : ℝ)) → leastFavorableWitness g N k c epsilon alpha t0 rho n h (geometryWitnessFamily g t0 h)
Proof (Lean source)
lemma geometryWitnessFamily_eventually_leastFavorable
(g : Geometry 𝒳) (N k : ℕ → ℕ) (c epsilon alpha t0 rho : ℝ)
(hc : 0 < c) (hepsilon : 0 < epsilon ∧ epsilon < 1 / 2)
(halpha : 0 < alpha ∧ alpha < 1) (ht0 : 0 < t0)
(hrho : rho = (1 - alpha) / 8)
(hg : AdmissibleGeometry g k epsilon)
(hN : Tendsto (fun n : ℕ => (N n : ℝ) / (n : ℝ)) atTop (𝓝 c))
(hkPos : ∀ n, 0 < k n)
(hkInf : Tendsto (fun n : ℕ => (k n : ℝ)) atTop atTop)
(hkRoot : Tendsto (fun n : ℕ => (k n : ℝ) / sqrt n)
atTop (𝓝 0)) :
∀ᶠ n in atTop,
∀ h, |h| ≤ min (1 / 4) (rho * t0 ^ (-1 / 2 : ℝ)) →
leastFavorableWitness g N k c epsilon alpha t0 rho n h
(geometryWitnessFamily g t0 h) := by
let μS : ℕ → Measure 𝒳 := g.sourceX
let μT : ℕ → Measure 𝒳 := g.targetX
let w : ℕ → 𝒳 → ℝ := g.weight
let p : ℕ → 𝒳 → ℝ := fun m x =>
min 1 (geometryCompliance g t0 m x)
let e : ℕ → 𝒳 → ℝ := g.propensity
have hpmeas : ∀ m, Measurable (p m) := fun m =>
measurable_const.min (geometryCompliance_measurable g t0 m)
have hemeas : ∀ m, Measurable (e m) := g.propensity_measurable
have hwmeas : ∀ m, Measurable (w m) := g.weight_measurable
have hμS : ∀ m, IsProbabilityMeasure (μS m) := hg.1
have hμT : ∀ m, IsProbabilityMeasure (μT m) := hg.2.1
have hp : ∀ m x, 0 ≤ p m x ∧ p m x ≤ 1 := by
intro m x
have hkish := geometryKish_pos g k epsilon hg m
have hcomp : 0 ≤ geometryCompliance g t0 m x := by
exact div_nonneg
(mul_nonneg (Real.sqrt_nonneg _) (hg.2.2.2.2.2.1 m x).1)
hkish.le
exact ⟨le_min (by norm_num) hcomp, min_le_left _ _⟩
have he : ∀ m x, epsilon ≤ e m x ∧ e m x ≤ 1 - epsilon :=
hg.2.2.2.2.1
have hw : ∀ m x, 0 ≤ w m x ∧ w m x ≤ 2 * (k m : ℝ) :=
hg.2.2.2.2.2.1
have hw2 : ∀ m, ∫ x, (w m x) ^ 2 ∂μS m ≤ (k m : ℝ) :=
hg.2.2.2.2.2.2.2.1
have htransport : ∀ m A, MeasurableSet A →
μT m A = ofReal (∫ x in A, w m x ∂μS m) :=
hg.2.2.2.2.2.2.2.2
have hvalid :=
geometryCompliance_eventually_valid g k epsilon t0 ht0 hg hkRoot
have hfirst : Tendsto (fun m => ∫ x, w m x * p m x ∂μS m)
atTop (𝓝 0) := by
apply (geometryMu_tendsto_zero g k epsilon t0 ht0 hg hkRoot).congr'
filter_upwards [hvalid] with m hm
have hpm : p m = geometryCompliance g t0 m := by
funext x
exact min_eq_right (hm x).2
rw [show w m = g.weight m from rfl, hpm,
show μS m = g.sourceX m from rfl]
exact (geometry_firstStage g k epsilon t0 hg m).symm
filter_upwards [hvalid,
eventually_atTop.2 ⟨1, fun _ hn => hn⟩] with n hnvalid hn
have hnpos : 0 < n := by omega
have hpm : p n = geometryCompliance g t0 n := by
funext x
exact min_eq_right (hnvalid x).2
have he01 : ∀ m x, 0 ≤ e m x ∧ e m x ≤ 1 := fun m x =>
⟨hepsilon.1.le.trans (he m x).1,
(he m x).2.trans (sub_le_self 1 hepsilon.1.le)⟩
have hwint (m : ℕ) : Integrable (w m) (μS m) := by
letI : IsProbabilityMeasure (μS m) := hμS m
refine Integrable.of_bound (hwmeas m).aestronglyMeasurable
(2 * (k m : ℝ)) ?_
filter_upwards with x
rw [Real.norm_eq_abs, abs_of_nonneg (hw m x).1]
exact (hw m x).2
have hrn (m : ℕ) :
(fun x => ((μT m).rnDeriv (μS m) x).toReal) =ᵐ[μS m] w m := by
letI : IsProbabilityMeasure (μS m) := hμS m
exact transportWeight_eq_of_integral (hwmeas m) (hwint m)
(fun x => (hw m x).1) (htransport m)
have hmuPos : 0 < geometryMu g t0 n := by
rw [geometryMu]
exact Real.sqrt_pos.2 (div_pos
(mul_pos ht0 (geometryKish_pos g k epsilon hg n))
(by exact_mod_cast hnpos))
have hpos : 0 < ∫ x, p n x ∂μT n := by
letI : IsProbabilityMeasure (μS n) := hμS n
rw [target_eq_withDensity_of_integral (hwmeas n) (hwint n)
(Filter.Eventually.of_forall fun x => (hw n x).1) (htransport n)]
change 0 < ∫ x, p n x
∂(μS n).withDensity (fun x => ofReal (w n x))
rw [show (fun x => ofReal (w n x)) =
ofReal ∘ w n by rfl]
rw [integral_withDensity_eq_integral_toReal_smul
(ENNReal.measurable_ofReal.comp (hwmeas n))
(Filter.Eventually.of_forall fun x => ENNReal.ofReal_lt_top) (p n)]
have hint :
(∫ x, ((ofReal ∘ w n) x).toReal • p n x ∂μS n) =
∫ x, w n x * p n x ∂μS n := by
apply integral_congr_ae
filter_upwards with x
simp only [Function.comp_apply, ENNReal.toReal_ofReal (hw n x).1,
smul_eq_mul]
rw [hint, hpm]
rw [show (∫ x, w n x * geometryCompliance g t0 n x ∂μS n) =
geometryMu g t0 n by
simpa [w, μS] using geometry_firstStage g k epsilon t0 hg n]
exact hmuPos
let Q : ℝ → TransportedArray 𝒳 := geometryWitnessFamily g t0
have hQ (a : ℝ) :
Q a = witnessArrayH a μS μT p e hpmeas hemeas := by
rfl
intro h hh
change leastFavorableWitness g N k c epsilon alpha t0 rho n h (Q h)
have hhquarter : |h| ≤ 1 / 4 :=
hh.trans (min_le_left _ _)
have hIV : TransportedIVClass (Q h) N k c epsilon n := by
rw [hQ]
exact witnessArrayH_mem_transportedIVClass
h μS μT w p e N k c epsilon hpmeas hemeas hwmeas
hμS hμT hhquarter hp hepsilon he hw hw2 htransport
hc hN hkInf hkRoot hfirst n hpos
have hsource : sourceXLaw (Q h) n = g.sourceX n := by
unfold Q geometryWitnessFamily
rw [witnessArrayH_sourceXLaw h μS μT p e hpmeas hemeas
hμS hμT hhquarter hp he01 n]
have htarget : targetXLaw (Q h) n = g.targetX n := by
unfold Q geometryWitnessFamily
rw [witnessArrayH_targetXLaw h μS μT p e hpmeas hemeas
hμS hμT hhquarter hp n]
have hweight :
transportWeight (Q h) n =ᵐ[sourceXLaw (Q h) n] g.weight n := by
rw [hsource]
unfold transportWeight
rw [hsource, htarget]
exact hrn n
have hweightG :
transportWeight (Q h) n =ᵐ[g.sourceX n] g.weight n := by
simpa only [hsource] using hweight
have hpropensity : (Q h).propensity n = g.propensity n := by
rfl
have hfirstExact :
transportedFirstStage (Q h) n = geometryMu g t0 n := by
letI : IsProbabilityMeasure (μS n) := hμS n
letI : IsProbabilityMeasure (μT n) := hμT n
rw [transportedFirstStage_eq_weighted_deltaD (Q h) k epsilon n
(sourceObservationFacts_of_class (Q h) N k c epsilon n hIV)
hIV.instrumentOverlap hIV.weightEnvelope]
change (∫ x, transportWeight (Q h) n x * p n x
∂sourceXLaw (Q h) n) = _
rw [hsource]
calc
(∫ x, transportWeight (Q h) n x * p n x ∂g.sourceX n) =
∫ x, w n x * p n x ∂μS n := by
change (∫ x, transportWeight (Q h) n x * p n x ∂μS n) = _
apply integral_congr_ae
filter_upwards [hweightG] with x hx
rw [hx]
_ = geometryMu g t0 n := by
rw [hpm]
simpa [w, μS] using geometry_firstStage g k epsilon t0 hg n
have hkishExact :
kishDispersion (Q h) n = geometryKish g n := by
unfold kishDispersion geometryKish
rw [hsource]
exact integral_congr_ae (hweightG.fun_comp fun z => z ^ 2)
have hslice : fixedGeometrySlice
(Q h) g N k c epsilon n :=
⟨hIV, hsource, htarget, hweight, hpropensity⟩
refine ⟨hg, halpha.1, halpha.2, ht0, hrho, hh, hslice, ?_,
hfirstExact, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_, ?_⟩
· rw [hQ]
exact witnessArrayH_equalConditionalKernel h μS μT p e hpmeas
hemeas hμS hμT hhquarter hp g n rfl rfl
· rw [effectiveStrength, hfirstExact, hkishExact,
geometry_strength g k epsilon t0 ht0 hg n hnpos]
· rw [hQ]
exact witnessArrayH_targetCACE h μS μT p e hpmeas hemeas
hμS hμT hhquarter hp n hpos
· intro s
rw [hQ, witnessArrayH_populationLaw h μS μT p e hpmeas hemeas
hμS hμT hhquarter hp n s]
cases s
· exact witnessPopulationMeasureH_ae_good h false hhquarter
(hpmeas n) (hp n) |>.mono fun o ho => ⟨ho.1, ho.2.1⟩
· exact witnessPopulationMeasureH_ae_good h true hhquarter
(hpmeas n) (hp n) |>.mono fun o ho => ⟨ho.1, ho.2.1⟩
· intro s A hA
rw [hQ, witnessArrayH_populationLaw h μS μT p e hpmeas hemeas
hμS hμT hhquarter hp n s]
cases s
· simpa [geometryPopulationX, hpm, μT] using
witnessPopulationMeasureH_complier_on
(μ := μT n) h false hhquarter (hpmeas n) (hp n) A hA
· simpa [geometryPopulationX, hpm, μS] using
witnessPopulationMeasureH_complier_on
(μ := μS n) h true hhquarter (hpmeas n) (hp n) A hA
· intro s A hA
rw [hQ, witnessArrayH_populationLaw h μS μT p e hpmeas hemeas
hμS hμT hhquarter hp n s]
cases s
· simpa [geometryPopulationX, hpm, μT] using
witnessPopulationMeasureH_always_on
(μ := μT n) h false hhquarter (hpmeas n) (hp n) A hA
· simpa [geometryPopulationX, hpm, μS] using
witnessPopulationMeasureH_always_on
(μ := μS n) h true hhquarter (hpmeas n) (hp n) A hA
· intro s A hA
rw [hQ, witnessArrayH_populationLaw h μS μT p e hpmeas hemeas
hμS hμT hhquarter hp n s]
cases s
· simpa [geometryPopulationX, hpm, μT] using
witnessPopulationMeasureH_never_on
(μ := μT n) h false hhquarter (hpmeas n) (hp n) A hA
· simpa [geometryPopulationX, hpm, μS] using
witnessPopulationMeasureH_never_on
(μ := μS n) h true hhquarter (hpmeas n) (hp n) A hA
· intro s A hA
rw [hQ, witnessArrayH_populationLaw h μS μT p e hpmeas hemeas
hμS hμT hhquarter hp n s]
cases s
· simpa [geometryPopulationX, hpm, μT] using
witnessPopulationMeasureH_complierY_on
(μ := μT n) h false hhquarter (hpmeas n) (hp n) A hA
· simpa [geometryPopulationX, hpm, μS] using
witnessPopulationMeasureH_complierY_on
(μ := μS n) h true hhquarter (hpmeas n) (hp n) A hA
· intro s A hA
rw [hQ, witnessArrayH_populationLaw h μS μT p e hpmeas hemeas
hμS hμT hhquarter hp n s]
cases s
· simpa [geometryPopulationX, hpm, μT] using
witnessPopulationMeasureH_alwaysY_on
(μ := μT n) h false hhquarter (hpmeas n) (hp n) A hA
· simpa [geometryPopulationX, hpm, μS] using
witnessPopulationMeasureH_alwaysY_on
(μ := μS n) h true hhquarter (hpmeas n) (hp n) A hA
· intro s A hA
rw [hQ, witnessArrayH_populationLaw h μS μT p e hpmeas hemeas
hμS hμT hhquarter hp n s]
cases s
· simpa [geometryPopulationX, hpm, μT] using
witnessPopulationMeasureH_neverY_on
(μ := μT n) h false hhquarter (hpmeas n) (hp n) A hA
· simpa [geometryPopulationX, hpm, μS] using
witnessPopulationMeasureH_neverY_on
(μ := μS n) h true hhquarter (hpmeas n) (hp n) A hA
theorem leastFavorableWitness_eventually_inhabited
At every sufficiently late index, the clamped construction gives the paper's full least-favourable family throughout its prescribed local range.
Formal statement
hypotheses
g :
Geometry 𝒳
N k :
ℕ → ℕ
c epsilon alpha t0 rho :
ℝ
hc :
0 < c
hepsilon :
0 < epsilon ∧ epsilon < 1 / 2
halpha :
0 < alpha ∧ alpha < 1
ht0 :
0 < t0
hrho :
rho = (1 - alpha) / 8
hg :
AdmissibleGeometry g k epsilon
hkPos :
∀ n, 0 < k n
conclusion
∀ᶠ n in atTop,
∃ Q : ℝ → TransportedArray 𝒳,
∀ h,
|h|
≤ min (1 / 4) (rho * t0 ^ (-1 / 2 : ℝ)) → leastFavorableWitness g N k c epsilon alpha t0 rho n h (Q h)
Proof (Lean source)
lemma leastFavorableWitness_eventually_inhabited
(g : Geometry 𝒳) (N k : ℕ → ℕ) (c epsilon alpha t0 rho : ℝ)
(hc : 0 < c) (hepsilon : 0 < epsilon ∧ epsilon < 1 / 2)
(halpha : 0 < alpha ∧ alpha < 1) (ht0 : 0 < t0)
(hrho : rho = (1 - alpha) / 8)
(hg : AdmissibleGeometry g k epsilon)
(hN : Tendsto (fun n : ℕ => (N n : ℝ) / (n : ℝ)) atTop (𝓝 c))
(hkPos : ∀ n, 0 < k n)
(hkInf : Tendsto (fun n : ℕ => (k n : ℝ)) atTop atTop)
(hkRoot : Tendsto (fun n : ℕ => (k n : ℝ) / sqrt n)
atTop (𝓝 0)) :
∀ᶠ n in atTop, ∃ Q : ℝ → TransportedArray 𝒳,
∀ h, |h| ≤ min (1 / 4) (rho * t0 ^ (-1 / 2 : ℝ)) →
leastFavorableWitness g N k c epsilon alpha t0 rho n h (Q h) := by
filter_upwards [
geometryWitnessFamily_eventually_leastFavorable g N k c epsilon alpha
t0 rho hc hepsilon halpha ht0 hrho hg hN hkPos hkInf hkRoot
] with n hn
exact ⟨geometryWitnessFamily g t0, hn⟩
The named least-favourable family supplies the complete geometry handle, including its chi-square and total-variation calibration, eventually in n.
Formal statement
hypotheses
g :
Geometry 𝒳
N k :
ℕ → ℕ
c epsilon alpha t0 rho :
ℝ
hc :
0 < c
hepsilon :
0 < epsilon ∧ epsilon < 1 / 2
halpha :
0 < alpha ∧ alpha < 1
ht0 :
0 < t0
hrho :
rho = (1 - alpha) / 8
hg :
AdmissibleGeometry g k epsilon
hkPos :
∀ n, 0 < k n
conclusion
∀ᶠ n in atTop, geometryHandle g N k c epsilon alpha t0 rho n
Proof (Lean source)
lemma geometryHandle_eventually_inhabited
(g : Geometry 𝒳) (N k : ℕ → ℕ) (c epsilon alpha t0 rho : ℝ)
(hc : 0 < c) (hepsilon : 0 < epsilon ∧ epsilon < 1 / 2)
(halpha : 0 < alpha ∧ alpha < 1) (ht0 : 0 < t0)
(hrho : rho = (1 - alpha) / 8)
(hg : AdmissibleGeometry g k epsilon)
(hN : Tendsto (fun n : ℕ => (N n : ℝ) / (n : ℝ)) atTop (𝓝 c))
(hkPos : ∀ n, 0 < k n)
(hkInf : Tendsto (fun n : ℕ => (k n : ℝ)) atTop atTop)
(hkRoot : Tendsto (fun n : ℕ => (k n : ℝ) / sqrt n)
atTop (𝓝 0)) :
∀ᶠ n in atTop,
geometryHandle g N k c epsilon alpha t0 rho n := by
have hvalid :=
geometryCompliance_eventually_valid g k epsilon t0 ht0 hg hkRoot
have hleast :=
geometryWitnessFamily_eventually_leastFavorable g N k c epsilon alpha
t0 rho hc hepsilon halpha ht0 hrho hg hN hkPos hkInf hkRoot
filter_upwards [hvalid, hleast,
eventually_atTop.2 ⟨1, fun _ hn => hn⟩] with n hnvalid hnleast hn
have hnpos : 0 < n := by omega
refine ⟨hg, halpha.1, halpha.2, ht0, hrho, hnvalid,
geometry_firstStage g k epsilon t0 hg n,
geometry_strength g k epsilon t0 ht0 hg n hnpos,
geometryWitnessFamily g t0, hnleast, ?_, ?_⟩
· intro h hh
exact geometryWitnessFamily_twoSample_chiSq_bound g N k epsilon t0
hepsilon.1 ht0 hg n hnpos h (hh.trans (min_le_left _ _)) hnvalid
· intro h hh
exact geometryWitnessFamily_twoSample_tv_bound g N k epsilon alpha t0 rho
hepsilon.1 halpha ht0 hrho hg n hnpos h hh hnvalid
The uniform finite-cell construction is realizable on any measurable carrier admitting the required rowwise finite injections. The injected Fin (k n) image carries full mass, so growing support constrains the measure rather than identifying the ambient carrier with a finite type.
Formal statement
hypotheses
N k :
ℕ → ℕ
c epsilon :
ℝ
hCarrier :
∀ n, ∃ cell : Fin (k n) ↪ 𝒳, ∀ i, MeasurableSet {cell i}
hc :
0 < c
hepsilon :
0 < epsilon ∧ epsilon < 1 / 2
hkPos :
∀ n, 0 < k n
conclusion
∀ n, ∃ P : TransportedArray 𝒳, FiniteCellClass P N k c epsilon n
Proof (Lean source)
lemma finiteCellClass_inhabited
(N k : ℕ → ℕ) (c epsilon : ℝ)
(hCarrier : ∀ n, ∃ cell : Fin (k n) ↪ 𝒳, ∀ i, MeasurableSet {cell i})
(hc : 0 < c) (hepsilon : 0 < epsilon ∧ epsilon < 1 / 2)
(hN : Tendsto (fun n : ℕ => (N n : ℝ) / (n : ℝ)) atTop (𝓝 c))
(hkPos : ∀ n, 0 < k n)
(hkInf : Tendsto (fun n : ℕ => (k n : ℝ)) atTop atTop)
(hkRoot : Tendsto (fun n : ℕ => (k n : ℝ) / sqrt n) atTop (𝓝 0)) :
∀ n, ∃ P : TransportedArray 𝒳,
FiniteCellClass P N k c epsilon n := by
exact uniformFiniteCellClass_inhabited N k c epsilon hCarrier hc hepsilon
hN hkPos hkInf hkRoot
theorem transportedIVClass_inhabited
Forgetting the finite-cell field gives an inhabited main class on the same arbitrary carrier.
Formal statement
hypotheses
N k :
ℕ → ℕ
c epsilon :
ℝ
hCarrier :
∀ n, ∃ cell : Fin (k n) ↪ 𝒳, ∀ i, MeasurableSet {cell i}
hc :
0 < c
hepsilon :
0 < epsilon ∧ epsilon < 1 / 2
hkPos :
∀ n, 0 < k n
conclusion
∀ n, ∃ P : TransportedArray 𝒳, TransportedIVClass P N k c epsilon n
Proof (Lean source)
lemma transportedIVClass_inhabited
(N k : ℕ → ℕ) (c epsilon : ℝ)
(hCarrier : ∀ n, ∃ cell : Fin (k n) ↪ 𝒳, ∀ i, MeasurableSet {cell i})
(hc : 0 < c) (hepsilon : 0 < epsilon ∧ epsilon < 1 / 2)
(hN : Tendsto (fun n : ℕ => (N n : ℝ) / (n : ℝ)) atTop (𝓝 c))
(hkPos : ∀ n, 0 < k n)
(hkInf : Tendsto (fun n : ℕ => (k n : ℝ)) atTop atTop)
(hkRoot : Tendsto (fun n : ℕ => (k n : ℝ) / sqrt n) atTop (𝓝 0)) :
∀ n, ∃ P : TransportedArray 𝒳,
TransportedIVClass P N k c epsilon n := by
intro n
obtain ⟨P, hP⟩ :=
finiteCellClass_inhabited N k c epsilon hCarrier hc hepsilon hN hkPos hkInf
hkRoot n
exact ⟨P, hP.toTransportedIVClass⟩
The same uniform witness inhabits every regular finite-cell class whose fixed constants contain the uniform mass 1 / k_n.
Formal statement
hypotheses
N k :
ℕ → ℕ
c epsilon cminus cplus :
ℝ
hCarrier :
∀ n, ∃ cell : Fin (k n) ↪ 𝒳, ∀ i, MeasurableSet {cell i}
hc :
0 < c
hepsilon :
0 < epsilon ∧ epsilon < 1 / 2
hcminus :
0 < cminus ∧ cminus ≤ 1
hcplus :
1 ≤ cplus
hkPos :
∀ n, 0 < k n
conclusion
∀ n, ∃ P : TransportedArray 𝒳, RegularFiniteCellClass P N k c epsilon cminus cplus n
Proof (Lean source)
lemma regularFiniteCellClass_inhabited
(N k : ℕ → ℕ) (c epsilon cminus cplus : ℝ)
(hCarrier : ∀ n, ∃ cell : Fin (k n) ↪ 𝒳, ∀ i, MeasurableSet {cell i})
(hc : 0 < c) (hepsilon : 0 < epsilon ∧ epsilon < 1 / 2)
(hcminus : 0 < cminus ∧ cminus ≤ 1) (hcplus : 1 ≤ cplus)
(hN : Tendsto (fun n : ℕ => (N n : ℝ) / (n : ℝ)) atTop (𝓝 c))
(hkPos : ∀ n, 0 < k n)
(hkInf : Tendsto (fun n : ℕ => (k n : ℝ)) atTop atTop)
(hkRoot : Tendsto (fun n : ℕ => (k n : ℝ) / sqrt n) atTop (𝓝 0)) :
∀ n, ∃ P : TransportedArray 𝒳,
RegularFiniteCellClass P N k c epsilon cminus cplus n := by
intro n
obtain ⟨P, hP⟩ :=
uniformFiniteCellClass_inhabited N k c epsilon hCarrier hc hepsilon hN
hkPos hkInf hkRoot n
rcases hP.finiteCellSource with
⟨hk, hprob, cell, hcell, hrange, hatom, _, he⟩
refine ⟨P, hP.toTransportedIVClass, hk, hcminus.1, hcminus.2, hcplus,
cell, hcell, hrange, ?_⟩
intro i
rw [hatom i]
have hinv : 0 ≤ (k n : ℝ)⁻¹ := by positivity
constructor
· simpa [div_eq_mul_inv] using
mul_le_mul_of_nonneg_right hcminus.2 hinv
· simpa [div_eq_mul_inv] using
mul_le_mul_of_nonneg_right hcplus hinv
T_CompactCausalRange 6 declarations
theorem measurable_observeSource
The map from assigned full data to the observed source-data record is measurable.
Formal statement
conclusion
Proof (Lean source)
lemma measurable_observeSource :
Measurable (observeSource : AssignedFullData 𝒳 → SourceObs 𝒳) := by
have hx : Measurable fun q : AssignedFullData 𝒳 => q.1.2.1 := by
fun_prop
have hz : Measurable fun q : AssignedFullData 𝒳 => q.2 := by
fun_prop
have hd : Measurable fun q : AssignedFullData 𝒳 =>
if q.2 then q.1.2.2.2.1 else q.1.2.2.1 := by
exact Measurable.ite
(hz (MeasurableSet.singleton true)) (by fun_prop) (by fun_prop)
have hy : Measurable fun q : AssignedFullData 𝒳 =>
if (if q.2 then q.1.2.2.2.1 else q.1.2.2.1)
then q.1.2.2.2.2.2 else q.1.2.2.2.2.1 := by
exact Measurable.ite
(hd (MeasurableSet.singleton true)) (by fun_prop) (by fun_prop)
exact hx.prodMk (hz.prodMk (hd.prodMk hy))
theorem sourceXLaw_eq_populationXLaw_source Lemma sourceXLaw_eq_populationXLaw_source in the paper ↗
Under the source-observation conditions, the source covariate distribution equals the source population covariate distribution.
Formal statement
hypotheses
conclusion
Proof (Lean source)
lemma sourceXLaw_eq_populationXLaw_source
(P : TransportedArray 𝒳) (n : ℕ)
(hSourceFacts : SourceObservationFacts P n) :
sourceXLaw P n = populationXLaw P n true := by
have hObserve :
Measurable (observeSource : AssignedFullData 𝒳 → SourceObs 𝒳) :=
measurable_observeSource
have hFullX : Measurable (fullX : FullData 𝒳 → 𝒳) := by
unfold fullX
fun_prop
rw [sourceXLaw, sourceObsLaw, Measure.map_map measurable_fst hObserve]
rw [populationXLaw, ← hSourceFacts.2.2.2.1]
rw [Measure.map_map hFullX measurable_fst]
congr 1
theorem measure_eq_withDensity_of_toReal_setIntegral Lemma measure_eq_withDensity_of_toReal_setIntegral in the paper ↗
A finite measure equals the measure obtained by weighting another measure when its mass on every measurable set is the corresponding integral of a nonnegative integrable weight.
Formal statement
hypotheses
α :
w :
α → ℝ
hwmeas :
hwint :
Integrable w μ
hw0 :
∀ᵐ x ∂μ, 0 ≤ w x
hν :
∀ A
if
then
(ν A).toReal = ∫ x in A, w x ∂μ
conclusion
ν = μ.withDensity (fun x => ofReal (w x))
Proof (Lean source)
lemma measure_eq_withDensity_of_toReal_setIntegral
{α : Type*} [MeasurableSpace α] {μ ν : Measure α} [IsFiniteMeasure ν]
{w : α → ℝ} (hwmeas : Measurable w) (hwint : Integrable w μ)
(hw0 : ∀ᵐ x ∂μ, 0 ≤ w x)
(hν : ∀ A, MeasurableSet A →
(ν A).toReal = ∫ x in A, w x ∂μ) :
ν = μ.withDensity (fun x => ofReal (w x)) := by
exact measure_eq_withDensity_of_toReal_setIntegral
hwint hν
theorem ivRandomization_slice_eq_withDensity Lemma ivRandomization_slice_eq_withDensity in the paper ↗
Under IV randomization and overlap, the full-data distribution conditional on each instrument value is the source population law reweighted by that instrument's propensity probability.
Formal statement
hypotheses
P :
n :
ℕ
epsilon :
ℝ
hSourceFacts :
hRandom :
IVRandomization P n
hOverlap :
InstrumentOverlap P n epsilon
z :
conclusion
= (populationLaw P n true).withDensity (fun o => ofReal (if z then P.propensity n (fullX o) else 1 - P.propensity n (fullX o)))
Proof (Lean source)
lemma ivRandomization_slice_eq_withDensity
(P : TransportedArray 𝒳) (n : ℕ) (epsilon : ℝ)
(hSourceFacts : SourceObservationFacts P n)
(hRandom : IVRandomization P n)
(hOverlap : InstrumentOverlap P n epsilon)
(z : Bool) :
((P.assignedSourceLaw n).restrict {q | q.2 = z}).map Prod.fst =
(populationLaw P n true).withDensity (fun o =>
ofReal
(if z then P.propensity n (fullX o)
else 1 - P.propensity n (fullX o))) := by
let μ := populationLaw P n true
let ν := ((P.assignedSourceLaw n).restrict {q | q.2 = z}).map Prod.fst
let w : FullData 𝒳 → ℝ := fun o =>
if z then P.propensity n (fullX o)
else 1 - P.propensity n (fullX o)
have hFullX : Measurable (fullX : FullData 𝒳 → 𝒳) := by
unfold fullX
fun_prop
have hwmeas : Measurable w := by
have hecomp : Measurable fun o : FullData 𝒳 =>
P.propensity n (fullX o) :=
(P.propensity_measurable n).comp hFullX
cases z
· simpa [w] using measurable_const.fun_sub hecomp
· simpa [w] using hecomp
have hsourceX := sourceXLaw_eq_populationXLaw_source P n hSourceFacts
have hoverlapPop : ∀ᵐ o ∂μ,
epsilon ≤ P.propensity n (fullX o) ∧
P.propensity n (fullX o) ≤ 1 - epsilon := by
have hx := hOverlap.2.2
rw [hsourceX] at hx
exact ae_of_ae_map hFullX.aemeasurable hx
have hw0 : ∀ᵐ o ∂μ, 0 ≤ w o := by
filter_upwards [hoverlapPop] with o ho
unfold w
cases z
· simp
linarith [hOverlap.1]
· simpa using (le_trans hOverlap.1.le ho.1)
have hwle : ∀ᵐ o ∂μ, w o ≤ 1 := by
filter_upwards [hoverlapPop] with o ho
unfold w
cases z
· simp
linarith [ho.1, hOverlap.1]
· simpa using ho.2.trans (sub_le_self 1 hOverlap.1.le)
haveI : IsProbabilityMeasure (P.assignedSourceLaw n) := hSourceFacts.1
haveI : IsProbabilityMeasure μ := by
unfold μ
rw [← hSourceFacts.2.2.2.1]
exact Measure.isProbabilityMeasure_map measurable_fst.aemeasurable
have hwint : Integrable w μ := by
refine Integrable.of_bound hwmeas.aestronglyMeasurable 1 ?_
filter_upwards [hw0, hwle] with o h0 h1
rw [Real.norm_eq_abs, abs_of_nonneg h0]
exact h1
haveI : IsFiniteMeasure ν := by
unfold ν
exact Measure.isFiniteMeasure_map
((P.assignedSourceLaw n).restrict {q | q.2 = z}) Prod.fst
apply measure_eq_withDensity_of_toReal_setIntegral hwmeas hwint hw0
intro A hA
rw [Measure.map_apply measurable_fst hA]
rw [Measure.restrict_apply (hA.preimage measurable_fst)]
rw [show Prod.fst ⁻¹' A ∩ {q | q.2 = z} =
{q | q.1 ∈ A ∧ q.2 = z} by
ext q
simp]
simpa [μ, w] using hRandom A hA z
Under IV randomization and overlap, the inverse-propensity-weighted contrast over a covariate set equals the source-population integral of the difference between the two full-data outcomes.
Formal statement
hypotheses
P :
n :
ℕ
epsilon :
ℝ
hSourceFacts :
hRandom :
IVRandomization P n
hOverlap :
InstrumentOverlap P n epsilon
F0 F1 :
FullData 𝒳 → ℝ
hF0 :
Measurable F0
hF1 :
Measurable F1
hF0int :
hF1int :
hInt :
Integrable (fun q : AssignedFullData 𝒳 => if q.2 then (1 / P.propensity n (fullX q.1)) * F1 q.1 else (-1 / (1 - P.propensity n (fullX q.1))) * F0 q.1) (P.assignedSourceLaw n)
A :
Set 𝒳
hA :
Proof (Lean source)
lemma ivRandomization_ipw_contrast
(P : TransportedArray 𝒳) (n : ℕ) (epsilon : ℝ)
(hSourceFacts : SourceObservationFacts P n)
(hRandom : IVRandomization P n)
(hOverlap : InstrumentOverlap P n epsilon)
(F0 F1 : FullData 𝒳 → ℝ)
(hF0 : Measurable F0) (hF1 : Measurable F1)
(hF0int : Integrable F0 (populationLaw P n true))
(hF1int : Integrable F1 (populationLaw P n true))
(hInt : Integrable (fun q : AssignedFullData 𝒳 =>
if q.2 then
(1 / P.propensity n (fullX q.1)) * F1 q.1
else
(-1 / (1 - P.propensity n (fullX q.1))) * F0 q.1)
(P.assignedSourceLaw n))
(A : Set 𝒳) (hA : MeasurableSet A) :
(∫ q in {q | fullX q.1 ∈ A},
(if q.2 then
(1 / P.propensity n (fullX q.1)) * F1 q.1
else
(-1 / (1 - P.propensity n (fullX q.1))) * F0 q.1)
∂P.assignedSourceLaw n) =
∫ o in {o | fullX o ∈ A}, (F1 o - F0 o)
∂populationLaw P n true := by
let μ := populationLaw P n true
let e : FullData 𝒳 → ℝ := fun o => P.propensity n (fullX o)
let S : Set (AssignedFullData 𝒳) := {q | fullX q.1 ∈ A}
let E : Set (AssignedFullData 𝒳) := {q | q.2 = true}
let W : AssignedFullData 𝒳 → ℝ := fun q =>
if q.2 then (1 / e q.1) * F1 q.1
else (-1 / (1 - e q.1)) * F0 q.1
have hFullX : Measurable (fullX : FullData 𝒳 → 𝒳) := by
unfold fullX
fun_prop
have hS : MeasurableSet S :=
hA.preimage (hFullX.comp measurable_fst)
have hE : MeasurableSet E :=
measurable_snd (MeasurableSet.singleton true)
have hsplit := integral_add_compl
(μ := (P.assignedSourceLaw n).restrict S) hE hInt.integrableOn
have hemeas : Measurable e :=
(P.propensity_measurable n).comp hFullX
have hoverlapPop : ∀ᵐ o ∂μ,
epsilon ≤ e o ∧ e o ≤ 1 - epsilon := by
have hx := hOverlap.2.2
rw [sourceXLaw_eq_populationXLaw_source P n hSourceFacts] at hx
exact ae_of_ae_map hFullX.aemeasurable hx
have htrue :
(∫ q in S ∩ E, W q ∂P.assignedSourceLaw n) =
∫ o in {o | fullX o ∈ A}, F1 o ∂μ := by
have hslice :=
ivRandomization_slice_eq_withDensity P n epsilon hSourceFacts hRandom
hOverlap true
have hmap :
(∫ o in {o | fullX o ∈ A}, (1 / e o) * F1 o
∂(((P.assignedSourceLaw n).restrict E).map fst)) =
∫ q in S ∩ E, W q ∂P.assignedSourceLaw n := by
calc
_ = ∫ q in fst ⁻¹' {o | fullX o ∈ A},
(1 / e q.1) * F1 q.1
∂(P.assignedSourceLaw n).restrict E :=
setIntegral_map (μ := (P.assignedSourceLaw n).restrict E)
(hA.preimage hFullX)
((measurable_const.div hemeas).mul hF1).aestronglyMeasurable
measurable_fst.aemeasurable
_ = _ := by
change (∫ q, (1 / e q.1) * F1 q.1
∂((P.assignedSourceLaw n).restrict E).restrict
(fst ⁻¹' {o | fullX o ∈ A})) =
∫ q in S ∩ E, W q ∂P.assignedSourceLaw n
rw [Measure.restrict_restrict' hE]
change (∫ q in S ∩ E, (1 / e q.1) * F1 q.1
∂P.assignedSourceLaw n) =
∫ q in S ∩ E, W q ∂P.assignedSourceLaw n
apply integral_congr_ae
filter_upwards [ae_restrict_mem (hS.inter hE)] with q hq
have hqtrue : q.2 = true := hq.2
simp [W, hqtrue]
rw [← hmap, hslice]
change (∫ o in {o | fullX o ∈ A}, (1 / e o) * F1 o
∂μ.withDensity (fun o => ofReal (e o))) =
∫ o in {o | fullX o ∈ A}, F1 o ∂μ
calc
_ = ∫ o in {o | fullX o ∈ A},
(ofReal (e o)).toReal • ((1 / e o) * F1 o) ∂μ :=
setIntegral_withDensity_eq_setIntegral_toReal_smul
(ENNReal.measurable_ofReal.comp hemeas)
(Filter.Eventually.of_forall fun _ => ENNReal.ofReal_lt_top)
(fun o => (1 / e o) * F1 o) (hA.preimage hFullX)
_ = _ := by
apply integral_congr_ae
filter_upwards [ae_restrict_of_ae hoverlapPop] with o ho
have hepos : 0 < e o := lt_of_lt_of_le hOverlap.1 ho.1
simp [ENNReal.toReal_ofReal hepos.le, smul_eq_mul, hepos.ne']
have hfalse :
(∫ q in S ∩ Eᶜ, W q ∂P.assignedSourceLaw n) =
∫ o in {o | fullX o ∈ A}, -F0 o ∂μ := by
have hslice :=
ivRandomization_slice_eq_withDensity P n epsilon hSourceFacts hRandom
hOverlap false
have hmap :
(∫ o in {o | fullX o ∈ A}, (-1 / (1 - e o)) * F0 o
∂(((P.assignedSourceLaw n).restrict {q | q.2 = false}).map
fst)) =
∫ q in S ∩ Eᶜ, W q ∂P.assignedSourceLaw n := by
calc
_ = ∫ q in fst ⁻¹' {o | fullX o ∈ A},
(-1 / (1 - e q.1)) * F0 q.1
∂(P.assignedSourceLaw n).restrict {q | q.2 = false} :=
setIntegral_map
(μ := (P.assignedSourceLaw n).restrict {q | q.2 = false})
(hA.preimage hFullX)
((measurable_const.neg.div (measurable_const.sub hemeas)).mul
hF0).aestronglyMeasurable
measurable_fst.aemeasurable
_ = _ := by
have hEf : {q : AssignedFullData 𝒳 | q.2 = false} = Eᶜ := by
ext q
cases q.2 <;> simp [E]
rw [hEf]
change (∫ q, (-1 / (1 - e q.1)) * F0 q.1
∂((P.assignedSourceLaw n).restrict Eᶜ).restrict
(fst ⁻¹' {o | fullX o ∈ A})) =
∫ q in S ∩ Eᶜ, W q ∂P.assignedSourceLaw n
rw [Measure.restrict_restrict' hE.compl]
change (∫ q in S ∩ Eᶜ, (-1 / (1 - e q.1)) * F0 q.1
∂P.assignedSourceLaw n) =
∫ q in S ∩ Eᶜ, W q ∂P.assignedSourceLaw n
apply integral_congr_ae
filter_upwards [ae_restrict_mem (hS.inter hE.compl)] with q hq
have hqfalse : q.2 = false := by
cases hqz : q.2
· rfl
· exfalso
exact hq.2 (by simp [E, hqz])
simp [W, hqfalse]
rw [← hmap, hslice]
change (∫ o in {o | fullX o ∈ A}, (-1 / (1 - e o)) * F0 o
∂μ.withDensity (fun o => ofReal (1 - e o))) =
∫ o in {o | fullX o ∈ A}, -F0 o ∂μ
calc
_ = ∫ o in {o | fullX o ∈ A},
(ofReal (1 - e o)).toReal •
((-1 / (1 - e o)) * F0 o) ∂μ :=
setIntegral_withDensity_eq_setIntegral_toReal_smul
(ENNReal.measurable_ofReal.comp (measurable_const.sub hemeas))
(Filter.Eventually.of_forall fun _ => ENNReal.ofReal_lt_top)
(fun o => (-1 / (1 - e o)) * F0 o) (hA.preimage hFullX)
_ = _ := by
apply integral_congr_ae
filter_upwards [ae_restrict_of_ae hoverlapPop] with o ho
have hden : 0 < 1 - e o := by
linarith [ho.2, hOverlap.1]
rw [ENNReal.toReal_ofReal hden.le]
simp only [smul_eq_mul]
field_simp [hden.ne']
have hsplitW :
(∫ q in S ∩ E, W q ∂P.assignedSourceLaw n) +
(∫ q in S ∩ Eᶜ, W q ∂P.assignedSourceLaw n) =
∫ q in S, W q ∂P.assignedSourceLaw n := by
rw [Measure.restrict_restrict hE,
Measure.restrict_restrict hE.compl] at hsplit
simpa [S, E, W, e, Set.inter_comm] using hsplit
change (∫ q in S, W q ∂P.assignedSourceLaw n) =
∫ o in {o | fullX o ∈ A}, F1 o - F0 o ∂μ
rw [← hsplitW, htrue, hfalse]
calc
(∫ o in {o | fullX o ∈ A}, F1 o ∂μ) +
∫ o in {o | fullX o ∈ A}, -F0 o ∂μ =
∫ o in {o | fullX o ∈ A}, F1 o + -F0 o ∂μ := by
simpa only [Pi.neg_apply] using
(integral_add hF1int.integrableOn hF0int.neg.integrableOn).symm
_ = _ := by
apply integral_congr_ae
filter_upwards with o
ring
Source randomization identifies the two conditional contrasts; transport identifies their target means; their Wald ratio is the target CACE and belongs to the forced interval [-1,1].
Formal statement
hypotheses
P :
N k :
ℕ → ℕ
c epsilon :
ℝ
n :
ℕ
hAssignmentContrastIntegrable :
hReceiptContrastIntegrable :
hP :
TransportedIVClass P N k c epsilon n
conclusion
OutcomeContrastRepresentation P n (P.deltaY n) ∧
ReceiptContrastRepresentation P n (P.deltaD n) ∧
= ∫ o, (fullY1 o - fullY0 o) * (if fullD1 o = true ∧ fullD0 o = false then 1 else 0) ∂populationLaw P n false ∧
transportedFirstStage P n = targetComplierShare P n ∧
targetCACE P n ∈ parameterSpace
Proof (Lean source)
theorem compact_causal_range
(P : TransportedArray 𝒳) (N k : ℕ → ℕ) (c epsilon : ℝ) (n : ℕ)
(hAssignmentContrastIntegrable :
Integrable (P.assignmentContrast n true) (sourceXLaw P n))
(hReceiptContrastIntegrable :
Integrable (P.receiptContrast n true) (sourceXLaw P n))
(hP : TransportedIVClass P N k c epsilon n) :
OutcomeContrastRepresentation P n (P.deltaY n) ∧
ReceiptContrastRepresentation P n (P.deltaD n) ∧
transportedOutcomeITT P n =
∫ o, (fullY1 o - fullY0 o) *
(if fullD1 o = true ∧ fullD0 o = false then 1 else 0)
∂populationLaw P n false ∧
transportedFirstStage P n = targetComplierShare P n ∧
transportedOutcomeITT P n / transportedFirstStage P n =
targetCACE P n ∧
targetCACE P n ∈ parameterSpace := by
have hSourceFacts :=
sourceObservationFacts_of_class P N k c epsilon n hP
have hSupport := hP.fullDataSupport
have hPresence := hP.populationPresence
have hTwo := hP.twoSampleArray
have hOverlap := hP.instrumentOverlap
have hRandom := hP.ivRandomization
have hExclusion := hP.ivExclusion
have hMono := hP.ivMonotonicity
have hOutcome := hP.outcomeTransport
have hReceipt := hP.receiptTransport
have hPositive := hP.targetComplierPositivity
have hDom := hP.transportDomination
letI : IsProbabilityMeasure (sourceObsLaw P n) := hSourceFacts.2.1
letI : IsProbabilityMeasure (sourceXLaw P n) := by
unfold sourceXLaw
exact Measure.isProbabilityMeasure_map measurable_fst.aemeasurable
have hOverlapObs : ∀ᵐ o ∂sourceObsLaw P n,
epsilon ≤ P.propensity n o.1 ∧
P.propensity n o.1 ≤ 1 - epsilon := by
have hx := hOverlap.2.2
unfold sourceXLaw at hx
exact ae_of_ae_map measurable_fst.aemeasurable hx
have hInstrumentScoreMeasurable :
Measurable (instrumentScore P n) := by
have hz : Measurable fun o : SourceObs 𝒳 => o.2.1 := by
fun_prop
have he : Measurable fun o : SourceObs 𝒳 => P.propensity n o.1 :=
(P.propensity_measurable n).comp measurable_fst
unfold instrumentScore
exact Measurable.ite (hz (MeasurableSet.singleton true))
(measurable_const.div he)
(measurable_const.neg.div (measurable_const.sub he))
have hInstrumentScoreBound : ∀ᵐ o ∂sourceObsLaw P n,
|instrumentScore P n o| ≤ 1 / epsilon := by
filter_upwards [hOverlapObs] with o ho
rcases o with ⟨x, z, d, y⟩
cases z
· simp only [instrumentScore, Bool.false_eq_true, ↓reduceIte, abs_neg,
abs_div, abs_one]
rw [abs_of_pos (sub_pos.mpr (lt_of_le_of_lt ho.2
(by linarith [hOverlap.1])))]
exact one_div_le_one_div_of_le hOverlap.1 (by linarith [ho.2])
· simp only [instrumentScore, ↓reduceIte, abs_div, abs_one]
rw [abs_of_pos (lt_of_lt_of_le hOverlap.1 ho.1)]
exact one_div_le_one_div_of_le hOverlap.1 ho.1
have hScoreIntegrable
(G : SourceObs 𝒳 → ℝ) (hGmeas : Measurable G)
(hGbound : ∀ᵐ o ∂sourceObsLaw P n, |G o| ≤ 1) :
Integrable (fun o => instrumentScore P n o * G o)
(sourceObsLaw P n) := by
refine Integrable.of_bound
((hInstrumentScoreMeasurable.mul hGmeas).aestronglyMeasurable)
(1 / epsilon) ?_
filter_upwards [hInstrumentScoreBound, hGbound] with o hs hG
rw [Real.norm_eq_abs, abs_mul]
calc
|instrumentScore P n o| * |G o| ≤
(1 / epsilon) * 1 :=
mul_le_mul hs hG (abs_nonneg _)
(one_div_nonneg.mpr hOverlap.1.le)
_ = 1 / epsilon := mul_one _
have hOutcomeScoreIntegrable :
Integrable (fun o => instrumentScore P n o * o.2.2.2)
(sourceObsLaw P n) := by
apply hScoreIntegrable (fun o => o.2.2.2) (by fun_prop)
filter_upwards [hSourceFacts.2.2.1] with o ho
rw [abs_of_nonneg ho.1]
exact ho.2
have hReceiptScoreIntegrable :
Integrable (fun o => instrumentScore P n o * boolReal o.2.2.1)
(sourceObsLaw P n) := by
apply hScoreIntegrable (fun o => boolReal o.2.2.1)
(by
unfold boolReal
have hd : Measurable fun o : SourceObs 𝒳 => o.2.2.1 := by
fun_prop
exact Measurable.ite (hd (MeasurableSet.singleton true))
measurable_const measurable_const)
filter_upwards with o
cases o.2.2.1 <;> simp [boolReal]
have hDeltaY :
P.deltaY n =ᵐ[sourceXLaw P n] P.assignmentContrast n true := by
have hDeltaYint : Integrable (P.deltaY n) (sourceXLaw P n) := by
refine Integrable.of_bound
(P.deltaY_measurable n).aestronglyMeasurable 1 ?_
filter_upwards [hSourceFacts.2.2.2.2.2.2.1] with x hx
rw [Real.norm_eq_abs]
exact (abs_le).2 hx
refine Integrable.ae_eq_of_forall_setIntegral_eq
(P.deltaY n) (P.assignmentContrast n true) hDeltaYint
hAssignmentContrastIntegrable ?_
intro A hA hAfin
have hFullX : Measurable (fullX : FullData 𝒳 → 𝒳) := by
unfold fullX
fun_prop
have hDerivedMeas (z : Bool) :
Measurable (fun o : FullData 𝒳 => derivedAssignmentOutcome o z) := by
have hd : Measurable fun o : FullData 𝒳 => potentialReceipt o z := by
cases z
· have hd0 : Measurable (fullD0 : FullData 𝒳 → Bool) := by
unfold fullD0
fun_prop
simpa [potentialReceipt] using hd0
· have hd1 : Measurable (fullD1 : FullData 𝒳 → Bool) := by
unfold fullD1
fun_prop
simpa [potentialReceipt] using hd1
unfold derivedAssignmentOutcome potentialOutcome
have hy0 : Measurable (fullY0 : FullData 𝒳 → ℝ) := by
unfold fullY0
fun_prop
have hy1 : Measurable (fullY1 : FullData 𝒳 → ℝ) := by
unfold fullY1
fun_prop
exact Measurable.ite (hd (MeasurableSet.singleton true)) hy1 hy0
haveI : IsProbabilityMeasure (P.assignedSourceLaw n) := hSourceFacts.1
haveI : IsProbabilityMeasure (populationLaw P n true) := by
rw [← hSourceFacts.2.2.2.1]
exact Measure.isProbabilityMeasure_map measurable_fst.aemeasurable
have hSupportSource : ∀ᵐ o ∂populationLaw P n true,
fullY0 o ∈ Icc (0 : ℝ) 1 ∧ fullY1 o ∈ Icc (0 : ℝ) 1 :=
ProbabilityTheory.cond_absolutelyContinuous.ae_le hSupport.2
have hDerivedBound (z : Bool) :
∀ᵐ o ∂populationLaw P n true,
|derivedAssignmentOutcome o z| ≤ 1 := by
filter_upwards [hSupportSource] with o ho
unfold derivedAssignmentOutcome potentialOutcome potentialReceipt
cases z <;> cases fullD0 o <;> cases fullD1 o <;>
simp [abs_of_nonneg ho.1.1, abs_of_nonneg ho.2.1, ho.1.2, ho.2.2]
have hF0int :
Integrable (fun o : FullData 𝒳 => derivedAssignmentOutcome o false)
(populationLaw P n true) := by
exact Integrable.of_bound (hDerivedMeas false).aestronglyMeasurable 1
(hDerivedBound false)
have hF1int :
Integrable (fun o : FullData 𝒳 => derivedAssignmentOutcome o true)
(populationLaw P n true) := by
exact Integrable.of_bound (hDerivedMeas true).aestronglyMeasurable 1
(hDerivedBound true)
have hpull :
Integrable (fun q : AssignedFullData 𝒳 =>
instrumentScore P n (observeSource q) * (observeSource q).2.2.2)
(P.assignedSourceLaw n) := by
unfold sourceObsLaw at hOutcomeScoreIntegrable
exact hOutcomeScoreIntegrable.comp_measurable measurable_observeSource
have hWint :
Integrable (fun q : AssignedFullData 𝒳 =>
if q.2 then
(1 / P.propensity n (fullX q.1)) *
derivedAssignmentOutcome q.1 true
else
(-1 / (1 - P.propensity n (fullX q.1))) *
derivedAssignmentOutcome q.1 false)
(P.assignedSourceLaw n) := by
apply hpull.congr
filter_upwards with q
cases hz : q.2 <;>
cases hd0 : fullD0 q.1 <;> cases hd1 : fullD1 q.1 <;>
simp [observeSource, instrumentScore, derivedAssignmentOutcome,
potentialOutcome, potentialReceipt, hz, hd0, hd1]
calc
(∫ x in A, P.deltaY n x ∂sourceXLaw P n) =
∫ o in {o | o.1 ∈ A},
instrumentScore P n o * o.2.2.2 ∂sourceObsLaw P n :=
(hSourceFacts.2.2.2.2.2.1 A hA).symm
_ = ∫ q in {q | fullX q.1 ∈ A},
(if q.2 then
(1 / P.propensity n (fullX q.1)) *
derivedAssignmentOutcome q.1 true
else
(-1 / (1 - P.propensity n (fullX q.1))) *
derivedAssignmentOutcome q.1 false)
∂P.assignedSourceLaw n := by
unfold sourceObsLaw
calc
_ = ∫ q in observeSource ⁻¹' {o | o.1 ∈ A},
instrumentScore P n (observeSource q) *
(observeSource q).2.2.2 ∂P.assignedSourceLaw n :=
setIntegral_map (μ := P.assignedSourceLaw n)
(hA.preimage measurable_fst)
(hInstrumentScoreMeasurable.mul (by fun_prop)).aestronglyMeasurable
measurable_observeSource.aemeasurable
_ = _ := by
apply integral_congr_ae
filter_upwards with q
cases hz : q.2 <;>
cases hd0 : fullD0 q.1 <;> cases hd1 : fullD1 q.1 <;>
simp [observeSource, instrumentScore, derivedAssignmentOutcome,
potentialOutcome, potentialReceipt, fullX, hz, hd0, hd1]
_ = ∫ o in {o | fullX o ∈ A},
(derivedAssignmentOutcome o true -
derivedAssignmentOutcome o false) ∂populationLaw P n true :=
ivRandomization_ipw_contrast P n epsilon hSourceFacts hRandom hOverlap
(fun o => derivedAssignmentOutcome o false)
(fun o => derivedAssignmentOutcome o true)
(hDerivedMeas false) (hDerivedMeas true) hF0int hF1int hWint A hA
_ = ∫ o in {o | fullX o ∈ A},
(P.assignmentOutcome n true o -
P.assignmentOutcome n false o) ∂populationLaw P n true := by
apply integral_congr_ae
filter_upwards [ae_restrict_of_ae (hExclusion true true),
ae_restrict_of_ae (hExclusion true false)]
with o h1 h0
rw [h1, h0]
_ = ∫ x in A, P.assignmentContrast n true x ∂sourceXLaw P n := by
rw [sourceXLaw_eq_populationXLaw_source P n hSourceFacts]
simpa using hOutcome.1 true A hA
have hDeltaD :
P.deltaD n =ᵐ[sourceXLaw P n] P.receiptContrast n true := by
have hDeltaDint : Integrable (P.deltaD n) (sourceXLaw P n) := by
refine Integrable.of_bound
(P.deltaD_measurable n).aestronglyMeasurable 1 ?_
filter_upwards [hSourceFacts.2.2.2.2.2.2.2.2] with x hx
rw [Real.norm_eq_abs, abs_of_nonneg hx.1]
exact hx.2
refine Integrable.ae_eq_of_forall_setIntegral_eq
(P.deltaD n) (P.receiptContrast n true) hDeltaDint
hReceiptContrastIntegrable ?_
intro A hA hAfin
have hFullX : Measurable (fullX : FullData 𝒳 → 𝒳) := by
unfold fullX
fun_prop
have hD0meas : Measurable fun o : FullData 𝒳 => boolReal (fullD0 o) := by
unfold boolReal
have hd : Measurable (fullD0 : FullData 𝒳 → Bool) := by
unfold fullD0
fun_prop
exact Measurable.ite (hd (MeasurableSet.singleton true))
measurable_const measurable_const
-- … truncated; follow the source link for the rest …
T_FiniteCellUnknownWeightAttainment 1 declarations
The uniform finite-cell submodel's sample-only collision-scaled rule is honest and matches the oracle converse order, on the same arbitrary measurable carrier as the regular-cell and global results.
Formal statement
hypotheses
N k :
ℕ → ℕ
c epsilon alpha :
ℝ
hc :
0 < c
@realizes c(c∈(0,∞))
hepsilon :
0 < epsilon ∧ epsilon < 1 / 2
halpha :
0 < alpha ∧ alpha < 1
@realizes \alpha(noncoverage in (0,1))
@realizes N_n(N_n/n→c)
hkPos :
∀ n, 0 < k n
@realizes k_n(positive, diverging, and o(√n))
hCarrier :
∀ n, ∃ cell : Fin (k n) ↪ 𝒳, ∀ i, MeasurableSet {cell i}
hTwo :
∀ n (P : TransportedArray 𝒳)
if
FiniteCellClass P N k c epsilon n
then
TwoSampleArray P N c
hCell :
∀ n (P : TransportedArray 𝒳)
if
FiniteCellClass P N k c epsilon n
then
FiniteCellSource P k n
hDegrade :
∀ n (P : TransportedArray 𝒳)
if
FiniteCellClass P N k c epsilon n
then
DegradingArray P k
conclusion
let Bc := 32 * (1 + c⁻¹) let L := sqrt (2 * Bc / alpha) let C0 := max 2 (4 * sqrt 2 * L + 8 * Bc) 0 < C0 ∧
∃ C : FiniteCellProcedure 𝒳 N k,
(∀ n (P : TransportedArray 𝒳), FiniteCellClass P N k c epsilon n → ∀ᵐ s ∂(twoSampleLaw P N n), C.set n s = finiteCellInversion (k n) n (N n) L s.1 s.2) ∧
FiniteCellHonest N k c epsilon alpha C ∧
(∀ t0 : ℝ, 0 < t0 → feasibleFiniteCellRisk N k c epsilon C t0 ≤ C0 * min 1 (t0 ^ (-1 / 2 : ℝ))) ∧
(∀ t0 : ℝ, 0 < t0 → 3 * (1 - alpha) ^ 2 / 16 * min 1 (t0 ^ (-1 / 2 : ℝ)) ≤ ⨅ C : {C : FiniteCellProcedure 𝒳 N k // FiniteCellHonest N k c epsilon alpha C}, feasibleFiniteCellRisk N k c epsilon C.1 t0)
Proof (Lean source)
theorem finite_cell_unknown_weight_attainment
(N k : ℕ → ℕ) (c epsilon alpha : ℝ)
(hc : 0 < c) -- @realizes c(c∈(0,∞))
(hepsilon : 0 < epsilon ∧ epsilon < 1 / 2)
(halpha : 0 < alpha ∧ alpha < 1)
-- @realizes \alpha(noncoverage in (0,1))
(hN : Tendsto (fun n : ℕ => (N n : ℝ) / (n : ℝ)) atTop (𝓝 c))
-- @realizes N_n(N_n/n→c)
(hkPos : ∀ n, 0 < k n)
(hkInf : Tendsto (fun n : ℕ => (k n : ℝ)) atTop atTop)
(hkRoot : Tendsto (fun n : ℕ =>
(k n : ℝ) / sqrt n) atTop (𝓝 0))
-- @realizes k_n(positive, diverging, and o(√n))
(hCarrier : ∀ n, ∃ cell : Fin (k n) ↪ 𝒳, ∀ i, MeasurableSet {cell i})
(hTwo : ∀ n (P : TransportedArray 𝒳),
FiniteCellClass P N k c epsilon n →
TwoSampleArray P N c)
(hCell : ∀ n (P : TransportedArray 𝒳),
FiniteCellClass P N k c epsilon n →
FiniteCellSource P k n)
(hDegrade : ∀ n (P : TransportedArray 𝒳),
FiniteCellClass P N k c epsilon n →
DegradingArray P k) :
let Bc := 32 * (1 + c⁻¹)
let L := sqrt (2 * Bc / alpha)
let C0 := max 2 (4 * sqrt 2 * L + 8 * Bc)
-- @realizes C_0(constructive finite-cell upper constant)
0 < C0 ∧
∃ C : FiniteCellProcedure 𝒳 N k,
(∀ n (P : TransportedArray 𝒳), FiniteCellClass P N k c epsilon n →
∀ᵐ s ∂(twoSampleLaw P N n),
C.set n s = finiteCellInversion (k n) n (N n) L s.1 s.2) ∧
FiniteCellHonest N k c epsilon alpha C ∧
(∀ t0 : ℝ, 0 < t0 → -- @realizes t_0(positive frontier threshold)
feasibleFiniteCellRisk N k c epsilon C t0 ≤
C0 *
min 1 (t0 ^ (-1 / 2 : ℝ))) ∧
(∀ t0 : ℝ, 0 < t0 → -- @realizes t_0(positive frontier threshold)
3 * (1 - alpha) ^ 2 / 16 *
min 1 (t0 ^ (-1 / 2 : ℝ)) ≤
⨅ C : {C : FiniteCellProcedure 𝒳 N k //
FiniteCellHonest N k c epsilon alpha C},
feasibleFiniteCellRisk N k c epsilon C.1 t0) := by
have hClass :=
finiteCellClass_inhabited N k c epsilon hCarrier hc hepsilon hN hkPos hkInf
hkRoot
dsimp
classical
let Bc : ℝ := 32 * (1 + c⁻¹)
let L : ℝ := sqrt (2 * Bc / alpha)
let C0 : ℝ := max 2 (4 * sqrt 2 * L + 8 * Bc)
let x : ℝ := 4 + 3 / c
let a : ℝ := (sqrt x)⁻¹
let epsilonBar : ℝ := 1 / 4 + (1 / 2) * a
have hx : 4 < x := by
dsimp [x]
have hdiv : 0 < 3 / c := div_pos (by norm_num) hc
nlinarith
have hsqrtSq : (sqrt x) ^ 2 = x :=
Real.sq_sqrt (le_trans (by norm_num) hx.le)
have hsqrtPos : 0 < sqrt x := by positivity
have hsqrtTwo : 2 < sqrt x := by
nlinarith [Real.sqrt_nonneg x]
have haPos : 0 < a := by
dsimp [a]
positivity
have haHalf : a < 1 / 2 := by
dsimp [a]
rw [inv_eq_one_div, div_lt_iff₀ hsqrtPos]
nlinarith
have hBar : 0 < epsilonBar ∧ epsilonBar < 1 / 2 := by
dsimp [epsilonBar]
constructor <;> nlinarith
have haMul : a * sqrt x = 1 := by
dsimp [a]
exact inv_mul_cancel₀ hsqrtPos.ne'
have hepsSqrt : 1 ≤ epsilonBar * sqrt x := by
have hsqrtNonneg := Real.sqrt_nonneg x
dsimp [epsilonBar]
nlinarith
have hinvLeSqrt : epsilonBar⁻¹ ≤ sqrt x := by
rw [inv_eq_one_div, div_le_iff₀ hBar.1]
simpa [mul_comm] using hepsSqrt
have hinvSq : epsilonBar⁻¹ ^ 2 ≤ x := by
calc
epsilonBar⁻¹ ^ 2 ≤ (sqrt x) ^ 2 := by
gcongr
_ = x := hsqrtSq
let B : ℝ := 8 * (epsilonBar⁻¹ ^ 2 + c⁻¹)
have hBnonneg : 0 ≤ B := by
dsimp [B]
positivity
have hBLe : B ≤ Bc := by
dsimp [B, Bc, x] at hinvSq ⊢
rw [div_eq_mul_inv] at hinvSq
nlinarith
have hBcPos : 0 < Bc := by
dsimp [Bc]
have hcinv : 0 < c⁻¹ := inv_pos.mpr hc
positivity
have hLreg : sqrt (2 * B / alpha) ≤ L := by
dsimp [L]
apply Real.sqrt_le_sqrt
have halphaPos := halpha.1
exact div_le_div_of_nonneg_right
(mul_le_mul_of_nonneg_left hBLe (by norm_num)) halphaPos.le
have hreg :=
regular_cell_unknown_weight_attainment
(𝒳 := 𝒳) N k c epsilonBar alpha 1 1 hc hBar halpha
⟨by norm_num, by norm_num⟩ (by norm_num) hN hkPos hkInf hkRoot hCarrier
(fun _ _ hP => hP.1.twoSampleArray)
(fun _ _ hP => hP.1.instrumentOverlap)
(fun _ _ hP => hP.1.degradingArray)
dsimp only at hreg
obtain ⟨hCregPos, Creg, hCregSet, hCregHonest, hCregRisk, _⟩ :=
hreg L hLreg
have hupperBridge :=
finiteCellProcedure_upper_bridge
(𝒳 := 𝒳) N k c epsilon epsilonBar alpha L hBar hClass
Creg (by
intro n P hP
filter_upwards
[finiteCellProcedure_eq_regularCellSet_ae N k L n P hP,
hCregSet n P (hP.toRegularFiniteCellClass hBar),
regularCellProcedure_set_eq_ambient_ae L n P
(hP.toRegularFiniteCellClass hBar)]
with s hfin hreg hproc
exact hfin.trans (hproc.trans hreg.symm))
hCregHonest
have hC0pos : 0 < C0 :=
lt_of_lt_of_le (by norm_num) (le_max_left _ _)
refine ⟨hC0pos, finiteCellProcedure (𝒳 := 𝒳) N k L, ?_, ?_,
?_, ?_⟩
· intro n P hP
filter_upwards with s
rfl
· exact hupperBridge.1
· intro t0 ht0
have hrisk :
regularCellRisk N k c epsilonBar 1 1 Creg t0 ≤
max 2 (4 * sqrt 2 * L + 8 * B) *
min 1 (t0 ^ (-1 / 2 : ℝ)) := by
simpa [B] using hCregRisk t0 ht0
have hconst :
max 2 (4 * sqrt 2 * L + 8 * B) ≤ C0 := by
dsimp [C0]
apply max_le_max le_rfl
gcongr
have hminNonneg :
0 ≤ min 1 (t0 ^ (-1 / 2 : ℝ)) :=
le_min (by norm_num) (Real.rpow_nonneg ht0.le _)
exact (hupperBridge.2 t0).trans
(hrisk.trans (mul_le_mul_of_nonneg_right hconst hminNonneg))
· intro t0 ht0
choose cell hcell using hCarrier
let μ : ℕ → Measure 𝒳 := fun n =>
∑ i : Fin (k n), (k n : ENNReal)⁻¹ • Measure.dirac (cell n i)
have hkinv (n : ℕ) :
(k n : ENNReal) * (k n : ENNReal)⁻¹ = 1 :=
ENNReal.mul_inv_cancel
(Nat.cast_ne_zero.mpr (Nat.ne_of_gt (hkPos n)))
(ENNReal.natCast_ne_top (k n))
have hμ : ∀ n, IsProbabilityMeasure (μ n) := by
intro n
rw [isProbabilityMeasure_iff]
simp [μ, hkinv n]
let g : Geometry 𝒳 := {
sourceX := μ
targetX := μ
weight := fun _ _ => 1
propensity := fun _ _ => 1 / 2
weight_measurable := fun _ => measurable_const
propensity_measurable := fun _ => measurable_const
}
have hg : AdmissibleGeometry g k epsilon := by
refine ⟨hμ, hμ, hepsilon.1, hepsilon.2, ?_, ?_, ?_, ?_, ?_⟩
· intro n x
dsimp [g]
constructor
· exact hepsilon.2.le
· linarith [hepsilon.2]
· intro n x
dsimp [g]
constructor
· norm_num
· have hk1 : (1 : ℝ) ≤ k n := by exact_mod_cast hkPos n
linarith
· intro n
letI : IsProbabilityMeasure (μ n) := hμ n
simp [g]
· intro n
letI : IsProbabilityMeasure (μ n) := hμ n
have hk1 : (1 : ℝ) ≤ k n := by exact_mod_cast hkPos n
simpa [g] using hk1
· intro n A hA
letI : IsProbabilityMeasure (μ n) := hμ n
simpa [g, measureReal_def] using
(ofReal_setIntegral_one (μ n) A).symm
have hrange (n : ℕ) : μ n (range (cell n)) = 1 := by
rw [show range (cell n) = ⋃ i, {cell n i} by
ext y
simp]
simp only [μ, Measure.finset_sum_apply, Measure.smul_apply]
simp [Measure.dirac_apply_of_mem, hkinv n]
have hatom (n : ℕ) (i : Fin (k n)) :
(μ n {cell n i}).toReal = (k n : ℝ)⁻¹ := by
simp only [μ, Measure.finset_sum_apply, Measure.smul_apply]
rw [Finset.sum_eq_single i]
· simp [ENNReal.toReal_inv]
· intro j _ hji
have hne : cell n j ≠ cell n i :=
fun h => hji ((cell n).injective h)
rw [Measure.dirac_apply' _ (hcell n i)]
simp [hne]
· simp
have hsliceFinite :
∀ n P, fixedGeometrySlice P g N k c epsilon n →
FiniteCellClass P N k c epsilon n := by
intro n P hP
have hsource : FiniteCellSource P k n := by
refine ⟨hkPos n, ?_, cell n, hcell n, ?_, ?_, ?_, ?_⟩
· rw [hP.2.1]
exact hμ n
· rw [hP.2.1]
exact hrange n
· intro i
rw [hP.2.1]
exact hatom n i
· rw [hP.2.1]
rfl
· filter_upwards with y
rw [hP.2.2.2.2, show g.propensity n y = 1 / 2 by rfl]
exact { hP.1 with finiteCellSource := hsource }
have hrangeMeas (n : ℕ) : MeasurableSet (range (cell n)) := by
rw [show range (cell n) = ⋃ i, {cell n i} by
ext y
simp]
exact MeasurableSet.iUnion (hcell n)
let sampleSupport (n : ℕ) (s : TwoSample 𝒳 n (N n)) : Prop :=
(∀ i, (s.1 i).1 ∈ range (cell n)) ∧
∀ j, s.2 j ∈ range (cell n)
have hsampleSupportMeas (n : ℕ) :
MeasurableSet {s : TwoSample 𝒳 n (N n) | sampleSupport n s} := by
rw [show {s : TwoSample 𝒳 n (N n) | sampleSupport n s} =
(⋂ i, {s | (s.1 i).1 ∈ range (cell n)}) ∩
⋂ j, {s | s.2 j ∈ range (cell n)} by
ext s
-- … truncated; follow the source link for the rest …
T_FixedGeometryFrontier 1 declarations
For every admissible deterministic geometry, the conditional honest expected-length frontier has order min(1,t0⁻¹/²), with constants independent of the geometry.
Formal statement
hypotheses
N k :
ℕ → ℕ
c epsilon alpha :
ℝ
g :
Geometry 𝒳
hc :
0 < c
@realizes c(c∈(0,∞))
@realizes N_n(N_n/n→c)
hkPos :
∀ n, 0 < k n
@realizes k_n(positive, diverging, and o(√n))
hepsilon :
0 < epsilon ∧ epsilon < 1 / 2
hg :
AdmissibleGeometry g k epsilon
hTwo :
∀ n P
if
fixedGeometrySlice P g N k c epsilon n
then
TwoSampleArray P N c
hOverlap :
∀ n P
if
fixedGeometrySlice P g N k c epsilon n
then
InstrumentOverlap P n epsilon
hEnvelope :
∀ n P
if
fixedGeometrySlice P g N k c epsilon n
then
WeightEnvelope P k n
hSecond :
∀ n P
if
fixedGeometrySlice P g N k c epsilon n
then
WeightSecondMoment P k n
hDegrade :
∀ n P
if
fixedGeometrySlice P g N k c epsilon n
then
DegradingArray P k
hAlpha :
0 < alpha ∧ alpha < 1
conclusion
∀ (t0 : ℝ) (ht0 : 0 < t0),
let Lalpha := sqrt (8 / (alpha * epsilon ^ 2)) let calpha := 3 * (1 - alpha) ^ 2 / 16 let Calpha := max 2 (4 * Lalpha + 8 / epsilon ^ 2) calpha * min 1 (t0 ^ (-1 / 2 : ℝ))
≤ fixedGeometryValue N k c epsilon alpha ⟨g, hg⟩ ⟨t0, ht0⟩ ∧
fixedGeometryValue N k c epsilon alpha ⟨g, hg⟩ ⟨t0, ht0⟩
≤ Calpha * min 1 (t0 ^ (-1 / 2 : ℝ))
Proof (Lean source)
theorem fixed_geometry_frontier
(N k : ℕ → ℕ) (c epsilon alpha : ℝ) (g : Geometry 𝒳)
(hc : 0 < c) -- @realizes c(c∈(0,∞))
(hN : Tendsto (fun n : ℕ => (N n : ℝ) / (n : ℝ)) atTop (𝓝 c))
-- @realizes N_n(N_n/n→c)
(hkPos : ∀ n, 0 < k n)
(hkInf : Tendsto (fun n : ℕ => (k n : ℝ)) atTop atTop)
(hkRoot : Tendsto (fun n : ℕ =>
(k n : ℝ) / sqrt n) atTop (𝓝 0))
-- @realizes k_n(positive, diverging, and o(√n))
(hepsilon : 0 < epsilon ∧ epsilon < 1 / 2)
(hg : AdmissibleGeometry g k epsilon)
(hTwo : ∀ n P, fixedGeometrySlice P g N k c epsilon n →
TwoSampleArray P N c)
(hOverlap : ∀ n P, fixedGeometrySlice P g N k c epsilon n →
InstrumentOverlap P n epsilon)
(hEnvelope : ∀ n P, fixedGeometrySlice P g N k c epsilon n →
WeightEnvelope P k n)
(hSecond : ∀ n P, fixedGeometrySlice P g N k c epsilon n →
WeightSecondMoment P k n)
(hDegrade : ∀ n P, fixedGeometrySlice P g N k c epsilon n →
DegradingArray P k)
(hAlpha : 0 < alpha ∧ alpha < 1) :
-- @realizes \alpha(noncoverage in (0,1))
∀ (t0 : ℝ) (ht0 : 0 < t0), -- @realizes t_0(positive frontier threshold)
let Lalpha := sqrt (8 / (alpha * epsilon ^ 2))
let calpha := 3 * (1 - alpha) ^ 2 / 16
let Calpha := max 2 (4 * Lalpha + 8 / epsilon ^ 2)
calpha * min 1 (t0 ^ (-1 / 2 : ℝ)) ≤
fixedGeometryValue N k c epsilon alpha ⟨g, hg⟩ ⟨t0, ht0⟩ ∧
fixedGeometryValue N k c epsilon alpha ⟨g, hg⟩ ⟨t0, ht0⟩ ≤
Calpha * min 1 (t0 ^ (-1 / 2 : ℝ)) := by
intro t0 ht0
have hSlice :=
fixedGeometrySlice_eventually_inhabited g N k c epsilon t0 hc
hepsilon ht0 hg hN hkPos hkInf hkRoot
refine ⟨?_, ?_⟩
· classical
let rho : ℝ := (1 - alpha) / 8
let H : ℝ := min (1 / 4) (rho * t0 ^ (-1 / 2 : ℝ))
have hrho : rho = (1 - alpha) / 8 := rfl
have hrhoPos : 0 < rho := by
unfold rho
exact div_pos (sub_pos.mpr hAlpha.2) (by norm_num)
have hrpowPos : 0 < t0 ^ (-1 / 2 : ℝ) :=
Real.rpow_pos_of_pos ht0 _
have hHpos : 0 < H := by
unfold H
exact lt_min (by norm_num) (mul_pos hrhoPos hrpowPos)
have hHandle :=
geometryHandle_eventually_inhabited g N k c epsilon alpha t0 rho
hc hepsilon hAlpha ht0 hrho hg hN hkPos hkInf hkRoot
have hRiskLower : ∀ D : OracleProcedure 𝒳 N k c epsilon,
FixedGeometryOracleHonest N k c epsilon alpha ⟨g, hg⟩ D →
2 * H * (1 - alpha - 2 * rho) ≤
fixedGeometryRisk N k c epsilon g D t0 := by
intro D hD
let covRow : ℕ → ℝ := fun n =>
⨅ P : {P : TransportedArray 𝒳 //
fixedGeometrySlice P g N k c epsilon n},
fixedGeometryOracleCoverage D g P n
let riskRow : ℕ → ℝ := fun n =>
⨆ P : {P : TransportedArray 𝒳 //
fixedGeometrySlice P g N k c epsilon n ∧
t0 ≤ effectiveStrength P n},
fixedGeometryOracleExpectedLength D g P n
have hcoverageRange : ∀ n P,
fixedGeometrySlice P g N k c epsilon n →
0 ≤ fixedGeometryOracleCoverage D g P n ∧
fixedGeometryOracleCoverage D g P n ≤ 1 := by
intro n P hP
letI : IsProbabilityMeasure (sourceObsLaw P n) :=
hP.1.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (targetXLaw P n) :=
hP.1.twoSampleArray.2.2.1 n
letI : IsProbabilityMeasure (twoSampleLaw P N n) := by
unfold twoSampleLaw
infer_instance
exact ⟨ENNReal.toReal_nonneg, by
simpa [fixedGeometryOracleCoverage, Measure.real] using
(measureReal_le_one (μ := twoSampleLaw P N n))⟩
have hcovNonneg : ∀ n, 0 ≤ covRow n := by
intro n
let I := {P : TransportedArray 𝒳 //
fixedGeometrySlice P g N k c epsilon n}
cases isEmpty_or_nonempty I with
| inl hEmpty =>
letI : IsEmpty I := hEmpty
simp [covRow, I]
| inr hNonempty =>
letI : Nonempty I := hNonempty
apply le_ciInf
intro P
exact (hcoverageRange n P P.2).1
have hcovOne : ∀ n, covRow n ≤ 1 := by
intro n
let I := {P : TransportedArray 𝒳 //
fixedGeometrySlice P g N k c epsilon n}
cases isEmpty_or_nonempty I with
| inl hEmpty =>
letI : IsEmpty I := hEmpty
simp [covRow, I]
| inr hNonempty =>
letI : Nonempty I := hNonempty
let P : I := Classical.choice hNonempty
have hbdd : BddBelow (Set.range fun Q : I =>
fixedGeometryOracleCoverage D g Q n) :=
⟨0, by
rintro y ⟨Q, rfl⟩
exact (hcoverageRange n Q Q.2).1⟩
exact (ciInf_le hbdd P).trans
(hcoverageRange n P P.2).2
have hsetLengthTwo : ∀ A : Set ℝ, setLength A ≤ 2 := by
intro A
unfold setLength parameterSpace
calc
(volume (A ∩ Icc (-1 : ℝ) 1)).toReal ≤
(volume (Icc (-1 : ℝ) 1)).toReal :=
ENNReal.toReal_mono (by simp [Real.volume_Icc])
(measure_mono Set.inter_subset_right)
_ = 2 := by
rw [Real.volume_Icc, ENNReal.toReal_ofReal (by norm_num)]
norm_num
have hExpectedTwo : ∀ n P,
fixedGeometrySlice P g N k c epsilon n →
fixedGeometryOracleExpectedLength D g P n ≤ 2 := by
intro n P hP
letI : IsProbabilityMeasure (sourceObsLaw P n) :=
hP.1.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (targetXLaw P n) :=
hP.1.twoSampleArray.2.2.1 n
letI : IsProbabilityMeasure (twoSampleLaw P N n) := by
unfold twoSampleLaw
infer_instance
exact (integral_mono_of_nonneg
(Filter.Eventually.of_forall fun _ => ENNReal.toReal_nonneg)
(integrable_const 2)
(Filter.Eventually.of_forall fun s =>
hsetLengthTwo (fixedGeometryOracleSet D g n s))).trans_eq (by simp)
have hriskNonneg : ∀ n, 0 ≤ riskRow n := by
intro n
let I := {P : TransportedArray 𝒳 //
fixedGeometrySlice P g N k c epsilon n ∧
t0 ≤ effectiveStrength P n}
cases isEmpty_or_nonempty I with
| inl hEmpty =>
letI : IsEmpty I := hEmpty
simp [riskRow, I]
| inr hNonempty =>
letI : Nonempty I := hNonempty
let P : I := Classical.choice hNonempty
have hbdd : BddAbove (Set.range fun Q : I =>
fixedGeometryOracleExpectedLength D g Q n) :=
⟨2, by
rintro y ⟨Q, rfl⟩
exact hExpectedTwo n Q Q.2.1⟩
exact (integral_nonneg fun _ => ENNReal.toReal_nonneg).trans
(le_ciSup hbdd P)
have hriskTwo : ∀ n, riskRow n ≤ 2 := by
intro n
let I := {P : TransportedArray 𝒳 //
fixedGeometrySlice P g N k c epsilon n ∧
t0 ≤ effectiveStrength P n}
cases isEmpty_or_nonempty I with
| inl hEmpty =>
letI : IsEmpty I := hEmpty
simp [riskRow, I]
| inr hNonempty =>
letI : Nonempty I := hNonempty
apply ciSup_le
intro P
exact hExpectedTwo n P P.2.1
have hpoint : ∀ᶠ n in atTop,
2 * H * (covRow n - 2 * rho) ≤ riskRow n := by
filter_upwards [hHandle] with n hn
rcases hn with
⟨_hg, _ha0, _ha1, _ht, _hr, _hcomp, _hfirst, _hstrength,
Q, hQ, _hchi, hTV⟩
let I : Set ℝ := Icc (1 / 2 - H) (1 / 2 + H)
have hHle : H ≤ 1 / 4 := by
exact min_le_left _ _
have hI : MeasurableSet I := measurableSet_Icc
have hIsub : I ⊆ parameterSpace := by
intro u hu
change -1 ≤ u ∧ u ≤ 1
change 1 / 2 - H ≤ u ∧ u ≤ 1 / 2 + H at hu
constructor <;> linarith
have hshift (u : ℝ) (hu : u ∈ I) :
|u - 1 / 2| ≤ H := by
change 1 / 2 - H ≤ u ∧ u ≤ 1 / 2 + H at hu
rw [abs_le]
constructor <;> linarith
have hu0 (u : ℝ) (hu : u ∈ I) : u ≠ 0 := by
change 1 / 2 - H ≤ u ∧ u ≤ 1 / 2 + H at hu
intro hueq
subst u
linarith
have hwitness (u : ℝ) (hu : u ∈ I) :=
hQ (u - 1 / 2) (hshift u hu)
have hslice (u : ℝ) (hu : u ∈ I) :
fixedGeometrySlice (Q (u - 1 / 2)) g N k c epsilon n :=
(hwitness u hu).2.2.2.2.2.2.1
have htarget (u : ℝ) (hu : u ∈ I) :
targetCACE (Q (u - 1 / 2)) n = u := by
have h :=
(hwitness u hu).2.2.2.2.2.2.2.2.2.2.1
linarith
have hprobLocal (u : ℝ) (hu : u ∈ I) :
IsProbabilityMeasure
(twoSampleLaw (Q (u - 1 / 2)) N n) := by
letI : IsProbabilityMeasure
(sourceObsLaw (Q (u - 1 / 2)) n) :=
(hslice u hu).1.twoSampleArray.2.1 n
letI : IsProbabilityMeasure
(targetXLaw (Q (u - 1 / 2)) n) :=
(hslice u hu).1.twoSampleArray.2.2.1 n
unfold twoSampleLaw
infer_instance
have hzero : |(0 : ℝ)| ≤ H := by
simpa using hHpos.le
have hwitnessZero := hQ 0 hzero
have hsliceZero :
fixedGeometrySlice (Q 0) g N k c epsilon n :=
hwitnessZero.2.2.2.2.2.2.1
have hstrengthZero : effectiveStrength (Q 0) n = t0 :=
hwitnessZero.2.2.2.2.2.2.2.2.2.1
have hprobZero :
IsProbabilityMeasure (twoSampleLaw (Q 0) N n) := by
letI : IsProbabilityMeasure (sourceObsLaw (Q 0) n) :=
hsliceZero.1.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (targetXLaw (Q 0) n) :=
hsliceZero.1.twoSampleArray.2.2.1 n
unfold twoSampleLaw
infer_instance
let Qm : ℝ → Measure (TwoSample 𝒳 n (N n)) := fun u =>
if u = 0 then twoSampleLaw (Q 0) N n
else if u ∈ I then twoSampleLaw (Q (u - 1 / 2)) N n
else twoSampleLaw (Q 0) N n
have hQmProb : ∀ u, IsProbabilityMeasure (Qm u) := by
intro u
by_cases hz : u = 0
· simpa [Qm, hz] using hprobZero
by_cases hu : u ∈ I
· simpa [Qm, hz, hu] using hprobLocal u hu
· simpa [Qm, hz, hu] using hprobZero
let Cset : TwoSample 𝒳 n (N n) → Set ℝ := fun s =>
fixedGeometryOracleSet D g n s
-- … truncated; follow the source link for the rest …
T_NoShiftReduction 1 declarations
If the transport weight is one, the target and source covariate laws agree, Kish dispersion is one, and effective strength reduces to n μ_n². The conditional frontier is therefore the compact single-population weak-ratio frontier.
Formal statement
hypotheses
N k :
ℕ → ℕ
c epsilon alpha :
ℝ
g :
Geometry 𝒳
hc :
0 < c
hepsilon :
0 < epsilon ∧ epsilon < 1 / 2
halpha :
0 < alpha ∧ alpha < 1
@realizes \alpha(noncoverage in (0,1))
@realizes N_n(N_n/n→c)
hkPos :
∀ n, 0 < k n
@realizes k_n(positive, diverging, and o(√n))
hg :
AdmissibleGeometry g k epsilon
hw :
∀ n, g.weight n =ᵐ[g.sourceX n] fun _ => 1
conclusion
(∀ n, geometryKish g n = 1 ∧ g.targetX n = g.sourceX n) ∧
(∀ (P : TransportedArray 𝒳) n, fixedGeometrySlice P g N k c epsilon n → effectiveStrength P n = (n : ℝ) * transportedFirstStage P n ^ 2) ∧
(∀ (t0 : ℝ) (ht0 : 0 < t0), let Lalpha := sqrt (8 / (alpha * epsilon ^ 2)) let calpha := 3 * (1 - alpha) ^ 2 / 16 let Calpha := max 2 (4 * Lalpha + 8 / epsilon ^ 2) calpha * min 1 (t0 ^ (-1 / 2 : ℝ)) ≤ fixedGeometryValue N k c epsilon alpha ⟨g, hg⟩ ⟨t0, ht0⟩ ∧ fixedGeometryValue N k c epsilon alpha ⟨g, hg⟩ ⟨t0, ht0⟩ ≤ Calpha * min 1 (t0 ^ (-1 / 2 : ℝ)))
Proof (Lean source)
theorem no_shift_reduction
(N k : ℕ → ℕ) (c epsilon alpha : ℝ) (g : Geometry 𝒳)
(hc : 0 < c)
(hepsilon : 0 < epsilon ∧ epsilon < 1 / 2)
(halpha : 0 < alpha ∧ alpha < 1)
-- @realizes \alpha(noncoverage in (0,1))
(hN : Tendsto (fun n : ℕ => (N n : ℝ) / (n : ℝ)) atTop (𝓝 c))
-- @realizes N_n(N_n/n→c)
(hkPos : ∀ n, 0 < k n)
(hkInf : Tendsto (fun n : ℕ => (k n : ℝ)) atTop atTop)
(hkRoot : Tendsto (fun n : ℕ =>
(k n : ℝ) / sqrt n) atTop (𝓝 0))
-- @realizes k_n(positive, diverging, and o(√n))
(hg : AdmissibleGeometry g k epsilon)
(hw : ∀ n, g.weight n =ᵐ[g.sourceX n] fun _ => 1) :
(∀ n, geometryKish g n = 1 ∧ g.targetX n = g.sourceX n) ∧
(∀ (P : TransportedArray 𝒳) n,
fixedGeometrySlice P g N k c epsilon n →
effectiveStrength P n =
(n : ℝ) * transportedFirstStage P n ^ 2) ∧
(∀ (t0 : ℝ) (ht0 : 0 < t0), -- @realizes t_0(positive frontier threshold)
let Lalpha := sqrt (8 / (alpha * epsilon ^ 2))
let calpha := 3 * (1 - alpha) ^ 2 / 16
let Calpha := max 2 (4 * Lalpha + 8 / epsilon ^ 2)
calpha * min 1 (t0 ^ (-1 / 2 : ℝ)) ≤
fixedGeometryValue N k c epsilon alpha ⟨g, hg⟩ ⟨t0, ht0⟩ ∧
fixedGeometryValue N k c epsilon alpha ⟨g, hg⟩ ⟨t0, ht0⟩ ≤
Calpha * min 1 (t0 ^ (-1 / 2 : ℝ))) := by
have hg' := hg
rcases hg' with ⟨hSourceProb, hTargetProb, heps0, heps1, hprop,
hweight, hweightMean, hweightSecond, hchange⟩
have hGeom :
∀ n, geometryKish g n = 1 ∧ g.targetX n = g.sourceX n := by
intro n
letI := hSourceProb n
constructor
· rw [geometryKish]
calc
∫ x, g.weight n x ^ 2 ∂g.sourceX n =
∫ _x, (1 : ℝ) ∂g.sourceX n := by
apply integral_congr_ae
filter_upwards [hw n] with x hx
simp [hx]
_ = 1 := by simp
· ext A hA
rw [hchange n A hA]
have hrestr :
g.weight n =ᵐ[(g.sourceX n).restrict A] fun _ => 1 :=
ae_restrict_of_ae (hw n)
rw [integral_congr_ae hrestr]
simp [measureReal_def]
refine ⟨hGeom, ?_, ?_⟩
· intro P n hslice
have hkish : kishDispersion P n = geometryKish g n := by
have hwP := hslice.2.2.2.1
rw [hslice.2.1] at hwP
rw [kishDispersion, geometryKish, hslice.2.1]
apply integral_congr_ae
filter_upwards [hwP] with x hx
rw [hx]
rw [effectiveStrength, hkish, (hGeom n).1]
ring
· intro t0 ht0
apply fixed_geometry_frontier N k c epsilon alpha g hc hN hkPos
hkInf hkRoot hepsilon hg
· intro n P hP
exact hP.1.twoSampleArray
· intro n P hP
exact hP.1.instrumentOverlap
· intro n P hP
exact hP.1.weightEnvelope
· intro n P hP
exact hP.1.weightSecondMoment
· intro n P hP
exact hP.1.degradingArray
· exact halpha
T_OracleConverse 1 declarations
Universal lower-frontier constants apply both globally and to the finite-cell oracle submodel and are invariant to the decomposition of effective strength into first stage and Kish dispersion.
Formal statement
hypotheses
epsilon alpha c :
ℝ
hc :
0 < c
hepsilon :
0 < epsilon ∧ epsilon < 1 / 2
halpha :
0 < alpha ∧ alpha < 1
conclusion
∃ c0 tc : ℝ,
0 < c0 ∧
0 < tc ∧
∀ {𝒳 : Type*} [MeasurableSpace 𝒳] (N k : ℕ → ℕ) (hN : Tendsto (fun n : ℕ => (N n : ℝ) / (n : ℝ)) atTop (𝓝 c)) (hkPos : ∀ n, 0 < k n) (hkInf : Tendsto (fun n : ℕ => (k n : ℝ)) atTop atTop) (hkRoot : Tendsto (fun n : ℕ => (k n : ℝ) / sqrt n) atTop (𝓝 0)) (hCarrier : ∀ n, ∃ cell : Fin (k n) ↪ 𝒳, ∀ i, MeasurableSet {cell i}) (hFullData : ∀ n (P : TransportedArray 𝒳), TransportedIVClass P N k c epsilon n → FullDataSupport P n) (hPresence : ∀ n (P : TransportedArray 𝒳), TransportedIVClass P N k c epsilon n → PopulationPresence P n) (hTwo : ∀ n (P : TransportedArray 𝒳), TransportedIVClass P N k c epsilon n → TwoSampleArray P N c) (hOverlap : ∀ n (P : TransportedArray 𝒳), TransportedIVClass P N k c epsilon n → InstrumentOverlap P n epsilon) (hObservation : ∀ n (P : TransportedArray 𝒳), TransportedIVClass P N k c epsilon n → SourceAssignmentConsistency P n) (hRandomization : ∀ n (P : TransportedArray 𝒳), TransportedIVClass P N k c epsilon n → IVRandomization P n) (hExclusion : ∀ n (P : TransportedArray 𝒳), TransportedIVClass P N k c epsilon n → IVExclusion P n) (hMonotonicity : ∀ n (P : TransportedArray 𝒳), TransportedIVClass P N k c epsilon n → IVMonotonicity P n) (hOutcomeTransport : ∀ n (P : TransportedArray 𝒳), TransportedIVClass P N k c epsilon n → OutcomeTransport P n) (hReceiptTransport : ∀ n (P : TransportedArray 𝒳), TransportedIVClass P N k c epsilon n → ReceiptTransport P n) (hComplierPositive : ∀ n (P : TransportedArray 𝒳), TransportedIVClass P N k c epsilon n → TargetComplierPositivity P n) (hDomination : ∀ n (P : TransportedArray 𝒳), TransportedIVClass P N k c epsilon n → TransportDomination P n) (hEnvelope : ∀ n (P : TransportedArray 𝒳), TransportedIVClass P N k c epsilon n → WeightEnvelope P k n) (hSecond : ∀ n (P : TransportedArray 𝒳), TransportedIVClass P N k c epsilon n → WeightSecondMoment P k n) (hDegrade : ∀ n (P : TransportedArray 𝒳), TransportedIVClass P N k c epsilon n → DegradingArray P k) (hFiniteCell : ∀ n (P : TransportedArray 𝒳), FiniteCellClass P N k c epsilon n → FiniteCellSource P k n) (t0 : ℝ),
∀ ht0 : 0 < t0,
c0 * min 1 (t0 ^ (-1 / 2 : ℝ)) ≤ oracleValue (𝒳 := 𝒳) N k c epsilon alpha ⟨t0, ht0⟩ ∧
c0 * min 1 (t0 ^ (-1 / 2 : ℝ))
≤ finiteCellOracleValue (𝒳 := 𝒳) N k c epsilon alpha ⟨t0, ht0⟩ ∧
(t0 ≤ tc → c0 ≤ oracleValue (𝒳 := 𝒳) N k c epsilon alpha ⟨t0, ht0⟩)
Proof (Lean source)
theorem oracle_converse
(epsilon alpha c : ℝ)
(hc : 0 < c)
(hepsilon : 0 < epsilon ∧ epsilon < 1 / 2)
(halpha : 0 < alpha ∧ alpha < 1) :
-- @realizes \alpha(noncoverage in (0,1))
∃ c0 tc : ℝ,
0 < c0 ∧ -- @realizes c_0(positive lower-frontier constant)
0 < tc ∧ -- @realizes t_c(positive transition threshold)
∀ {𝒳 : Type*} [MeasurableSpace 𝒳]
(N k : ℕ → ℕ)
(hN : Tendsto (fun n : ℕ => (N n : ℝ) / (n : ℝ)) atTop (𝓝 c))
-- @realizes N_n(N_n/n→c)
(hkPos : ∀ n, 0 < k n)
(hkInf : Tendsto (fun n : ℕ => (k n : ℝ)) atTop atTop)
(hkRoot : Tendsto (fun n : ℕ =>
(k n : ℝ) / sqrt n) atTop (𝓝 0))
-- @realizes k_n(positive, diverging, and o(√n))
(hCarrier : ∀ n, ∃ cell : Fin (k n) ↪ 𝒳, ∀ i, MeasurableSet {cell i})
(hFullData : ∀ n (P : TransportedArray 𝒳),
TransportedIVClass P N k c epsilon n → FullDataSupport P n)
(hPresence : ∀ n (P : TransportedArray 𝒳),
TransportedIVClass P N k c epsilon n → PopulationPresence P n)
(hTwo : ∀ n (P : TransportedArray 𝒳),
TransportedIVClass P N k c epsilon n → TwoSampleArray P N c)
(hOverlap : ∀ n (P : TransportedArray 𝒳),
TransportedIVClass P N k c epsilon n → InstrumentOverlap P n epsilon)
(hObservation : ∀ n (P : TransportedArray 𝒳),
TransportedIVClass P N k c epsilon n →
SourceAssignmentConsistency P n)
(hRandomization : ∀ n (P : TransportedArray 𝒳),
TransportedIVClass P N k c epsilon n → IVRandomization P n)
(hExclusion : ∀ n (P : TransportedArray 𝒳),
TransportedIVClass P N k c epsilon n → IVExclusion P n)
(hMonotonicity : ∀ n (P : TransportedArray 𝒳),
TransportedIVClass P N k c epsilon n → IVMonotonicity P n)
(hOutcomeTransport : ∀ n (P : TransportedArray 𝒳),
TransportedIVClass P N k c epsilon n → OutcomeTransport P n)
(hReceiptTransport : ∀ n (P : TransportedArray 𝒳),
TransportedIVClass P N k c epsilon n → ReceiptTransport P n)
(hComplierPositive : ∀ n (P : TransportedArray 𝒳),
TransportedIVClass P N k c epsilon n → TargetComplierPositivity P n)
(hDomination : ∀ n (P : TransportedArray 𝒳),
TransportedIVClass P N k c epsilon n → TransportDomination P n)
(hEnvelope : ∀ n (P : TransportedArray 𝒳),
TransportedIVClass P N k c epsilon n → WeightEnvelope P k n)
(hSecond : ∀ n (P : TransportedArray 𝒳),
TransportedIVClass P N k c epsilon n → WeightSecondMoment P k n)
(hDegrade : ∀ n (P : TransportedArray 𝒳),
TransportedIVClass P N k c epsilon n → DegradingArray P k)
(hFiniteCell : ∀ n (P : TransportedArray 𝒳),
FiniteCellClass P N k c epsilon n → FiniteCellSource P k n)
(t0 : ℝ), ∀ ht0 : 0 < t0, -- @realizes t_0(positive frontier threshold)
c0 * min 1 (t0 ^ (-1 / 2 : ℝ)) ≤
oracleValue (𝒳 := 𝒳) N k c epsilon alpha ⟨t0, ht0⟩ ∧
c0 * min 1 (t0 ^ (-1 / 2 : ℝ)) ≤
finiteCellOracleValue (𝒳 := 𝒳) N k c epsilon alpha ⟨t0, ht0⟩ ∧
(t0 ≤ tc →
c0 ≤ oracleValue (𝒳 := 𝒳) N k c epsilon alpha ⟨t0, ht0⟩) := by
refine ⟨3 * (1 - alpha) ^ 2 / 16, 1, ?_, by norm_num, ?_⟩
· have : 0 < 1 - alpha := sub_pos.mpr halpha.2
positivity
intro 𝒳 _ N k hN hkPos hkInf hkRoot hCarrier hFullData hPresence
hTwo hOverlap hObservation hRandomization hExclusion hMonotonicity
hOutcomeTransport hReceiptTransport hComplierPositive hDomination hEnvelope
hSecond hDegrade hFiniteCell t0 ht0
have hFiniteInhabited :=
finiteCellClass_inhabited N k c epsilon hCarrier hc hepsilon hN hkPos hkInf
hkRoot
have hMainInhabited :=
transportedIVClass_inhabited N k c epsilon hCarrier hc hepsilon hN hkPos
hkInf hkRoot
classical
choose cell hcell using hCarrier
let μ : ℕ → Measure 𝒳 := fun n =>
∑ i : Fin (k n), (k n : ENNReal)⁻¹ • Measure.dirac (cell n i)
have hkinv (n : ℕ) : (k n : ENNReal) * (k n : ENNReal)⁻¹ = 1 :=
ENNReal.mul_inv_cancel (Nat.cast_ne_zero.mpr (Nat.ne_of_gt (hkPos n)))
(ENNReal.natCast_ne_top (k n))
have hμ : ∀ n, IsProbabilityMeasure (μ n) := by
intro n
rw [isProbabilityMeasure_iff]
simp [μ, hkinv n]
let g : Geometry 𝒳 :=
{ sourceX := μ
targetX := μ
weight := fun _ _ => 1
propensity := fun _ _ => 1 / 2
weight_measurable := fun _ => measurable_const
propensity_measurable := fun _ => measurable_const }
have hg : AdmissibleGeometry g k epsilon := by
refine ⟨hμ, hμ, hepsilon.1, hepsilon.2, ?_, ?_, ?_, ?_, ?_⟩
· intro n x
dsimp [g]
constructor
· exact hepsilon.2.le
· linarith [hepsilon.2]
· intro n x
dsimp [g]
constructor
· norm_num
· have hk1 : (1 : ℝ) ≤ k n := by exact_mod_cast hkPos n
linarith
· intro n
letI : IsProbabilityMeasure (μ n) := hμ n
simp [g]
· intro n
letI : IsProbabilityMeasure (μ n) := hμ n
have hk1 : (1 : ℝ) ≤ k n := by exact_mod_cast hkPos n
simpa [g] using hk1
· intro n A hA
letI : IsProbabilityMeasure (μ n) := hμ n
simpa [g] using (ofReal_setIntegral_one (μ n) A).symm
have hSliceFinite : ∀ n (P : TransportedArray 𝒳),
fixedGeometrySlice P g N k c epsilon n →
FiniteCellClass P N k c epsilon n := by
intro n P hP
refine { hP.1 with finiteCellSource := ?_ }
refine ⟨hkPos n, ?_, cell n, hcell n, ?_, ?_, ?_, ?_⟩
· rw [hP.2.1]
exact hμ n
· rw [hP.2.1]
rw [show range (cell n) = ⋃ i, {cell n i} by
ext x
simp]
simp only [g, μ, Measure.finset_sum_apply, Measure.smul_apply]
simp [Measure.dirac_apply_of_mem, hkinv n]
· intro i
rw [hP.2.1]
simp only [g, μ, Measure.finset_sum_apply, Measure.smul_apply]
rw [Finset.sum_eq_single i]
· simp [ENNReal.toReal_inv]
· intro j _ hji
have hne : cell n j ≠ cell n i :=
fun h => hji ((cell n).injective h)
rw [Measure.dirac_apply' _ (hcell n i)]
simp [hne]
· simp
· rw [hP.2.1]
rfl
· filter_upwards with x
rw [hP.2.2.2.2]
let Dfull : OracleProcedure 𝒳 N k c epsilon :=
{ set := fun _ _ => parameterSpace
subset := fun _ _ => Set.Subset.rfl
measurableGraph := by
intro _ _ _ _ _
exact measurableSet_Icc.preimage measurable_snd
weightAEInvariant := by
intro _ _ _ _ _
exact Filter.Eventually.of_forall (fun _ => rfl) }
have hfullCoverage : ∀ n P,
TransportedIVClass P N k c epsilon n →
oracleCoverage Dfull P n = 1 := by
intro n P hP
letI : IsProbabilityMeasure (sourceObsLaw P n) :=
hP.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (targetXLaw P n) :=
hP.twoSampleArray.2.2.1 n
letI : IsProbabilityMeasure (twoSampleLaw P N n) := by
unfold twoSampleLaw
infer_instance
have htheta : targetCACE P n ∈ parameterSpace :=
(scoreRiskClass_compact_causal_range
(transportedIVScoreRiskAtoms N k c epsilon) hP).2.2.2.2.2
simp [oracleCoverage, oracleSet, Dfull, htheta]
have hfullCoverageClassInf : ∀ n,
oracleCoverageClassInf N k c epsilon Dfull n = 1 := by
intro n
let P := (hMainInhabited n).choose
letI : Nonempty
{Q : TransportedArray 𝒳 //
TransportedIVClass Q N k c epsilon n} :=
⟨⟨P, (hMainInhabited n).choose_spec⟩⟩
apply le_antisymm
· exact (oracleCoverageClassInf_le_canonical N k c epsilon Dfull P n
(hMainInhabited n).choose_spec).trans_eq
(hfullCoverage n P (hMainInhabited n).choose_spec)
· rw [oracleCoverageClassInf, coverageInfOrOne_of_nonempty]
refine le_ciInf (fun Q => ?_)
rw [hfullCoverage n Q.1 Q.2]
have hDfullHonest :
OracleHonest N k c epsilon alpha Dfull := by
refine ⟨halpha.1, halpha.2, ?_⟩
rw [show oracleCoverageClassInf N k c epsilon Dfull =
fun _ => 1 by
funext n
exact hfullCoverageClassInf n]
rw [liminf_const]
exact sub_le_self 1 (le_of_lt halpha.1)
letI : Nonempty {C : OracleProcedure 𝒳 N k c epsilon //
OracleHonest N k c epsilon alpha C} :=
⟨⟨Dfull, hDfullHonest⟩⟩
have hFixedLower : ∀ t0 : ℝ, 0 < t0 →
3 * (1 - alpha) ^ 2 / 16 * min 1 (t0 ^ (-1 / 2 : ℝ)) ≤
fixedGeometryValueTotal N k c epsilon alpha ⟨g, hg⟩ t0 := by
intro t0 ht0
have hw : ∀ n, g.weight n =ᵐ[g.sourceX n] fun _ => 1 := by
intro n
filter_upwards with x
rfl
exact (no_shift_reduction N k c epsilon alpha g hc hepsilon halpha hN
hkPos hkInf hkRoot hg hw).2.2 t0 ht0 |>.1
have hOracleLower : ∀ t0 : ℝ, 0 < t0 →
3 * (1 - alpha) ^ 2 / 16 * min 1 (t0 ^ (-1 / 2 : ℝ)) ≤
oracleValueTotal (𝒳 := 𝒳) N k c epsilon alpha t0 := by
intro t0 ht0
unfold oracleValueTotal
apply le_ciInf
intro C
exact (hFixedLower t0 ht0).trans
(fixedGeometryValue_le_frontierRisk N k c epsilon alpha g C.1 C.2 t0
hc hepsilon ht0 hg hN hkPos hkInf hkRoot)
have hFiniteLower : ∀ t0 : ℝ, 0 < t0 →
3 * (1 - alpha) ^ 2 / 16 * min 1 (t0 ^ (-1 / 2 : ℝ)) ≤
finiteCellOracleValueTotal (𝒳 := 𝒳) N k c epsilon alpha t0 := by
intro t0 ht0
have hSliceInhabited :
∀ᶠ n in atTop, ∃ P : TransportedArray 𝒳,
fixedGeometrySlice P g N k c epsilon n ∧
t0 ≤ effectiveStrength P n :=
fixedGeometrySlice_eventually_inhabited g N k c epsilon t0 hc
hepsilon ht0 hg hN hkPos hkInf hkRoot
have hsetLength_nonneg : ∀ A : Set ℝ, 0 ≤ setLength A :=
fun _ => ENNReal.toReal_nonneg
have hsetLength_le_two : ∀ A : Set ℝ, setLength A ≤ 2 := by
intro A
unfold setLength parameterSpace
calc
(volume (A ∩ Icc (-1 : ℝ) 1)).toReal ≤
(volume (Icc (-1 : ℝ) 1)).toReal :=
ENNReal.toReal_mono (by simp [Real.volume_Icc])
(measure_mono Set.inter_subset_right)
_ = 2 := by
rw [Real.volume_Icc, ENNReal.toReal_ofReal (by norm_num)]
norm_num
have hfixedRisk_nonneg : ∀ E : OracleProcedure 𝒳 N k c epsilon,
0 ≤ fixedGeometryRisk N k c epsilon g E t0 := by
intro E
have hrow_nonneg : ∀ n,
0 ≤ (⨆ P : {P : TransportedArray 𝒳 //
fixedGeometrySlice P g N k c epsilon n ∧
t0 ≤ effectiveStrength P n},
fixedGeometryOracleExpectedLength E g P n) := by
intro n
cases isEmpty_or_nonempty
{P : TransportedArray 𝒳 //
-- … truncated; follow the source link for the rest …
T_OracleScoreInversionAttainment 1 declarations
One oracle transported-score inversion sequence is honest and attains the frontier simultaneously at every fixed positive strength threshold.
Formal statement
hypotheses
N k :
ℕ → ℕ
c epsilon alpha :
ℝ
hc :
0 < c
hepsilon :
0 < epsilon ∧ epsilon < 1 / 2
halpha :
0 < alpha ∧ alpha < 1
@realizes \alpha(noncoverage in (0,1))
@realizes N_n(N_n/n→c)
hkPos :
∀ n, 0 < k n
@realizes k_n(positive, diverging, and o(√n))
hGeometry :
∃ g : Geometry 𝒳, AdmissibleGeometry g k epsilon
the carrier admits the paper's own deterministic geometry
hTwo :
∀ n (P : TransportedArray 𝒳)
if
TransportedIVClass P N k c epsilon n
then
TwoSampleArray P N c
hOverlap :
∀ n (P : TransportedArray 𝒳)
if
TransportedIVClass P N k c epsilon n
then
InstrumentOverlap P n epsilon
hEnvelope :
∀ n (P : TransportedArray 𝒳)
if
TransportedIVClass P N k c epsilon n
then
WeightEnvelope P k n
hSecond :
∀ n (P : TransportedArray 𝒳)
if
TransportedIVClass P N k c epsilon n
then
WeightSecondMoment P k n
hDegrade :
∀ n (P : TransportedArray 𝒳)
if
TransportedIVClass P N k c epsilon n
then
DegradingArray P k
conclusion
let Lalpha := sqrt (8 / (alpha * epsilon ^ 2)) let C0 := max 2 (4 * Lalpha + 8 / epsilon ^ 2) 0 < C0 ∧
∃ C : OracleProcedure 𝒳 N k c epsilon,
(∀ n x, C.set n x = inversionHandle x.2.1.1 x.2.2 n Lalpha x.1.1) ∧
OracleHonest N k c epsilon alpha C ∧
∀ (t0 : ℝ) (ht0 : 0 < t0),
frontierRisk N k c epsilon C ⟨t0, ht0⟩ ≤ C0 * min 1 (t0 ^ (-1 / 2 : ℝ)) ∧
3 * (1 - alpha) ^ 2 / 16 * min 1 (t0 ^ (-1 / 2 : ℝ))
≤ oracleValue (𝒳 := 𝒳) N k c epsilon alpha ⟨t0, ht0⟩ ∧
oracleValue (𝒳 := 𝒳) N k c epsilon alpha ⟨t0, ht0⟩ ≤ C0 * min 1 (t0 ^ (-1 / 2 : ℝ)) ∧
∀ (g : Geometry 𝒳) (hg : AdmissibleGeometry g k epsilon),
3 * (1 - alpha) ^ 2 / 16 * min 1 (t0 ^ (-1 / 2 : ℝ))
≤ fixedGeometryValue N k c epsilon alpha ⟨g, hg⟩ ⟨t0, ht0⟩ ∧
fixedGeometryValue N k c epsilon alpha ⟨g, hg⟩ ⟨t0, ht0⟩ ≤ C0 * min 1 (t0 ^ (-1 / 2 : ℝ))
Proof (Lean source)
theorem oracle_score_inversion_attainment
(N k : ℕ → ℕ) (c epsilon alpha : ℝ)
(hc : 0 < c)
(hepsilon : 0 < epsilon ∧ epsilon < 1 / 2)
(halpha : 0 < alpha ∧ alpha < 1) -- @realizes \alpha(noncoverage in (0,1))
(hN : Tendsto (fun n : ℕ => (N n : ℝ) / (n : ℝ)) atTop (𝓝 c))
-- @realizes N_n(N_n/n→c)
(hkPos : ∀ n, 0 < k n)
(hkInf : Tendsto (fun n : ℕ => (k n : ℝ)) atTop atTop)
(hkRoot : Tendsto (fun n : ℕ =>
(k n : ℝ) / sqrt n) atTop (𝓝 0))
-- @realizes k_n(positive, diverging, and o(√n))
(hGeometry : ∃ g : Geometry 𝒳, AdmissibleGeometry g k epsilon)
-- the carrier admits the paper's own deterministic geometry
(hTwo : ∀ n (P : TransportedArray 𝒳),
TransportedIVClass P N k c epsilon n →
TwoSampleArray P N c)
(hOverlap : ∀ n (P : TransportedArray 𝒳),
TransportedIVClass P N k c epsilon n →
InstrumentOverlap P n epsilon)
(hEnvelope : ∀ n (P : TransportedArray 𝒳),
TransportedIVClass P N k c epsilon n →
WeightEnvelope P k n)
(hSecond : ∀ n (P : TransportedArray 𝒳),
TransportedIVClass P N k c epsilon n →
WeightSecondMoment P k n)
(hDegrade : ∀ n (P : TransportedArray 𝒳),
TransportedIVClass P N k c epsilon n →
DegradingArray P k) :
let Lalpha := sqrt (8 / (alpha * epsilon ^ 2))
let C0 := max 2 (4 * Lalpha + 8 / epsilon ^ 2)
-- @realizes C_0(max{2,4L_alpha+8/epsilon^2})
0 < C0 ∧
∃ C : OracleProcedure 𝒳 N k c epsilon,
(∀ n x, C.set n x =
inversionHandle x.2.1.1 x.2.2 n Lalpha x.1.1) ∧
OracleHonest N k c epsilon alpha C ∧
∀ (t0 : ℝ) (ht0 : 0 < t0), -- @realizes t_0(positive frontier threshold)
frontierRisk N k c epsilon C ⟨t0, ht0⟩ ≤
C0 * min 1 (t0 ^ (-1 / 2 : ℝ)) ∧
3 * (1 - alpha) ^ 2 / 16 *
min 1 (t0 ^ (-1 / 2 : ℝ)) ≤
oracleValue (𝒳 := 𝒳) N k c epsilon alpha ⟨t0, ht0⟩ ∧
oracleValue (𝒳 := 𝒳) N k c epsilon alpha ⟨t0, ht0⟩ ≤
C0 * min 1 (t0 ^ (-1 / 2 : ℝ)) ∧
∀ (g : Geometry 𝒳) (hg : AdmissibleGeometry g k epsilon),
3 * (1 - alpha) ^ 2 / 16 *
min 1 (t0 ^ (-1 / 2 : ℝ)) ≤
fixedGeometryValue N k c epsilon alpha ⟨g, hg⟩
⟨t0, ht0⟩ ∧
fixedGeometryValue N k c epsilon alpha ⟨g, hg⟩
⟨t0, ht0⟩ ≤
C0 * min 1 (t0 ^ (-1 / 2 : ℝ)) := by
obtain ⟨g, hg⟩ := hGeometry
have hClass : ∀ t0 : ℝ, 0 < t0 →
∀ᶠ n in atTop, ∃ P : TransportedArray 𝒳,
TransportedIVClass P N k c epsilon n := by
intro t0 ht0
filter_upwards [
fixedGeometrySlice_eventually_inhabited
g N k c epsilon t0 hc hepsilon ht0 hg hN hkPos hkInf hkRoot
] with n hn
obtain ⟨P, hP, _⟩ := hn
exact ⟨P, hP.1⟩
dsimp only
have hC0pos :
0 < max 2
(4 * sqrt (8 / (alpha * epsilon ^ 2)) +
8 / epsilon ^ 2) :=
lt_of_lt_of_le (by norm_num) (le_max_left _ _)
refine ⟨hC0pos, ?_⟩
classical
let L : ℝ := sqrt (8 / (alpha * epsilon ^ 2))
let C : OracleProcedure 𝒳 N k c epsilon :=
{ set := fun n x =>
inversionHandle x.2.1.1 x.2.2 n L x.1.1
subset := by
intro n x theta htheta
exact htheta.1
measurableGraph := by
intro n w e hw he
have hscore : Measurable (oracleInstrumentScore e) := by
have he' : Measurable (fun o : SourceObs 𝒳 => e o.1) :=
he.comp measurable_fst
unfold oracleInstrumentScore
exact Measurable.ite (by measurability)
(measurable_const.div he')
(measurable_const.neg.div (measurable_const.sub he'))
have hA : Measurable (fun s : SourceSample 𝒳 n =>
scoreOutcomeMean w e n s) := by
unfold scoreOutcomeMean
fun_prop
have hB : Measurable (fun s : SourceSample 𝒳 n =>
scoreReceiptMean w e n s) := by
unfold scoreReceiptMean
fun_prop
have hK : Measurable (fun s : SourceSample 𝒳 n =>
empiricalKish w n s) := by
unfold empiricalKish empiricalKishDispersion
fun_prop
unfold inversionHandle
refine (measurableSet_Icc.preimage measurable_snd).inter ?_
exact measurableSet_le
((hA.comp (measurable_fst.fst)).sub
(measurable_snd.mul
(hB.comp (measurable_fst.fst)))).abs
(measurable_const.mul
((hK.comp (measurable_fst.fst)).div measurable_const).sqrt)
weightAEInvariant := by
intro n P hP w w'
letI : IsProbabilityMeasure (sourceObsLaw P n) :=
hP.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (targetXLaw P n) :=
hP.twoSampleArray.2.2.1 n
let μS := Measure.pi (fun _ : Fin n => sourceObsLaw P n)
let μT := Measure.pi (fun _ : Fin (N n) => targetXLaw P n)
have hmap : Measure.map fst (μS.prod μT) = μS := by
simp [μS, μT]
have hpull (v : TransportWeightVersion P n) :
(fun z : TwoSample 𝒳 n (N n) =>
inversionHandle v.1.1 (P.propensity n) n L z.1) =ᵐ[
twoSampleLaw P N n]
(fun z : TwoSample 𝒳 n (N n) =>
inversionHandle (transportWeight P n) (P.propensity n) n L z.1) := by
have hv := inversionHandle_weight_version_ae_eq P n v L
have hv' :
(fun s : SourceSample 𝒳 n =>
inversionHandle v.1.1 (P.propensity n) n L s) =ᵐ[
Measure.map fst (μS.prod μT)]
(fun s : SourceSample 𝒳 n =>
inversionHandle (transportWeight P n) (P.propensity n) n L s) := by
rw [hmap]
exact hv
simpa only [twoSampleLaw, μS, μT, Function.comp_def] using
(MeasureTheory.ae_eq_comp measurable_fst.aemeasurable hv')
exact (hpull w).trans (hpull w').symm }
have hCset : ∀ n x, C.set n x =
inversionHandle x.2.1.1 x.2.2 n L x.1.1 := by
intro n x
rfl
have hUpper :
OracleHonest N k c epsilon alpha C ∧
∀ t0 : ℝ, 0 < t0 →
frontierRiskTotal N k c epsilon C t0 ≤
max 2 (4 * L + 8 / epsilon ^ 2) *
min 1 (t0 ^ (-1 / 2 : ℝ)) := by
have atoms :=
transportedIVScoreRiskAtoms (𝒳 := 𝒳) N k c epsilon
have pullout
(P : TransportedArray 𝒳) (n : ℕ)
(F : SourceObs 𝒳 → ℝ) (d w : 𝒳 → ℝ)
(hprob : IsProbabilityMeasure (sourceObsLaw P n))
(hF : Integrable F (sourceObsLaw P n))
(hd : Integrable d (sourceXLaw P n))
(hdmeas : Measurable d)
(hw : Measurable w)
(hWF : Integrable (fun o => w o.1 * F o) (sourceObsLaw P n))
(hwd : Integrable (fun x => w x * d x) (sourceXLaw P n))
(hset : ∀ A, MeasurableSet A →
∫ o in {o | o.1 ∈ A}, F o ∂sourceObsLaw P n =
∫ x in A, d x ∂sourceXLaw P n) :
(∫ o, w o.1 * F o ∂sourceObsLaw P n) =
∫ x, w x * d x ∂sourceXLaw P n := by
let m0 : MeasurableSpace (SourceObs 𝒳) := inferInstance
let mX : MeasurableSpace (SourceObs 𝒳) :=
comap (fun o => o.1)
(inferInstance : MeasurableSpace 𝒳)
letI : MeasurableSpace (SourceObs 𝒳) := m0
letI : IsProbabilityMeasure (sourceObsLaw P n) := hprob
have hfst :
@Measurable (SourceObs 𝒳) 𝒳 m0 inferInstance (fun o => o.1) :=
measurable_fst
have hm : mX ≤ m0 :=
hfst.comap_le
have hfstM :
@Measurable (SourceObs 𝒳) 𝒳 mX inferInstance (fun o => o.1) := by
intro A hA
exact MeasurableSpace.measurableSet_comap.mpr ⟨A, hA, rfl⟩
have hdcomp : Integrable (fun o => d o.1) (sourceObsLaw P n) := by
have hdmap : Integrable d
(Measure.map (fun o : SourceObs 𝒳 => o.1) (sourceObsLaw P n)) := by
simpa [sourceXLaw] using hd
simpa [Function.comp_def] using
(integrable_map_measure hdmap.1 hfst.aemeasurable).mp hdmap
have hcond : (fun o => d o.1) =ᵐ[sourceObsLaw P n]
(sourceObsLaw P n)[F | mX] := by
refine ae_eq_condExp_of_forall_setIntegral_eq
(μ := sourceObsLaw P n) (f := F) (g := fun o => d o.1)
hm hF ?_ ?_ ?_
· intro s hs _hfin
exact hdcomp.integrableOn
· intro s hs _hfin
rcases MeasurableSpace.measurableSet_comap.mp hs with
⟨A, hA, rfl⟩
calc
(∫ x in (fun o : SourceObs 𝒳 => o.1) ⁻¹' A,
d x.1 ∂sourceObsLaw P n) =
∫ x in A, d x ∂sourceXLaw P n := by
rw [sourceXLaw]
exact (setIntegral_map hA hd.1 hfst.aemeasurable).symm
_ = ∫ x in (fun o : SourceObs 𝒳 => o.1) ⁻¹' A,
F x ∂sourceObsLaw P n := (hset A hA).symm
· exact (hdmeas.comp hfstM).aestronglyMeasurable
have hwM : @StronglyMeasurable (SourceObs 𝒳) ℝ _ mX
(fun o => w o.1) :=
(hw.comp hfstM).stronglyMeasurable
have hpull := condExp_mul_of_stronglyMeasurable_left
(m := mX) (μ := sourceObsLaw P n) hwM hWF hF
have hpull' :
(sourceObsLaw P n)[(fun o => w o.1 * F o) | mX] =ᵐ[
sourceObsLaw P n]
fun o => w o.1 * (sourceObsLaw P n)[F | mX] o := by
exact hpull
calc
(∫ o, w o.1 * F o ∂sourceObsLaw P n) =
∫ o, (sourceObsLaw P n)[(fun o => w o.1 * F o) | mX] o
∂sourceObsLaw P n := (integral_condExp hm).symm
_ = ∫ o, w o.1 * d o.1 ∂sourceObsLaw P n := by
apply integral_congr_ae
filter_upwards [hpull', hcond] with o hp hc
rw [hp, ← hc]
_ = ∫ x, w x * d x ∂sourceXLaw P n := by
rw [sourceXLaw]
exact (integral_map measurable_fst.aemeasurable hwd.1).symm
have hRiskPoint : ∀ n P, 0 < n →
TransportedIVClass P N k c epsilon n →
oracleExpectedLength C P n ≤
max 2 (4 * L + 8 / epsilon ^ 2) *
min 1 (effectiveStrength P n ^ (-1 / 2 : ℝ)) := by
intro n P hn hP
have hIV := atoms.toTransportedIVClass hP
letI : IsProbabilityMeasure (sourceObsLaw P n) :=
hIV.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (sourceXLaw P n) := by
unfold sourceXLaw
exact Measure.isProbabilityMeasure_map measurable_fst.aemeasurable
letI : IsProbabilityMeasure (targetXLaw P n) :=
hIV.twoSampleArray.2.2.1 n
letI : IsProbabilityMeasure (twoSampleLaw P N n) := by
unfold twoSampleLaw
infer_instance
have hcompact := scoreRiskClass_compact_causal_range atoms hP
have htheta : targetCACE P n ∈ parameterSpace :=
hcompact.2.2.2.2.2
have hfirstEq :
transportedFirstStage P n = targetComplierShare P n :=
hcompact.2.2.2.1
have hmu : 0 < transportedFirstStage P n := by
-- … truncated; follow the source link for the rest …
T_RegularCellUnknownWeightAttainment 1 declarations
With known regular source-cell probabilities and known propensity, the cross-averaged rule does not use the transport weight and attains the oracle frontier order; the uniform fixed geometry supplies the matching converse.
Formal statement
hypotheses
N k :
ℕ → ℕ
c epsilon alpha cminus cplus :
ℝ
hc :
0 < c
@realizes c(c∈(0,∞))
hepsilon :
0 < epsilon ∧ epsilon < 1 / 2
halpha :
0 < alpha ∧ alpha < 1
@realizes \alpha(noncoverage in (0,1))
hcminus :
0 < cminus ∧ cminus ≤ 1
hcplus :
1 ≤ cplus
@realizes N_n(N_n/n→c)
hkPos :
∀ n, 0 < k n
@realizes k_n(positive, diverging, and o(√n))
hCarrier :
∀ n, ∃ cell : Fin (k n) ↪ 𝒳, ∀ i, MeasurableSet {cell i}
hTwo :
∀ n (P : TransportedArray 𝒳)
if
RegularFiniteCellClass P N k c epsilon cminus cplus n
then
TwoSampleArray P N c
hOverlap :
∀ n (P : TransportedArray 𝒳)
if
RegularFiniteCellClass P N k c epsilon cminus cplus n
then
InstrumentOverlap P n epsilon
hDegrade :
∀ n (P : TransportedArray 𝒳)
if
RegularFiniteCellClass P N k c epsilon cminus cplus n
then
DegradingArray P k
conclusion
let B := 8 * (epsilon⁻¹ ^ 2 + c⁻¹) ∀ L : ℝ, sqrt (2 * B / alpha)
∃ C : RegularCellProcedure 𝒳 N k,
(∀ n (P : TransportedArray 𝒳), ∀ hP : RegularFiniteCellClass P N k c epsilon cminus cplus n, ∀ᵐ s ∂(twoSampleLaw P N n), C.set n (regularCellInputOfClass P hP s) = regularCellInversion (sourceCellMass P n) (P.propensity n) L s.1 s.2) ∧
RegularCellHonest N k c epsilon alpha cminus cplus C ∧
(∀ t0 : ℝ, 0 < t0 → regularCellRisk N k c epsilon cminus cplus C t0 ≤ C0 * min 1 (t0 ^ (-1 / 2 : ℝ))) ∧
(∀ t0 : ℝ, 0 < t0 → 3 * (1 - alpha) ^ 2 / 16 * min 1 (t0 ^ (-1 / 2 : ℝ)) ≤ ⨅ C : {C : RegularCellProcedure 𝒳 N k // RegularCellHonest N k c epsilon alpha cminus cplus C}, regularCellRisk N k c epsilon cminus cplus C.1 t0)
Proof (Lean source)
theorem regular_cell_unknown_weight_attainment
(N k : ℕ → ℕ) (c epsilon alpha cminus cplus : ℝ)
(hc : 0 < c) -- @realizes c(c∈(0,∞))
(hepsilon : 0 < epsilon ∧ epsilon < 1 / 2)
(halpha : 0 < alpha ∧ alpha < 1)
-- @realizes \alpha(noncoverage in (0,1))
(hcminus : 0 < cminus ∧ cminus ≤ 1)
(hcplus : 1 ≤ cplus)
(hN : Tendsto (fun n : ℕ => (N n : ℝ) / (n : ℝ)) atTop (𝓝 c))
-- @realizes N_n(N_n/n→c)
(hkPos : ∀ n, 0 < k n)
(hkInf : Tendsto (fun n : ℕ => (k n : ℝ)) atTop atTop)
(hkRoot : Tendsto (fun n : ℕ =>
(k n : ℝ) / sqrt n) atTop (𝓝 0))
-- @realizes k_n(positive, diverging, and o(√n))
(hCarrier : ∀ n, ∃ cell : Fin (k n) ↪ 𝒳, ∀ i, MeasurableSet {cell i})
(hTwo : ∀ n (P : TransportedArray 𝒳),
RegularFiniteCellClass P N k c epsilon cminus cplus n →
TwoSampleArray P N c)
(hOverlap : ∀ n (P : TransportedArray 𝒳),
RegularFiniteCellClass P N k c epsilon cminus cplus n →
InstrumentOverlap P n epsilon)
(hDegrade : ∀ n (P : TransportedArray 𝒳),
RegularFiniteCellClass P N k c epsilon cminus cplus n →
DegradingArray P k) :
let B := 8 * (epsilon⁻¹ ^ 2 + c⁻¹)
∀ L : ℝ, sqrt (2 * B / alpha) ≤ L →
let C0 := max 2 (4 * sqrt 2 * L + 8 * B)
-- @realizes C_0(constructive regular-cell upper constant)
0 < C0 ∧
∃ C : RegularCellProcedure 𝒳 N k,
(∀ n (P : TransportedArray 𝒳),
∀ hP : RegularFiniteCellClass P N k c epsilon cminus cplus n,
∀ᵐ s ∂(twoSampleLaw P N n),
C.set n (regularCellInputOfClass P hP s) =
regularCellInversion (sourceCellMass P n) (P.propensity n)
L s.1 s.2) ∧
RegularCellHonest N k c epsilon alpha cminus cplus C ∧
(∀ t0 : ℝ, 0 < t0 → -- @realizes t_0(positive frontier threshold)
regularCellRisk N k c epsilon cminus cplus C t0 ≤
C0 *
min 1 (t0 ^ (-1 / 2 : ℝ))) ∧
(∀ t0 : ℝ, 0 < t0 → -- @realizes t_0(positive frontier threshold)
3 * (1 - alpha) ^ 2 / 16 *
min 1 (t0 ^ (-1 / 2 : ℝ)) ≤
⨅ C : {C : RegularCellProcedure 𝒳 N k //
RegularCellHonest N k c epsilon alpha cminus cplus C},
regularCellRisk N k c epsilon cminus cplus C.1 t0) := by
have hClass :=
regularFiniteCellClass_inhabited N k c epsilon cminus cplus hCarrier hc
hepsilon hcminus hcplus hN hkPos hkInf hkRoot
dsimp
intro L hL
let B : ℝ := 8 * (epsilon⁻¹ ^ 2 + c⁻¹)
let C0 : ℝ := max 2 (4 * sqrt 2 * L + 8 * B)
have hB : B = regularCellVarianceConstant epsilon c := rfl
have hBnonneg : 0 ≤ B := by
unfold B
positivity
have hLnonneg : 0 ≤ L := by
exact (Real.sqrt_nonneg _).trans hL
have hC0pos : 0 < C0 := by
exact lt_of_lt_of_le (by norm_num) (le_max_left _ _)
refine ⟨hC0pos, regularCellProcedure (𝒳 := 𝒳) N k L, ?_, ?_, ?_, ?_⟩
· intro n P hP
exact regularCellProcedure_set_eq_ambient_ae L n P hP
· refine ⟨halpha.1, halpha.2, ?_⟩
have hrowEq :
(fun n => coverageInfOrOne fun P : {P : TransportedArray 𝒳 //
RegularFiniteCellClass P N k c epsilon cminus cplus n} =>
regularCellCoverage
(regularCellProcedure (𝒳 := 𝒳) N k L) n P) =
(fun n => ⨅ P : {P : TransportedArray 𝒳 //
RegularFiniteCellClass P N k c epsilon cminus cplus n},
regularCellCoverage
(regularCellProcedure (𝒳 := 𝒳) N k L) n P) := by
funext n
obtain ⟨P, hP⟩ := hClass n
simp [coverageInfOrOne, coverageInfOrOne, show Nonempty {P : TransportedArray 𝒳 //
RegularFiniteCellClass P N k c epsilon cminus cplus n} from
⟨⟨P, hP⟩⟩]
rw [hrowEq]
exact regularCellProcedure_coverage_liminf
N k c epsilon alpha cminus cplus L hc hepsilon halpha
hcminus hcplus hN hkPos hkInf hkRoot hCarrier (hB ▸ hL)
· intro t0 ht0
classical
have hnpos : ∀ᶠ n : ℕ in atTop, 0 < n :=
eventually_atTop.2 ⟨1, fun n hn => Nat.zero_lt_of_lt hn⟩
have hNratio :
∀ᶠ n : ℕ in atTop, c / 2 < (N n : ℝ) / (n : ℝ) :=
(tendsto_order.1 hN).1 _ (by linarith)
obtain ⟨m : ℕ, hm : 2 / c < m⟩ := exists_nat_gt (2 / c)
have hm_event : ∀ᶠ n : ℕ in atTop, m ≤ n :=
eventually_atTop.2 ⟨m, fun _ hn => hn⟩
have hNtwo : ∀ᶠ n : ℕ in atTop, 2 ≤ N n := by
filter_upwards [hnpos, hNratio, hm_event] with n hn hratio hmn
have hnreal : 0 < (n : ℝ) := by exact_mod_cast hn
have hmnreal : (m : ℝ) ≤ n := by exact_mod_cast hmn
have hc_n : 2 < c * (n : ℝ) := by
have hcm : 2 < (m : ℝ) * c := (div_lt_iff₀ hc).mp hm
have hmcn : (m : ℝ) * c ≤ (n : ℝ) * c :=
mul_le_mul_of_nonneg_right hmnreal hc.le
nlinarith
have hNreal : c * (n : ℝ) / 2 < (N n : ℝ) := by
apply (lt_div_iff₀ hnreal).mp at hratio
nlinarith
exact_mod_cast (show (1 : ℝ) < N n by linarith)
have hbad := regularCell_firstStage_bad_probability
(𝒳 := 𝒳) N k c epsilon cminus cplus hc hepsilon hcminus hcplus
hN hkPos hkRoot
obtain ⟨n0, hgood⟩ :=
eventually_atTop.1 (hbad.and (hNtwo.and hnpos))
let ell : ℕ → TransportedArray 𝒳 → ℝ := fun n P =>
if n0 ≤ n then
if hP : RegularFiniteCellClass P N k c epsilon cminus cplus n then
regularCellExpectedLength
(regularCellProcedure (𝒳 := 𝒳) N k L) n ⟨P, hP⟩
else 0
else 0
have hell_nonneg : ∀ n P,
RegularFiniteCellClass P N k c epsilon cminus cplus n →
0 ≤ ell n P := by
intro n P hP
unfold ell
split_ifs <;> simp_all
· exact integral_nonneg fun _ => ENNReal.toReal_nonneg
have hell :
∀ n P, RegularFiniteCellClass P N k c epsilon cminus cplus n →
ell n P ≤ C0 *
min 1 (effectiveStrength P n ^ (-1 / 2 : ℝ)) := by
intro n P hP
unfold ell
split_ifs with hn0 hclass
· have hgoodn := hgood n hn0
have hbadn := hgoodn.1 P hP
have hN2 := hgoodn.2.1
have hn : 0 < n := hgoodn.2.2
have hNpos : 0 < N n := lt_of_lt_of_le (by omega) hN2
letI : IsProbabilityMeasure (sourceObsLaw P n) :=
hP.1.twoSampleArray.2.1 n
letI : IsProbabilityMeasure (targetXLaw P n) :=
hP.1.twoSampleArray.2.2.1 n
letI : IsProbabilityMeasure (twoSampleLaw P N n) := by
unfold twoSampleLaw
infer_instance
have hcompact := scoreRiskClass_compact_causal_range
(transportedIVScoreRiskAtoms (𝒳 := 𝒳) N k c epsilon) hP.1
have hmu : 0 < transportedFirstStage P n := by
rw [hcompact.2.2.2.1]
exact hP.1.targetComplierPositivity
obtain ⟨cell, hcell, hrange, _hmass⟩ := hP.2.2.2.2.2
have hkappaOne := one_le_regularCell_kish
P N k c epsilon n hP.1 cell hcell hrange
have hkappa : 0 < kishDispersion P n :=
lt_of_lt_of_le zero_lt_one hkappaOne
have hmem := regularCell_moments_memLp
P N k c epsilon cminus cplus 0 n hn hNpos hP
(by simp [parameterSpace])
have hKintTarget :
Integrable (regularCellKhat (sourceCellMass P n))
(Measure.pi (fun _ : Fin (N n) => targetXLaw P n)) := by
have hconst :
Integrable
(fun _ : TargetSample 𝒳 (N n) => (1 : ℝ))
(Measure.pi (fun _ : Fin (N n) => targetXLaw P n)) :=
integrable_const 1
change Integrable
(fun target => 1 +
collisionScale (sourceCellMass P n) target) _
exact hconst.add (hmem.2.2.integrable (by norm_num))
have hKint :
Integrable
(fun s : TwoSample 𝒳 n (N n) =>
regularCellKhat (sourceCellMass P n) s.2)
(twoSampleLaw P N n) := by
exact hKintTarget.comp_snd _
have hKmean := regularCell_Khat_mean
P N k c epsilon cminus cplus n hN2 hP
have hKbar :
(∫ s : TwoSample 𝒳 n (N n),
regularCellKhat (sourceCellMass P n) s.2
∂twoSampleLaw P N n) =
1 + kishDispersion P n := by
unfold twoSampleLaw
rw [MeasureTheory.integral_fun_snd]
simpa using hKmean.1
have hcollision :
∀ target : TargetSample 𝒳 (N n),
0 ≤ collisionScale (sourceCellMass P n) target := by
intro target
unfold collisionScale collisionScale
apply mul_nonneg
· positivity
· apply sum_nonneg
intro j hj
apply sum_nonneg
intro l hl
split_ifs
· unfold collisionKernel
split_ifs
· exact one_div_nonneg.mpr ENNReal.toReal_nonneg
· norm_num
· norm_num
have hpoint := expectedLength_affineInversion_frontier_le_inflated
(twoSampleLaw P N n)
(fun s => regularCellOutcomeMoment (sourceCellMass P n)
(P.propensity n) s.1 s.2)
(fun s => regularCellReceiptMoment (sourceCellMass P n)
(P.propensity n) s.1 s.2)
(fun s => regularCellKhat (sourceCellMass P n) s.2)
n L (transportedFirstStage P n)
(1 + kishDispersion P n)
(4 * B / effectiveStrength P n)
(kishDispersion P n) (8 * B) (effectiveStrength P n)
hLnonneg hmu hn
(fun s => by
unfold regularCellKhat
linarith [hcollision s.2])
hKint hKbar.le
hbadn
hkappa (mul_nonneg (by norm_num) hBnonneg)
hKmean.2
(by
have ht : 0 < effectiveStrength P n := by
unfold effectiveStrength
positivity
field_simp [ht.ne']
ring_nf
rfl)
rfl
have hset :
regularCellExpectedLength
(regularCellProcedure (𝒳 := 𝒳) N k L) n ⟨P, hP⟩ =
∫ s, setLength
(affineInversionSet
(regularCellOutcomeMoment (sourceCellMass P n)
(P.propensity n) s.1 s.2)
(regularCellReceiptMoment (sourceCellMass P n)
(P.propensity n) s.1 s.2)
(L * sqrt
(regularCellKhat (sourceCellMass P n) s.2 / n)))
∂twoSampleLaw P N n := by
apply integral_congr_ae
filter_upwards [regularCellProcedure_set_eq_ambient_ae L n P hP]
with s hs
simp only [regularCellExpectedLength, regularCellSet]
-- … truncated; follow the source link for the rest …