Estimation.CATE.Orthogonal­Learning.Local­Emp­Process

Empirical-process modulus results for the CATE DR-Learner: Rademacher, localized, honest-split, random-nuisance, and end-to-end oracle bounds.

DRLearner 5 core · 1 supporting This file proves that bounded outcomes, bounded target evaluations, strict propensity overlap, bounded nuisance regressions, and a Rademacher-complexity bound imply the local empirical-process modulus for the doubly robu ★ localEmpProcessModulus_drLearner

DR-Learner Empirical Modulus

This file proves that bounded outcomes, bounded target evaluations, strict propensity overlap, bounded nuisance regressions, and a Rademacher-complexity bound imply the local empirical-process modulus for the doubly robust learner for conditional treatment effects. It connects the generic orthogonal statistical-learning modulus to the causal conditional-effect system through the assumption predicates DREvalBounded, DROutcomeBounded, DRNuisanceOverlap, and DRNuisanceMuBounded, the bounded-loss bridge dr_loss_uniformly_bounded, and the modulus theorem localEmpProcessModulus_drLearner.

def DREvalBounded reviewed
Causalean.Estimation.OrthogonalLearning

Boundedness of the target candidate evaluation map: |eval θ x| ≤ M_Θ uniformly over θ ∈ Θ_set and x : γ.

Definition (Lean source)
def DREvalBounded {Θ : Type*} (Θ_set : Set Θ) (eval : Θ → γ → ℝ) (M_Θ : ℝ) : Prop := 0 ≤ M_Θ ∧ ∀ θ ∈ Θ_set, ∀ x : γ, |eval θ x| ≤ M_Θ
Causalean.Estimation.OrthogonalLearning.DREvalBounded · Causalean/Estimation/CATE/OrthogonalLearning/LocalEmpProcess/DRLearner.lean:56
def DROutcomeBounded reviewed
Causalean.Estimation.OrthogonalLearning

Bounded outcome assumption: the outcome coordinate is bounded by M_Y under the observed-data law P_Z, almost everywhere.

Definition (Lean source)
def DROutcomeBounded [StandardBorelSpace P.Ω] [IsFiniteMeasure P.μ] (S : CATEEstimationSystem P γ) (M_Y : ℝ) : Prop := 0 ≤ M_Y ∧ ∀ᵐ z ∂(S.toBackdoorEstimationSystem.P_Z), |z.2.2| ≤ M_Y
def DRNuisanceOverlap reviewed
Causalean.Estimation.OrthogonalLearning

Strict overlap floor on the realised nuisance h: the propensity component h.e_fn is bounded inside [ε, 1 − ε] uniformly in x.

Definition (Lean source)
def DRNuisanceOverlap {Θ : Type*} [StandardBorelSpace P.Ω] [IsFiniteMeasure P.μ] (_S : CATEEstimationSystem P γ) (_Θ_set : Set Θ) (h : NuisanceVec γ) (ε : ℝ) : Prop := 0 < ε ∧ ε ≤ 1 / 2 ∧ ∀ x : γ, ε ≤ h.e_fn x ∧ h.e_fn x ≤ 1 - ε
def DRNuisanceMuBounded reviewed
Causalean.Estimation.OrthogonalLearning

Uniform L∞-bound on the outcome-regression component of the realised nuisance h: |h.μ_fn b x| ≤ M_μ for both treatment arms.

Definition (Lean source)
def DRNuisanceMuBounded (h : NuisanceVec γ) (M_μ : ℝ) : Prop := 0 ≤ M_μ ∧ ∀ b : Bool, ∀ x : γ, |h.μ_fn b x| ≤ M_μ
Causalean.Estimation.OrthogonalLearning.DRNuisanceMuBounded · Causalean/Estimation/CATE/OrthogonalLearning/LocalEmpProcess/DRLearner.lean:81 · uses NuisanceVec
theorem localEmpProcessModulus_drLearner reviewed
Causalean.Estimation.OrthogonalLearning

DR-Learner bounded-loss Rademacher modulus. Consider the doubly robust orthogonal-learning system for conditional treatment effects, cross-fitted via a one-shot sample split, whose truth-identifying candidate belongs to the target set, has a measurable evaluation map, and recovers the population CATE function through that evaluation map. Suppose the candidate evaluation maps are uniformly bounded, the observed outcome is almost-surely bounded, the realised nuisance outcome-regression is uniformly bounded on both treatment arms, and the realised propensity stays away from 0 and 1 by a fixed margin; suppose also that the resulting doubly robust loss is continuous in the candidate over the target set at that realised nuisance, a sequence R n bounds the Rademacher complexity of the centred loss class on the estimation fold, the truth-identifying candidate still minimizes the clamped population loss at the realised nuisance, and the confidence level δ lies strictly above 0 and at most 1. Then there is a nonnegative constant b for which the system obeys the local empirical-process modulus condition at the explicit rate ρ n = √(2 R n + 2b · √(2 log(1/δ) / |B(n)|)) on nonempty estimation folds, with boundary value ρ n = √(2b) on empty folds.

Formal statement
Θ_set :
Set Θ
Θ_convex :
Convex ℝ Θ_set
θ₀ :
Θ
θ₀_mem :
θ₀ ∈ Θ_set
eval :
Θ → γ → ℝ
eval_meas :
∀ θ, Measurable (eval θ)
eval_θ₀ :
∀ x, eval θ₀ x = S.τ_val x
θ₀_minimizes :
DRThetaMinimizes S Θ_set θ₀ eval
S_iid :
IIDSample P.Ω (γ × Bool × ℝ) P.μ S.toBackdoorEstimationSystem.P_Z
split :
M_Θ M_Y M_μ ε :
hM_Θ :
DREvalBounded Θ_set eval M_Θ
hM_Y :
h :
hM_μ :
hOverlap :
DRNuisanceOverlap S Θ_set h ε
hLoss_cont :
LossContinuousOnΘset (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) h
idx :
ℕ → (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set
idx_dense :
R :
ℕ → ℝ
hR :
RademacherBound (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) S_iid split h idx R
hclamp_minimizes :
DRClampedThetaMinimizes S Θ_set θ₀ eval ((M_Θ + 2 * M_μ + 2 * (M_Y + M_μ) / ε) ^ 2)
δ :
:
0 < δ
hδ' :
δ ≤ 1
∃ b : ℝ,
0 ≤ b ∧
LocalEmpProcessModulus (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) S_iid split (fun n => sqrt (if (split.foldB n).card = 0 then 2 * b else 2 * R n + 2 * b * sqrt (2 * log (1 / δ) / (split.foldB n).card))) δ h
Proof (Lean source)
theorem localEmpProcessModulus_drLearner [StandardBorelSpace P.Ω] [IsFiniteMeasure P.μ] [IsProbabilityMeasure P.μ] (S : CATEEstimationSystem P γ) (Θ : Type*) [NormedAddCommGroup Θ] [InnerProductSpace ℝ Θ] (Θ_set : Set Θ) (Θ_convex : Convex ℝ Θ_set) (θ₀ : Θ) (θ₀_mem : θ₀ ∈ Θ_set) (eval : Θ → γ → ℝ) (eval_meas : ∀ θ, Measurable (eval θ)) (eval_θ₀ : ∀ x, eval θ₀ x = S.τ_val x) (θ₀_minimizes : DRThetaMinimizes S Θ_set θ₀ eval) (S_iid : IIDSample P.Ω (γ × Bool × ℝ) P.μ S.toBackdoorEstimationSystem.P_Z) (split : OneShotSplit S_iid) {M_Θ M_Y M_μ ε : ℝ} (hM_Θ : DREvalBounded Θ_set eval M_Θ) (hM_Y : DROutcomeBounded S M_Y) (h : NuisanceVec γ) (hM_μ : DRNuisanceMuBounded h M_μ) (hOverlap : DRNuisanceOverlap S Θ_set h ε) (hLoss_cont : LossContinuousOnΘset (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) h) (idx : ℕ → (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set) (idx_dense : DenseRange idx) (R : ℕ → ℝ) (hR : RademacherBound (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) S_iid split h idx R) (hclamp_minimizes : DRClampedThetaMinimizes S Θ_set θ₀ eval ((M_Θ + 2 * M_μ + 2 * (M_Y + M_μ) / ε) ^ 2)) {δ : ℝ} (hδ : 0 < δ) (hδ' : δ ≤ 1) : ∃ b : ℝ, 0 ≤ b ∧ LocalEmpProcessModulus (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) S_iid split (fun n => sqrt (if (split.foldB n).card = 0 then 2 * b else 2 * R n + 2 * b * sqrt (2 * log (1 / δ) / (split.foldB n).card))) δ h := by have hb_loss := dr_loss_uniformly_bounded S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes hM_Θ hM_Y h hM_μ hOverlap set b : ℝ := (M_Θ + 2 * M_μ + 2 * (M_Y + M_μ) / ε) ^ 2 with hb_def have hb_nonneg : 0 ≤ b := sq_nonneg _ refine ⟨b, hb_nonneg, ?_⟩ exact localEmpProcessModulus_of_bounded_rademacher_ae (S := drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) (S_iid := S_iid) (split := split) (hb := hb_nonneg) (g := h) (hg_bdd_ae := hb_loss) (hg_cont := hLoss_cont) (idx := idx) (idx_dense := idx_dense) (R := R) hR (by simpa [DRClampedThetaMinimizes, hb_def, drLearningSystem] using hclamp_minimizes) hδ hδ'
1 supporting declaration (lemmas, instances)
  • dr_loss_uniformly_bounded theorem — Bridge: DR-Learner squared loss is bounded under (M_Θ, M_Y, M_μ, ε).
    Θ_set :
    Set Θ
    Θ_convex :
    Convex ℝ Θ_set
    θ₀ :
    Θ
    θ₀_mem :
    θ₀ ∈ Θ_set
    eval :
    Θ → γ → ℝ
    eval_meas :
    ∀ θ, Measurable (eval θ)
    eval_θ₀ :
    ∀ x, eval θ₀ x = S.τ_val x
    θ₀_minimizes :
    DRThetaMinimizes S Θ_set θ₀ eval
    M_Θ M_Y M_μ ε :
    hM_Θ :
    DREvalBounded Θ_set eval M_Θ
    hM_Y :
    h :
    hM_μ :
    hOverlap :
    DRNuisanceOverlap S Θ_set h ε
    UniformlyBoundedLossAE (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) h ((M_Θ + 2 * M_μ + 2 * (M_Y + M_μ) / ε) ^ 2)
    Proof (Lean source)
    theorem dr_loss_uniformly_bounded [StandardBorelSpace P.Ω] [IsFiniteMeasure P.μ] (S : CATEEstimationSystem P γ) (Θ : Type*) [NormedAddCommGroup Θ] [InnerProductSpace ℝ Θ] (Θ_set : Set Θ) (Θ_convex : Convex ℝ Θ_set) (θ₀ : Θ) (θ₀_mem : θ₀ ∈ Θ_set) (eval : Θ → γ → ℝ) (eval_meas : ∀ θ, Measurable (eval θ)) (eval_θ₀ : ∀ x, eval θ₀ x = S.τ_val x) (θ₀_minimizes : DRThetaMinimizes S Θ_set θ₀ eval) {M_Θ M_Y M_μ ε : ℝ} (hM_Θ : DREvalBounded Θ_set eval M_Θ) (hM_Y : DROutcomeBounded S M_Y) (h : NuisanceVec γ) (hM_μ : DRNuisanceMuBounded h M_μ) (hOverlap : DRNuisanceOverlap S Θ_set h ε) : UniformlyBoundedLossAE (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) h ((M_Θ + 2 * M_μ + 2 * (M_Y + M_μ) / ε) ^ 2) := by let Mφ : ℝ := 2 * M_μ + 2 * (M_Y + M_μ) / ε have hMφ_eq : M_Θ + 2 * M_μ + 2 * (M_Y + M_μ) / ε = M_Θ + Mφ := by change M_Θ + 2 * M_μ + 2 * (M_Y + M_μ) / ε = M_Θ + (2 * M_μ + 2 * (M_Y + M_μ) / ε) ring rw [hMφ_eq] rcases hM_Θ with ⟨hMΘ_nonneg, hEval⟩ rcases hM_Y with ⟨hMY_nonneg, hY_ae⟩ rcases hM_μ with ⟨hMμ_nonneg, hMu⟩ rcases hOverlap with ⟨hε_pos, _hε_half, hOverlap'⟩ filter_upwards [hY_ae] with z hz intro θ hθ let x : γ := z.1 let I : ℝ := BackdoorEstimationSystem.indA z have hI_nonneg : 0 ≤ I := by by_cases hz : z.2.1 = true · simp [I, BackdoorEstimationSystem.indA, BackdoorEstimationSystem.projA, hz] · simp [I, BackdoorEstimationSystem.indA, BackdoorEstimationSystem.projA, hz] have hI_le_one : I ≤ 1 := by by_cases hz : z.2.1 = true · simp [I, BackdoorEstimationSystem.indA, BackdoorEstimationSystem.projA, hz] · simp [I, BackdoorEstimationSystem.indA, BackdoorEstimationSystem.projA, hz] have hOneSubI_nonneg : 0 ≤ 1 - I := by linarith have hOneSubI_le_one : 1 - I ≤ 1 := by linarith have he_lower : ε ≤ h.e_fn x := (hOverlap' x).1 have he_upper : h.e_fn x ≤ 1 - ε := (hOverlap' x).2 have he_pos : 0 < h.e_fn x := lt_of_lt_of_le hε_pos he_lower have hden_lower : ε ≤ 1 - h.e_fn x := by linarith have hden_pos : 0 < 1 - h.e_fn x := lt_of_lt_of_le hε_pos hden_lower have hsum_nonneg : 0 ≤ M_Y + M_μ := by linarith have hdiff_mu : |h.μ_fn true x - h.μ_fn false x| ≤ 2 * M_μ := by have htri := abs_sub (h.μ_fn true x) (h.μ_fn false x) have ht := hMu true x have hf := hMu false x nlinarith have hres_true : |z.2.2 - h.μ_fn true x| ≤ M_Y + M_μ := by have htri := abs_sub z.2.2 (h.μ_fn true x) have hy := hz have hm := hMu true x nlinarith have hres_false : |z.2.2 - h.μ_fn false x| ≤ M_Y + M_μ := by have htri := abs_sub z.2.2 (h.μ_fn false x) have hy := hz have hm := hMu false x nlinarith have hratio_true : |I / h.e_fn x| ≤ 1 / ε := by have hinv_le : (h.e_fn x)⁻¹ ≤ ε⁻¹ := (inv_le_inv₀ he_pos hε_pos).2 he_lower have hmul₁ : I * (h.e_fn x)⁻¹ ≤ 1 * (h.e_fn x)⁻¹ := mul_le_mul_of_nonneg_right hI_le_one (inv_nonneg.mpr he_pos.le) have hmul₂ : 1 * (h.e_fn x)⁻¹ ≤ 1 * ε⁻¹ := mul_le_mul_of_nonneg_left hinv_le zero_le_one have hmul : I * (h.e_fn x)⁻¹ ≤ 1 * ε⁻¹ := le_trans hmul₁ hmul₂ simpa [div_eq_mul_inv, one_div, abs_of_nonneg hI_nonneg, abs_of_pos he_pos] using hmul have hratio_false : |(1 - I) / (1 - h.e_fn x)| ≤ 1 / ε := by have hinv_le : (1 - h.e_fn x)⁻¹ ≤ ε⁻¹ := (inv_le_inv₀ hden_pos hε_pos).2 hden_lower have hmul₁ : (1 - I) * (1 - h.e_fn x)⁻¹ ≤ 1 * (1 - h.e_fn x)⁻¹ := mul_le_mul_of_nonneg_right hOneSubI_le_one (inv_nonneg.mpr hden_pos.le) have hmul₂ : 1 * (1 - h.e_fn x)⁻¹ ≤ 1 * ε⁻¹ := mul_le_mul_of_nonneg_left hinv_le zero_le_one have hmul : (1 - I) * (1 - h.e_fn x)⁻¹ ≤ 1 * ε⁻¹ := le_trans hmul₁ hmul₂ simpa [div_eq_mul_inv, one_div, abs_of_nonneg hOneSubI_nonneg, abs_of_pos hden_pos] using hmul have hterm_true : |(I / h.e_fn x) * (z.2.2 - h.μ_fn true x)| ≤ (M_Y + M_μ) / ε := by rw [abs_mul] have hmul := mul_le_mul hratio_true hres_true (abs_nonneg _) (one_div_nonneg.mpr hε_pos.le) simpa [div_eq_mul_inv, one_div, mul_comm, mul_left_comm, mul_assoc] using hmul have hterm_false : |((1 - I) / (1 - h.e_fn x)) * (z.2.2 - h.μ_fn false x)| ≤ (M_Y + M_μ) / ε := by rw [abs_mul] have hmul := mul_le_mul hratio_false hres_false (abs_nonneg _) (one_div_nonneg.mpr hε_pos.le) simpa [div_eq_mul_inv, one_div, mul_comm, mul_left_comm, mul_assoc] using hmul have hphi : |phi_eta z h| ≤ Mφ := by have htri₁ := abs_sub ((h.μ_fn true x - h.μ_fn false x) + (I / h.e_fn x) * (z.2.2 - h.μ_fn true x)) (((1 - I) / (1 - h.e_fn x)) * (z.2.2 - h.μ_fn false x)) have htri₂ := abs_add_le (h.μ_fn true x - h.μ_fn false x) ((I / h.e_fn x) * (z.2.2 - h.μ_fn true x)) have hraw : |(h.μ_fn true x - h.μ_fn false x) + (I / h.e_fn x) * (z.2.2 - h.μ_fn true x) - ((1 - I) / (1 - h.e_fn x)) * (z.2.2 - h.μ_fn false x)| ≤ 2 * M_μ + (M_Y + M_μ) / ε + (M_Y + M_μ) / ε := by nlinarith have hraw' : |(h.μ_fn true x - h.μ_fn false x) + (I / h.e_fn x) * (z.2.2 - h.μ_fn true x) - ((1 - I) / (1 - h.e_fn x)) * (z.2.2 - h.μ_fn false x) - 0| ≤ Mφ := by have hraw₀ : |(h.μ_fn true x - h.μ_fn false x) + (I / h.e_fn x) * (z.2.2 - h.μ_fn true x) - ((1 - I) / (1 - h.e_fn x)) * (z.2.2 - h.μ_fn false x) - 0| ≤ 2 * M_μ + (M_Y + M_μ) / ε + (M_Y + M_μ) / ε := by simpa using hraw calc |(h.μ_fn true x - h.μ_fn false x) + (I / h.e_fn x) * (z.2.2 - h.μ_fn true x) - ((1 - I) / (1 - h.e_fn x)) * (z.2.2 - h.μ_fn false x) - 0| ≤ 2 * M_μ + (M_Y + M_μ) / ε + (M_Y + M_μ) / ε := hraw₀ _ = Mφ := by dsimp [Mφ] ring simpa [phi_eta, BackdoorEstimationSystem.aipwMoment, BackdoorEstimationSystem.projX, BackdoorEstimationSystem.projY, x, I] using hraw' have hdiff_loss : |phi_eta z h - eval θ z.1| ≤ Mφ + M_Θ := by have htri := abs_sub (phi_eta z h) (eval θ z.1) have heval := hEval θ hθ z.1 nlinarith have hsq : (phi_eta z h - eval θ z.1) ^ 2 ≤ (M_Θ + Mφ) ^ 2 := by have hbound : |phi_eta z h - eval θ z.1| ≤ M_Θ + Mφ := by linarith have hneg : -(M_Θ + Mφ) ≤ |phi_eta z h - eval θ z.1| := by have hnonneg := abs_nonneg (phi_eta z h - eval θ z.1) nlinarith simpa [sq_abs] using sq_le_sq' hneg hbound have hloss_nonneg : 0 ≤ (phi_eta z h - eval θ z.1) ^ 2 := sq_nonneg _ simpa [UniformlyBoundedLossAE, drLearningSystem, abs_of_nonneg hloss_nonneg] using hsq
    Causalean.Estimation.OrthogonalLearning.dr_loss_uniformly_bounded · Causalean/Estimation/CATE/OrthogonalLearning/LocalEmpProcess/DRLearner.lean:86
Localized­DRLearner 2 core · 1 supporting This file applies the localized empirical-process modulus theorem to the DR-Learner for conditional average treatment effects. ★ localEmpProcessModulus_localized_drLearner

Localized Modulus for the DR-Learner

This file applies the localized empirical-process modulus theorem to the DR-Learner for conditional average treatment effects. It formulates the critical-radius hypothesis for the centered quadratic DR-loss class and derives both the sharp localized modulus and a bounded-loss fallback modulus.

def DRCriticalRadius reviewed
Causalean.Estimation.OrthogonalLearning

Critical-radius hypothesis for the DR-Learner.

Definition (Lean source)
def DRCriticalRadius {Θ : Type*} [StandardBorelSpace P.Ω] [IsFiniteMeasure P.μ] (S : CATEEstimationSystem P γ) (Θ_set : Set Θ) (eval : Θ → γ → ℝ) (idx : ℕ → Θ_set) (h : NuisanceVec γ) (norm : ((γ × Bool × ℝ) → ℝ) → ℝ) (ψ : ℕ → ℝ → ℝ) : Prop := (∀ n, SubRoot (ψ n)) ∧ ∀ n : ℕ, RademacherUpperBound (fun (k : ℕ) (z : γ × Bool × ℝ) => (phi_eta z h - eval (idx k).val z.1) ^ 2 - (phi_eta z h - S.τ_val z.1) ^ 2) norm S.toBackdoorEstimationSystem.P_Z (id : (γ × Bool × ℝ) → γ × Bool × ℝ) n (ψ n)
theorem localEmpProcessModulus_localized_drLearner reviewed
Causalean.Estimation.OrthogonalLearning

DR-Learner localized modulus, sharp Foster–Syrgkanis form. For the DR-Learner CATE estimation system, suppose the evaluation functional is measurable in its parameter and recovers the true CATE at a parameter θ₀ in the constraint set, and that the evaluation functional, outcome, and fixed nuisance conditional-mean are uniformly bounded while the nuisance's propensity score satisfies ε-overlap. Assume the centred DR-loss is continuous in θ, a clamped version of θ₀ minimizes it, its population Rademacher complexity along a dense index sequence is controlled by a sub-root envelope ψ with respect to a seminorm that is invariant under almost-everywhere modification, and the same Rademacher upper bound extends to loss differences across the whole constraint set, together with Lipschitz and diameter control of the centred loss increments — nonnegative Lipschitz constant L, a diameter bound Rmax dominating every critical radius criticalRadius (ψ m), and the sub-root fixed-point property, plus boundedness and integrability of the empirical star-hull Rademacher process needed by the localization bridge and a confidence level in (0,1](0,1] together with the Foster–Syrgkanis critical-radius domination inequality across dyadic shell counts. Then there is a nonnegative envelope b such that the DR-Learner system satisfies the localized empirical-process modulus predicate at rate ρ n = (8L+3)·criticalRadius (ψ |B(n)|) on nonempty validation folds, falling back to √(2b) when the fold is empty.

Formal statement
Θ_set :
Set Θ
Θ_convex :
Convex ℝ Θ_set
Nonempty Θ_set
Countable Θ_set
θ₀ :
Θ
θ₀_mem :
θ₀ ∈ Θ_set
eval :
Θ → γ → ℝ
eval_meas :
∀ θ, Measurable (eval θ)
eval_θ₀ :
∀ x, eval θ₀ x = S.τ_val x
θ₀_minimizes :
DRThetaMinimizes S Θ_set θ₀ eval
S_iid :
IIDSample P.Ω (γ × Bool × ℝ) P.μ S.toBackdoorEstimationSystem.P_Z
split :
M_Θ M_Y M_μ ε :
hM_Θ :
DREvalBounded Θ_set eval M_Θ
hM_Y :
h :
hM_μ :
hOverlap :
DRNuisanceOverlap S Θ_set h ε
_hLoss_cont :
∀ z,
Continuous fun (θ : (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set)
=> (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z θ.val h
idx :
ℕ → (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set
_idx_dense :
norm :
((γ × Bool × ℝ) → ℝ) → ℝ
ψ :
ℕ → ℝ → ℝ
:
DRCriticalRadius S Θ_set eval (fun k => ⟨((idx k).val), (idx k).property⟩) h norm ψ
hnorm_ae :
∀ F F' : (γ × Bool × ℝ) → ℝ
if
F =ᵐ[S.toBackdoorEstimationSystem.P_Z] F'
then
norm F = norm F'
Rmax L :
hL_nonneg :
0 ≤ L
hF_lip :
∀ θ ∈ (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set,
norm (fun z => (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z θ h - (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).θ₀ h)
≤ L * ‖θ - (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).θ₀‖
hF_diam :
∀ θ ∈ (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set,
norm (fun z => (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z θ h - (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).θ₀ h)
≤ Rmax
hRmax_lb :
∀ m : ℕ, criticalRadius (ψ m) ≤ Rmax
hcrit_pos :
∀ m : ℕ, 0 < criticalRadius (ψ m)
hcrit_fp :
∀ m : ℕ, ψ m (criticalRadius (ψ m)) ≤ (criticalRadius (ψ m)) ^ 2
hψ_ub :
∀ m : ℕ,
RademacherUpperBound (fun (θ : (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set) (z : γ × Bool × ℝ) => (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z θ.val h - (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).θ₀ h) norm S.toBackdoorEstimationSystem.P_Z (id : (γ × Bool × ℝ) → γ × Bool × ℝ) m (ψ m)
BddAbove hypothesis needed by the bridge lemma inside `localized_uniform_deviation`.
hrad_bdd :
∀ m r,
∀ S_fin : Fin m → γ × Bool × ℝ,
∀ σ : Signs m,
BddAbove (Set.range fun p : starHullParam (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set => |(m : ℝ)⁻¹ * ∑ k : Fin m, (σ k : ℝ) * starHullZeroOut (fun (θ : (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set) (z : γ × Bool × ℝ) => (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z θ.val h - (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).θ₀ h) norm r p (S_fin k)|)
Integrability of the upper empirical Rademacher process (bridge prerequisite).
hrad_int :
∀ m r,
Integrable (fun ω : Fin m → γ × Bool × ℝ => empiricalRademacherComplexity m (starHullZeroOut (fun (θ : (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set) (z : γ × Bool × ℝ) => (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z θ.val h - (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).θ₀ h) norm r) ((id : (γ × Bool × ℝ) → (γ × Bool × ℝ)) ∘ ω)) (Measure.pi (fun _ => S.toBackdoorEstimationSystem.P_Z))
hclamp_minimizes :
CenteredClampedThetaMinimizes (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) (2 * (M_Θ + 2 * M_μ + 2 * (M_Y + M_μ) / ε) ^ 2)
δ :
:
0 < δ
hδ' :
δ ≤ 1
Foster–Syrgkanis Lemma 29 critical-radius lower bound (peeling-aware): for any dyadic shell count `K` covering `Rmax`, the McDiarmid slack at the union-bound-adjusted confidence `δ / (2 (K + 1))` (against the centred-loss bound `2 * (M_Θ + 2*M_μ + 2*(M_Y + M_μ)/ε)^2`) is dominated by the squared critical radius. Forwarded directly to `localEmpProcessModulus_of_localized_sharp`.
hδ_dom :
∀ n K : ℕ
if
0 < (split.foldB n).card
and
Rmax ≤ (criticalRadius (ψ (split.foldB n).card)) * (2 : ℝ) ^ K
then
2 * (M_Θ + 2 * M_μ + 2 * (M_Y + M_μ) / ε) ^ 2 * sqrt (2 * log (2 * ((K : ℝ) + 1) / δ) / (split.foldB n).card)
≤ (criticalRadius (ψ (split.foldB n).card)) ^ 2
∃ b : ℝ,
0 ≤ b ∧
LocalEmpProcessModulus (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) S_iid split (fun n => if (split.foldB n).card = 0 then sqrt (2 * b) else (8 * L + 3) * criticalRadius (ψ (split.foldB n).card)) δ h
Proof (Lean source)
theorem localEmpProcessModulus_localized_drLearner [StandardBorelSpace P.Ω] [IsFiniteMeasure P.μ] [IsProbabilityMeasure P.μ] (S : CATEEstimationSystem P γ) (Θ : Type*) [NormedAddCommGroup Θ] [InnerProductSpace ℝ Θ] (Θ_set : Set Θ) (Θ_convex : Convex ℝ Θ_set) [Nonempty Θ_set] [Countable Θ_set] (θ₀ : Θ) (θ₀_mem : θ₀ ∈ Θ_set) (eval : Θ → γ → ℝ) (eval_meas : ∀ θ, Measurable (eval θ)) (eval_θ₀ : ∀ x, eval θ₀ x = S.τ_val x) (θ₀_minimizes : DRThetaMinimizes S Θ_set θ₀ eval) (S_iid : IIDSample P.Ω (γ × Bool × ℝ) P.μ S.toBackdoorEstimationSystem.P_Z) (split : OneShotSplit S_iid) {M_Θ M_Y M_μ ε : ℝ} (hM_Θ : DREvalBounded Θ_set eval M_Θ) (hM_Y : DROutcomeBounded S M_Y) (h : NuisanceVec γ) (hM_μ : DRNuisanceMuBounded h M_μ) (hOverlap : DRNuisanceOverlap S Θ_set h ε) (_hLoss_cont : ∀ z, Continuous fun (θ : (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set) => (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z θ.val h) (idx : ℕ → (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set) (_idx_dense : DenseRange idx) {norm : ((γ × Bool × ℝ) → ℝ) → ℝ} {ψ : ℕ → ℝ → ℝ} (hψ : DRCriticalRadius S Θ_set eval (fun k => ⟨((idx k).val), (idx k).property⟩) h norm ψ) (hnorm_ae : ∀ F F' : (γ × Bool × ℝ) → ℝ, F =ᵐ[S.toBackdoorEstimationSystem.P_Z] F' → norm F = norm F') {Rmax L : ℝ} (hL_nonneg : 0 ≤ L) (hF_lip : ∀ θ ∈ (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set, norm (fun z => (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z θ h - (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).θ₀ h) ≤ L * ‖θ - (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).θ₀‖) (hF_diam : ∀ θ ∈ (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set, norm (fun z => (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z θ h - (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).θ₀ h) ≤ Rmax) (hRmax_lb : ∀ m : ℕ, criticalRadius (ψ m) ≤ Rmax) (hcrit_pos : ∀ m : ℕ, 0 < criticalRadius (ψ m)) (hcrit_fp : ∀ m : ℕ, ψ m (criticalRadius (ψ m)) ≤ (criticalRadius (ψ m)) ^ 2) (hψ_ub : ∀ m : ℕ, RademacherUpperBound (fun (θ : (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set) (z : γ × Bool × ℝ) => (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z θ.val h - (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).θ₀ h) norm S.toBackdoorEstimationSystem.P_Z (id : (γ × Bool × ℝ) → γ × Bool × ℝ) m (ψ m)) -- BddAbove hypothesis needed by the bridge lemma inside `localized_uniform_deviation`. (hrad_bdd : ∀ m r, ∀ S_fin : Fin m → γ × Bool × ℝ, ∀ σ : Signs m, BddAbove (Set.range fun p : starHullParam (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set => |(m : ℝ)⁻¹ * ∑ k : Fin m, (σ k : ℝ) * starHullZeroOut (fun (θ : (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set) (z : γ × Bool × ℝ) => (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z θ.val h - (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).θ₀ h) norm r p (S_fin k)|)) -- Integrability of the upper empirical Rademacher process (bridge prerequisite). (hrad_int : ∀ m r, Integrable (fun ω : Fin m → γ × Bool × ℝ => empiricalRademacherComplexity m (starHullZeroOut (fun (θ : (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set) (z : γ × Bool × ℝ) => (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z θ.val h - (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).θ₀ h) norm r) ((id : (γ × Bool × ℝ) → (γ × Bool × ℝ)) ∘ ω)) (Measure.pi (fun _ => S.toBackdoorEstimationSystem.P_Z))) (hclamp_minimizes : CenteredClampedThetaMinimizes (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) (2 * (M_Θ + 2 * M_μ + 2 * (M_Y + M_μ) / ε) ^ 2)) {δ : ℝ} (hδ : 0 < δ) (hδ' : δ ≤ 1) -- Foster–Syrgkanis Lemma 29 critical-radius lower bound (peeling-aware): -- for any dyadic shell count `K` covering `Rmax`, the McDiarmid slack at -- the union-bound-adjusted confidence `δ / (2 (K + 1))` (against the -- centred-loss bound `2 * (M_Θ + 2*M_μ + 2*(M_Y + M_μ)/ε)^2`) is -- dominated by the squared critical radius. Forwarded directly to -- `localEmpProcessModulus_of_localized_sharp`. (hδ_dom : ∀ n K : ℕ, 0 < (split.foldB n).card → Rmax ≤ (criticalRadius (ψ (split.foldB n).card)) * (2 : ℝ) ^ K → 2 * (M_Θ + 2 * M_μ + 2 * (M_Y + M_μ) / ε) ^ 2 * sqrt (2 * log (2 * ((K : ℝ) + 1) / δ) / (split.foldB n).card) ≤ (criticalRadius (ψ (split.foldB n).card)) ^ 2) : ∃ b : ℝ, 0 ≤ b ∧ LocalEmpProcessModulus (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) S_iid split (fun n => if (split.foldB n).card = 0 then sqrt (2 * b) else (8 * L + 3) * criticalRadius (ψ (split.foldB n).card)) δ h := by classical let Ssys := drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes haveI : Nonempty Ssys.Θ_set := inferInstanceAs (Nonempty Θ_set) haveI : Countable Ssys.Θ_set := inferInstanceAs (Countable Θ_set) haveI hPZ : IsProbabilityMeasure S.toBackdoorEstimationSystem.P_Z := by rw [← S_iid.law] exact Measure.isProbabilityMeasure_map (S_iid.meas 0).aemeasurable set b_loss : ℝ := (M_Θ + 2 * M_μ + 2 * (M_Y + M_μ) / ε) ^ 2 with hb_loss_def have hb_loss_nonneg : 0 ≤ b_loss := sq_nonneg _ have hb_loss : UniformlyBoundedLossAE Ssys h b_loss := dr_loss_uniformly_bounded S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes hM_Θ hM_Y h hM_μ hOverlap refine ⟨2 * b_loss, by linarith, ?_⟩ have h_centred_ae : ∀ᵐ z ∂S.toBackdoorEstimationSystem.P_Z, ∀ θ ∈ Ssys.Θ_set, |Ssys.ℓ z θ h - Ssys.ℓ z Ssys.θ₀ h| ≤ 2 * b_loss := by filter_upwards [hb_loss] with z hz intro θ hθ have h1 : |Ssys.ℓ z θ h| ≤ b_loss := hz θ hθ have h2 : |Ssys.ℓ z Ssys.θ₀ h| ≤ b_loss := hz Ssys.θ₀ Ssys.θ₀_mem have htri := abs_sub (Ssys.ℓ z θ h) (Ssys.ℓ z Ssys.θ₀ h) linarith have hℓ_meas_sys : ∀ θ ∈ Ssys.Θ_set, Measurable (fun z => Ssys.ℓ z θ h) := by intro θ _hθ exact Ssys.ℓ_meas θ h have hℓ_int_sys : ∀ θ ∈ Ssys.Θ_set, Integrable (fun z => Ssys.ℓ z θ h) S.toBackdoorEstimationSystem.P_Z := by intro θ hθ exact Integrable.of_bound (Ssys.ℓ_meas θ h).aestronglyMeasurable b_loss (by filter_upwards [hb_loss] with z hz simpa [Real.norm_eq_abs] using hz θ hθ) have hloss_eq : (fun (k : ℕ) (z : γ × Bool × ℝ) => Ssys.ℓ z (idx k).val h - Ssys.ℓ z Ssys.θ₀ h) = fun (k : ℕ) (z : γ × Bool × ℝ) => (phi_eta z h - eval (idx k).val z.1) ^ 2 - (phi_eta z h - S.τ_val z.1) ^ 2 := by funext k z change (phi_eta z h - eval (idx k).val z.1) ^ 2 - (phi_eta z h - eval θ₀ z.1) ^ 2 = (phi_eta z h - eval (idx k).val z.1) ^ 2 - (phi_eta z h - S.τ_val z.1) ^ 2 rw [eval_θ₀ z.1] have hreg : LocalizedRademacherRegimeAE Ssys S_iid split h idx norm ψ (2 * b_loss) := by refine ⟨by linarith, h_centred_ae, hψ.1, ?_⟩ intro n have hub := hψ.2 (split.foldB n).card rw [hloss_eq] exact hub -- The caller's `hδ_dom` is stated against `2 * (M_Θ + Mφ)^2 = 2 * b_loss` (by `hb_loss_def`). exact localEmpProcessModulus_of_localized_sharp_ae (S := Ssys) (S_iid := S_iid) (split := split) (g := h) (_hg_cont := _hLoss_cont) (idx := idx) (_idx_dense := _idx_dense) (norm := norm) (hnorm_ae := hnorm_ae) (ψ := ψ) (L := L) (b := 2 * b_loss) (Rmax := Rmax) (hreg := hreg) (hL_nonneg := hL_nonneg) (hF_lip := hF_lip) (hℓ_meas := hℓ_meas_sys) (hℓ_int := hℓ_int_sys) (hF_diam := hF_diam) (hRmax_lb := hRmax_lb) (hcrit_pos := hcrit_pos) (hcrit_fp := hcrit_fp) (hψ_ub := hψ_ub) (hrad_bdd := fun m r S_fin σ => hrad_bdd m r S_fin σ) (hrad_int := fun m r => hrad_int m r) (hclamp_minimizes := by simpa [Ssys, hb_loss_def] using hclamp_minimizes) (hδ := hδ) (hδ' := hδ') (hδ_dom := hδ_dom)
1 supporting declaration (lemmas, instances)
  • localEmpProcessModulus_localized_drLearner_bounded theorem — DR-Learner bounded-loss localized modulus.
    Θ_set :
    Set Θ
    Θ_convex :
    Convex ℝ Θ_set
    Nonempty Θ_set
    θ₀ :
    Θ
    θ₀_mem :
    θ₀ ∈ Θ_set
    eval :
    Θ → γ → ℝ
    eval_meas :
    ∀ θ, Measurable (eval θ)
    eval_θ₀ :
    ∀ x, eval θ₀ x = S.τ_val x
    θ₀_minimizes :
    DRThetaMinimizes S Θ_set θ₀ eval
    S_iid :
    IIDSample P.Ω (γ × Bool × ℝ) P.μ S.toBackdoorEstimationSystem.P_Z
    split :
    M_Θ M_Y M_μ ε :
    hM_Θ :
    DREvalBounded Θ_set eval M_Θ
    hM_Y :
    h :
    hM_μ :
    hOverlap :
    DRNuisanceOverlap S Θ_set h ε
    _hLoss_cont :
    ∀ z,
    Continuous fun (θ : (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set)
    => (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z θ.val h
    idx :
    ℕ → (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set
    _idx_dense :
    norm :
    ((γ × Bool × ℝ) → ℝ) → ℝ
    ψ :
    ℕ → ℝ → ℝ
    :
    DRCriticalRadius S Θ_set eval (fun k => ⟨((idx k).val), (idx k).property⟩) h norm ψ
    δ :
    :
    0 < δ
    hδ' :
    δ ≤ 1
    ∃ b : ℝ,
    0 ≤ b ∧
    LocalEmpProcessModulus (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) S_iid split (fun _n => sqrt (2 * (2 * b))) δ h
    Proof (Lean source)
    theorem localEmpProcessModulus_localized_drLearner_bounded [StandardBorelSpace P.Ω] [IsFiniteMeasure P.μ] [IsProbabilityMeasure P.μ] (S : CATEEstimationSystem P γ) (Θ : Type*) [NormedAddCommGroup Θ] [InnerProductSpace ℝ Θ] (Θ_set : Set Θ) (Θ_convex : Convex ℝ Θ_set) [Nonempty Θ_set] (θ₀ : Θ) (θ₀_mem : θ₀ ∈ Θ_set) (eval : Θ → γ → ℝ) (eval_meas : ∀ θ, Measurable (eval θ)) (eval_θ₀ : ∀ x, eval θ₀ x = S.τ_val x) (θ₀_minimizes : DRThetaMinimizes S Θ_set θ₀ eval) (S_iid : IIDSample P.Ω (γ × Bool × ℝ) P.μ S.toBackdoorEstimationSystem.P_Z) (split : OneShotSplit S_iid) {M_Θ M_Y M_μ ε : ℝ} (hM_Θ : DREvalBounded Θ_set eval M_Θ) (hM_Y : DROutcomeBounded S M_Y) (h : NuisanceVec γ) (hM_μ : DRNuisanceMuBounded h M_μ) (hOverlap : DRNuisanceOverlap S Θ_set h ε) (_hLoss_cont : ∀ z, Continuous fun (θ : (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set) => (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z θ.val h) (idx : ℕ → (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set) (_idx_dense : DenseRange idx) {norm : ((γ × Bool × ℝ) → ℝ) → ℝ} {ψ : ℕ → ℝ → ℝ} (hψ : DRCriticalRadius S Θ_set eval (fun k => ⟨((idx k).val), (idx k).property⟩) h norm ψ) {δ : ℝ} (hδ : 0 < δ) (hδ' : δ ≤ 1) : ∃ b : ℝ, 0 ≤ b ∧ LocalEmpProcessModulus (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) S_iid split (fun _n => sqrt (2 * (2 * b))) δ h := by classical let Ssys := drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes -- Nonempty instance for the system's parameter set. haveI : Nonempty Ssys.Θ_set := inferInstanceAs (Nonempty Θ_set) -- Probability-measure instance for P_Z (used by populationRisk integrability). haveI hPZ : IsProbabilityMeasure S.toBackdoorEstimationSystem.P_Z := by rw [← S_iid.law] exact Measure.isProbabilityMeasure_map (S_iid.meas 0).aemeasurable -- Step 1: extract uniform loss bound `b`. set b : ℝ := (M_Θ + 2 * M_μ + 2 * (M_Y + M_μ) / ε) ^ 2 with hb_def have hb_nonneg : 0 ≤ b := sq_nonneg _ have hb_loss : UniformlyBoundedLossAE Ssys h b := dr_loss_uniformly_bounded S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes hM_Θ hM_Y h hM_μ hOverlap refine ⟨b, hb_nonneg, ?_⟩ -- Step 2: derive the centred a.e. bound (factor 2). have h_centred_ae : ∀ᵐ z ∂S.toBackdoorEstimationSystem.P_Z, ∀ θ ∈ Ssys.Θ_set, |Ssys.ℓ z θ h - Ssys.ℓ z Ssys.θ₀ h| ≤ 2 * b := by filter_upwards [hb_loss] with z hz intro θ hθ have h1 : |Ssys.ℓ z θ h| ≤ b := hz θ hθ have h2 : |Ssys.ℓ z Ssys.θ₀ h| ≤ b := hz Ssys.θ₀ Ssys.θ₀_mem have htri := abs_sub (Ssys.ℓ z θ h) (Ssys.ℓ z Ssys.θ₀ h) linarith -- Step 3: derive the centred population bound via integration. have h_centred_pop : ∀ θ ∈ Ssys.Θ_set, |Ssys.L θ h - Ssys.L Ssys.θ₀ h| ≤ 2 * b := by intro θ hθ have hint_θ : Integrable (fun z => Ssys.ℓ z θ h) S.toBackdoorEstimationSystem.P_Z := Integrable.of_bound (Ssys.ℓ_meas θ h).aestronglyMeasurable b (by filter_upwards [hb_loss] with z hz simpa [Real.norm_eq_abs] using hz θ hθ) have hint_0 : Integrable (fun z => Ssys.ℓ z Ssys.θ₀ h) S.toBackdoorEstimationSystem.P_Z := Integrable.of_bound (Ssys.ℓ_meas Ssys.θ₀ h).aestronglyMeasurable b (by filter_upwards [hb_loss] with z hz simpa [Real.norm_eq_abs] using hz Ssys.θ₀ Ssys.θ₀_mem) have hL_eq : Ssys.L θ h - Ssys.L Ssys.θ₀ h = ∫ z, Ssys.ℓ z θ h - Ssys.ℓ z Ssys.θ₀ h ∂S.toBackdoorEstimationSystem.P_Z := by change (∫ z, Ssys.ℓ z θ h ∂S.toBackdoorEstimationSystem.P_Z) - (∫ z, Ssys.ℓ z Ssys.θ₀ h ∂S.toBackdoorEstimationSystem.P_Z) = _ rw [← integral_sub hint_θ hint_0] rw [hL_eq] calc |∫ z, Ssys.ℓ z θ h - Ssys.ℓ z Ssys.θ₀ h ∂S.toBackdoorEstimationSystem.P_Z| ≤ ∫ z, |Ssys.ℓ z θ h - Ssys.ℓ z Ssys.θ₀ h| ∂S.toBackdoorEstimationSystem.P_Z := abs_integral_le_integral_abs _ ≤ ∫ _z, 2 * b ∂S.toBackdoorEstimationSystem.P_Z := by apply integral_mono_ae · exact (hint_θ.sub hint_0).abs · exact integrable_const _ · filter_upwards [h_centred_ae] with z hz exact hz θ hθ _ = 2 * b := by simp -- Step 4: package the LocalizedRademacherRegime for Ssys. have hloss_eq : (fun (k : ℕ) (z : γ × Bool × ℝ) => Ssys.ℓ z (idx k).val h - Ssys.ℓ z Ssys.θ₀ h) = fun (k : ℕ) (z : γ × Bool × ℝ) => (phi_eta z h - eval (idx k).val z.1) ^ 2 - (phi_eta z h - S.τ_val z.1) ^ 2 := by funext k z change (phi_eta z h - eval (idx k).val z.1) ^ 2 - (phi_eta z h - eval θ₀ z.1) ^ 2 = (phi_eta z h - eval (idx k).val z.1) ^ 2 - (phi_eta z h - S.τ_val z.1) ^ 2 rw [eval_θ₀ z.1] have hreg : LocalizedRademacherRegimeAE Ssys S_iid split h idx norm ψ (2 * b) := by refine ⟨by linarith, h_centred_ae, hψ.1, ?_⟩ intro n have hub := hψ.2 (split.foldB n).card rw [hloss_eq] exact hub -- Step 5: apply the bounded fallback bridge. exact localEmpProcessModulus_of_localized_bounded_ae (S := Ssys) (S_iid := S_iid) (split := split) (g := h) (idx := idx) (norm := norm) (ψ := ψ) (b := 2 * b) (hreg := hreg) (hpop_center := h_centred_pop) (_hδ := hδ) (_hδ' := hδ')
    Causalean.Estimation.OrthogonalLearning.localEmpProcessModulus_localized_drLearner_bounded · Causalean/Estimation/CATE/OrthogonalLearning/LocalEmpProcess/LocalizedDRLearner.lean:306
DRLearner­End­To­End 1 core · 0 supporting This file composes the generic plug-in oracle inequality with the global Rademacher modulus for the doubly robust learner for conditional treatment effects. ★ oracle_inequality_drLearner_highProb

DR-Learner Oracle Chain

This file composes the generic plug-in oracle inequality with the global Rademacher modulus for the doubly robust learner for conditional treatment effects. It yields a high-probability squared-error bound whose leading term is the empirical-process modulus and whose remaining terms are nuisance bias and optimization slack.

theorem oracle_inequality_drLearner_highProb reviewed
Causalean.Estimation.OrthogonalLearning

DR-Learner end-to-end high-probability oracle inequality. Consider the same doubly robust orthogonal-learning system, one-shot sample split, and boundedness / overlap / continuity / Rademacher package as in localEmpProcessModulus_drLearnertruth-identifying admissibility and evaluation correctness, uniform bounds on the candidate evaluations, the outcome, and the realised nuisance regression together with propensity overlap, and loss continuity, a Rademacher bound R n, the clamped-loss minimizer property, and a confidence level δ in (0, 1] . Assume in addition that a plug-in empirical-risk-minimisation estimator sequence τhat, evaluated against the same realised nuisance on every cross-fitting fold, attains the empirical risk up to an optimization slack r_opt n, that the population risk is strongly convex at the realised nuisance with modulus σ > 0, and that the truth-identifying candidate satisfies the first-order optimality inequality for the population risk's directional derivative at the truth. Then there is a nonnegative constant b such that, for every sample size n, with P.μ-probability at least 1 − δ the estimation error obeys ‖τhat n ω − θ₀‖² ≤ (4(1+σ)/σ²) · ρ_{n,δ}² + (4/σ) · Bias_n + (4/σ) · r_opt n, where ρ_{n,δ} is the modulus rate realised by localEmpProcessModulus_drLearner.

Formal statement
Θ_set :
Set Θ
Θ_convex :
Convex ℝ Θ_set
θ₀ :
Θ
θ₀_mem :
θ₀ ∈ Θ_set
eval :
Θ → γ → ℝ
eval_meas :
∀ θ, Measurable (eval θ)
eval_θ₀ :
∀ x, eval θ₀ x = S.τ_val x
θ₀_minimizes :
DRThetaMinimizes S Θ_set θ₀ eval
S_iid :
IIDSample P.Ω (γ × Bool × ℝ) P.μ S.toBackdoorEstimationSystem.P_Z
split :
M_Θ M_Y M_μ ε :
hM_Θ :
DREvalBounded Θ_set eval M_Θ
hM_Y :
h :
hM_μ :
hOverlap :
DRNuisanceOverlap S Θ_set h ε
hLoss_cont :
LossContinuousOnΘset (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) h
idx :
ℕ → (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set
idx_dense :
R :
ℕ → ℝ
hR :
RademacherBound (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) S_iid split h idx R
hclamp_minimizes :
DRClampedThetaMinimizes S Θ_set θ₀ eval ((M_Θ + 2 * M_μ + 2 * (M_Y + M_μ) / ε) ^ 2)
δ :
:
0 < δ
hδ' :
δ ≤ 1
Generic oracle-inequality ingredients specialised to the DR system.
Dθ_truth :
HasDirDerivTheta (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).g₀
Dθ_at_h :
HasDirDerivTheta (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) h
τhat :
ℕ → P.Ω → Θ
r_opt :
ℕ → ℝ
hPluginERM :
SampleSplitPluginERM (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) S_iid split τhat (fun _ _ => h) r_opt
σ :
:
0 < σ
hSC :
∀ θ ∈ Θ_set,
(drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).L θ h
- (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).L θ₀ h
≥ (∫ z, Dθ_at_h.dℓ_θ θ z ∂S.toBackdoorEstimationSystem.P_Z) + (σ / 2) * ‖θ - θ₀‖ ^ 2
hFOI :
FirstOrderInequality (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) Dθ_truth
∃ b : ℝ,
0 ≤ b ∧
∀ n : ℕ,
∃ E : Set P.Ω,
P.μ E ≥ 1 - ofReal δ ∧
∀ ω ∈ E,
‖τhat n ω - θ₀‖ ^ 2
≤ (4 * (1 + σ) / σ ^ 2) * (sqrt (if (split.foldB n).card = 0 then 2 * b else 2 * R n + 2 * b * sqrt (2 * log (1 / δ) / (split.foldB n).card))) ^ 2
+ (4 / σ) * Bias_n (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) Dθ_truth Dθ_at_h (τhat n ω)
+ (4 / σ) * r_opt n
Proof (Lean source)
theorem oracle_inequality_drLearner_highProb [StandardBorelSpace P.Ω] [IsFiniteMeasure P.μ] [IsProbabilityMeasure P.μ] (S : CATEEstimationSystem P γ) (Θ : Type*) [NormedAddCommGroup Θ] [InnerProductSpace ℝ Θ] (Θ_set : Set Θ) (Θ_convex : Convex ℝ Θ_set) (θ₀ : Θ) (θ₀_mem : θ₀ ∈ Θ_set) (eval : Θ → γ → ℝ) (eval_meas : ∀ θ, Measurable (eval θ)) (eval_θ₀ : ∀ x, eval θ₀ x = S.τ_val x) (θ₀_minimizes : DRThetaMinimizes S Θ_set θ₀ eval) (S_iid : IIDSample P.Ω (γ × Bool × ℝ) P.μ S.toBackdoorEstimationSystem.P_Z) (split : OneShotSplit S_iid) {M_Θ M_Y M_μ ε : ℝ} (hM_Θ : DREvalBounded Θ_set eval M_Θ) (hM_Y : DROutcomeBounded S M_Y) (h : NuisanceVec γ) (hM_μ : DRNuisanceMuBounded h M_μ) (hOverlap : DRNuisanceOverlap S Θ_set h ε) (hLoss_cont : LossContinuousOnΘset (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) h) (idx : ℕ → (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set) (idx_dense : DenseRange idx) (R : ℕ → ℝ) (hR : RademacherBound (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) S_iid split h idx R) (hclamp_minimizes : DRClampedThetaMinimizes S Θ_set θ₀ eval ((M_Θ + 2 * M_μ + 2 * (M_Y + M_μ) / ε) ^ 2)) {δ : ℝ} (hδ : 0 < δ) (hδ' : δ ≤ 1) -- Generic oracle-inequality ingredients specialised to the DR system. (Dθ_truth : HasDirDerivTheta (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).g₀) (Dθ_at_h : HasDirDerivTheta (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) h) (τhat : ℕ → P.Ω → Θ) (r_opt : ℕ → ℝ) (hPluginERM : SampleSplitPluginERM (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) S_iid split τhat (fun _ _ => h) r_opt) (σ : ℝ) (hσ : 0 < σ) (hSC : ∀ θ ∈ Θ_set, (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).L θ h - (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).L θ₀ h ≥ (∫ z, Dθ_at_h.dℓ_θ θ z ∂S.toBackdoorEstimationSystem.P_Z) + (σ / 2) * ‖θ - θ₀‖ ^ 2) (hFOI : FirstOrderInequality (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) Dθ_truth) : ∃ b : ℝ, 0 ≤ b ∧ ∀ n : ℕ, ∃ E : Set P.Ω, MeasurableSet E ∧ P.μ E ≥ 1 - ofReal δ ∧ ∀ ω ∈ E, ‖τhat n ω - θ₀‖ ^ 2 ≤ (4 * (1 + σ) / σ ^ 2) * (sqrt (if (split.foldB n).card = 0 then 2 * b else 2 * R n + 2 * b * sqrt (2 * log (1 / δ) / (split.foldB n).card))) ^ 2 + (4 / σ) * Bias_n (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) Dθ_truth Dθ_at_h (τhat n ω) + (4 / σ) * r_opt n := by rcases localEmpProcessModulus_drLearner S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes S_iid split hM_Θ hM_Y h hM_μ hOverlap hLoss_cont idx idx_dense R hR hclamp_minimizes hδ hδ' with ⟨b, hb_nonneg, hMod⟩ refine ⟨b, hb_nonneg, ?_⟩ exact oracle_inequality_plugin_ERM_highProb (S := drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) (S_iid := S_iid) (split := split) (Dθ_truth := Dθ_truth) (g := h) (Dθ_at_g := Dθ_at_h) (θhat := τhat) (r_opt := r_opt) (hPluginERM := hPluginERM) (σ := σ) (hσ := hσ) (hSC := hSC) (hFOI := hFOI) (ρ := fun n => sqrt (if (split.foldB n).card = 0 then 2 * b else 2 * R n + 2 * b * sqrt (2 * log (1 / δ) / (split.foldB n).card))) (δ := δ) hMod
DRLearner­Honest 1 core · 0 supporting This file combines the doubly robust learner oracle inequality with the second-order product-bias bound. ★ oracle_inequality_drLearner_highProb_honest

Product-Bias DR-Learner Oracle Bound

This file combines the doubly robust learner oracle inequality with the second-order product-bias bound. The resulting high-probability statement makes the error contribution from the two nuisance estimators explicit as a product of their conditional mean and propensity-score errors. The main theorem is oracle_inequality_drLearner_highProb_honest; the declaration keeps the historical suffix _honest, where "honest" means that the bound exposes the second-order product term instead of leaving it as the symbolic Bias_n.

theorem oracle_inequality_drLearner_highProb_honest reviewed
Causalean.Estimation.OrthogonalLearning

Product-bias DR-Learner oracle inequality. Consider a CATE estimation system satisfying the underlying backdoor identification assumptions, with an evaluation functional that is measurable in its parameter and recovers the true CATE at a parameter θ₀ belonging to the constraint set, and suppose the evaluation functional, the outcome, and a fixed plug-in nuisance's conditional-mean function are uniformly bounded while that nuisance's propensity score satisfies ε-overlap. Assume further the DR-loss is continuous in the parameter, its localized empirical process admits a Rademacher complexity bound, and a clamped version of θ₀ still minimizes the loss, the confidence level lies in (0,1](0,1], the overlap parameter is positive, and both the true and the plug-in nuisance lie in the ε-overlap slice, and that the estimator sequence stays within the constraint set as an approximate sample-split plug-in empirical-risk minimizer (slack r_opt) whose excess population risk satisfies a strong-convexity-type lower bound with constant σ>0 and a first-order orthogonality inequality at the true nuisance's directional derivative. Finally, a battery of boundedness and integrability conditions bounds the directional-derivative envelope and controls the two nuisance-error terms entering the second-order bias decomposition. Then for every sample fold n, with probability at least 1-δ the squared estimation error of the DR-Learner is bounded by an oracle/Rademacher term plus an explicit second-order product-bias term (2B/ε)·Σ_a ‖Δμ_a‖₂·‖Δe‖₂ in the two nuisance L²-errors, plus an optimization slack.

Formal statement
hA :
S.toPOBackdoorSystem.Assumptions
Θ_set :
Set Θ
Θ_convex :
Convex ℝ Θ_set
θ₀ :
Θ
θ₀_mem :
θ₀ ∈ Θ_set
eval :
Θ → γ → ℝ
eval_meas :
∀ θ, Measurable (eval θ)
eval_θ₀ :
∀ x, eval θ₀ x = S.τ_val x
θ₀_minimizes :
DRThetaMinimizes S Θ_set θ₀ eval
S_iid :
IIDSample P.Ω (γ × Bool × ℝ) P.μ S.toBackdoorEstimationSystem.P_Z
split :
M_Θ M_Y M_μ ε :
hM_Θ :
DREvalBounded Θ_set eval M_Θ
hM_Y :
h :
hM_μ :
hOverlap :
DRNuisanceOverlap S Θ_set h ε
hLoss_cont :
LossContinuousOnΘset (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) h
idx :
ℕ → (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set
idx_dense :
R :
ℕ → ℝ
hR :
RademacherBound (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) S_iid split h idx R
hclamp_minimizes :
DRClampedThetaMinimizes S Θ_set θ₀ eval ((M_Θ + 2 * M_μ + 2 * (M_Y + M_μ) / ε) ^ 2)
δ :
:
0 < δ
hδ' :
δ ≤ 1
hε_pos :
0 < ε
h_overlap_η₀ :
S.toBackdoorEstimationSystem.η₀ ∈ BackdoorEstimationSystem.H_ε (γ := γ) ε
h_overlap_h :
h ∈ BackdoorEstimationSystem.H_ε (γ := γ) ε
D :
EvalDirDeriv Θ_set θ₀ eval
ND :
NuisanceDirDeriv S.toBackdoorEstimationSystem.η₀
τhat :
ℕ → P.Ω → Θ
hτ_mem :
∀ n ω, τhat n ω ∈ Θ_set
r_opt :
ℕ → ℝ
hPluginERM :
SampleSplitPluginERM (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) S_iid split τhat (fun _ _ => h) r_opt
σ :
:
0 < σ
hSC :
∀ θ ∈ Θ_set,
(drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).L θ h
- (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).L θ₀ h
≥ (∫ z, ((drMixedDirDeriv S hA hε_pos h_overlap_η₀ Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes D ND).Dθ_at h).dℓ_θ θ z ∂S.toBackdoorEstimationSystem.P_Z)
+ (σ / 2) * ‖θ - θ₀‖ ^ 2
hFOI :
FirstOrderInequality (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) ((drMixedDirDeriv S hA hε_pos h_overlap_η₀ Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes D ND).Dθ_at S.toBackdoorEstimationSystem.η₀)
B :
hB_nonneg :
0 ≤ B
hdEval_unif :
∀ θ ∈ Θ_set, ∀ x, |D.dEval θ x| ≤ B
h_μ_h_int :
∀ a : Bool, Integrable (fun ω => h.μ_fn a (S.toBackdoorEstimationSystem.factualX ω)) P.μ
h_phi_int :
Integrable (fun ω => phi_eta (S.toBackdoorEstimationSystem.factualZ ω) h - phi₀ S (S.toBackdoorEstimationSystem.factualZ ω)) P.μ
h_phiw_int :
∀ n ω,
Integrable (fun ω' => (phi_eta (S.toBackdoorEstimationSystem.factualZ ω') h - phi₀ S (S.toBackdoorEstimationSystem.factualZ ω')) * D.dEval (τhat n ω) (S.toBackdoorEstimationSystem.factualX ω')) P.μ
hΔμ_memLp :
∀ a, MemLp (fun x => h.μ_fn a x - S.μ_val a x) 2 S.toBackdoorEstimationSystem.P_X
hΔe_memLp :
MemLp (fun x => h.e_fn x - S.e_val x) 2 S.toBackdoorEstimationSystem.P_X
hA_int :
∀ n ω,
Integrable (fun z => ((drMixedDirDeriv S hA hε_pos h_overlap_η₀ Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes D ND).Dθ_at S.toBackdoorEstimationSystem.η₀).dℓ_θ (τhat n ω) z) S.toBackdoorEstimationSystem.P_Z
hB_int :
∀ n ω,
Integrable (fun z => ((drMixedDirDeriv S hA hε_pos h_overlap_η₀ Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes D ND).Dθ_at h).dℓ_θ (τhat n ω) z) S.toBackdoorEstimationSystem.P_Z
∃ b : ℝ,
0 ≤ b ∧
∀ n : ℕ,
∃ E : Set P.Ω,
P.μ E ≥ 1 - ofReal δ ∧
∀ ω ∈ E,
‖τhat n ω - θ₀‖ ^ 2
≤ (4 * (1 + σ) / σ ^ 2) * (sqrt (if (split.foldB n).card = 0 then 2 * b else 2 * R n + 2 * b * sqrt (2 * log (1 / δ) / (split.foldB n).card))) ^ 2
+ (4 / σ) * ((2 * B / ε) * ∑ a : Bool, (eLpNorm (fun x => h.μ_fn a x - S.μ_val a x) 2 S.toBackdoorEstimationSystem.P_X).toReal * (eLpNorm (fun x => h.e_fn x - S.e_val x) 2 S.toBackdoorEstimationSystem.P_X).toReal)
+ (4 / σ) * r_opt n
Proof (Lean source)
theorem oracle_inequality_drLearner_highProb_honest [StandardBorelSpace P.Ω] [IsFiniteMeasure P.μ] [IsProbabilityMeasure P.μ] (S : CATEEstimationSystem P γ) (hA : S.toPOBackdoorSystem.Assumptions) (Θ : Type*) [NormedAddCommGroup Θ] [InnerProductSpace ℝ Θ] (Θ_set : Set Θ) (Θ_convex : Convex ℝ Θ_set) (θ₀ : Θ) (θ₀_mem : θ₀ ∈ Θ_set) (eval : Θ → γ → ℝ) (eval_meas : ∀ θ, Measurable (eval θ)) (eval_θ₀ : ∀ x, eval θ₀ x = S.τ_val x) (θ₀_minimizes : DRThetaMinimizes S Θ_set θ₀ eval) (S_iid : IIDSample P.Ω (γ × Bool × ℝ) P.μ S.toBackdoorEstimationSystem.P_Z) (split : OneShotSplit S_iid) {M_Θ M_Y M_μ ε : ℝ} (hM_Θ : DREvalBounded Θ_set eval M_Θ) (hM_Y : DROutcomeBounded S M_Y) (h : NuisanceVec γ) (hM_μ : DRNuisanceMuBounded h M_μ) (hOverlap : DRNuisanceOverlap S Θ_set h ε) (hLoss_cont : LossContinuousOnΘset (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) h) (idx : ℕ → (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set) (idx_dense : DenseRange idx) (R : ℕ → ℝ) (hR : RademacherBound (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) S_iid split h idx R) (hclamp_minimizes : DRClampedThetaMinimizes S Θ_set θ₀ eval ((M_Θ + 2 * M_μ + 2 * (M_Y + M_μ) / ε) ^ 2)) {δ : ℝ} (hδ : 0 < δ) (hδ' : δ ≤ 1) (hε_pos : 0 < ε) (h_overlap_η₀ : S.toBackdoorEstimationSystem.η₀ ∈ BackdoorEstimationSystem.H_ε (γ := γ) ε) (h_overlap_h : h ∈ BackdoorEstimationSystem.H_ε (γ := γ) ε) (D : EvalDirDeriv Θ_set θ₀ eval) (ND : NuisanceDirDeriv S.toBackdoorEstimationSystem.η₀) (τhat : ℕ → P.Ω → Θ) (hτ_mem : ∀ n ω, τhat n ω ∈ Θ_set) (r_opt : ℕ → ℝ) (hPluginERM : SampleSplitPluginERM (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) S_iid split τhat (fun _ _ => h) r_opt) (σ : ℝ) (hσ : 0 < σ) (hSC : ∀ θ ∈ Θ_set, (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).L θ h - (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).L θ₀ h ≥ (∫ z, ((drMixedDirDeriv S hA hε_pos h_overlap_η₀ Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes D ND).Dθ_at h).dℓ_θ θ z ∂S.toBackdoorEstimationSystem.P_Z) + (σ / 2) * ‖θ - θ₀‖ ^ 2) (hFOI : FirstOrderInequality (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) ((drMixedDirDeriv S hA hε_pos h_overlap_η₀ Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes D ND).Dθ_at S.toBackdoorEstimationSystem.η₀)) {B : ℝ} (hB_nonneg : 0 ≤ B) (hdEval_unif : ∀ θ ∈ Θ_set, ∀ x, |D.dEval θ x| ≤ B) (h_μ_h_int : ∀ a : Bool, Integrable (fun ω => h.μ_fn a (S.toBackdoorEstimationSystem.factualX ω)) P.μ) (h_phi_int : Integrable (fun ω => phi_eta (S.toBackdoorEstimationSystem.factualZ ω) h - phi₀ S (S.toBackdoorEstimationSystem.factualZ ω)) P.μ) (h_phiw_int : ∀ n ω, Integrable (fun ω' => (phi_eta (S.toBackdoorEstimationSystem.factualZ ω') h - phi₀ S (S.toBackdoorEstimationSystem.factualZ ω')) * D.dEval (τhat n ω) (S.toBackdoorEstimationSystem.factualX ω')) P.μ) (hΔμ_memLp : ∀ a, MemLp (fun x => h.μ_fn a x - S.μ_val a x) 2 S.toBackdoorEstimationSystem.P_X) (hΔe_memLp : MemLp (fun x => h.e_fn x - S.e_val x) 2 S.toBackdoorEstimationSystem.P_X) (hA_int : ∀ n ω, Integrable (fun z => ((drMixedDirDeriv S hA hε_pos h_overlap_η₀ Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes D ND).Dθ_at S.toBackdoorEstimationSystem.η₀).dℓ_θ (τhat n ω) z) S.toBackdoorEstimationSystem.P_Z) (hB_int : ∀ n ω, Integrable (fun z => ((drMixedDirDeriv S hA hε_pos h_overlap_η₀ Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes D ND).Dθ_at h).dℓ_θ (τhat n ω) z) S.toBackdoorEstimationSystem.P_Z) : ∃ b : ℝ, 0 ≤ b ∧ ∀ n : ℕ, ∃ E : Set P.Ω, MeasurableSet E ∧ P.μ E ≥ 1 - ofReal δ ∧ ∀ ω ∈ E, ‖τhat n ω - θ₀‖ ^ 2 ≤ (4 * (1 + σ) / σ ^ 2) * (sqrt (if (split.foldB n).card = 0 then 2 * b else 2 * R n + 2 * b * sqrt (2 * log (1 / δ) / (split.foldB n).card))) ^ 2 + (4 / σ) * ((2 * B / ε) * ∑ a : Bool, (eLpNorm (fun x => h.μ_fn a x - S.μ_val a x) 2 S.toBackdoorEstimationSystem.P_X).toReal * (eLpNorm (fun x => h.e_fn x - S.e_val x) 2 S.toBackdoorEstimationSystem.P_X).toReal) + (4 / σ) * r_opt n := by obtain ⟨b, hb, hbound⟩ := oracle_inequality_drLearner_highProb S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes S_iid split hM_Θ hM_Y h hM_μ hOverlap hLoss_cont idx idx_dense R hR hclamp_minimizes hδ hδ' ((drMixedDirDeriv S hA hε_pos h_overlap_η₀ Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes D ND).Dθ_at S.toBackdoorEstimationSystem.η₀) ((drMixedDirDeriv S hA hε_pos h_overlap_η₀ Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes D ND).Dθ_at h) τhat r_opt hPluginERM σ hσ hSC hFOI refine ⟨b, hb, fun n => ?_⟩ obtain ⟨E, hEm, hEge, hE⟩ := hbound n refine ⟨E, hEm, hEge, fun ω hω => ?_⟩ have hbias := drBias_le_product S hA hε_pos h_overlap_η₀ Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes D ND h h_overlap_h (τhat n ω) hB_nonneg (fun x => hdEval_unif (τhat n ω) (hτ_mem n ω) x) h_μ_h_int h_phi_int (h_phiw_int n ω) hΔμ_memLp hΔe_memLp (hA_int n ω) (hB_int n ω) have hbias' : (4 / σ) * Bias_n (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) ((drMixedDirDeriv S hA hε_pos h_overlap_η₀ Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes D ND).Dθ_at S.toBackdoorEstimationSystem.η₀) ((drMixedDirDeriv S hA hε_pos h_overlap_η₀ Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes D ND).Dθ_at h) (τhat n ω) ≤ (4 / σ) * ((2 * B / ε) * ∑ a : Bool, (eLpNorm (fun x => h.μ_fn a x - S.μ_val a x) 2 S.toBackdoorEstimationSystem.P_X).toReal * (eLpNorm (fun x => h.e_fn x - S.e_val x) 2 S.toBackdoorEstimationSystem.P_X).toReal) := mul_le_mul_of_nonneg_left ((le_abs_self _).trans hbias) (by positivity) linarith [hE ω hω, hbias']
DRLearner­Random­Nuisance 1 core · 0 supporting This file extends the doubly robust learner oracle inequality from a fixed nuisance value to a cross-fitted random nuisance estimate. ★ oracle_inequality_drLearner_random_nuisance

Random-Nuisance DR-Learner Bound

This file extends the doubly robust learner oracle inequality from a fixed nuisance value to a cross-fitted random nuisance estimate. The resulting bound keeps the empirical-process control as an event and evaluates the second-order bias at the realized nuisance estimate. The main theorem oracle_inequality_drLearner_random_nuisance combines the deterministic plug-in ERM oracle inequality with the product-bias bound applied pointwise to the realized nuisance.

theorem oracle_inequality_drLearner_random_nuisance reviewed
Causalean.Estimation.OrthogonalLearning

DR-Learner oracle inequality at a random cross-fitted nuisance. Consider a CATE estimation system satisfying the backdoor identification assumptions, with an evaluation functional measurable in its parameter and recovering the true CATE at a parameter θ₀ in the constraint set, and suppose the overlap parameter is positive while the true nuisance and every realization of the random, fold-A-estimated nuisance ĥ n ω lie in the ε-overlap slice. Assume the estimator sequence stays in the constraint set as an approximate sample-split plug-in empirical-risk minimizer with respect to the random nuisance (slack r_opt), whose excess population risk at each realized ĥ n ω obeys a strong-convexity-type lower bound with constant σ>0, together with a first-order orthogonality inequality at the true nuisance's directional derivative, and that on a high-probability event the centred excess empirical risk at the realized estimator and nuisance is controlled by a rate ρ n. Finally, a battery of boundedness and integrability conditions governs the directional-derivative envelope and bounds the two random nuisance-error terms. Then for every n, with probability at least 1-δ the squared estimation error is bounded by the oracle/Rademacher term (4(1+σ)/σ²)·(ρ n)² plus a random second-order product-bias term (4/σ)·(2B/ε)·Σ_a ‖(ĥ n ω).μ_fn a − μ_val a‖₂·‖(ĥ n ω).e_fn − e_val‖₂ evaluated at the realized nuisance, plus an optimization slack (4/σ)·r_opt n.

Formal statement
hA :
S.toPOBackdoorSystem.Assumptions
Θ_set :
Set Θ
Θ_convex :
Convex ℝ Θ_set
θ₀ :
Θ
θ₀_mem :
θ₀ ∈ Θ_set
eval :
Θ → γ → ℝ
eval_meas :
∀ θ, Measurable (eval θ)
eval_θ₀ :
∀ x, eval θ₀ x = S.τ_val x
θ₀_minimizes :
DRThetaMinimizes S Θ_set θ₀ eval
S_iid :
IIDSample P.Ω (γ × Bool × ℝ) P.μ S.toBackdoorEstimationSystem.P_Z
split :
ε :
hε_pos :
0 < ε
h_overlap_η₀ :
S.toBackdoorEstimationSystem.η₀ ∈ BackdoorEstimationSystem.H_ε (γ := γ) ε
D :
EvalDirDeriv Θ_set θ₀ eval
ND :
NuisanceDirDeriv S.toBackdoorEstimationSystem.η₀
ĥ :
ℕ → P.Ω → NuisanceVec γ
hĥ_overlap :
∀ n ω, ĥ n ω ∈ BackdoorEstimationSystem.H_ε (γ := γ) ε
τhat :
ℕ → P.Ω → Θ
hτ_mem :
∀ n ω, τhat n ω ∈ Θ_set
r_opt :
ℕ → ℝ
hPluginERM :
SampleSplitPluginERM (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) S_iid split τhat ĥ r_opt
σ :
:
0 < σ
hSC :
∀ n ω,
∀ θ ∈ Θ_set,
(drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).L θ (ĥ n ω)
- (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).L θ₀ (ĥ n ω)
≥ (∫ z, ((drMixedDirDeriv S hA hε_pos h_overlap_η₀ Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes D ND).Dθ_at (ĥ n ω)).dℓ_θ θ z ∂S.toBackdoorEstimationSystem.P_Z)
+ (σ / 2) * ‖θ - θ₀‖ ^ 2
hFOI :
FirstOrderInequality (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) ((drMixedDirDeriv S hA hε_pos h_overlap_η₀ Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes D ND).Dθ_at S.toBackdoorEstimationSystem.η₀)
ρ :
ℕ → ℝ
δ :
hMod :
∀ n,
∃ E : Set P.Ω,
P.μ E ≥ 1 - ofReal δ ∧
∀ ω ∈ E,
((drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).L (τhat n ω) (ĥ n ω) - (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).L θ₀ (ĥ n ω))
- (empRiskFoldB (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) S_iid split n ω (τhat n ω) (ĥ n ω) - empRiskFoldB (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) S_iid split n ω θ₀ (ĥ n ω))
≤ ρ n * ‖τhat n ω - θ₀‖ + (ρ n) ^ 2
B :
hB_nonneg :
0 ≤ B
hdEval_unif :
∀ θ ∈ Θ_set, ∀ x, |D.dEval θ x| ≤ B
h_μ_ĥ_int :
∀ n ω,
∀ a : Bool,
Integrable (fun ω' => (ĥ n ω).μ_fn a (S.toBackdoorEstimationSystem.factualX ω')) P.μ
h_phi_int :
∀ n ω,
Integrable (fun ω' => phi_eta (S.toBackdoorEstimationSystem.factualZ ω') (ĥ n ω) - phi₀ S (S.toBackdoorEstimationSystem.factualZ ω')) P.μ
h_phiw_int :
∀ n ω,
Integrable (fun ω' => (phi_eta (S.toBackdoorEstimationSystem.factualZ ω') (ĥ n ω) - phi₀ S (S.toBackdoorEstimationSystem.factualZ ω')) * D.dEval (τhat n ω) (S.toBackdoorEstimationSystem.factualX ω')) P.μ
hΔμ_memLp :
∀ n ω,
∀ a, MemLp (fun x => (ĥ n ω).μ_fn a x - S.μ_val a x) 2 S.toBackdoorEstimationSystem.P_X
hΔe_memLp :
∀ n ω, MemLp (fun x => (ĥ n ω).e_fn x - S.e_val x) 2 S.toBackdoorEstimationSystem.P_X
hA_int :
∀ n ω,
Integrable (fun z => ((drMixedDirDeriv S hA hε_pos h_overlap_η₀ Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes D ND).Dθ_at S.toBackdoorEstimationSystem.η₀).dℓ_θ (τhat n ω) z) S.toBackdoorEstimationSystem.P_Z
hB_int :
∀ n ω,
Integrable (fun z => ((drMixedDirDeriv S hA hε_pos h_overlap_η₀ Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes D ND).Dθ_at (ĥ n ω)).dℓ_θ (τhat n ω) z) S.toBackdoorEstimationSystem.P_Z
∀ n : ℕ,
∃ E : Set P.Ω,
P.μ E ≥ 1 - ofReal δ ∧
∀ ω ∈ E,
‖τhat n ω - θ₀‖ ^ 2
≤ (4 * (1 + σ) / σ ^ 2) * (ρ n) ^ 2
+ (4 / σ) * ((2 * B / ε) * ∑ a : Bool, (eLpNorm (fun x => (ĥ n ω).μ_fn a x - S.μ_val a x) 2 S.toBackdoorEstimationSystem.P_X).toReal * (eLpNorm (fun x => (ĥ n ω).e_fn x - S.e_val x) 2 S.toBackdoorEstimationSystem.P_X).toReal)
+ (4 / σ) * r_opt n
Proof (Lean source)
theorem oracle_inequality_drLearner_random_nuisance [StandardBorelSpace P.Ω] [IsFiniteMeasure P.μ] (S : CATEEstimationSystem P γ) (hA : S.toPOBackdoorSystem.Assumptions) (Θ : Type*) [NormedAddCommGroup Θ] [InnerProductSpace ℝ Θ] (Θ_set : Set Θ) (Θ_convex : Convex ℝ Θ_set) (θ₀ : Θ) (θ₀_mem : θ₀ ∈ Θ_set) (eval : Θ → γ → ℝ) (eval_meas : ∀ θ, Measurable (eval θ)) (eval_θ₀ : ∀ x, eval θ₀ x = S.τ_val x) (θ₀_minimizes : DRThetaMinimizes S Θ_set θ₀ eval) (S_iid : IIDSample P.Ω (γ × Bool × ℝ) P.μ S.toBackdoorEstimationSystem.P_Z) (split : OneShotSplit S_iid) {ε : ℝ} (hε_pos : 0 < ε) (h_overlap_η₀ : S.toBackdoorEstimationSystem.η₀ ∈ BackdoorEstimationSystem.H_ε (γ := γ) ε) (D : EvalDirDeriv Θ_set θ₀ eval) (ND : NuisanceDirDeriv S.toBackdoorEstimationSystem.η₀) (ĥ : ℕ → P.Ω → NuisanceVec γ) (hĥ_overlap : ∀ n ω, ĥ n ω ∈ BackdoorEstimationSystem.H_ε (γ := γ) ε) (τhat : ℕ → P.Ω → Θ) (hτ_mem : ∀ n ω, τhat n ω ∈ Θ_set) (r_opt : ℕ → ℝ) (hPluginERM : SampleSplitPluginERM (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) S_iid split τhat ĥ r_opt) (σ : ℝ) (hσ : 0 < σ) (hSC : ∀ n ω, ∀ θ ∈ Θ_set, (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).L θ (ĥ n ω) - (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).L θ₀ (ĥ n ω) ≥ (∫ z, ((drMixedDirDeriv S hA hε_pos h_overlap_η₀ Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes D ND).Dθ_at (ĥ n ω)).dℓ_θ θ z ∂S.toBackdoorEstimationSystem.P_Z) + (σ / 2) * ‖θ - θ₀‖ ^ 2) (hFOI : FirstOrderInequality (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) ((drMixedDirDeriv S hA hε_pos h_overlap_η₀ Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes D ND).Dθ_at S.toBackdoorEstimationSystem.η₀)) (ρ : ℕ → ℝ) {δ : ℝ} (hMod : ∀ n, ∃ E : Set P.Ω, MeasurableSet E ∧ P.μ E ≥ 1 - ofReal δ ∧ ∀ ω ∈ E, ((drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).L (τhat n ω) (ĥ n ω) - (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).L θ₀ (ĥ n ω)) - (empRiskFoldB (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) S_iid split n ω (τhat n ω) (ĥ n ω) - empRiskFoldB (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) S_iid split n ω θ₀ (ĥ n ω)) ≤ ρ n * ‖τhat n ω - θ₀‖ + (ρ n) ^ 2) {B : ℝ} (hB_nonneg : 0 ≤ B) (hdEval_unif : ∀ θ ∈ Θ_set, ∀ x, |D.dEval θ x| ≤ B) (h_μ_ĥ_int : ∀ n ω, ∀ a : Bool, Integrable (fun ω' => (ĥ n ω).μ_fn a (S.toBackdoorEstimationSystem.factualX ω')) P.μ) (h_phi_int : ∀ n ω, Integrable (fun ω' => phi_eta (S.toBackdoorEstimationSystem.factualZ ω') (ĥ n ω) - phi₀ S (S.toBackdoorEstimationSystem.factualZ ω')) P.μ) (h_phiw_int : ∀ n ω, Integrable (fun ω' => (phi_eta (S.toBackdoorEstimationSystem.factualZ ω') (ĥ n ω) - phi₀ S (S.toBackdoorEstimationSystem.factualZ ω')) * D.dEval (τhat n ω) (S.toBackdoorEstimationSystem.factualX ω')) P.μ) (hΔμ_memLp : ∀ n ω, ∀ a, MemLp (fun x => (ĥ n ω).μ_fn a x - S.μ_val a x) 2 S.toBackdoorEstimationSystem.P_X) (hΔe_memLp : ∀ n ω, MemLp (fun x => (ĥ n ω).e_fn x - S.e_val x) 2 S.toBackdoorEstimationSystem.P_X) (hA_int : ∀ n ω, Integrable (fun z => ((drMixedDirDeriv S hA hε_pos h_overlap_η₀ Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes D ND).Dθ_at S.toBackdoorEstimationSystem.η₀).dℓ_θ (τhat n ω) z) S.toBackdoorEstimationSystem.P_Z) (hB_int : ∀ n ω, Integrable (fun z => ((drMixedDirDeriv S hA hε_pos h_overlap_η₀ Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes D ND).Dθ_at (ĥ n ω)).dℓ_θ (τhat n ω) z) S.toBackdoorEstimationSystem.P_Z) : ∀ n : ℕ, ∃ E : Set P.Ω, MeasurableSet E ∧ P.μ E ≥ 1 - ofReal δ ∧ ∀ ω ∈ E, ‖τhat n ω - θ₀‖ ^ 2 ≤ (4 * (1 + σ) / σ ^ 2) * (ρ n) ^ 2 + (4 / σ) * ((2 * B / ε) * ∑ a : Bool, (eLpNorm (fun x => (ĥ n ω).μ_fn a x - S.μ_val a x) 2 S.toBackdoorEstimationSystem.P_X).toReal * (eLpNorm (fun x => (ĥ n ω).e_fn x - S.e_val x) 2 S.toBackdoorEstimationSystem.P_X).toReal) + (4 / σ) * r_opt n := by intro n obtain ⟨E, hEm, hEge, hE⟩ := hMod n refine ⟨E, hEm, hEge, fun ω hω => ?_⟩ -- Deterministic oracle inequality at the realised random nuisance. have hdet := oracle_inequality_plugin_ERM (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) S_iid split ((drMixedDirDeriv S hA hε_pos h_overlap_η₀ Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes D ND).Dθ_at S.toBackdoorEstimationSystem.η₀) ĥ (fun n ω => (drMixedDirDeriv S hA hε_pos h_overlap_η₀ Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes D ND).Dθ_at (ĥ n ω)) τhat r_opt hPluginERM σ hσ hSC hFOI ρ n ω (hE ω hω) -- Random second-order product bias bound at `ĥ n ω`. have hbias := drBias_le_product S hA hε_pos h_overlap_η₀ Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes D ND (ĥ n ω) (hĥ_overlap n ω) (τhat n ω) hB_nonneg (fun x => hdEval_unif (τhat n ω) (hτ_mem n ω) x) (h_μ_ĥ_int n ω) (h_phi_int n ω) (h_phiw_int n ω) (hΔμ_memLp n ω) (hΔe_memLp n ω) (hA_int n ω) (hB_int n ω) have hbias' : (4 / σ) * Bias_n (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) ((drMixedDirDeriv S hA hε_pos h_overlap_η₀ Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes D ND).Dθ_at S.toBackdoorEstimationSystem.η₀) ((drMixedDirDeriv S hA hε_pos h_overlap_η₀ Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes D ND).Dθ_at (ĥ n ω)) (τhat n ω) ≤ (4 / σ) * ((2 * B / ε) * ∑ a : Bool, (eLpNorm (fun x => (ĥ n ω).μ_fn a x - S.μ_val a x) 2 S.toBackdoorEstimationSystem.P_X).toReal * (eLpNorm (fun x => (ĥ n ω).e_fn x - S.e_val x) 2 S.toBackdoorEstimationSystem.P_X).toReal) := mul_le_mul_of_nonneg_left ((le_abs_self _).trans hbias) (by positivity) have hθ₀ : (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).θ₀ = θ₀ := rfl simp only [hθ₀] at hdet linarith [hdet, hbias']
Localized­DRLearner­End­To­End 1 core · 0 supporting This file composes the localized empirical-process modulus for the DR-Learner with the generic orthogonal statistical learning oracle inequality. ★ oracle_inequality_localized_drLearner_highProb

Localized DR-Learner Oracle Chain

This file composes the localized empirical-process modulus for the DR-Learner with the generic orthogonal statistical learning oracle inequality. It provides the high-probability CATE bound whose rate is governed by the localized critical radius, with a separate bounded branch for an empty validation fold.

theorem oracle_inequality_localized_drLearner_highProb reviewed
Causalean.Estimation.OrthogonalLearning

DR-Learner localized high-probability oracle inequality. For the DR-Learner CATE estimation system, suppose the evaluation functional is measurable in its parameter and recovers the true CATE at a parameter θ₀ in the constraint set, and that the evaluation functional, outcome, and fixed nuisance conditional-mean are uniformly bounded while the nuisance's propensity score satisfies ε-overlap. Assume the centred DR-loss is continuous in θ, a clamped version of θ₀ minimizes it, its population Rademacher complexity along a dense index sequence is controlled by a sub-root envelope ψ with respect to a seminorm invariant under almost-everywhere modification, and the same Rademacher upper bound extends to loss differences across the whole constraint set, together with Lipschitz and diameter control of the centred loss increments — nonnegative Lipschitz constant L, a diameter bound Rmax dominating every critical radius, and the sub-root fixed-point property, plus boundedness and integrability of the empirical star-hull Rademacher process needed by the localization bridge and a confidence level in (0,1](0,1] together with the Foster–Syrgkanis critical-radius domination inequality across dyadic shell counts. Finally, suppose the estimator sequence is an approximate sample-split plug-in empirical-risk minimizer at the fixed nuisance h (slack r_opt), whose excess population risk obeys a strong-convexity-type lower bound with constant σ>0, together with a first-order orthogonality inequality at the truth's directional derivative. Then there is a nonnegative envelope b such that, for every n, with probability at least 1-δ the squared estimation error is bounded by the oracle term (4(1+σ)/σ²)·(ρ n)² at the sharp localized Foster–Syrgkanis rate ρ n = (8L+3)·criticalRadius (ψ |B(n)|) (falling back to √(2b) on an empty validation fold), plus the nuisance bias term (4/σ)·Bias_n and an optimization slack (4/σ)·r_opt n.

Formal statement
Θ_set :
Set Θ
Θ_convex :
Convex ℝ Θ_set
Nonempty Θ_set
Countable Θ_set
θ₀ :
Θ
θ₀_mem :
θ₀ ∈ Θ_set
eval :
Θ → γ → ℝ
eval_meas :
∀ θ, Measurable (eval θ)
eval_θ₀ :
∀ x, eval θ₀ x = S.τ_val x
θ₀_minimizes :
DRThetaMinimizes S Θ_set θ₀ eval
S_iid :
IIDSample P.Ω (γ × Bool × ℝ) P.μ S.toBackdoorEstimationSystem.P_Z
split :
M_Θ M_Y M_μ ε :
hM_Θ :
DREvalBounded Θ_set eval M_Θ
hM_Y :
h :
hM_μ :
hOverlap :
DRNuisanceOverlap S Θ_set h ε
hclamp_minimizes :
CenteredClampedThetaMinimizes (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) (2 * (M_Θ + 2 * M_μ + 2 * (M_Y + M_μ) / ε) ^ 2)
hLoss_cont :
∀ z,
Continuous fun (θ : (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set)
=> (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z θ.val h
idx :
ℕ → (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set
idx_dense :
norm :
((γ × Bool × ℝ) → ℝ) → ℝ
ψ :
ℕ → ℝ → ℝ
:
DRCriticalRadius S Θ_set eval (fun k => ⟨((idx k).val), (idx k).property⟩) h norm ψ
hnorm_ae :
∀ F F' : (γ × Bool × ℝ) → ℝ
if
F =ᵐ[S.toBackdoorEstimationSystem.P_Z] F'
then
norm F = norm F'
Rmax L :
hL_nonneg :
0 ≤ L
hF_lip :
∀ θ ∈ (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set,
norm (fun z => (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z θ h - (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).θ₀ h)
≤ L * ‖θ - (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).θ₀‖
hF_diam :
∀ θ ∈ (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set,
norm (fun z => (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z θ h - (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).θ₀ h)
≤ Rmax
hRmax_lb :
∀ m : ℕ, criticalRadius (ψ m) ≤ Rmax
hcrit_pos :
∀ m : ℕ, 0 < criticalRadius (ψ m)
hcrit_fp :
∀ m : ℕ, ψ m (criticalRadius (ψ m)) ≤ (criticalRadius (ψ m)) ^ 2
hψ_ub :
∀ m : ℕ,
RademacherUpperBound (fun (θ : (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set) (z : γ × Bool × ℝ) => (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z θ.val h - (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).θ₀ h) norm S.toBackdoorEstimationSystem.P_Z (id : (γ × Bool × ℝ) → γ × Bool × ℝ) m (ψ m)
hrad_bdd :
∀ m r,
∀ S_fin : Fin m → γ × Bool × ℝ,
∀ σ : Signs m,
BddAbove (Set.range fun p : starHullParam (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set => |(m : ℝ)⁻¹ * ∑ k : Fin m, (σ k : ℝ) * starHullZeroOut (fun (θ : (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set) (z : γ × Bool × ℝ) => (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z θ.val h - (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).θ₀ h) norm r p (S_fin k)|)
hrad_int :
∀ m r,
Integrable (fun ω : Fin m → γ × Bool × ℝ => empiricalRademacherComplexity m (starHullZeroOut (fun (θ : (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set) (z : γ × Bool × ℝ) => (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z θ.val h - (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).θ₀ h) norm r) ((id : (γ × Bool × ℝ) → (γ × Bool × ℝ)) ∘ ω)) (Measure.pi (fun _ => S.toBackdoorEstimationSystem.P_Z))
δ :
:
0 < δ
hδ' :
δ ≤ 1
hδ_dom :
∀ n K : ℕ
if
0 < (split.foldB n).card
and
Rmax ≤ (criticalRadius (ψ (split.foldB n).card)) * (2 : ℝ) ^ K
then
2 * (M_Θ + 2 * M_μ + 2 * (M_Y + M_μ) / ε) ^ 2 * sqrt (2 * log (2 * ((K : ℝ) + 1) / δ) / (split.foldB n).card)
≤ (criticalRadius (ψ (split.foldB n).card)) ^ 2
Oracle-inequality ingredients specialised to the DR system.
Dθ_truth :
HasDirDerivTheta (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).g₀
Dθ_at_h :
HasDirDerivTheta (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) h
τhat :
ℕ → P.Ω → Θ
r_opt :
ℕ → ℝ
hPluginERM :
SampleSplitPluginERM (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) S_iid split τhat (fun _ _ => h) r_opt
σ :
:
0 < σ
hSC :
∀ θ ∈ Θ_set,
(drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).L θ h
- (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).L θ₀ h
≥ (∫ z, Dθ_at_h.dℓ_θ θ z ∂S.toBackdoorEstimationSystem.P_Z) + (σ / 2) * ‖θ - θ₀‖ ^ 2
hFOI :
FirstOrderInequality (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) Dθ_truth
∃ b : ℝ,
0 ≤ b ∧
∀ n : ℕ,
∃ E : Set P.Ω,
P.μ E ≥ 1 - ofReal δ ∧
∀ ω ∈ E,
‖τhat n ω - θ₀‖ ^ 2
≤ (4 * (1 + σ) / σ ^ 2) * (if (split.foldB n).card = 0 then sqrt (2 * b) else (8 * L + 3) * criticalRadius (ψ (split.foldB n).card)) ^ 2
+ (4 / σ) * Bias_n (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) Dθ_truth Dθ_at_h (τhat n ω)
+ (4 / σ) * r_opt n
Proof (Lean source)
theorem oracle_inequality_localized_drLearner_highProb [StandardBorelSpace P.Ω] [IsFiniteMeasure P.μ] [IsProbabilityMeasure P.μ] (S : CATEEstimationSystem P γ) (Θ : Type*) [NormedAddCommGroup Θ] [InnerProductSpace ℝ Θ] (Θ_set : Set Θ) (Θ_convex : Convex ℝ Θ_set) [Nonempty Θ_set] [Countable Θ_set] (θ₀ : Θ) (θ₀_mem : θ₀ ∈ Θ_set) (eval : Θ → γ → ℝ) (eval_meas : ∀ θ, Measurable (eval θ)) (eval_θ₀ : ∀ x, eval θ₀ x = S.τ_val x) (θ₀_minimizes : DRThetaMinimizes S Θ_set θ₀ eval) (S_iid : IIDSample P.Ω (γ × Bool × ℝ) P.μ S.toBackdoorEstimationSystem.P_Z) (split : OneShotSplit S_iid) {M_Θ M_Y M_μ ε : ℝ} (hM_Θ : DREvalBounded Θ_set eval M_Θ) (hM_Y : DROutcomeBounded S M_Y) (h : NuisanceVec γ) (hM_μ : DRNuisanceMuBounded h M_μ) (hOverlap : DRNuisanceOverlap S Θ_set h ε) (hclamp_minimizes : CenteredClampedThetaMinimizes (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) (2 * (M_Θ + 2 * M_μ + 2 * (M_Y + M_μ) / ε) ^ 2)) (hLoss_cont : ∀ z, Continuous fun (θ : (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set) => (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z θ.val h) (idx : ℕ → (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set) (idx_dense : DenseRange idx) {norm : ((γ × Bool × ℝ) → ℝ) → ℝ} {ψ : ℕ → ℝ → ℝ} (hψ : DRCriticalRadius S Θ_set eval (fun k => ⟨((idx k).val), (idx k).property⟩) h norm ψ) (hnorm_ae : ∀ F F' : (γ × Bool × ℝ) → ℝ, F =ᵐ[S.toBackdoorEstimationSystem.P_Z] F' → norm F = norm F') {Rmax L : ℝ} (hL_nonneg : 0 ≤ L) (hF_lip : ∀ θ ∈ (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set, norm (fun z => (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z θ h - (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).θ₀ h) ≤ L * ‖θ - (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).θ₀‖) (hF_diam : ∀ θ ∈ (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set, norm (fun z => (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z θ h - (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).θ₀ h) ≤ Rmax) (hRmax_lb : ∀ m : ℕ, criticalRadius (ψ m) ≤ Rmax) (hcrit_pos : ∀ m : ℕ, 0 < criticalRadius (ψ m)) (hcrit_fp : ∀ m : ℕ, ψ m (criticalRadius (ψ m)) ≤ (criticalRadius (ψ m)) ^ 2) (hψ_ub : ∀ m : ℕ, RademacherUpperBound (fun (θ : (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set) (z : γ × Bool × ℝ) => (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z θ.val h - (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).θ₀ h) norm S.toBackdoorEstimationSystem.P_Z (id : (γ × Bool × ℝ) → γ × Bool × ℝ) m (ψ m)) (hrad_bdd : ∀ m r, ∀ S_fin : Fin m → γ × Bool × ℝ, ∀ σ : Signs m, BddAbove (Set.range fun p : starHullParam (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set => |(m : ℝ)⁻¹ * ∑ k : Fin m, (σ k : ℝ) * starHullZeroOut (fun (θ : (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set) (z : γ × Bool × ℝ) => (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z θ.val h - (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).θ₀ h) norm r p (S_fin k)|)) (hrad_int : ∀ m r, Integrable (fun ω : Fin m → γ × Bool × ℝ => empiricalRademacherComplexity m (starHullZeroOut (fun (θ : (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).Θ_set) (z : γ × Bool × ℝ) => (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z θ.val h - (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).ℓ z (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).θ₀ h) norm r) ((id : (γ × Bool × ℝ) → (γ × Bool × ℝ)) ∘ ω)) (Measure.pi (fun _ => S.toBackdoorEstimationSystem.P_Z))) {δ : ℝ} (hδ : 0 < δ) (hδ' : δ ≤ 1) (hδ_dom : ∀ n K : ℕ, 0 < (split.foldB n).card → Rmax ≤ (criticalRadius (ψ (split.foldB n).card)) * (2 : ℝ) ^ K → 2 * (M_Θ + 2 * M_μ + 2 * (M_Y + M_μ) / ε) ^ 2 * sqrt (2 * log (2 * ((K : ℝ) + 1) / δ) / (split.foldB n).card) ≤ (criticalRadius (ψ (split.foldB n).card)) ^ 2) -- Oracle-inequality ingredients specialised to the DR system. (Dθ_truth : HasDirDerivTheta (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).g₀) (Dθ_at_h : HasDirDerivTheta (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) h) (τhat : ℕ → P.Ω → Θ) (r_opt : ℕ → ℝ) (hPluginERM : SampleSplitPluginERM (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) S_iid split τhat (fun _ _ => h) r_opt) (σ : ℝ) (hσ : 0 < σ) (hSC : ∀ θ ∈ Θ_set, (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).L θ h - (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes).L θ₀ h ≥ (∫ z, Dθ_at_h.dℓ_θ θ z ∂S.toBackdoorEstimationSystem.P_Z) + (σ / 2) * ‖θ - θ₀‖ ^ 2) (hFOI : FirstOrderInequality (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) Dθ_truth) : ∃ b : ℝ, 0 ≤ b ∧ ∀ n : ℕ, ∃ E : Set P.Ω, MeasurableSet E ∧ P.μ E ≥ 1 - ofReal δ ∧ ∀ ω ∈ E, ‖τhat n ω - θ₀‖ ^ 2 ≤ (4 * (1 + σ) / σ ^ 2) * (if (split.foldB n).card = 0 then sqrt (2 * b) else (8 * L + 3) * criticalRadius (ψ (split.foldB n).card)) ^ 2 + (4 / σ) * Bias_n (drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) Dθ_truth Dθ_at_h (τhat n ω) + (4 / σ) * r_opt n := by rcases localEmpProcessModulus_localized_drLearner (S := S) (Θ := Θ) (Θ_set := Θ_set) (Θ_convex := Θ_convex) (θ₀ := θ₀) (θ₀_mem := θ₀_mem) (eval := eval) (eval_meas := eval_meas) (eval_θ₀ := eval_θ₀) (θ₀_minimizes := θ₀_minimizes) (S_iid := S_iid) (split := split) (hM_Θ := hM_Θ) (hM_Y := hM_Y) (h := h) (hM_μ := hM_μ) (hOverlap := hOverlap) (hclamp_minimizes := hclamp_minimizes) (_hLoss_cont := hLoss_cont) (idx := idx) (_idx_dense := idx_dense) (hψ := hψ) (hnorm_ae := hnorm_ae) (Rmax := Rmax) (L := L) (hL_nonneg := hL_nonneg) (hF_lip := hF_lip) (hF_diam := hF_diam) (hRmax_lb := hRmax_lb) (hcrit_pos := hcrit_pos) (hcrit_fp := hcrit_fp) (hψ_ub := hψ_ub) (hrad_bdd := hrad_bdd) (hrad_int := hrad_int) (hδ := hδ) (hδ' := hδ') (hδ_dom := hδ_dom) with ⟨b, hb_nonneg, hMod⟩ refine ⟨b, hb_nonneg, ?_⟩ exact oracle_inequality_plugin_ERM_highProb (S := drLearningSystem S Θ Θ_set Θ_convex θ₀ θ₀_mem eval eval_meas eval_θ₀ θ₀_minimizes) (S_iid := S_iid) (split := split) (Dθ_truth := Dθ_truth) (g := h) (Dθ_at_g := Dθ_at_h) (θhat := τhat) (r_opt := r_opt) (hPluginERM := hPluginERM) (σ := σ) (hσ := hσ) (hSC := hSC) (hFOI := hFOI) (ρ := fun n => if (split.foldB n).card = 0 then sqrt (2 * b) else (8 * L + 3) * criticalRadius (ψ (split.foldB n).card)) (δ := δ) hMod