Stat.Nonparametric.Series­Sieve

Jackson-type approximation for the piecewise-polynomial sieve

Jackson-type approximation bounds for piecewise-polynomial sieves on one-dimensional Hölder classes.

This file proves a from-scratch Jackson theorem for a concrete one-dimensional sieve: the piecewise-Taylor approximant on a uniform J-cell partition of a window [lo, hi]. Splitting [lo, hi] into J cells of width δ = (hi − lo)/J and replacing f on each cell by its degree-p Taylor polynomial, where p = holderDerivOrder β, expanded at the cell's left endpoint yields a piecewise polynomial whose uniform (sup-norm) approximation error obeys

sup_{x ∈ [lo,hi]} |f x − g x| ≤ (M / p!) · ((hi − lo)/J)^β = C · J^{−β}

under the standard Hölder convention used here: f has p continuous derivatives and its p-th derivative satisfies the displayed Hölder-type bound with exponent β − p. For positive integer β = m, this means p = m - 1 and exponent 1.

The result is the deterministic approximation-error half of the series least-squares prediction analysis. It feeds the empirical projection reduction in SeriesSieve/Prediction.lean, where a sup-norm approximation bound can control a weighted empirical least-squares objective via projection optimality (lstsq_objective_le_of_orthogonal).

Series / sieve least-squares prediction error

Prediction-error decompositions for series least squares, combining deterministic approximation error with the spherical-noise variance of the projected fit.

This file assembles the series least-squares prediction rate O(J^{−s/d} + √(J/N)) from the two reusable substrate pieces:

* approximation (deterministic, bias side): the empirical best-approximation error of the least-squares fit is controlled by the sup-norm error of any comparator, so an externally supplied approximation bound can transfer to the weighted empirical objective; * stochastic (variance side): the exact orthogonal (Pythagorean) decomposition of the prediction error splits off the projected noise, whose expected weighted quadratic form is σ² · (effective degrees of freedom).

Conditioning on the design, the data fit Φ ĉ and the noise-free projection Φ ĉ⁰ differ by a deterministic linear image of the noise (the hat matrix), so the stochastic term's expectation reduces, via the Gauss–Markov spherical-variance identity (linearSmoother_variance_spherical), to σ²·∑ᵢ wᵢ ∑ₖ aᵢₖ². If a separate design argument bounds that effective-degree-of-freedom sum by V, and if a separate approximation argument bounds the noise-free projection error by A, the main theorem gives the oracle inequality A + σ² V.

All results are generic in the design matrix Φ (Newey 1997; Chen 2007; Belloni–Chernozhukov– Chetverikov–Kato 2015).

Series / sieve least-squares prediction rate O(J^{−2s/d} + J/N)

The Causalean file Causalean.Stat.Nonparametric.SeriesSieve.Prediction proves a conditional oracle inequality seriesLS_expected_prediction_le:

𝔼[‖f − Φ·ĉ‖²_w] ≤ A + σ² · V,

where A is any bound on the noise-free least-squares objective (bias side) and V is any bound on the effective-degrees-of-freedom sum ∑ᵢ wᵢ ∑ₖ aᵢₖ² (variance side). This module supplies the two missing rate ingredients and combines them into the standard series/sieve prediction rate.

Main results

* jacksonSeriesBias_le — squares the Jackson best-approximation bound A ≤ (∑ᵢ wᵢ)·(C_J·J^{−s/d})² into the explicit bias rate A ≤ ((∑ᵢ wᵢ)·C_J²)·J^{−2s/d}. The Jackson approximation power of the sieve is taken as an explicit hypothesis (no attempt to prove Jackson's theorem for a concrete basis). * seriesEffectiveDoF_le — for normalized weights wᵢ = 1/N and a Frobenius / trace hypothesis on the hat map ∑ᵢ ∑ₖ aᵢₖ² ≤ Cvar·J, the effective-degrees-of-freedom sum obeys ∑ᵢ wᵢ ∑ₖ aᵢₖ² ≤ Cvar·J/N, i.e. it is controlled by (basis dimension)/(sample size). The trace bound is an explicit hypothesis (not derived from an explicit Gram matrix). * seriesLS_prediction_rate — feeds the two suppliers into seriesLS_expected_prediction_le to obtain the full rate 𝔼[‖f − Φ·ĉ‖²_w] ≤ ((∑ᵢ wᵢ)·C_J²)·J^{−2s/d} + σ²·(Cvar·J/N).

Here J^{−s/d} is the real power (J : ℝ) ^ (-(s/d)) (Real.rpow), keeping s, d, J, N, σ as free parameters; no concrete basis is assumed. See Newey (1997), Chen (2007), Belloni–Chernozhukov–Chetverikov–Kato (2015) for the econometric series-estimator theory and DeVore–Lorentz, Constructive Approximation, for the Jackson/Bernstein bias term.