误差分析报告 — 北京多站点 PM2.5 下一小时预测(验证集)

本报告由 scripts/render_error_analysis.py 从 reports/error_analysis_numbers.json 程序化渲染, 全部数值直接取自该 JSON,未做任何重算或补充(数值按统一规则四舍五入显示)。 数据范围: 验证集 12 个站点、逐小时, 覆盖 2015-09 → 2016-02(供暖季), 共 51349 行; 目标: 预测下一小时 PM2.5, 误差按 真实值−预测值 或 预测值−真实值 口径见各表说明。

一、总体误差(全验证集)

预测均值低于真实均值,OLS 斜率小于 1、截距大于 0:高值段系统性低估、低值段被拉高,详见第二节分箱表。

残差(真实值 − 预测值)分布

残差均值接近零但右偏(偏度 4.93)、峰度极高(70.92),说明绝大多数误差很小,尾部由少数极端低估行贡献(见第五节 SSE 集中度)。

图 1: 残差分布与预测-真实散点(fig_e2_residuals.png)

二、按真实浓度分箱的误差分布

按真实 PM2.5 值分 9 箱。偏差 = 预测均值 − 真实均值(正 = 平均高估,负 = 平均低估);偏差率 = 偏差 / 真实均值;SSE 占比 = 该箱平方误差和占全验证集比例。

图 2: 按真实浓度分箱的误差构成(fig_e1_error_bins.png)

SSE 高度集中: 行占比最高的区间是 0-20(32.81% 的行),只贡献 3.00% 的 SSE;而 SSE 占比最高的区间 >300 仅占 4.48% 的行,却贡献 60.08% 的 SSE,其平均偏差达 -88.19(低估)。

三、分站点误差

站点名沿用 JSON 中的英文原名(北京 12 个国控站点)。偏差 = 预测均值 − 真实均值;RMSE/标准差 < 1 表示站点内误差小于其自身波动幅度。

四、按小时与月份

图 3: 站点、小时、月份三个维度的误差结构(fig_e3_error_structures.png)

4.1 按小时(0–11 时)

4.2 按小时(12–23 时)

4.3 凌晨 0–3 时汇总

0–3 时合计 8568 行,占全部行的 16.69%,却贡献了 27.31% 的 SSE;该时段 RMSE 为 41.97(高于总体 32.80),真实均值 86.64。

4.4 按月

五、平方误差集中度与高值端表现

5.1 高浓度阈值段: 行占比 vs SSE 占比

验证集误差高度集中于高浓度段:真实值 ≥150 的行占 18.37%,却贡献了 80.33% 的平方误差;≥200 的行占 12.21% 贡献 74.49%;≥300 的行占 4.48% 贡献 60.08%。

各阈值段偏差均为负(低估),且阈值越高偏差绝对值越大:≥300 段平均低估 -88.19。

5.2 极端值(真实值 >600)

真实值 >600 的 163 行仅占全部行的 0.32%,却贡献了 24.26% 的 SSE;该段真实均值 660.97、预测均值仅 414.80,RMSE 高达 286.75。

5.3 绝对误差最大的 1% / 5% 行

绝对误差最大的前 1% 行(513 行,最小绝对误差 133.47)贡献了 47.52% 的 SSE,其真实均值达 484.78;前 5% 行(2567 行,最小绝对误差 59.79)贡献了 75.24% 的 SSE。

5.4 重污染检出与清洁日误报

检出能力: 真实值 ≥150 的 9432 行中,预测值 ≥150 的占 83.00%,预测值 <100 的仅占 1.08%(平均预测值 220.11)。

清洁日误报: 真实值 ≤50 的 27951 行中,预测值 >100 的占 0.30%,预测值 >150 的占 0.07%。

5.5 各误差场景下的输入环境剖面

比较全体 / 真实 ≥150 / 真实 ≥300 / |误差|>75 / 真实 ≥150 且被低估 五类行 (under_pred_ge150 为真实值 ≥150 且预测低于真实的行)的平均污染物与气象输入。

| 输入变量 | 全体 | 真实 ≥150 | 真实 ≥300 | |误差|>75 | 真实≥150且被低估 | |---|---|---|---|---|---| | PM10 (μg/m³) | 96.28 | 267.08 | 418.42 | 377.26 | 160.06 | | SO2 (μg/m³) | 12.82 | 25.88 | 29.29 | 32.68 | 21.75 | | NO2 (μg/m³) | 52.12 | 99.51 | 130.32 | 117.62 | 76.89 | | CO (μg/m³) | 1501.98 | 3751.71 | 5630.98 | 5306.48 | 2374.42 | | O3 (μg/m³) | 34.45 | 20.29 | 19.16 | 12.49 | 20.69 | | 气温 (℃) | 5.56 | 4.44 | 2.56 | 0.34 | 4.88 | | 风速 (m/s) | 1.80 | 1.05 | 1.04 | 1.18 | 1.17 | | 露点 (℃) | -4.04 | 0.58 | -0.30 | -3.70 | -0.20 | | 气压 (hPa) | 1018.95 | 1017.58 | 1016.33 | 1017.34 | 1018.48 | | 降雨 (mm) | 0.04 | 0.0085 | 0.0012 | 0.04 | 0.0031 | | 样本数 n | 51349 | 9432 | 2298 | 1683 | 1603 | | 真实均值 | 83.15 | 260.31 | 411.96 | 376.40 | 174.88 | | 预测均值 | 76.06 | 220.11 | 323.77 | 273.89 | 129.29 |

误差大(真实 ≥150 / ≥300 / |误差|>75)的行对应显著更高的 PM10、SO2、NO2、CO 与更低的风速:高浓度静稳天气正是低估集中的场景。

5.6 训练 / 验证分布差异

验证集(供暖季)的重污染占比明显高于训练集(>150: 14.96% vs 18.20%;>300: 1.80% vs 4.45%),分布漂移放大了高值段误差。

六、重污染事件(日级峰值)检出

以下为验证期内真实均值最高的 8 个污染日(日级事件,按日期聚合)。

最极端的污染日 2016-02-08 真实均值 559.65,而当日平均预测仅 252.24 —— 事件级别的大幅低估是整体误差的主要来源之一。

6.1 绝对误差最大的 10 行

七、校正场景模拟

7.1 假设性场景: 若高值段误差被消除 / 减半,总体 RMSE 会降到多少

若真实 ≥150 的误差完全消除,总体 RMSE 可由 32.80 降至 14.55;仅处理 ≥300 段也可降至 20.73 —— 高值段是改进空间最大的方向。

7.2 分段乘法校准(样本外验证)

按预测值分箱在校准集上估计乘法因子,再应用到时间上不重叠的评估集:

校准集: 25458 行(2015-09-01→2015-11-30);评估集: 25891 行(2015-12-01→2016-02-29,真实均值 87.36,原始预测均值 78.35)。校准集样本内 RMSE 为 20.56。

在样本外评估集上,分段校准把 RMSE 从 39.86 降到 34.60(降幅 -13.19%),MAE 从 18.75 降到 16.30。

7.3 参照: Box-Cox 变换目标尺度

作为参照,以 Box-Cox 变换目标训练并在原始尺度评估的模型 RMSE 为 38.06(MAE 18.03),高于当前模型 32.80 / 16.38 的水平。

八、结论与改进建议

1. 误差集中于高浓度段与极端污染事件,应优先专项治理。 真实值 ≥150 的行仅占 18.37% 却贡献 80.33% 的 SSE;>600 的 163 行贡献 24.26% 的 SSE,当日真实均值最高的事件(2016-02-08)真实 559.65 vs 预测 252.24。建议: 针对重污染过程单独建模或加权(如对真实 ≥150 行做误差敏感的训练加权 / 上采样),并为极端事件构建阈值预警专用模型。

2. 夜间(0–3 时)误差偏高。 该时段行占 16.69% 却贡献 27.31% 的 SSE,RMSE 41.97 高于总体 32.80。建议: 增加夜间气象与逆温相关特征,或按小时分模型、给夜间段更大的误差权重。

3. 分段乘法校正在样本外有效,可继续沿用并扩展。 评估集 RMSE 由 39.86 降至 34.60(降幅 -13.19%),其中 >300 段的校准因子达 1.18;高值段假设场景也显示消除 ≥150 误差可将总体 RMSE 从 32.80 降至 14.55。建议: 在模型预测后叠加分段校准,并针对校准因子最大的高值段继续降低原始模型偏差。

注: 本报告数值均按 JSON 原值统一舍入显示(整数保持整数,其余保留两位小数,小于 0.01 的值保留四位小数);百分比字段在 JSON 中即已为百分数。

Error Analysis Report — Next-Hour PM2.5 Prediction Across Beijing Stations (Validation Set)

This report is programmatically rendered by scripts/render_error_analysis.py from reports/error_analysis_numbers.json; all values are taken directly from that JSON without any recomputation or supplementation (values are displayed rounded according to a uniform rule). Data scope: validation set of 12 stations, hourly, covering 2015-09 → 2016-02 (heating season), 51349 rows in total; target: predict next-hour PM2.5. Error definitions follow 真值−预测值 (actual − predicted) or 预测值−真值 (predicted − actual) conventions as noted in each table.

1. Overall Error (Full Validation Set)

The predicted mean is below the actual mean; the OLS slope is less than 1 and the intercept greater than 0: high-value segments are systematically underestimated while low values are pulled upward — see the bin tables in Section 2.

Residual (Actual − Predicted) Distribution

The residual mean is near zero but right-skewed (skewness 4.93) with extremely high kurtosis (70.92), indicating that most errors are very small while the tails are driven by a few rows with extreme underestimation (see SSE concentration in Section 5).

Figure 1: Residual distribution and predicted-actual scatter (fig_e2_residuals.png)

2. Error Distribution by Bins of Actual Concentration

Rows are split into 9 bins by actual PM2.5 value. Bias = predicted mean − actual mean (positive = average overestimation, negative = average underestimation); bias rate = bias / actual mean; SSE share = the bin's sum of squared errors as a share of the full validation set.

Figure 2: Error composition by bins of actual concentration (fig_e1_error_bins.png)

SSE is highly concentrated: the 0-20 range holds the largest row share (32.81% of rows) yet contributes only 3.00% of the SSE, while the >300 range — the largest SSE contributor — holds only 4.48% of rows yet contributes 60.08% of the SSE, with an average bias of -88.19 (underestimation).

3. Error by Station

Station names follow the original English names in the JSON (12 national control stations in Beijing). Bias = predicted mean − actual mean; RMSE/std. dev. < 1 indicates that within-station error is smaller than the station's own variability.

4. By Hour and Month

Figure 3: Error structure across the station, hour, and month dimensions (fig_e3_error_structures.png)

4.1 By Hour (0–11)

4.2 By Hour (12–23)

4.3 Early-Morning Hours (0–3) Summary

Hours 0–3 total 8568 rows — 16.69% of all rows — yet contribute 27.31% of the SSE; the RMSE for this period is 41.97 (above the overall 32.80), with an actual mean of 86.64.

4.4 By Month

5. Squared-Error Concentration and Performance on High Values

5.1 High-Concentration Threshold Segments: Row Share vs. SSE Share

Validation errors are heavily concentrated in the high-concentration segments: rows with actual ≥150 account for 18.37% of rows yet contribute 80.33% of the squared error; rows with ≥200 account for 12.21% and contribute 74.49%; rows with ≥300 account for 4.48% and contribute 60.08%.

The bias is negative (underestimation) in every threshold segment, and the higher the threshold the larger the absolute bias: the ≥300 segment is underestimated by -88.19 on average.

5.2 Extreme Values (Actual >600)

The 163 rows with actual >600 account for only 0.32% of all rows yet contribute 24.26% of the SSE; for this segment the actual mean is 660.97 while the predicted mean is only 414.80, with an RMSE as high as 286.75.

5.3 The 1% / 5% Rows with the Largest Absolute Errors

The top 1% of rows by absolute error (513 rows, minimum absolute error 133.47) contribute 47.52% of the SSE, with an actual mean of 484.78; the top 5% of rows (2567 rows, minimum absolute error 59.79) contribute 75.24% of the SSE.

5.4 Heavy-Pollution Detection and Clean-Day False Alarms

Detection capability: among the 9432 rows with actual ≥150, 83.00% have predictions ≥150 and only 1.08% have predictions <100 (mean prediction 220.11).

Clean-day false alarms: among the 27951 rows with actual ≤50, 0.30% have predictions >100 and 0.07% have predictions >150.

5.5 Input Environment Profiles Across Error Scenarios

Compares five groups of rows — all / actual ≥150 / actual ≥300 / |error|>75 / actual ≥150 and underestimated (under_pred_ge150 denotes rows with actual ≥150 whose prediction is below the actual) — in terms of mean pollutant and meteorological inputs.

Rows with large errors (actual ≥150 / ≥300 / |error|>75) correspond to markedly higher PM10, SO2, NO2, CO and lower wind speed: high-concentration stagnant-weather conditions are precisely the scenario where underestimation concentrates.

5.6 Training / Validation Distribution Differences

The validation set (heating season) has a markedly higher heavy-pollution share than the training set (>150: 14.96% vs. 18.20%; >300: 1.80% vs. 4.45%); the distribution shift amplifies errors in the high-value segments.

6. Heavy-Pollution Event (Daily Peak) Detection

The 8 pollution days with the highest actual means in the validation period (daily-level events, aggregated by date) are listed below.

On the most extreme pollution day, 2016-02-08, the actual mean was 559.65 while the day's average prediction was only 252.24 — event-scale severe underestimation is one of the main sources of overall error.

6.1 The 10 Rows with the Largest Absolute Errors

7. Calibration Scenario Simulations

7.1 Hypothetical Scenario: How Much the Overall RMSE Would Drop If High-Value-Segment Errors Were Eliminated / Halved

If errors on rows with actual ≥150 were completely eliminated, the overall RMSE could drop from 32.80 to 14.55; addressing only the ≥300 segment would still bring it down to 20.73 — the high-value segment is the direction with the greatest room for improvement.

7.2 Segmented Multiplicative Calibration (Out-of-Sample Validation)

Multiplicative factors are estimated per bin of predicted value on a calibration set, then applied to a temporally non-overlapping evaluation set:

Calibration set: 25458 rows (2015-09-01→2015-11-30); evaluation set: 25891 rows (2015-12-01→2016-02-29, actual mean 87.36, original predicted mean 78.35). The in-sample RMSE on the calibration set is 20.56.

On the out-of-sample evaluation set, segmented calibration reduces RMSE from 39.86 to 34.60 (a drop of -13.19%) and MAE from 18.75 to 16.30.

7.3 Reference: Box-Cox Transformed Target Scale

As a reference, the model trained with a Box-Cox-transformed target and evaluated on the original scale has an RMSE of 38.06 (MAE 18.03), higher than the current model's 32.80 / 16.38.

8. Conclusions and Improvement Recommendations

1. Errors concentrate in the high-concentration segment and extreme pollution events and should be prioritized for targeted treatment. Rows with actual ≥150 account for only 18.37% yet contribute 80.33% of the SSE; the 163 rows >600 contribute 24.26% of the SSE, and on the event day with the highest daily actual mean (2016-02-08), actual was 559.65 vs. predicted 252.24. Recommendation: model heavy-pollution episodes separately or weight them (e.g., error-sensitive training weighting / upsampling on rows with actual ≥150), and build dedicated threshold-alert models for extreme events.

2. Errors are elevated during the nighttime hours (0–3). This period accounts for 16.69% of rows yet contributes 27.31% of the SSE, with an RMSE of 41.97 above the overall 32.80. Recommendation: add nighttime meteorological and temperature-inversion-related features, or fit per-hour sub-models and give the nighttime segment greater error weight.

3. Segmented multiplicative calibration is effective out of sample and can continue to be used and extended. On the evaluation set, RMSE drops from 39.86 to 34.60 (a drop of -13.19%), with the calibration factor reaching 1.18 for the >300 segment; the high-value-segment hypothetical scenario also shows that eliminating errors on ≥150 could reduce the overall RMSE from 32.80 to 14.55. Recommendation: stack segmented calibration after model prediction, and continue reducing the original model's bias in the high-value segments that have the largest calibration factors.

Note: All values in this report are rounded from the raw JSON values under a uniform rule (integers stay integers, other values are kept to two decimal places, and values below 0.01 are kept to four decimal places); percentage fields are already expressed as percentages in the JSON.