Axis-aligned decision trees are fast and stable but struggle on datasets with rotated or interaction-dependent decision boundaries, where informative splits require linear combinations of features rather than single-feature thresholds. Oblique forests address this with per-node hyperplane splits, but at added computational cost and implementation complexity. We propose a simple alternative: JARF, Jacobian-Aligned Random Forests. Concretely, we first fit an axis-aligned forest to estimate class probabilities or regression outputs, compute finite-difference gradients of these predictions with respect to each feature, aggregate them into an expected Jacobian outer product that generalizes the expected gradient outer product (EGOP), and use it as a single global linear preconditioner for all inputs. This supervised preconditioner applies a single global rotation of the feature space, then hands the transformed data back to a standard axis-aligned forest, preserving off-the-shelf training pipelines while capturing oblique boundaries and feature interactions that would otherwise require many axis-aligned splits to approximate. The same construction applies to any model that provides gradients, though we focus on random forests and gradient-boosted trees in this work. On tabular classification and regression benchmarks, this preconditioning consistently improves axis-aligned forests and often matches or surpasses oblique baselines while improving training time. Our experimental results and theoretical analysis together indicate that supervised preconditioning can recover much of the accuracy of oblique forests while retaining the simplicity and robustness of axis-aligned trees.

翻译：轴对齐决策树具有快速且稳定的特性，但在处理具有旋转或交互依赖决策边界的数据集时表现不佳，因为信息丰富的分割需要特征的线性组合而非单一特征的阈值。倾斜森林通过每个节点的超平面分割解决了这一问题，但增加了计算成本和实现复杂度。我们提出一种简单的替代方案：JARF，即雅可比对齐随机森林。具体而言，我们首先拟合一个轴对齐森林以估计类别概率或回归输出，计算这些预测相对于每个特征的有限差分梯度，将它们聚合为一个推广了期望梯度外积（EGOP）的期望雅可比外积，并将其用作所有输入的单一全局线性预处理器。这种有监督的预处理器对特征空间应用单一的全局旋转，然后将转换后的数据传递回标准的轴对齐森林，从而保留现成的训练流程，同时捕获倾斜边界和特征交互，而这些原本需要许多轴对齐分割来近似。相同的构造适用于任何提供梯度的模型，尽管本工作主要关注随机森林和梯度提升树。在表格分类和回归基准测试中，这种预处理方法持续改进了轴对齐森林的性能，并经常匹配或超越倾斜基线，同时提升了训练速度。我们的实验结果和理论分析共同表明，有监督的预处理可以在保持轴对齐树简洁性和鲁棒性的同时，恢复倾斜森林的大部分准确性。