In recent years, the rapid growth of mobile data demands has introduced many stringent requirements on latency and convergence performance in wireless network optimization. To address these challenges, several momentum-based algorithms have been proposed to improve the classical queue-length-based algorithmic framework (QLA). By combining queue-length updates and one-slot weight changes (known as the first-order momentum), it has been shown that these algorithms dramatically improve delay and convergence compared to QLA, while maintaining the same throughput-optimality and low-complexity. These exciting attempts have sparked a lot of conjectures about whether it is useful to further exploit high-order momentum information to improve delay and convergence speed. In this paper, we show that the answer is yes. Specifically, we first propose a new weight updating scheme that enables the incorporation of high-order momentum. We then prove the throughput-optimality and queue-stability of the proposed high-order momentum-based approach and characterize its delay and convergence performances. Through these analytical results, we finally show that delay and convergence would continue to improve as more high-order momentum information is utilized.