In this paper, we propose a joint learning and control approach for optimizing performance in stochastic queueing networks. Our approach is motivated by the recently developed Lyapunov optimization technique and utilizes the connection between queueing network control and Lagrange multiplier learning. We show that by incorporating a learning phase, which estimates an optimal Lagrange multiplier for a carefully constructed deterministic optimization problem, algorithms constructed based on Lyapunov optimization can achieve near-optimal utility-delay tradeoff for general stochastic queueing networks. Moreover, the learning-aided algorithms process faster convergence speed compared to the ones that do not utilize such information.