We consider a remote estimation problem where a non-stochastic source is observed by an energy-constrained sensor. The sensor can communicate with a remote estimator which needs to produce real-time estimates of the source. We consider the problem of finding a communication scheduling strategy for the sensor and an estimation strategy for the estimator that jointly minimizes the worst-case instantaneous estimation cost. Our methodology makes use of decentralized minimax control methods to find optimal strategies.