We consider the setting of a dynamic database, where data is generated over time, and an analyst tracks statistics. We model privacy using differential privacy, which ensures privacy by distorting data at the expense of a loss of accuracy in the computation, resulting in a privacy-accuracy trade-off. Assuming that the analyst observes the private outputs continually, we propose two methods to improve the trade-off. First, we decay the input data relevance, by applying a sliding window or a decay function to the input before statistics computation. We provide algorithms and bounds on the tradeoff for sums. Second we decay the privacy guarantees over time. We derive a framework to model privacy expiration, design a scheme for linear functions, and show improvements in the trade-off for sums.