Fermi's golden rule is widely used, and the resulting transition rates are an important part of the thermal behaviour of open quantum systems. But this rule is curious because it is valid outside the regime in which it is derived: It is derived only for short times and for off-resonant transitions but works for all times and for resonant transitions.
Here we show analytically that an interaction with a resonant, dense spectrum induces a rate equation for all times, giving essentially exact exponential decay in the appropriate regime. From this analysis we are able to extract the decay rate, which is indeed the rate of Fermi's golden rule (with a small correction), the short, non-Markovian time period before which the rate equation sets in, and determine the parameter regime required for this behavior. Our analysis provides the start of a more solid foundation on which to model thermal baths in terms of interactions with dense spectra.