Recently my jenkins.log has started getting very large, very quickly, full of exceptions about DNS resolution. I attempted to use logrotate, but the log file grows too quick
You can plug in on Jenkins initialization with init.groovy file in Jenkins home folder and change the logging levels permanently with it. The changes will be kept even if Jenkins is restarted. There is a simple content of the file:
import java.util.logging.Level
import java.util.logging.Logger
Logger.getLogger("").setLevel(Level.SEVERE)
Logger.getLogger("org.apache.sshd").setLevel(Level.SEVERE)
Logger.getLogger("winstone").setLevel(Level.SEVERE)
You can change the name of the logger and the level so that it suits your needs. See my article on this topic for more details.