I have a metric varnish_main_client_req of type counter and I want to set up an alert that triggers if the rate of requests drops/raises by a certain amount in a gi
Yes, you need to use a recording rule for this.
Prometheus calculates the rate of client requests over the last 2 mins and returns a derivative of the resulting values over the last 5 mins.
Herein lies the problem - at what interval should Prometheus synthesise this data?