Retry

Hertz provides users with customized retry logic,Let’s take a look at how to use Client Retry. Note: Hertz version >= v0.4.0

Retry times and delay policy configuration

First create a Client, and use the configuration item WithRetryConfig() to configure the Retry related logic. (This section mainly configures the times of retry and the delay policy)

package main

import (
	"github.com/cloudwego/hertz/pkg/app/client"
	"github.com/cloudwego/hertz/pkg/app/client/retry"
)

func main() {
	cli, err := client.NewClient(
		client.WithRetryConfig(
			retry.WithXxx(), // the method of setting retry config
		),
	)
}
Configuration Name Type Description
WithMaxAttemptTimes uint Set the maximum attempts times. Default:1 times (That is, only request once without retry)
WithInitDelay time.Duration Set initial delay time. Default: 1ms
WithMaxDelay time.Duration Set the maximum delay time. Default: 100ms
WithMaxJitter time.Duration Set the maximum jitter time, which needs to be used in conjunction with RandomDelayPolicy, and will generate a random time not exceeding the maximum jitter time. Default: 20ms
WithDelayPolicy type DelayPolicyFunc func(attempts uint, err error, retryConfig *Config) time.Duration Set the delay policy, you can use any combination of the following four policies, FixedDelayPolicy, BackOffDelayPolicy, RandomDelayPolicy, DefaultDelayPolicy ( See the next section Delay Policy for details ) . Default: DefaultDelayPolicy (That is, the retry delay is 0)

Delay Policy

retry.WithDelayPolicy() usage

cli, err := client.NewClient(
		client.WithRetryConfig(
			...
			retry.WithDelayPolicy(retry.CombineDelay(retry.FixedDelayPolicy, retry.BackOffDelayPolicy, retry.RandomDelayPolicy)),
    		...
		),
	)
Function Name Description
CombineDelay It is used to combine any of the following four policies and sum the values calculated by the selected policy. When you only need one of the following four policies, you can choose to use CombineDelay or directly pass any policy into WithDelayPolicy as a parameter
FixedDelayPolicy Set the fixed delay time and use the value set by WithInitDelay to generate an equivalent delay time
BackOffDelayPolicy Set the exponential delay time. Use the value set by WithInitDelay. The exponential delay time is generated according to the number of retries currently
RandomDelayPolicy Set the random delay time. Use the value set by WithMaxJitter to generate a random delay time that does not exceed this value
DefaultDelayPolicy Set the default delay time (That is, 0) . Generally, it is used alone and has no effect when combined with other policies

Complete example

package main

import (
	"github.com/cloudwego/hertz/pkg/app/client"
	"github.com/cloudwego/hertz/pkg/app/client/retry"
)
func main() {

	cli, err := client.NewClient(
		client.WithRetryConfig(
			retry.WithMaxAttemptTimes(3), // Maximum number of attempts, including initial calls
			retry.WithInitDelay(1*time.Millisecond), // Initial delay
			retry.WithMaxDelay(6*time.Millisecond), // Maximum delay.No matter how many retries and what the policy is, the delay will not exceed this delay
			retry.WithMaxJitter(2*time.Millisecond), // Maximum jitter delay, which will have effect only when combined with RandomDelayPolicy
			/*
			   To configure the delay policy, you can select any combination of the following four, and the final result is the sum of each delay policy.
			   FixedDelayPolicy uses the value set by retry.WithInitDelay,
			   BackOffDelayPolicy increases exponentially with the number of retries based on the value set by retry.WithInitDelay,
			   RandomDelayPolicy generates a random value of [0, 2*time.Millisecond). 2*time.Millisecond is the value set by retry.WithMaxJitter,
			   DefaultDelayPolicy generates a value of 0. If it is used alone, retry again immediately,
			   retry.CombineDelay() sums the values generated by the set delay policy, and the final result is the delay time of the current retry,
			   The first call failed -> Retry delay:1 + 1<<1 + rand[0,2)ms -> The second call failed -> Retry delay:min(1 + 1<<2 + rand[0,2) , 6)ms -> The third call succeeded/failed
			*/
			retry.WithDelayPolicy(retry.CombineDelay(retry.FixedDelayPolicy, retry.BackOffDelayPolicy, retry.RandomDelayPolicy)),
		),
	)
}

Retry condition configuration

If you want to customize the conditions under which retries occur, you can use client SetRetryIfFunc() configuration. The parameter of this function is a function, and the signature is:

func(req *protocol.Request, resp *protocol.Response, err error) bool

Relevant parameters include the req, resp and err fields in the Hertz request. You can use these parameters to determine whether the request should be retried. In the following example, when the status code returned by the request is not 200 or err!=nil during the call, we return true, that is, we retry.

cli.SetRetryIfFunc(func(req *protocol.Request, resp *protocol.Response, err error) bool {
   return resp.StatusCode() != 200 || err != nil
})

Note that if you do not set client SetRetryIfFunc() . We will judge according to Hertz’s default retry conditions, that is, whether the request meets the following DefaultRetryIf() function and whether the call is idempotent. ( Idempotent call: when canIdempotentRetry is true in pkg/protocol/http1/client.go::Do() and pkg/protocol/http1/client.go::doNonNilReqResp() )

// DefaultRetryIf Default retry condition, mainly used for idempotent requests.
// If this cannot be satisfied, you can implement your own retry condition.
func DefaultRetryIf(req *protocol.Request, resp *protocol.Response, err error) bool {
   // cannot retry if the request body is not rewindable
   if req.IsBodyStream() {
      return false
   }

   if isIdempotent(req, resp, err) {
      return true
   }
   // Retry non-idempotent requests if the server closes
   // the connection before sending the response.
   //
   // This case is possible if the server closes the idle
   // keep-alive connection on timeout.
   //
   // Apache and nginx usually do this.
   if err == io.EOF {
      return true
   }

   return false
}
func isIdempotent(req *protocol.Request, resp *protocol.Response, err error) bool {
   return req.Header.IsGet() ||
      req.Header.IsHead() ||
      req.Header.IsPut() ||
      req.Header.IsDelete() ||
      req.Header.IsOptions() ||
      req.Header.IsTrace()
}

Table - 1 When canIdempotentRetry in Hertz source code doNonNilReqResp() is true.

doNonNilReqResp() return true
err = conn.SetWriteDeadline(currentTime.Add(c.WriteTimeout))
err = reqI.Write(req, zw)
err = reqI.ProxyWrite(req, zw)
err = zw.Flush()
err = conn.SetReadTimeout(c.ReadTimeout)
( err = respI.ReadHeaderAndLimitBody() || err = respI.ReadBodyStream() ) && (err != errs.ErrBodyTooLarge)

Last modified April 18, 2023 : docs(wip): request context handler (36ea2d4)