Skip to content

Commit

Permalink
ipvs: add sysctl_run_estimation to support disable estimation
Browse files Browse the repository at this point in the history
estimation_timer will iterate the est_list to do estimation
for each ipvs stats. When there are lots of services, the
list can be very large.
We found that estimation_timer() run for more then 200ms on a
machine with 104 CPU and 50K services.

yunhong-cgl jiang report the same phenomenon before:
https://www.spinics.net/lists/lvs-devel/msg05426.html

In some cases(for example a large K8S cluster with many ipvs services),
ipvs estimation may not be needed. So adding a sysctl blob to allow
users to disable this completely.

Default is: 1 (enable)

Cc: yunhong-cgl jiang <xintian1976@gmail.com>
Signed-off-by: Dust Li <dust.li@linux.alibaba.com>
Acked-by: Julian Anastasov <ja@ssi.bg>
Acked-by: Simon Horman <horms@verge.net.au>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
  • Loading branch information
Dust Li authored and Pablo Neira Ayuso committed Oct 7, 2021
1 parent c514fbb commit 2232642
Show file tree
Hide file tree
Showing 4 changed files with 35 additions and 0 deletions.
11 changes: 11 additions & 0 deletions Documentation/networking/ipvs-sysctl.rst
Original file line number Diff line number Diff line change
Expand Up @@ -300,3 +300,14 @@ sync_version - INTEGER

Kernels with this sync_version entry are able to receive messages
of both version 1 and version 2 of the synchronisation protocol.

run_estimation - BOOLEAN
0 - disabled
not 0 - enabled (default)

If disabled, the estimation will be stop, and you can't see
any update on speed estimation data.

You can always re-enable estimation by setting this value to 1.
But be careful, the first estimation after re-enable is not
accurate.
11 changes: 11 additions & 0 deletions include/net/ip_vs.h
Original file line number Diff line number Diff line change
Expand Up @@ -931,6 +931,7 @@ struct netns_ipvs {
int sysctl_conn_reuse_mode;
int sysctl_schedule_icmp;
int sysctl_ignore_tunneled;
int sysctl_run_estimation;

/* ip_vs_lblc */
int sysctl_lblc_expiration;
Expand Down Expand Up @@ -1071,6 +1072,11 @@ static inline int sysctl_cache_bypass(struct netns_ipvs *ipvs)
return ipvs->sysctl_cache_bypass;
}

static inline int sysctl_run_estimation(struct netns_ipvs *ipvs)
{
return ipvs->sysctl_run_estimation;
}

#else

static inline int sysctl_sync_threshold(struct netns_ipvs *ipvs)
Expand Down Expand Up @@ -1163,6 +1169,11 @@ static inline int sysctl_cache_bypass(struct netns_ipvs *ipvs)
return 0;
}

static inline int sysctl_run_estimation(struct netns_ipvs *ipvs)
{
return 1;
}

#endif

/* IPVS core functions
Expand Down
8 changes: 8 additions & 0 deletions net/netfilter/ipvs/ip_vs_ctl.c
Original file line number Diff line number Diff line change
Expand Up @@ -2017,6 +2017,12 @@ static struct ctl_table vs_vars[] = {
.mode = 0644,
.proc_handler = proc_dointvec,
},
{
.procname = "run_estimation",
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec,
},
#ifdef CONFIG_IP_VS_DEBUG
{
.procname = "debug_level",
Expand Down Expand Up @@ -4090,6 +4096,8 @@ static int __net_init ip_vs_control_net_init_sysctl(struct netns_ipvs *ipvs)
tbl[idx++].data = &ipvs->sysctl_conn_reuse_mode;
tbl[idx++].data = &ipvs->sysctl_schedule_icmp;
tbl[idx++].data = &ipvs->sysctl_ignore_tunneled;
ipvs->sysctl_run_estimation = 1;
tbl[idx++].data = &ipvs->sysctl_run_estimation;

ipvs->sysctl_hdr = register_net_sysctl(net, "net/ipv4/vs", tbl);
if (ipvs->sysctl_hdr == NULL) {
Expand Down
5 changes: 5 additions & 0 deletions net/netfilter/ipvs/ip_vs_est.c
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,9 @@ static void estimation_timer(struct timer_list *t)
u64 rate;
struct netns_ipvs *ipvs = from_timer(ipvs, t, est_timer);

if (!sysctl_run_estimation(ipvs))
goto skip;

spin_lock(&ipvs->est_lock);
list_for_each_entry(e, &ipvs->est_list, list) {
s = container_of(e, struct ip_vs_stats, est);
Expand Down Expand Up @@ -131,6 +134,8 @@ static void estimation_timer(struct timer_list *t)
spin_unlock(&s->lock);
}
spin_unlock(&ipvs->est_lock);

skip:
mod_timer(&ipvs->est_timer, jiffies + 2*HZ);
}

Expand Down

0 comments on commit 2232642

Please sign in to comment.