Skip to content

Commit

Permalink
knfsd: remove the nfsd thread busy histogram
Browse files Browse the repository at this point in the history
Stop gathering the data that feeds the 'th' line in /proc/net/rpc/nfsd
because the questionable data provided is not worth the scalability
impact of calculating it.  Instead, always report zeroes.  The current
approach suffers from three major issues:

1. update_thread_usage() increments buckets by call service
   time or call arrival time...in jiffies.  On lightly loaded
   machines, call service times are usually < 1 jiffy; on
   heavily loaded machines call arrival times will be << 1 jiffy.
   So a large portion of the updates to the buckets are rounded
   down to zero, and the histogram is undercounting.

2. As seen previously on the nfs mailing list, the format in which
   the histogram is presented is cryptic, difficult to explain,
   and difficult to use.

3. Updating the histogram requires taking a global spinlock and
   dirtying the global variables nfsd_last_call, nfsd_busy, and
   nfsdstats *twice* on every RPC call, which is a significant
   scaling limitation.

Testing on a 4 CPU 4 NIC Altix using 4 IRIX clients each doing
1K streaming reads at full line rate, shows the stats update code
(inlined into nfsd()) takes about 1.7% of each CPU.  This patch drops
the contribution from nfsd() into the profile noise.

This patch is a forward-ported version of knfsd-remove-nfsd-threadstats
which has been shipping in the SGI "Enhanced NFS" product since 2006.
In that time, exactly one customer has noticed that the threadstats
were missing.  It has been previously posted:

http://article.gmane.org/gmane.linux.nfs/10376

and more recently requested to be posted again.

Signed-off-by: Greg Banks <gnb@sgi.com>
Signed-off-by: J. Bruce Fields <bfields@citi.umich.edu>
  • Loading branch information
Greg Banks authored and J. Bruce Fields committed Mar 18, 2009
1 parent 5cb031b commit 8bbfa9f
Showing 1 changed file with 0 additions and 28 deletions.
28 changes: 0 additions & 28 deletions fs/nfsd/nfssvc.c
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,6 @@
extern struct svc_program nfsd_program;
static int nfsd(void *vrqstp);
struct timeval nfssvc_boot;
static atomic_t nfsd_busy;
static unsigned long nfsd_last_call;
static DEFINE_SPINLOCK(nfsd_call_lock);

/*
* nfsd_mutex protects nfsd_serv -- both the pointer itself and the members
Expand Down Expand Up @@ -227,7 +224,6 @@ int nfsd_create_serv(void)
nfsd_max_blksize /= 2;
}

atomic_set(&nfsd_busy, 0);
nfsd_serv = svc_create_pooled(&nfsd_program, nfsd_max_blksize,
AF_INET,
nfsd_last_thread, nfsd, THIS_MODULE);
Expand Down Expand Up @@ -376,26 +372,6 @@ nfsd_svc(unsigned short port, int nrservs)
return error;
}

static inline void
update_thread_usage(int busy_threads)
{
unsigned long prev_call;
unsigned long diff;
int decile;

spin_lock(&nfsd_call_lock);
prev_call = nfsd_last_call;
nfsd_last_call = jiffies;
decile = busy_threads*10/nfsdstats.th_cnt;
if (decile>0 && decile <= 10) {
diff = nfsd_last_call - prev_call;
if ( (nfsdstats.th_usage[decile-1] += diff) >= NFSD_USAGE_WRAP)
nfsdstats.th_usage[decile-1] -= NFSD_USAGE_WRAP;
if (decile == 10)
nfsdstats.th_fullcnt++;
}
spin_unlock(&nfsd_call_lock);
}

/*
* This is the NFS server kernel thread
Expand Down Expand Up @@ -464,8 +440,6 @@ nfsd(void *vrqstp)
continue;
}

update_thread_usage(atomic_read(&nfsd_busy));
atomic_inc(&nfsd_busy);

/* Lock the export hash tables for reading. */
exp_readlock();
Expand All @@ -474,8 +448,6 @@ nfsd(void *vrqstp)

/* Unlock export hash tables */
exp_readunlock();
update_thread_usage(atomic_read(&nfsd_busy));
atomic_dec(&nfsd_busy);
}

/* Clear signals before calling svc_exit_thread() */
Expand Down

0 comments on commit 8bbfa9f

Please sign in to comment.