Skip to content

Commit

Permalink
Reduce nfs max_session_lots from 64 to 4
Browse files Browse the repository at this point in the history
The default number of session slots (nfs/max_session_lots) is 64. This
is the maximum number of parallel requests a nfs client can have
outstanding to a nfs server (per mount). Because we have 64 nfsd threads
on our servers, a single client is able to dominate all nsfd threads of
a server.

There seems to be a bad interaction between nfs and readahead. The
readahead size is set to max_session_lots-1 and is doubled when
posix_fadvice(..,POSIX_FADV_SEQUENTIAL) is used, which "cp" and "cat"
do.  Although the exact mechanism is not yet fully understood, the
readahead io-requests seem to be handled by multiple nfsd thread in
parallel, which are blocking each other while compeeting for the same
block device. Because all nfsd threads are used up (by a single client
doing a single copy) other nfs clients stall and we've even seen them
being pushed over their lease timeouts and losing their state.

Reduce this value via modprobe.d to avoid overload situations.

This parameter can also be read and written on a running system via
/sys/module/nfs/parameters/max_session_slots if nfs is loaded.  Changes
apply to future mounts only.
  • Loading branch information
donald committed May 24, 2019
1 parent 7749375 commit caadb40
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 0 deletions.
1 change: 1 addition & 0 deletions install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -148,4 +148,5 @@ install_data misc_systemd_units/network.service "$DESTDIR$systemdunitd
install_data misc_systemd_units/mxvlan.service "$DESTDIR$systemdunitdir/mxvlan.service"
mkdir -p "$DESTDIR$usr_prefix/share/applications"
install_data applications-defaults/defaults.list "$DESTDIR$usr_prefix/share/applications/defaults.list"
install_data misc_etc_files/modprobe.d/nfs_slotlimit.conf "$DESTDIR$sysconfdir/modprobe.d/nfs_slotlimit.conf"
exit
1 change: 1 addition & 0 deletions misc_etc_files/modprobe.d/nfs_slotlimit.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
options nfs max_session_slots=4

0 comments on commit caadb40

Please sign in to comment.