Skip to content

Commit

Permalink
Reduce sunrpc.tcp_max_slot_table_entries
Browse files Browse the repository at this point in the history
PR #93 did not work as expected.

Further analysis showed, that the parallelism of page io
(which includes buffered reads) is not limit by nfs.max_session_slots
but by sunrpc.tcp_max_slot_table_entries.

The default value is 65536 ( RPC_MAX_SLOT_TABLE_LIMIT ).

This parameter can be modified for the running system via
/proc/sys/sunrpc/tcp_max_slot_table_entries and is applied to
new server connections.

Experiment (on simsalabimbambasaladusaladim) :

    # preload server cache
    ssh mrtorgue cat /project/stresser/files/random_10G.dat \> /dev/null

    umount /project/stresser
    echo 3 > /proc/sys/vm/drop_caches
    time cat /project/stresser/files/random_10G.dat > /dev/null

    # result: 0m11.429s  ca. 14 nfsd visible active in `top` on server

    echo 4 > /proc/sys/sunrpc/tcp_max_slot_table_entries
    umount /project/stresser
    echo 3 > /proc/sys/vm/drop_caches
    time cat /project/stresser/files/random_10G.dat > /dev/null

    # 0m12.546s , 4 nfsd visible active in `top` on server

    echo 65536 > /proc/sys/sunrpc/tcp_max_slot_table_entries
    umount /project/stresser
    echo 3 > /proc/sys/vm/drop_caches
    time cat /project/stresser/files/random_10G.dat > /dev/null

    # result: 0m11.937s , ca. 16 nfsd visible active in `top` on server

Reduce sunrpc.tcp_max_slot_table_entries to 4 instead of nfs.max_session_slots
  • Loading branch information
donald committed Jun 28, 2019
1 parent 3c1cd23 commit 17a48dc
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion misc_etc_files/modprobe.d/nfs_slotlimit.conf
Original file line number Diff line number Diff line change
@@ -1 +1 @@
options nfs max_session_slots=4
options sunrpc tcp_max_slot_table_entries=4

0 comments on commit 17a48dc

Please sign in to comment.