From 0dd6b95f9afe57da89b21ca7011839f6401ae068 Mon Sep 17 00:00:00 2001 From: Paul Menzel Date: Fri, 18 Sep 2026 10:37:08 +0200 Subject: [PATCH] nvidiactl: Disable GSP firmware via hostconfig tag *nvidia-no-gsp* MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Driver 580.178.04 and earlier enable the GPU System Processor (GSP) firmware by default on Turing cards. Since hooking up [Nvidia’s EGL libraries][1], on *ploetze* (OptiPlex 7071, RTX 2080/TU104) this leads to a reproducible lockup cascade under Xorg load: Xid 38/8 channel errors, repeated Xid 109 (CTX SWITCH TIMEOUT), until the GSP firmware itself panics (Xid 120, RISC-V crash report) and stops answering RPCs (Xid 119). The GPU is then unrecoverable until reboot. [580377.250202] NVRM: GPU at PCI:0000:01:00: GPU-0cd39e0b-595f-07c2-e5b3-ef9a66092d64 [580377.257777] NVRM: Xid (PCI:0000:01:00): 62, 00011fab 00012007 00011b38 000159fb 00015e06 00013e17 00000011 00000000 [580383.269003] NVRM: Xid (PCI:0000:01:00): 119, Timeout after 6s of waiting for RPC response from GPU0 GSP! Expected function 76 (GSP_RM_CONTROL) (0x20800a56 0x5c). [580383.283524] NVRM: GPU0 GSP RPC buffer contains function 76 (GSP_RM_CONTROL) and data 0x0000000020800a56 0x000000000000005c. [580383.294732] NVRM: GPU0 RPC history (CPU -> GSP): [580383.299440] NVRM: entry function data0 data1 ts_start ts_end duration actively_polling [580383.313504] NVRM: 0 76 GSP_RM_CONTROL 0x0000000020800a56 0x000000000000005c 0x00065b93db1410f3 0x0000000000000000 y [580383.326278] NVRM: -1 76 GSP_RM_CONTROL 0x00000000c3700104 0x0000000000000014 0x00065b93dad6badd 0x00065b93dad6bced 528us [580383.339045] NVRM: -2 76 GSP_RM_CONTROL 0x00000000c3700104 0x0000000000000014 0x00065b93dad6acb0 0x00065b93dad6ae57 423us [580383.351812] NVRM: -3 76 GSP_RM_CONTROL 0x00000000c3700104 0x0000000000000014 0x00065b93dad69e94 0x00065b93dad6a177 739us [580383.364579] NVRM: -4 76 GSP_RM_CONTROL 0x00000000c3700104 0x0000000000000014 0x00065b93dad69754 0x00065b93dad69cca 1398us [580383.377344] NVRM: -5 76 GSP_RM_CONTROL 0x00000000c3700104 0x0000000000000014 0x00065b93dad68d89 0x00065b93dad68efb 370us [580383.390120] NVRM: -6 76 GSP_RM_CONTROL 0x00000000c3700104 0x0000000000000014 0x00065b93dad67e73 0x00065b93dad683a3 1328us [580383.402898] NVRM: -7 76 GSP_RM_CONTROL 0x00000000c3700104 0x0000000000000014 0x00065b93dad64e4b 0x00065b93dad651fe 947us [580383.415673] NVRM: GPU0 RPC event history (CPU <- GSP): [580383.420898] NVRM: entry function data0 data1 ts_start ts_end duration during_incomplete_rpc [580383.435394] NVRM: 0 4130 RECOVERY_ACTION 0x0000000000000000 0x0000000000000000 0x00065b93db1410b0 0x00065b93db1410b1 1us [580383.448160] NVRM: -1 4102 OS_ERROR_LOG 0x0000000000000000 0x0000000000000000 0x00065b93db13c9fc 0x00065b93db1410af 18099us [580383.460935] NVRM: -2 4128 GSP_POST_NOCAT_RECORD 0x0000000000000003 0x0000000000011fab 0x00065b93db13c9fa 0x00065b93db13c9fc 2us [580383.473703] NVRM: -3 4128 GSP_POST_NOCAT_RECORD 0x0000000000000005 0x00000285057f2624 0x00065b937828a266 0x00065b937828a267 1us [580383.486467] NVRM: -4 4128 GSP_POST_NOCAT_RECORD 0x0000000000000002 0x0000000000000025 0x00065b93780dc3f3 0x00065b93780dc3f8 5us [580383.499231] NVRM: -5 4099 POST_EVENT 0x0000000000000001 0x0000000000000000 0x00065b93780dc3db 0x00065b93780dc3f1 22us [580383.511995] NVRM: -6 4128 GSP_POST_NOCAT_RECORD 0x0000000000000005 0x00000285057f2624 0x00065b873abd27cf 0x00065b873abd27d0 1us [580383.524761] NVRM: -7 4128 GSP_POST_NOCAT_RECORD 0x0000000000000005 0x00000285057f2624 0x00065b873abc9137 0x00065b873abc9139 2us [580383.537527] CPU: 9 UID: 0 PID: 711 Comm: nv_queue Tainted: P O 6.12.74.mx64.489 #1 [580383.537530] Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE [580383.537530] Hardware name: Dell Inc. OptiPlex 7071/097YXY, BIOS 1.2.1 11/11/2019 [580383.537531] Call Trace: [580383.537533] [580383.537534] dump_stack_lvl+0x53/0x70 [580383.537539] _nv013207rm+0x508/0x5b0 [nvidia] Pass `NVreg_EnableGpuFirmware=0` to insmod on hosts carrying the new hostconfig tag *nvidia-no-gsp*, so the resource manager runs on the CPU again, as it did on Turing before the 580 branch. Off by default: other GSP-capable desktops (RTX A4000, RTX 3060, RTX 4090) have not reported crashes, so only affected hosts (*ploetze* now, the identically equipped pcpool machines when needed) get the tag in `/etc/hostconfig`. insmod does not consult `/etc/modprobe.d`, so the parameter has to be given on the command line. Probe the module with `modinfo` first, because `insmod` fails hard on unknown parameters and drivers before 510 do not have this one. [1]: https://github.molgen.mpg.de/mariux64/mxtools/pull/578 Assisted-by: Claude Fable 5 Resolves: https://github.molgen.mpg.de/mariux64/mariux64-issues/issues/229 --- nvidiactl/nvidiactl | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/nvidiactl/nvidiactl b/nvidiactl/nvidiactl index 4d69a6df..7944b088 100755 --- a/nvidiactl/nvidiactl +++ b/nvidiactl/nvidiactl @@ -65,7 +65,17 @@ test -e /lib/modules/$kver/kernel/drivers/gpu/drm/drm.ko && modprobe drm test -e /lib/modules/$kver/kernel/drivers/gpu/drm/drm_kms_helper.ko && modprobe drm_kms_helper test -e /lib/modules/$kver/kernel/drivers/gpu/drm/drm_ttm_helper.ko && modprobe drm_ttm_helper -insmod ${nvlink}/nvidia.ko +nvargs="" +if /usr/sbin/hostconfig nvidia-no-gsp 2>/dev/null ; then + if modinfo -p ${nvlink}/nvidia.ko 2>/dev/null | grep -q NVreg_EnableGpuFirmware ; then + echo >&2 "NOTE: tag 'nvidia-no-gsp' set, disabling GSP firmware and run resource manager on CPU" + nvargs="NVreg_EnableGpuFirmware=0" + else + echo >&2 "NOTE: tag 'nvidia-no-gsp' set, but driver has no NVreg_EnableGpuFirmware parameter, ignoring.." + fi +fi + +insmod ${nvlink}/nvidia.ko ${nvargs} # Count the number of NVIDIA controllers found. NVDEVS=$(lspci | grep -i NVIDIA)