-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs: fault-injection: convert docs to ReST and rename to *.rst
The conversion is actually: - add blank lines and identation in order to identify paragraphs; - fix tables markups; - add some lists markups; - mark literal blocks; - adjust title markups. At its new index.rst, let's add a :orphan: while this is not linked to the main index.rst file, in order to avoid build warnings. Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org> Acked-by: Federico Vaga <federico.vaga@vaga.pv.it> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
- Loading branch information
Mauro Carvalho Chehab
authored and
Jonathan Corbet
committed
Jun 14, 2019
1 parent
f0ba437
commit 10ffebb
Showing
14 changed files
with
352 additions
and
303 deletions.
There are no files selected for viewing
281 changes: 146 additions & 135 deletions
281
...ation/fault-injection/fault-injection.txt → ...ation/fault-injection/fault-injection.rst
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
:orphan: | ||
|
||
=============== | ||
fault-injection | ||
=============== | ||
|
||
.. toctree:: | ||
:maxdepth: 1 | ||
|
||
fault-injection | ||
notifier-error-inject | ||
nvme-fault-injection | ||
provoke-crashes | ||
|
||
.. only:: subproject and html | ||
|
||
Indices | ||
======= | ||
|
||
* :ref:`genindex` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,120 @@ | ||
NVMe Fault Injection | ||
==================== | ||
Linux's fault injection framework provides a systematic way to support | ||
error injection via debugfs in the /sys/kernel/debug directory. When | ||
enabled, the default NVME_SC_INVALID_OPCODE with no retry will be | ||
injected into the nvme_end_request. Users can change the default status | ||
code and no retry flag via the debugfs. The list of Generic Command | ||
Status can be found in include/linux/nvme.h | ||
|
||
Following examples show how to inject an error into the nvme. | ||
|
||
First, enable CONFIG_FAULT_INJECTION_DEBUG_FS kernel config, | ||
recompile the kernel. After booting up the kernel, do the | ||
following. | ||
|
||
Example 1: Inject default status code with no retry | ||
--------------------------------------------------- | ||
|
||
:: | ||
|
||
mount /dev/nvme0n1 /mnt | ||
echo 1 > /sys/kernel/debug/nvme0n1/fault_inject/times | ||
echo 100 > /sys/kernel/debug/nvme0n1/fault_inject/probability | ||
cp a.file /mnt | ||
|
||
Expected Result:: | ||
|
||
cp: cannot stat ‘/mnt/a.file’: Input/output error | ||
|
||
Message from dmesg:: | ||
|
||
FAULT_INJECTION: forcing a failure. | ||
name fault_inject, interval 1, probability 100, space 0, times 1 | ||
CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.15.0-rc8+ #2 | ||
Hardware name: innotek GmbH VirtualBox/VirtualBox, | ||
BIOS VirtualBox 12/01/2006 | ||
Call Trace: | ||
<IRQ> | ||
dump_stack+0x5c/0x7d | ||
should_fail+0x148/0x170 | ||
nvme_should_fail+0x2f/0x50 [nvme_core] | ||
nvme_process_cq+0xe7/0x1d0 [nvme] | ||
nvme_irq+0x1e/0x40 [nvme] | ||
__handle_irq_event_percpu+0x3a/0x190 | ||
handle_irq_event_percpu+0x30/0x70 | ||
handle_irq_event+0x36/0x60 | ||
handle_fasteoi_irq+0x78/0x120 | ||
handle_irq+0xa7/0x130 | ||
? tick_irq_enter+0xa8/0xc0 | ||
do_IRQ+0x43/0xc0 | ||
common_interrupt+0xa2/0xa2 | ||
</IRQ> | ||
RIP: 0010:native_safe_halt+0x2/0x10 | ||
RSP: 0018:ffffffff82003e90 EFLAGS: 00000246 ORIG_RAX: ffffffffffffffdd | ||
RAX: ffffffff817a10c0 RBX: ffffffff82012480 RCX: 0000000000000000 | ||
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 | ||
RBP: 0000000000000000 R08: 000000008e38ce64 R09: 0000000000000000 | ||
R10: 0000000000000000 R11: 0000000000000000 R12: ffffffff82012480 | ||
R13: ffffffff82012480 R14: 0000000000000000 R15: 0000000000000000 | ||
? __sched_text_end+0x4/0x4 | ||
default_idle+0x18/0xf0 | ||
do_idle+0x150/0x1d0 | ||
cpu_startup_entry+0x6f/0x80 | ||
start_kernel+0x4c4/0x4e4 | ||
? set_init_arg+0x55/0x55 | ||
secondary_startup_64+0xa5/0xb0 | ||
print_req_error: I/O error, dev nvme0n1, sector 9240 | ||
EXT4-fs error (device nvme0n1): ext4_find_entry:1436: | ||
inode #2: comm cp: reading directory lblock 0 | ||
|
||
Example 2: Inject default status code with retry | ||
------------------------------------------------ | ||
|
||
:: | ||
|
||
mount /dev/nvme0n1 /mnt | ||
echo 1 > /sys/kernel/debug/nvme0n1/fault_inject/times | ||
echo 100 > /sys/kernel/debug/nvme0n1/fault_inject/probability | ||
echo 1 > /sys/kernel/debug/nvme0n1/fault_inject/status | ||
echo 0 > /sys/kernel/debug/nvme0n1/fault_inject/dont_retry | ||
|
||
cp a.file /mnt | ||
|
||
Expected Result:: | ||
|
||
command success without error | ||
|
||
Message from dmesg:: | ||
|
||
FAULT_INJECTION: forcing a failure. | ||
name fault_inject, interval 1, probability 100, space 0, times 1 | ||
CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.15.0-rc8+ #4 | ||
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006 | ||
Call Trace: | ||
<IRQ> | ||
dump_stack+0x5c/0x7d | ||
should_fail+0x148/0x170 | ||
nvme_should_fail+0x30/0x60 [nvme_core] | ||
nvme_loop_queue_response+0x84/0x110 [nvme_loop] | ||
nvmet_req_complete+0x11/0x40 [nvmet] | ||
nvmet_bio_done+0x28/0x40 [nvmet] | ||
blk_update_request+0xb0/0x310 | ||
blk_mq_end_request+0x18/0x60 | ||
flush_smp_call_function_queue+0x3d/0xf0 | ||
smp_call_function_single_interrupt+0x2c/0xc0 | ||
call_function_single_interrupt+0xa2/0xb0 | ||
</IRQ> | ||
RIP: 0010:native_safe_halt+0x2/0x10 | ||
RSP: 0018:ffffc9000068bec0 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff04 | ||
RAX: ffffffff817a10c0 RBX: ffff88011a3c9680 RCX: 0000000000000000 | ||
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 | ||
RBP: 0000000000000001 R08: 000000008e38c131 R09: 0000000000000000 | ||
R10: 0000000000000000 R11: 0000000000000000 R12: ffff88011a3c9680 | ||
R13: ffff88011a3c9680 R14: 0000000000000000 R15: 0000000000000000 | ||
? __sched_text_end+0x4/0x4 | ||
default_idle+0x18/0xf0 | ||
do_idle+0x150/0x1d0 | ||
cpu_startup_entry+0x6f/0x80 | ||
start_secondary+0x187/0x1e0 | ||
secondary_startup_64+0xa5/0xb0 |
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,48 @@ | ||
=============== | ||
Provoke crashes | ||
=============== | ||
|
||
The lkdtm module provides an interface to crash or injure the kernel at | ||
predefined crashpoints to evaluate the reliability of crash dumps obtained | ||
using different dumping solutions. The module uses KPROBEs to instrument | ||
crashing points, but can also crash the kernel directly without KRPOBE | ||
support. | ||
|
||
|
||
You can provide the way either through module arguments when inserting | ||
the module, or through a debugfs interface. | ||
|
||
Usage:: | ||
|
||
insmod lkdtm.ko [recur_count={>0}] cpoint_name=<> cpoint_type=<> | ||
[cpoint_count={>0}] | ||
|
||
recur_count | ||
Recursion level for the stack overflow test. Default is 10. | ||
|
||
cpoint_name | ||
Crash point where the kernel is to be crashed. It can be | ||
one of INT_HARDWARE_ENTRY, INT_HW_IRQ_EN, INT_TASKLET_ENTRY, | ||
FS_DEVRW, MEM_SWAPOUT, TIMERADD, SCSI_DISPATCH_CMD, | ||
IDE_CORE_CP, DIRECT | ||
|
||
cpoint_type | ||
Indicates the action to be taken on hitting the crash point. | ||
It can be one of PANIC, BUG, EXCEPTION, LOOP, OVERFLOW, | ||
CORRUPT_STACK, UNALIGNED_LOAD_STORE_WRITE, OVERWRITE_ALLOCATION, | ||
WRITE_AFTER_FREE, | ||
|
||
cpoint_count | ||
Indicates the number of times the crash point is to be hit | ||
to trigger an action. The default is 10. | ||
|
||
You can also induce failures by mounting debugfs and writing the type to | ||
<mountpoint>/provoke-crash/<crashpoint>. E.g.:: | ||
|
||
mount -t debugfs debugfs /mnt | ||
echo EXCEPTION > /mnt/provoke-crash/INT_HARDWARE_ENTRY | ||
|
||
|
||
A special file is `DIRECT` which will induce the crash directly without | ||
KPROBE instrumentation. This mode is the only one available when the module | ||
is built on a kernel without KPROBEs support. |
Oops, something went wrong.