Skip to content

Commit

Permalink
s390/pci: add s390_iommu_aperture kernel parameter
Browse files Browse the repository at this point in the history
Some applications map the same memory area for DMA multiple times while
also mapping significant amounts of memory. With our current DMA code
these applications will run out of DMA addresses after mapping half of
the available memory because the number of DMA mappings is constrained
by the number of concurrently active DMA addresses we support which in
turn is limited by the minimum of hardware constraints and high_memory.

Limiting the number of active DMA addresses to high_memory is only
a heuristic to save memory used by the iommu_bitmap and DMA page tables
however. This was added under the assumption that it rarely makes sense
to DMA map more than system memory.

To accommodate special applications which insist on double mapping, which
works on other platforms, allow specifying a factor of how many times
installed memory is available as DMA address space. Use 0 as a special
value to apply no constraints beyond what hardware dictates at the
expense of significantly more memory use.

Reviewed-by: Matthew Rosato <mjrosato@linux.ibm.com>
Reviewed-by: Pierre Morel <pmorel@linux.ibm.com>
Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
  • Loading branch information
Niklas Schnelle authored and Vasily Gorbik committed Oct 26, 2021
1 parent 74e74f9 commit 6aefbf1
Show file tree
Hide file tree
Showing 2 changed files with 35 additions and 2 deletions.
12 changes: 12 additions & 0 deletions Documentation/admin-guide/kernel-parameters.txt
Original file line number Diff line number Diff line change
Expand Up @@ -4982,6 +4982,18 @@
an IOTLB flush. Default is lazy flushing before reuse,
which is faster.

s390_iommu_aperture= [KNL,S390]
Specifies the size of the per device DMA address space
accessible through the DMA and IOMMU APIs as a decimal
factor of the size of main memory.
The default is 1 meaning that one can concurrently use
as many DMA addresses as physical memory is installed,
if supported by hardware, and thus map all of memory
once. With a value of 2 one can map all of memory twice
and so on. As a special case a factor of 0 imposes no
restrictions other than those given by hardware at the
cost of significant additional memory use for tables.

sa1100ir [NET]
See drivers/net/irda/sa1100_ir.c.

Expand Down
25 changes: 23 additions & 2 deletions arch/s390/pci/pci_dma.c
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@
static struct kmem_cache *dma_region_table_cache;
static struct kmem_cache *dma_page_table_cache;
static int s390_iommu_strict;
static u64 s390_iommu_aperture;
static u32 s390_iommu_aperture_factor = 1;

static int zpci_refresh_global(struct zpci_dev *zdev)
{
Expand Down Expand Up @@ -565,15 +567,19 @@ int zpci_dma_init_device(struct zpci_dev *zdev)

/*
* Restrict the iommu bitmap size to the minimum of the following:
* - main memory size
* - s390_iommu_aperture which defaults to high_memory
* - 3-level pagetable address limit minus start_dma offset
* - DMA address range allowed by the hardware (clp query pci fn)
*
* Also set zdev->end_dma to the actual end address of the usable
* range, instead of the theoretical maximum as reported by hardware.
*
* This limits the number of concurrently usable DMA mappings since
* for each DMA mapped memory address we need a DMA address including
* extra DMA addresses for multiple mappings of the same memory address.
*/
zdev->start_dma = PAGE_ALIGN(zdev->start_dma);
zdev->iommu_size = min3((u64) high_memory,
zdev->iommu_size = min3(s390_iommu_aperture,
ZPCI_TABLE_SIZE_RT - zdev->start_dma,
zdev->end_dma - zdev->start_dma + 1);
zdev->end_dma = zdev->start_dma + zdev->iommu_size - 1;
Expand Down Expand Up @@ -660,6 +666,12 @@ static int __init dma_alloc_cpu_table_caches(void)

int __init zpci_dma_init(void)
{
s390_iommu_aperture = (u64)high_memory;
if (!s390_iommu_aperture_factor)
s390_iommu_aperture = ULONG_MAX;
else
s390_iommu_aperture *= s390_iommu_aperture_factor;

return dma_alloc_cpu_table_caches();
}

Expand Down Expand Up @@ -692,3 +704,12 @@ static int __init s390_iommu_setup(char *str)
}

__setup("s390_iommu=", s390_iommu_setup);

static int __init s390_iommu_aperture_setup(char *str)
{
if (kstrtou32(str, 10, &s390_iommu_aperture_factor))
s390_iommu_aperture_factor = 1;
return 1;
}

__setup("s390_iommu_aperture=", s390_iommu_aperture_setup);

0 comments on commit 6aefbf1

Please sign in to comment.