Skip to content

Commit

Permalink
Merge branch 'mm-everything' of git://git.kernel.org/pub/scm/linux/ke…
Browse files Browse the repository at this point in the history
…rnel/git/akpm/mm
  • Loading branch information
Stephen Rothwell committed Jul 28, 2022
2 parents d764339 + 4724a97 commit 40ef37d
Show file tree
Hide file tree
Showing 127 changed files with 2,358 additions and 1,821 deletions.
3 changes: 2 additions & 1 deletion .mailmap
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ Boris Brezillon <bbrezillon@kernel.org> <b.brezillon.dev@gmail.com>
Boris Brezillon <bbrezillon@kernel.org> <b.brezillon@overkiz.com>
Boris Brezillon <bbrezillon@kernel.org> <boris.brezillon@bootlin.com>
Boris Brezillon <bbrezillon@kernel.org> <boris.brezillon@free-electrons.com>
Brendan Higgins <brendan.higgins@linux.dev> <brendanhiggins@google.com>
Brian Avery <b.avery@hp.com>
Brian King <brking@us.ibm.com>
Brian Silverman <bsilver16384@gmail.com> <brian.silverman@bluerivertech.com>
Expand Down Expand Up @@ -230,7 +231,7 @@ Kees Cook <keescook@chromium.org> <kees@ubuntu.com>
Keith Busch <kbusch@kernel.org> <keith.busch@intel.com>
Keith Busch <kbusch@kernel.org> <keith.busch@linux.intel.com>
Kenneth W Chen <kenneth.w.chen@intel.com>
Kirill Tkhai <kirill.tkhai@openvz.org> <ktkhai@virtuozzo.com>
Kirill Tkhai <tkhai@ya.ru> <ktkhai@virtuozzo.com>
Konstantin Khlebnikov <koct9i@gmail.com> <khlebnikov@yandex-team.ru>
Konstantin Khlebnikov <koct9i@gmail.com> <k.khlebnikov@samsung.com>
Koushik <raghavendra.koushik@neterion.com>
Expand Down
7 changes: 7 additions & 0 deletions Documentation/admin-guide/cgroup-v2.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1237,6 +1237,13 @@ PAGE_SIZE multiple when read back.
the target cgroup. If less bytes are reclaimed than the
specified amount, -EAGAIN is returned.

Please note that the proactive reclaim (triggered by this
interface) is not meant to indicate memory pressure on the
memory cgroup. Therefore socket memory balancing triggered by
the memory reclaim normally is not exercised in this case.
This means that the networking layer will not adapt based on
reclaim induced by memory.reclaim.

memory.peak
A read-only single value file which exists on non-root
cgroups.
Expand Down
7 changes: 4 additions & 3 deletions Documentation/admin-guide/kernel-parameters.txt
Original file line number Diff line number Diff line change
Expand Up @@ -1735,12 +1735,13 @@
hugetlb_free_vmemmap=
[KNL] Reguires CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP
enabled.
Control if HugeTLB Vmemmap Optimization (HVO) is enabled.
Allows heavy hugetlb users to free up some more
memory (7 * PAGE_SIZE for each 2MB hugetlb page).
Format: { [oO][Nn]/Y/y/1 | [oO][Ff]/N/n/0 (default) }
Format: { on | off (default) }

[oO][Nn]/Y/y/1: enable the feature
[oO][Ff]/N/n/0: disable the feature
on: enable HVO
off: disable HVO

Built with CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP_DEFAULT_ON=y,
the default is on.
Expand Down
4 changes: 2 additions & 2 deletions Documentation/admin-guide/mm/hugetlbpage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -164,8 +164,8 @@ default_hugepagesz
will all result in 256 2M huge pages being allocated. Valid default
huge page size is architecture dependent.
hugetlb_free_vmemmap
When CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP is set, this enables optimizing
unused vmemmap pages associated with each HugeTLB page.
When CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP is set, this enables HugeTLB
Vmemmap Optimization (HVO).

When multiple huge page sizes are supported, ``/proc/sys/vm/nr_hugepages``
indicates the current number of pre-allocated huge pages of the default size.
Expand Down
4 changes: 2 additions & 2 deletions Documentation/admin-guide/mm/memory-hotplug.rst
Original file line number Diff line number Diff line change
Expand Up @@ -653,8 +653,8 @@ block might fail:
- Concurrent activity that operates on the same physical memory area, such as
allocating gigantic pages, can result in temporary offlining failures.

- Out of memory when dissolving huge pages, especially when freeing unused
vmemmap pages associated with each hugetlb page is enabled.
- Out of memory when dissolving huge pages, especially when HugeTLB Vmemmap
Optimization (HVO) is enabled.

Offlining code may be able to migrate huge page contents, but may not be able
to dissolve the source huge page because it fails allocating (unmovable) pages
Expand Down
3 changes: 1 addition & 2 deletions Documentation/admin-guide/sysctl/vm.rst
Original file line number Diff line number Diff line change
Expand Up @@ -569,8 +569,7 @@ This knob is not available when the size of 'struct page' (a structure defined
in include/linux/mm_types.h) is not power of two (an unusual system config could
result in this).

Enable (set to 1) or disable (set to 0) the feature of optimizing vmemmap pages
associated with each HugeTLB page.
Enable (set to 1) or disable (set to 0) HugeTLB Vmemmap Optimization (HVO).

Once enabled, the vmemmap pages of subsequent allocation of HugeTLB pages from
buddy allocator will be optimized (7 pages per 2MB HugeTLB page and 4095 pages
Expand Down
22 changes: 20 additions & 2 deletions Documentation/filesystems/proc.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1891,25 +1891,30 @@ if precise results are needed.
3.8 /proc/<pid>/fdinfo/<fd> - Information about opened file
---------------------------------------------------------------
This file provides information associated with an opened file. The regular
files have at least four fields -- 'pos', 'flags', 'mnt_id' and 'ino'.
files have at least five fields -- 'pos', 'flags', 'mnt_id', 'ino', and 'size'.

The 'pos' represents the current offset of the opened file in decimal
form [see lseek(2) for details], 'flags' denotes the octal O_xxx mask the
file has been created with [see open(2) for details] and 'mnt_id' represents
mount ID of the file system containing the opened file [see 3.5
/proc/<pid>/mountinfo for details]. 'ino' represents the inode number of
the file.
the file, and 'size' represents the size of the file in bytes.

A typical output is::

pos: 0
flags: 0100002
mnt_id: 19
ino: 63107
size: 0

All locks associated with a file descriptor are shown in its fdinfo too::

lock: 1: FLOCK ADVISORY WRITE 359 00:13:11691 0 EOF

Files with anonymous inodes have an additional 'path' field which represents
the anonymous file path.

The files such as eventfd, fsnotify, signalfd, epoll among the regular pos/flags
pair provide additional information particular to the objects they represent.

Expand All @@ -1922,6 +1927,8 @@ Eventfd files
flags: 04002
mnt_id: 9
ino: 63107
size: 0
path: anon_inode:[eventfd]
eventfd-count: 5a

where 'eventfd-count' is hex value of a counter.
Expand All @@ -1935,6 +1942,8 @@ Signalfd files
flags: 04002
mnt_id: 9
ino: 63107
size: 0
path: anon_inode:[signalfd]
sigmask: 0000000000000200

where 'sigmask' is hex value of the signal mask associated
Expand All @@ -1949,6 +1958,8 @@ Epoll files
flags: 02
mnt_id: 9
ino: 63107
size: 0
path: anon_inode:[eventpoll]
tfd: 5 events: 1d data: ffffffffffffffff pos:0 ino:61af sdev:7

where 'tfd' is a target file descriptor number in decimal form,
Expand All @@ -1967,6 +1978,8 @@ For inotify files the format is the following::
flags: 02000000
mnt_id: 9
ino: 63107
size: 0
path: anon_inode:inotify
inotify wd:3 ino:9e7e sdev:800013 mask:800afce ignored_mask:0 fhandle-bytes:8 fhandle-type:1 f_handle:7e9e0000640d1b6d

where 'wd' is a watch descriptor in decimal form, i.e. a target file
Expand All @@ -1990,6 +2003,8 @@ For fanotify files the format is::
flags: 02
mnt_id: 9
ino: 63107
size: 0
path: anon_inode:[fanotify]
fanotify flags:10 event-flags:0
fanotify mnt_id:12 mflags:40 mask:38 ignored_mask:40000003
fanotify ino:4f969 sdev:800013 mflags:0 mask:3b ignored_mask:40000000 fhandle-bytes:8 fhandle-type:1 f_handle:69f90400c275b5b4
Expand All @@ -2015,6 +2030,8 @@ Timerfd files
flags: 02
mnt_id: 9
ino: 63107
size: 0
path: anon_inode:[timerfd]
clockid: 0
ticks: 0
settime flags: 01
Expand All @@ -2039,6 +2056,7 @@ DMA Buffer files
mnt_id: 9
ino: 63107
size: 32768
path: /dmabuf:
count: 2
exp_name: system-heap

Expand Down
2 changes: 1 addition & 1 deletion Documentation/mm/balance.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Memory Balancing

Started Jan 2000 by Kanoj Sarcar <kanoj@sgi.com>

Memory balancing is needed for !__GFP_ATOMIC and !__GFP_KSWAPD_RECLAIM as
Memory balancing is needed for !__GFP_HIGH and !__GFP_KSWAPD_RECLAIM as
well as for non __GFP_IO allocations.

The first reason why a caller may avoid reclaim is that the caller can not
Expand Down
72 changes: 49 additions & 23 deletions Documentation/mm/vmemmap_dedup.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,23 +7,25 @@ A vmemmap diet for HugeTLB and Device DAX
HugeTLB
=======

The struct page structures (page structs) are used to describe a physical
page frame. By default, there is a one-to-one mapping from a page frame to
it's corresponding page struct.
This section is to explain how HugeTLB Vmemmap Optimization (HVO) works.

The ``struct page`` structures are used to describe a physical page frame. By
default, there is a one-to-one mapping from a page frame to it's corresponding
``struct page``.

HugeTLB pages consist of multiple base page size pages and is supported by many
architectures. See Documentation/admin-guide/mm/hugetlbpage.rst for more
details. On the x86-64 architecture, HugeTLB pages of size 2MB and 1GB are
currently supported. Since the base page size on x86 is 4KB, a 2MB HugeTLB page
consists of 512 base pages and a 1GB HugeTLB page consists of 4096 base pages.
For each base page, there is a corresponding page struct.
For each base page, there is a corresponding ``struct page``.

Within the HugeTLB subsystem, only the first 4 page structs are used to
contain unique information about a HugeTLB page. __NR_USED_SUBPAGE provides
this upper limit. The only 'useful' information in the remaining page structs
Within the HugeTLB subsystem, only the first 4 ``struct page`` are used to
contain unique information about a HugeTLB page. ``__NR_USED_SUBPAGE`` provides
this upper limit. The only 'useful' information in the remaining ``struct page``
is the compound_head field, and this field is the same for all tail pages.

By removing redundant page structs for HugeTLB pages, memory can be returned
By removing redundant ``struct page`` for HugeTLB pages, memory can be returned
to the buddy allocator for other uses.

Different architectures support different HugeTLB pages. For example, the
Expand All @@ -44,7 +46,7 @@ page.
| | 64KB | 2MB | 512MB | 16GB | |
+--------------+-----------+-----------+-----------+-----------+-----------+

When the system boot up, every HugeTLB page has more than one struct page
When the system boot up, every HugeTLB page has more than one ``struct page``
structs which size is (unit: pages)::

struct_size = HugeTLB_Size / PAGE_SIZE * sizeof(struct page) / PAGE_SIZE
Expand Down Expand Up @@ -74,10 +76,10 @@ Where n is how many pte entries which one page can contains. So the value of
n is (PAGE_SIZE / sizeof(pte_t)).

This optimization only supports 64-bit system, so the value of sizeof(pte_t)
is 8. And this optimization also applicable only when the size of struct page
is a power of two. In most cases, the size of struct page is 64 bytes (e.g.
is 8. And this optimization also applicable only when the size of ``struct page``
is a power of two. In most cases, the size of ``struct page`` is 64 bytes (e.g.
x86-64 and arm64). So if we use pmd level mapping for a HugeTLB page, the
size of struct page structs of it is 8 page frames which size depends on the
size of ``struct page`` structs of it is 8 page frames which size depends on the
size of the base page.

For the HugeTLB page of the pud level mapping, then::
Expand All @@ -86,15 +88,15 @@ For the HugeTLB page of the pud level mapping, then::
= PAGE_SIZE / 8 * 8 (pages)
= PAGE_SIZE (pages)

Where the struct_size(pmd) is the size of the struct page structs of a
Where the struct_size(pmd) is the size of the ``struct page`` structs of a
HugeTLB page of the pmd level mapping.

E.g.: A 2MB HugeTLB page on x86_64 consists in 8 page frames while 1GB
HugeTLB page consists in 4096.

Next, we take the pmd level mapping of the HugeTLB page as an example to
show the internal implementation of this optimization. There are 8 pages
struct page structs associated with a HugeTLB page which is pmd mapped.
``struct page`` structs associated with a HugeTLB page which is pmd mapped.

Here is how things look before optimization::

Expand Down Expand Up @@ -122,10 +124,10 @@ Here is how things look before optimization::
+-----------+

The value of page->compound_head is the same for all tail pages. The first
page of page structs (page 0) associated with the HugeTLB page contains the 4
page structs necessary to describe the HugeTLB. The only use of the remaining
pages of page structs (page 1 to page 7) is to point to page->compound_head.
Therefore, we can remap pages 1 to 7 to page 0. Only 1 page of page structs
page of ``struct page`` (page 0) associated with the HugeTLB page contains the 4
``struct page`` necessary to describe the HugeTLB. The only use of the remaining
pages of ``struct page`` (page 1 to page 7) is to point to page->compound_head.
Therefore, we can remap pages 1 to 7 to page 0. Only 1 page of ``struct page``
will be used for each HugeTLB page. This will allow us to free the remaining
7 pages to the buddy allocator.

Expand Down Expand Up @@ -167,13 +169,37 @@ entries that can be cached in a single TLB entry.

The contiguous bit is used to increase the mapping size at the pmd and pte
(last) level. So this type of HugeTLB page can be optimized only when its
size of the struct page structs is greater than 1 page.
size of the ``struct page`` structs is greater than **1** page.

Notice: The head vmemmap page is not freed to the buddy allocator and all
tail vmemmap pages are mapped to the head vmemmap page frame. So we can see
more than one struct page struct with PG_head (e.g. 8 per 2 MB HugeTLB page)
associated with each HugeTLB page. The compound_head() can handle this
correctly (more details refer to the comment above compound_head()).
more than one ``struct page`` struct with ``PG_head`` (e.g. 8 per 2 MB HugeTLB
page) associated with each HugeTLB page. The ``compound_head()`` can handle
this correctly. There is only **one** head ``struct page``, the tail
``struct page`` with ``PG_head`` are fake head ``struct page``. We need an
approach to distinguish between those two different types of ``struct page`` so
that ``compound_head()`` can return the real head ``struct page`` when the
parameter is the tail ``struct page`` but with ``PG_head``. The following code
snippet describes how to distinguish between real and fake head ``struct page``.

.. code-block:: c
if (test_bit(PG_head, &page->flags)) {
unsigned long head = READ_ONCE(page[1].compound_head);
if (head & 1) {
if (head == (unsigned long)page + 1)
/* head struct page */
else
/* tail struct page */
} else {
/* head struct page */
}
}
We can safely access the field of the **page[1]** with ``PG_head`` because the
page is a compound page composed with at least two contiguous pages.
The implementation refers to ``page_fixed_fake_head()``.

Device DAX
==========
Expand All @@ -187,7 +213,7 @@ PMD_SIZE (2M on x86_64) and PUD_SIZE (1G on x86_64).

The differences with HugeTLB are relatively minor.

It only use 3 page structs for storing all information as opposed
It only use 3 ``struct page`` for storing all information as opposed
to 4 on HugeTLB pages.

There's no remapping of vmemmap given that device-dax memory is not part of
Expand Down
1 change: 1 addition & 0 deletions MAINTAINERS
Original file line number Diff line number Diff line change
Expand Up @@ -11074,6 +11074,7 @@ F: fs/smbfs_common/

KERNEL UNIT TESTING FRAMEWORK (KUnit)
M: Brendan Higgins <brendanhiggins@google.com>
M: David Gow <davidgow@google.com>
L: linux-kselftest@vger.kernel.org
L: kunit-dev@googlegroups.com
S: Maintained
Expand Down
2 changes: 2 additions & 0 deletions arch/alpha/include/uapi/asm/mman.h
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,8 @@

#define MADV_DONTNEED_LOCKED 24 /* like DONTNEED, but drop locked pages too */

#define MADV_COLLAPSE 25 /* Synchronous hugepage collapse */

/* compatibility flags */
#define MAP_FILE 0

Expand Down
6 changes: 0 additions & 6 deletions arch/alpha/kernel/smp.c
Original file line number Diff line number Diff line change
Expand Up @@ -497,12 +497,6 @@ smp_cpus_done(unsigned int max_cpus)
((bogosum + 2500) / (5000/HZ)) % 100);
}

int
setup_profiling_timer(unsigned int multiplier)
{
return -EINVAL;
}

static void
send_ipi_message(const struct cpumask *to_whom, enum ipi_message_type operation)
{
Expand Down
8 changes: 0 additions & 8 deletions arch/arc/kernel/smp.c
Original file line number Diff line number Diff line change
Expand Up @@ -232,14 +232,6 @@ int __cpu_up(unsigned int cpu, struct task_struct *idle)
return 0;
}

/*
* not supported here
*/
int setup_profiling_timer(unsigned int multiplier)
{
return -EINVAL;
}

/*****************************************************************************/
/* Inter Processor Interrupt Handling */
/*****************************************************************************/
Expand Down
8 changes: 0 additions & 8 deletions arch/arm/kernel/smp.c
Original file line number Diff line number Diff line change
Expand Up @@ -787,14 +787,6 @@ void panic_smp_self_stop(void)
cpu_relax();
}

/*
* not supported here
*/
int setup_profiling_timer(unsigned int multiplier)
{
return -EINVAL;
}

#ifdef CONFIG_CPU_FREQ

static DEFINE_PER_CPU(unsigned long, l_p_j_ref);
Expand Down
Loading

0 comments on commit 40ef37d

Please sign in to comment.