Skip to content

Commit

Permalink
powerpc: Prevent gcc to re-read the pagetables
Browse files Browse the repository at this point in the history
GCC is very likely to read the pagetables just once and cache them in
the local stack or in a register, but it is can also decide to re-read
the pagetables. The problem is that the pagetable in those places can
change from under gcc.

With THP/hugetlbfs the pmd (and pgd for hugetlbfs giga pages) can
change under gup_fast. The pages won't be freed untill we finish
gup fast because we have irq disabled and we free these pages via
rcu callback.

Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
  • Loading branch information
Aneesh Kumar K.V authored and Benjamin Herrenschmidt committed Jun 21, 2013
1 parent 0ac52dd commit 7888b4d
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 5 deletions.
8 changes: 4 additions & 4 deletions arch/powerpc/mm/gup.c
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ static noinline int gup_pte_range(pmd_t pmd, unsigned long addr,

ptep = pte_offset_kernel(&pmd, addr);
do {
pte_t pte = *ptep;
pte_t pte = ACCESS_ONCE(*ptep);
struct page *page;

if ((pte_val(pte) & mask) != result)
Expand Down Expand Up @@ -63,7 +63,7 @@ static int gup_pmd_range(pud_t pud, unsigned long addr, unsigned long end,

pmdp = pmd_offset(&pud, addr);
do {
pmd_t pmd = *pmdp;
pmd_t pmd = ACCESS_ONCE(*pmdp);

next = pmd_addr_end(addr, end);
/*
Expand Down Expand Up @@ -97,7 +97,7 @@ static int gup_pud_range(pgd_t pgd, unsigned long addr, unsigned long end,

pudp = pud_offset(&pgd, addr);
do {
pud_t pud = *pudp;
pud_t pud = ACCESS_ONCE(*pudp);

next = pud_addr_end(addr, end);
if (pud_none(pud))
Expand Down Expand Up @@ -160,7 +160,7 @@ int get_user_pages_fast(unsigned long start, int nr_pages, int write,

pgdp = pgd_offset(mm, addr);
do {
pgd_t pgd = *pgdp;
pgd_t pgd = ACCESS_ONCE(*pgdp);

pr_devel(" %016lx: normal pgd %p\n", addr,
(void *)pgd_val(pgd));
Expand Down
2 changes: 1 addition & 1 deletion arch/powerpc/mm/hugetlbpage.c
Original file line number Diff line number Diff line change
Expand Up @@ -1024,7 +1024,7 @@ int gup_hugepte(pte_t *ptep, unsigned long sz, unsigned long addr,
if (pte_end < end)
end = pte_end;

pte = *ptep;
pte = ACCESS_ONCE(*ptep);
mask = _PAGE_PRESENT | _PAGE_USER;
if (write)
mask |= _PAGE_RW;
Expand Down

0 comments on commit 7888b4d

Please sign in to comment.