Ryan Roberts c4602f9fa7 mm/readahead: store folio order in struct file_ra_state ... Previously the folio order of the previous readahead request was inferred from the folio who's readahead marker was hit. But due to the way we have to round to non-natural boundaries sometimes, this first folio in the readahead block is often smaller than the preferred order for that request. This means that for cases where the initial sync readahead is poorly aligned, the folio order will ramp up much more slowly. So instead, let's store the order in struct file_ra_state so we are not affected by any required alignment. We previously made enough room in the struct for a 16 order field. This should be plenty big enough since we are limited to MAX_PAGECACHE_ORDER anyway, which is certainly never larger than ~20. Since we now pass order in struct file_ra_state, page_cache_ra_order() no longer needs it's new_order parameter, so let's remove that. Worked example: Here we are touching pages 17-256 sequentially just as we did in the previous commit, but now that we are remembering the preferred order explicitly, we no longer have the slow ramp up problem. Note specifically that we no longer have 2 rounds (2x ~128K) of order-2 folios: TYPE STARTOFFS ENDOFFS SIZE STARTPG ENDPG NRPG ORDER RA ----- ---------- ---------- ---------- ------- ------- ----- ----- -- HOLE 0x00000000 0x00001000 4096 0 1 1 FOLIO 0x00001000 0x00002000 4096 1 2 1 0 FOLIO 0x00002000 0x00003000 4096 2 3 1 0 FOLIO 0x00003000 0x00004000 4096 3 4 1 0 FOLIO 0x00004000 0x00005000 4096 4 5 1 0 FOLIO 0x00005000 0x00006000 4096 5 6 1 0 FOLIO 0x00006000 0x00007000 4096 6 7 1 0 FOLIO 0x00007000 0x00008000 4096 7 8 1 0 FOLIO 0x00008000 0x00009000 4096 8 9 1 0 FOLIO 0x00009000 0x0000a000 4096 9 10 1 0 FOLIO 0x0000a000 0x0000b000 4096 10 11 1 0 FOLIO 0x0000b000 0x0000c000 4096 11 12 1 0 FOLIO 0x0000c000 0x0000d000 4096 12 13 1 0 FOLIO 0x0000d000 0x0000e000 4096 13 14 1 0 FOLIO 0x0000e000 0x0000f000 4096 14 15 1 0 FOLIO 0x0000f000 0x00010000 4096 15 16 1 0 FOLIO 0x00010000 0x00011000 4096 16 17 1 0 FOLIO 0x00011000 0x00012000 4096 17 18 1 0 FOLIO 0x00012000 0x00013000 4096 18 19 1 0 FOLIO 0x00013000 0x00014000 4096 19 20 1 0 FOLIO 0x00014000 0x00015000 4096 20 21 1 0 FOLIO 0x00015000 0x00016000 4096 21 22 1 0 FOLIO 0x00016000 0x00017000 4096 22 23 1 0 FOLIO 0x00017000 0x00018000 4096 23 24 1 0 FOLIO 0x00018000 0x00019000 4096 24 25 1 0 FOLIO 0x00019000 0x0001a000 4096 25 26 1 0 FOLIO 0x0001a000 0x0001b000 4096 26 27 1 0 FOLIO 0x0001b000 0x0001c000 4096 27 28 1 0 FOLIO 0x0001c000 0x0001d000 4096 28 29 1 0 FOLIO 0x0001d000 0x0001e000 4096 29 30 1 0 FOLIO 0x0001e000 0x0001f000 4096 30 31 1 0 FOLIO 0x0001f000 0x00020000 4096 31 32 1 0 FOLIO 0x00020000 0x00021000 4096 32 33 1 0 FOLIO 0x00021000 0x00022000 4096 33 34 1 0 FOLIO 0x00022000 0x00024000 8192 34 36 2 1 FOLIO 0x00024000 0x00028000 16384 36 40 4 2 FOLIO 0x00028000 0x0002c000 16384 40 44 4 2 FOLIO 0x0002c000 0x00030000 16384 44 48 4 2 FOLIO 0x00030000 0x00034000 16384 48 52 4 2 FOLIO 0x00034000 0x00038000 16384 52 56 4 2 FOLIO 0x00038000 0x0003c000 16384 56 60 4 2 FOLIO 0x0003c000 0x00040000 16384 60 64 4 2 FOLIO 0x00040000 0x00050000 65536 64 80 16 4 FOLIO 0x00050000 0x00060000 65536 80 96 16 4 FOLIO 0x00060000 0x00080000 131072 96 128 32 5 FOLIO 0x00080000 0x000a0000 131072 128 160 32 5 FOLIO 0x000a0000 0x000c0000 131072 160 192 32 5 FOLIO 0x000c0000 0x000e0000 131072 192 224 32 5 FOLIO 0x000e0000 0x00100000 131072 224 256 32 5 FOLIO 0x00100000 0x00120000 131072 256 288 32 5 FOLIO 0x00120000 0x00140000 131072 288 320 32 5 Y HOLE 0x00140000 0x00800000 7077888 320 2048 1728 Link: https://lkml.kernel.org/r/20250609092729.274960-5-ryan.roberts@arm.com Signed-off-by: Ryan Roberts <ryan.roberts@arm.com> Reviewed-by: Jan Kara <jack@suse.cz> Cc: Chaitanya S Prakash <chaitanyas.prakash@arm.com> Cc: David Hildenbrand <david@redhat.com> Cc: Will Deacon <will@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>		2025-07-09 22:42:03 -07:00
..
damon	mm/damon/stat: calculate and expose idle time percentiles	2025-07-09 22:41:55 -07:00
kasan	hardening updates for v6.16-rc1	2025-05-28 07:47:10 -07:00
kfence	kfence: skip __GFP_THISNODE allocations on NUMA systems	2025-02-01 03:53:26 -08:00
kmsan	kmsan: test: add module description	2025-06-05 22:02:25 -07:00
backing-dev.c	treewide: Switch/rename to timer_delete[_sync]()	2025-04-05 10:30:12 +02:00
balloon_compaction.c	balloon_compaction: update the NR_BALLOON_PAGES state	2025-03-21 22:03:13 -07:00
bootmem_info.c	mm/sparse: allow for alternate vmemmap section init at boot	2025-03-16 22:06:27 -07:00
cma_debug.c	mm, cma: support multiple contiguous ranges, if requested	2025-03-16 22:06:25 -07:00
cma_sysfs.c	mm/cma: export total and free number of pages for CMA areas	2025-03-16 22:06:24 -07:00
cma.c	mm/cma: pair the trace_cma_alloc_start/finish	2025-07-09 22:41:58 -07:00
cma.h	mm: cma: set early_pfn and bitmap as a union in cma_memrange	2025-05-22 14:55:36 -07:00
compaction.c	mm: page_alloc: tighten up find_suitable_fallback()	2025-05-11 17:48:18 -07:00
debug_page_alloc.c	mm/debug_page_alloc: improve error message for invalid guardpage minorder	2025-05-12 23:50:38 -07:00
debug_page_ref.c
debug_vm_pgtable.c	mm: remove mk_huge_pte()	2025-05-11 17:48:04 -07:00
debug.c	mm/debug: fix parameter passed to page_mapcount_is_type()	2025-05-11 17:48:17 -07:00
dmapool_test.c
dmapool.c	dmapool: add NUMA affinity support	2025-05-20 05:34:27 +02:00
early_ioremap.c	mm/early_ioremap: add null pointer checks to prevent NULL-pointer dereference	2025-01-13 22:40:59 -08:00
execmem.c	Revert "mm/execmem: Unify early execmem_cache behaviour"	2025-06-11 11:20:52 +02:00
fadvise.c	fdget(), trivial conversions	2024-11-03 01:28:06 -05:00
fail_page_alloc.c	fault-inject: improve build for CONFIG_FAULT_INJECTION=n	2024-09-01 20:43:33 -07:00
failslab.c	fault-inject: improve build for CONFIG_FAULT_INJECTION=n	2024-09-01 20:43:33 -07:00
filemap.c	mm/readahead: store folio order in struct file_ra_state	2025-07-09 22:42:03 -07:00
folio-compat.c	mm: Remove grab_cache_page_write_begin()	2025-03-04 17:02:25 +00:00
gup_test.c
gup_test.h
gup.c	mm/gup: remove (VM_)BUG_ONs	2025-07-09 22:41:56 -07:00
highmem.c
hmm.c	RDMA/core: Avoid hmm_dma_map_alloc() for virtual DMA devices	2025-05-25 06:24:21 -04:00
huge_memory.c	mm: convert track_pfn_insert() to pfnmap_setup_cachemode*()	2025-05-22 14:55:36 -07:00
hugetlb_cgroup.c	page_counter: track failcnt only for legacy cgroups	2025-03-17 00:05:35 -07:00
hugetlb_cma.c	mm/hugetlb: use separate nodemask for bootmem allocations	2025-05-12 23:50:35 -07:00
hugetlb_cma.h	mm/hugetlb: move hugetlb CMA code in to its own file	2025-03-16 22:06:31 -07:00
hugetlb_vmemmap.c	mm, hugetlb: increment the number of pages to be reset on HVO	2025-04-17 20:10:08 -07:00
hugetlb_vmemmap.h	mm/hugetlb: do pre-HVO for bootmem allocated pages	2025-03-16 22:06:29 -07:00
hugetlb.c	mm/hugetlb: convert hugetlb_change_protection() to folios	2025-07-09 22:41:54 -07:00
hwpoison-inject.c
init-mm.c	mm: replace vm_lock and detached flag with a reference count	2025-03-16 22:06:20 -07:00
internal.h	mm/readahead: store folio order in struct file_ra_state	2025-07-09 22:42:03 -07:00
interval_tree.c
io-mapping.c	mm/io-mapping: track_pfn() -> "pfnmap tracking"	2025-05-22 14:55:37 -07:00
ioremap.c	mm/ioremap: pass pgprot_t to ioremap_prot() instead of unsigned long	2025-03-16 22:06:23 -07:00
Kconfig	mm: rename CONFIG_PAGE_BLOCK_ORDER to CONFIG_PAGE_BLOCK_MAX_ORDER	2025-07-09 22:41:56 -07:00
Kconfig.debug	mm: rename GENERIC_PTDUMP and PTDUMP_CORE	2025-03-17 00:05:32 -07:00
khugepaged.c	mm/khugepaged: clean up refcount check using folio_expected_ref_count()	2025-05-31 22:46:16 -07:00
kmemleak.c	mm/alloc_tag: fix the kmemleak false positive issue in the allocation of the percpu variable tag->counters	2025-06-25 15:55:03 -07:00
ksm.c	mm: prevent KSM from breaking VMA merging for new VMAs	2025-07-09 22:41:54 -07:00
list_lru.c	mm, list_lru: refactor the locking code	2025-07-09 22:41:56 -07:00
maccess.c	maccess: fix strncpy_from_user_nofault() empty string handling	2025-05-11 17:54:10 -07:00
madvise.c	mm: use per_vma lock for MADV_DONTNEED	2025-07-09 22:42:01 -07:00
Makefile	mm: perform VMA allocation, freeing, duplication in mm	2025-05-12 23:50:48 -07:00
mapping_dirty_helpers.c
memblock.c	memblock: add KHO support for reserve_mem	2025-05-12 23:50:42 -07:00
memcontrol-v1.c	memcg: make count_memcg_events re-entrant safe against irqs	2025-05-22 14:55:38 -07:00
memcontrol-v1.h	memcg: move do_memsw_account() to CONFIG_MEMCG_V1	2025-03-21 22:03:11 -07:00
memcontrol.c	- The 2 patch series "zram: support algorithm-specific parameters" from	2025-06-02 16:00:26 -07:00
memfd.c	mm: move folio_index to mm/swap.h and remove no longer needed helper	2025-05-12 23:50:50 -07:00
memory_hotplug.c	drivers/base/node: rename __register_one_node() to register_one_node()	2025-07-09 22:42:00 -07:00
memory-failure.c	mm: memory-failure: enhance comments for return value of memory_failure()	2025-03-17 22:07:05 -07:00
memory-tiers.c	memory tiers: use default_dram_perf_ref_source in log message	2024-09-26 14:01:44 -07:00
memory.c	mm/shmem, swap: fix softlockup with mTHP swapin	2025-06-19 20:48:01 -07:00
mempolicy.c	mm/mempolicy: skip unnecessary synchronize_rcu()	2025-07-09 22:42:02 -07:00
mempool.c
memremap.c	mm: introduce pfnmap_track() and pfnmap_untrack() and use them for memremap	2025-05-22 14:55:37 -07:00
memtest.c
migrate_device.c	- The 6 patch series "Enable strict percpu address space checks" from	2025-04-01 09:29:18 -07:00
migrate.c	- The 11 patch series "Add folio_mk_pte()" from Matthew Wilcox	2025-05-31 15:44:16 -07:00
mincore.c	mm: mincore: use pte_batch_hint() to batch process large folios	2025-05-22 14:55:36 -07:00
mlock.c	mm: allow compound zone device pages	2025-03-17 22:06:39 -07:00
mm_init.c	mm: rename CONFIG_PAGE_BLOCK_ORDER to CONFIG_PAGE_BLOCK_MAX_ORDER	2025-07-09 22:41:56 -07:00
mm_slot.h
mmap_lock.c	mm: move mmap/vma locking logic into specific files	2025-05-11 17:48:33 -07:00
mmap.c	mm: convert VM_PFNMAP tracking to pfnmap_track() + pfnmap_untrack()	2025-05-22 14:55:37 -07:00
mmu_gather.c	mmu_gather: move tlb flush for VM_PFNMAP/VM_MIXEDMAP vmas into free_pgtables()	2025-05-31 22:46:12 -07:00
mmu_notifier.c	Update Christoph's Email address and make it consistent	2025-05-12 23:50:31 -07:00
mmzone.c	mm: improve code consistency with zonelist_* helper functions	2024-09-01 20:25:55 -07:00
mprotect.c	mm/huge_memory: remove useless folio pointers passing	2025-05-12 23:50:34 -07:00
mremap.c	mm: expose abnormal new_pte during move_ptes	2025-06-05 21:55:41 -07:00
mseal.c	mseal: remove can_do_mseal()	2025-01-13 22:40:51 -08:00
msync.c
nommu.c	- The 11 patch series "Add folio_mk_pte()" from Matthew Wilcox	2025-05-31 15:44:16 -07:00
numa_emulation.c	mm/fake-numa: allow later numa node hotplug	2025-01-25 20:22:29 -08:00
numa_memblks.c	mm: numa_memblks: introduce numa_add_reserved_memblk	2025-05-22 14:55:36 -07:00
numa.c	mm/numa: remove unnecessary local variable in alloc_node_data()	2025-05-12 23:50:38 -07:00
oom_kill.c	mm/oom_kill: fix trivial typo in comment	2025-03-16 22:05:55 -07:00
page_alloc.c	mm: restore documentation for __free_pages()	2025-07-09 22:41:52 -07:00
page_counter.c	page_counter: track failcnt only for legacy cgroups	2025-03-17 00:05:35 -07:00
page_ext.c	mm: page_ext: add an iteration API for page extensions	2025-03-17 22:06:57 -07:00
page_frag_cache.c	mm/page_alloc: export free_frozen_pages() instead of free_unref_page()	2025-01-13 22:40:31 -08:00
page_idle.c	mm/page_idle: handle device-exclusive entries correctly in page_idle_clear_pte_refs_one()	2025-03-16 22:05:59 -07:00
page_io.c	mm: stop passing a writeback_control structure to swap_writeout	2025-07-09 22:41:58 -07:00
page_isolation.c	mm: page_isolation: avoid calling folio_hstate() without hugetlb_lock	2025-04-01 15:14:43 -07:00
page_owner.c	mm: rename try_alloc_pages() to alloc_pages_nolock()	2025-05-22 14:55:37 -07:00
page_poison.c
page_reporting.c
page_reporting.h
page_table_check.c	mm/page_table_check: Batch-check pmds/puds just like ptes	2025-05-09 13:43:07 +01:00
page_vma_mapped.c	mm: make page_mapped_in_vma() hugetlb walk aware	2025-03-16 22:06:42 -07:00
page-writeback.c	treewide, timers: Rename from_timer() to timer_container_of()	2025-06-08 09:07:37 +02:00
pagewalk.c	mm: pagewalk: add the ability to install PTEs	2024-11-11 00:26:44 -08:00
percpu-internal.h	mm: remove CONFIG_MEMCG_KMEM	2024-07-10 12:14:54 -07:00
percpu-km.c
percpu-stats.c
percpu-vm.c
percpu.c	- The 6 patch series "Enable strict percpu address space checks" from	2025-04-01 09:29:18 -07:00
pgalloc-track.h
pgtable-generic.c	mm: add RCU annotation to pte_offset_map(_lock)	2024-12-18 19:04:43 -08:00
process_vm_access.c	mm: refactor mm_access() to not return NULL	2024-11-05 16:56:23 -08:00
pt_reclaim.c	mm: pgtable: reclaim empty PTE page in madvise(MADV_DONTNEED)	2025-01-13 22:40:48 -08:00
ptdump.c	mm/ptdump: split effective_prot() into level specific callbacks	2025-05-11 17:48:19 -07:00
readahead.c	mm/readahead: store folio order in struct file_ra_state	2025-07-09 22:42:03 -07:00
rmap.c	mm/rmap: fix typo in comment in page_address_in_vma	2025-05-12 23:50:37 -07:00
rodata_test.c	mm/rodata_test: verify test data is unchanged, rather than non-zero	2025-01-13 22:40:38 -08:00
secretmem.c	fs: export anon_inode_make_secure_inode() and fix secretmem LSM bypass	2025-06-23 12:41:17 +02:00
shmem_quota.c	shmem_quota: build the object file conditionally to the config option	2024-09-01 20:25:45 -07:00
shmem.c	mm: stop passing a writeback_control structure to swap_writeout	2025-07-09 22:41:58 -07:00
show_mem.c	- The 11 patch series "Add folio_mk_pte()" from Matthew Wilcox	2025-05-31 15:44:16 -07:00
shrinker_debug.c	mm/shrinker: fix name consistency issue in shrinker_debugfs_rename()	2025-03-17 00:05:40 -07:00
shrinker.c	mm: shrinker: avoid memleak in alloc_shrinker_info	2024-10-31 20:27:04 -07:00
shuffle.c
shuffle.h
slab_common.c	Update Christoph's Email address and make it consistent	2025-05-12 23:50:31 -07:00
slab.h	Merge branch 'slab/for-6.15/kfree_rcu_tiny' into slab/for-next	2025-03-20 10:33:38 +01:00
slub.c	slab updates for 6.16	2025-06-04 08:59:59 -07:00
sparse-vmemmap.c	mm/hugetlb: do pre-HVO for bootmem allocated pages	2025-03-16 22:06:29 -07:00
sparse.c	drivers/base/memory: improve add_boot_memory_block()	2025-03-17 22:07:01 -07:00
swap_cgroup.c	mm: swap_cgroup: remove double initialization of locals	2025-03-17 22:06:58 -07:00
swap_state.c	- The 11 patch series "Add folio_mk_pte()" from Matthew Wilcox	2025-05-31 15:44:16 -07:00
swap.c	memcg: make count_memcg_events re-entrant safe against irqs	2025-05-22 14:55:38 -07:00
swap.h	mm: stop passing a writeback_control structure to swap_writeout	2025-07-09 22:41:58 -07:00
swapfile.c	- The 11 patch series "Add folio_mk_pte()" from Matthew Wilcox	2025-05-31 15:44:16 -07:00
truncate.c	- The 2 patch series "zram: support algorithm-specific parameters" from	2025-06-02 16:00:26 -07:00
usercopy.c	mm: security: Check early if HARDENED_USERCOPY is enabled	2025-02-28 11:51:31 -08:00
userfaultfd.c	userfaultfd: remove (VM_)BUG_ON()s	2025-07-09 22:42:01 -07:00
util.c	mm: add mmap_prepare() compatibility layer for nested file systems	2025-06-12 21:39:02 -07:00
vma_exec.c	mm: abstract initial stack setup to mm subsystem	2025-05-12 23:50:48 -07:00
vma_init.c	mm: convert VM_PFNMAP tracking to pfnmap_track() + pfnmap_untrack()	2025-05-22 14:55:37 -07:00
vma_internal.h	mm/vma: move brk() internals to mm/vma.c	2025-01-13 22:40:42 -08:00
vma.c	mm: prevent KSM from breaking VMA merging for new VMAs	2025-07-09 22:41:54 -07:00
vma.h	mm: add mmap_prepare() compatibility layer for nested file systems	2025-06-12 21:39:02 -07:00
vmalloc.c	- The 11 patch series "Add folio_mk_pte()" from Matthew Wilcox	2025-05-31 15:44:16 -07:00
vmpressure.c
vmscan.c	mm: stop passing a writeback_control structure to swap_writeout	2025-07-09 22:41:58 -07:00
vmstat.c	mm: fix vmstat after removing NR_BOUNCE	2025-06-05 22:02:22 -07:00
workingset.c	mm: workingset: simplify lockdep check in update_node	2025-05-12 23:50:44 -07:00
zpdesc.h	mm: rename page->index to page->__folio_index	2025-05-31 22:46:06 -07:00
zpool.c	zsmalloc: prefer the the original page's node for compressed data	2025-05-11 17:48:06 -07:00
zsmalloc.c	zsmalloc: cleanup headers includes	2025-05-11 17:48:16 -07:00
zswap.c	mm: stop passing a writeback_control structure to __swap_writepage	2025-07-09 22:41:57 -07:00