mirror of
git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
synced 2025-09-04 20:19:47 +08:00
db6a227416
There is a race condition that can trigger in dmflakey fstests that can result in asserts in xfs_ialloc_read_agi() and xfs_alloc_read_agf() firing. The asserts look like this: XFS: Assertion failed: pag->pagf_freeblks == be32_to_cpu(agf->agf_freeblks), file: fs/xfs/libxfs/xfs_alloc.c, line: 3440 ..... Call Trace: <TASK> xfs_alloc_read_agf+0x2ad/0x3a0 xfs_alloc_fix_freelist+0x280/0x720 xfs_alloc_vextent_prepare_ag+0x42/0x120 xfs_alloc_vextent_iterate_ags+0x67/0x260 xfs_alloc_vextent_start_ag+0xe4/0x1c0 xfs_bmapi_allocate+0x6fe/0xc90 xfs_bmapi_convert_delalloc+0x338/0x560 xfs_map_blocks+0x354/0x580 iomap_writepages+0x52b/0xa70 xfs_vm_writepages+0xd7/0x100 do_writepages+0xe1/0x2c0 __writeback_single_inode+0x44/0x340 writeback_sb_inodes+0x2d0/0x570 __writeback_inodes_wb+0x9c/0xf0 wb_writeback+0x139/0x2d0 wb_workfn+0x23e/0x4c0 process_scheduled_works+0x1d4/0x400 worker_thread+0x234/0x2e0 kthread+0x147/0x170 ret_from_fork+0x3e/0x50 ret_from_fork_asm+0x1a/0x30 I've seen the AGI variant from scrub running on the filesysetm after unmount failed due to systemd interference: XFS: Assertion failed: pag->pagi_freecount == be32_to_cpu(agi->agi_freecount) || xfs_is_shutdown(pag->pag_mount), file: fs/xfs/libxfs/xfs_ialloc.c, line: 2804 ..... Call Trace: <TASK> xfs_ialloc_read_agi+0xee/0x150 xchk_perag_drain_and_lock+0x7d/0x240 xchk_ag_init+0x34/0x90 xchk_inode_xref+0x7b/0x220 xchk_inode+0x14d/0x180 xfs_scrub_metadata+0x2e2/0x510 xfs_ioc_scrub_metadata+0x62/0xb0 xfs_file_ioctl+0x446/0xbf0 __se_sys_ioctl+0x6f/0xc0 __x64_sys_ioctl+0x1d/0x30 x64_sys_call+0x1879/0x2ee0 do_syscall_64+0x68/0x130 ? exc_page_fault+0x62/0xc0 entry_SYSCALL_64_after_hwframe+0x76/0x7e Essentially, it is the same problem. When _flakey_drop_and_remount() loads the drop-writes table, it makes all writes silently fail. Writes are reported to the fs as completed successfully, but they are not issued to the backing store. The filesystem sees the successful write completion and marks the metadata buffer clean and removes it from the AIL. If this happens at the same time as memory pressure is occuring, the now-clean AGF and/or AGI buffers can be reclaimed from memory. Shortly afterwards, but before _flakey_drop_and_remount() runs unmount, background writeback is kicked and it tries to allocate blocks for the dirty pages in memory. This then tries to access the AGF buffer we just turfed out of memory. It's not found, so it gets read in from disk. This is all fine, except for the fact that the last writeback of the AGF did not actually reach disk. The AGF on disk is stale compared to the in-memory state held by the perag, and so they don't match and the assert fires. Then other operations on that inode hang because the task was killed whilst holding inode locks. e.g: Workqueue: xfs-conv/dm-12 xfs_end_io Call Trace: <TASK> __schedule+0x650/0xb10 schedule+0x6d/0xf0 schedule_preempt_disabled+0x15/0x30 rwsem_down_write_slowpath+0x31a/0x5f0 down_write+0x43/0x60 xfs_ilock+0x1a8/0x210 xfs_trans_alloc_inode+0x9c/0x240 xfs_iomap_write_unwritten+0xe3/0x300 xfs_end_ioend+0x90/0x130 xfs_end_io+0xce/0x100 process_scheduled_works+0x1d4/0x400 worker_thread+0x234/0x2e0 kthread+0x147/0x170 ret_from_fork+0x3e/0x50 ret_from_fork_asm+0x1a/0x30 </TASK> and it's all down hill from there. Memory pressure is one way to trigger this, another is to run "echo 3 > /proc/sys/vm/drop_caches" randomly while tests are running. Regardless of how it is triggered, this effectively takes down the system once umount hangs because it's holding a sb->s_umount lock exclusive and now every sync(1) call gets stuck on it. Fix this by replacing the asserts with a corruption detection check and a shutdown. Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Carlos Maiolino <cmaiolino@redhat.com> Signed-off-by: Carlos Maiolino <cem@kernel.org> |
||
|---|---|---|
| .. | ||
| xfs_ag_resv.c | ||
| xfs_ag_resv.h | ||
| xfs_ag.c | ||
| xfs_ag.h | ||
| xfs_alloc_btree.c | ||
| xfs_alloc_btree.h | ||
| xfs_alloc.c | ||
| xfs_alloc.h | ||
| xfs_attr_leaf.c | ||
| xfs_attr_leaf.h | ||
| xfs_attr_remote.c | ||
| xfs_attr_remote.h | ||
| xfs_attr_sf.h | ||
| xfs_attr.c | ||
| xfs_attr.h | ||
| xfs_bit.c | ||
| xfs_bit.h | ||
| xfs_bmap_btree.c | ||
| xfs_bmap_btree.h | ||
| xfs_bmap.c | ||
| xfs_bmap.h | ||
| xfs_btree_mem.c | ||
| xfs_btree_mem.h | ||
| xfs_btree_staging.c | ||
| xfs_btree_staging.h | ||
| xfs_btree.c | ||
| xfs_btree.h | ||
| xfs_cksum.h | ||
| xfs_da_btree.c | ||
| xfs_da_btree.h | ||
| xfs_da_format.h | ||
| xfs_defer.c | ||
| xfs_defer.h | ||
| xfs_dir2_block.c | ||
| xfs_dir2_data.c | ||
| xfs_dir2_leaf.c | ||
| xfs_dir2_node.c | ||
| xfs_dir2_priv.h | ||
| xfs_dir2_sf.c | ||
| xfs_dir2.c | ||
| xfs_dir2.h | ||
| xfs_dquot_buf.c | ||
| xfs_errortag.h | ||
| xfs_exchmaps.c | ||
| xfs_exchmaps.h | ||
| xfs_format.h | ||
| xfs_fs.h | ||
| xfs_group.c | ||
| xfs_group.h | ||
| xfs_health.h | ||
| xfs_ialloc_btree.c | ||
| xfs_ialloc_btree.h | ||
| xfs_ialloc.c | ||
| xfs_ialloc.h | ||
| xfs_iext_tree.c | ||
| xfs_inode_buf.c | ||
| xfs_inode_buf.h | ||
| xfs_inode_fork.c | ||
| xfs_inode_fork.h | ||
| xfs_inode_util.c | ||
| xfs_inode_util.h | ||
| xfs_log_format.h | ||
| xfs_log_recover.h | ||
| xfs_log_rlimit.c | ||
| xfs_metadir.c | ||
| xfs_metadir.h | ||
| xfs_metafile.c | ||
| xfs_metafile.h | ||
| xfs_ondisk.h | ||
| xfs_parent.c | ||
| xfs_parent.h | ||
| xfs_quota_defs.h | ||
| xfs_refcount_btree.c | ||
| xfs_refcount_btree.h | ||
| xfs_refcount.c | ||
| xfs_refcount.h | ||
| xfs_rmap_btree.c | ||
| xfs_rmap_btree.h | ||
| xfs_rmap.c | ||
| xfs_rmap.h | ||
| xfs_rtbitmap.c | ||
| xfs_rtbitmap.h | ||
| xfs_rtgroup.c | ||
| xfs_rtgroup.h | ||
| xfs_rtrefcount_btree.c | ||
| xfs_rtrefcount_btree.h | ||
| xfs_rtrmap_btree.c | ||
| xfs_rtrmap_btree.h | ||
| xfs_sb.c | ||
| xfs_sb.h | ||
| xfs_shared.h | ||
| xfs_symlink_remote.c | ||
| xfs_symlink_remote.h | ||
| xfs_trans_inode.c | ||
| xfs_trans_resv.c | ||
| xfs_trans_resv.h | ||
| xfs_trans_space.c | ||
| xfs_trans_space.h | ||
| xfs_types.c | ||
| xfs_types.h | ||
| xfs_zones.c | ||
| xfs_zones.h | ||