Pull driver core fixes from Danilo Krummrich:
- Always inline I/O and IRQ methods using build_assert!() to avoid
false positive build errors
- Do not free the driver's device private data in I2C shutdown()
avoiding race conditions that can lead to UAF bugs
- Drop the driver's device private data after the driver has been
fully unbound from its device to avoid UAF bugs from &Device<Bound>
scopes, such as IRQ callbacks
* tag 'driver-core-6.19-rc7' of git://git.kernel.org/pub/scm/linux/kernel/git/driver-core/driver-core:
rust: driver: drop device private data post unbind
rust: driver: add DriverData type to the DriverLayout trait
rust: driver: add DEVICE_DRIVER_OFFSET to the DriverLayout trait
rust: driver: introduce a DriverLayout trait
rust: auxiliary: add Driver::unbind() callback
rust: i2c: do not drop device private data on shutdown()
rust: irq: always inline functions using build_assert with arguments
rust: io: always inline functions using build_assert with arguments
Currently, `dev_*` only works on the core `Device`, but not on any other
bus or class device objects. This causes a pattern of
`dev_info!(pdev.as_ref())` which is not ideal.
This adds support of using these devices directly with `dev_*` macros, by
adding `AsRef` call inside the macro. To make sure we can still use just
`kernel::device::Device`, as `AsRef` implementation is added for it; this
is typical for types that is designed to use with `AsRef` anyway, for
example, `str` implements `AsRef<str>` and `Path` implements `AsRef<Path>`.
Signed-off-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260123175854.176735-1-gary@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Drivers might need to access PCI config space for querying capability
structures and access the registers inside the structures.
For Rust drivers need to access PCI config space, the Rust PCI abstraction
needs to support it in a way that upholds Rust's safety principles.
Introduce a `ConfigSpace` wrapper in Rust PCI abstraction to provide safe
accessors for PCI config space. The new type implements the `Io` trait and
`IoCapable<T>` for u8, u16, and u32 to share offset validation and
bound-checking logic with other I/O backends.
The `ConfigSpace` type uses marker types (`Normal` and `Extended`) to
represent configuration space sizes at the type level.
Cc: Alexandre Courbot <acourbot@nvidia.com>
Cc: Danilo Krummrich <dakr@kernel.org>
Cc: Gary Guo <gary@garyguo.net>
Cc: Joel Fernandes <joelagnelf@nvidia.com>
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://lore.kernel.org/all/DFV4IJDQC2J6.1Q91JOAL6CJSG@kernel.org/ [1]
Link: https://patch.msgid.link/20260121202212.4438-5-zhiw@nvidia.com
[ Applied the diff from [1], considering subsequent comment; remove
#[expect(unused)] from define_{read,write}!(). - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Refactor the existing MMIO accessors to use common call macros
instead of inlining the bindings calls in each `define_{read,write}!`
expansion.
This factoring separates the common offset/bounds checks from the
low-level call pattern, making it easier to add additional I/O accessor
families.
No functional change intended.
Cc: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260121202212.4438-4-zhiw@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
The previous Io<SIZE> type combined both the generic I/O access helpers
and MMIO implementation details in a single struct. This coupling prevented
reusing the I/O helpers for other backends, such as PCI configuration
space.
Establish a clean separation between the I/O interface and concrete
backends by separating generic I/O helpers from MMIO implementation.
Introduce a new trait hierarchy to handle different access capabilities:
- IoCapable<T>: A marker trait indicating that a backend supports I/O
operations of a certain type (u8, u16, u32, or u64).
- Io trait: Defines fallible (try_read8, try_write8, etc.) and infallibile
(read8, write8, etc.) I/O methods with runtime bounds checking and
compile-time bounds checking.
- IoKnownSize trait: The marker trait for types support infallible I/O
methods.
Move the MMIO-specific logic into a dedicated Mmio<SIZE> type that
implements the Io traits. Rename IoRaw to MmioRaw and update consumers to
use the new types.
Cc: Alexandre Courbot <acourbot@nvidia.com>
Cc: Alice Ryhl <aliceryhl@google.com>
Cc: Bjorn Helgaas <helgaas@kernel.org>
Cc: Gary Guo <gary@garyguo.net>
Cc: Danilo Krummrich <dakr@kernel.org>
Cc: John Hubbard <jhubbard@nvidia.com>
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260121202212.4438-3-zhiw@nvidia.com
[ Add #[expect(unused)] to define_{read,write}!(). - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Currently, the driver's device private data is allocated and initialized
from driver core code called from bus abstractions after the driver's
probe() callback returned the corresponding initializer.
Similarly, the driver's device private data is dropped within the
remove() callback of bus abstractions after calling the remove()
callback of the corresponding driver.
However, commit 6f61a2637a ("rust: device: introduce
Device::drvdata()") introduced an accessor for the driver's device
private data for a Device<Bound>, i.e. a device that is currently bound
to a driver.
Obviously, this is in conflict with dropping the driver's device private
data in remove(), since a device can not be considered to be fully
unbound after remove() has finished:
We also have to consider registrations guarded by devres - such as IRQ
or class device registrations - which are torn down after remove() in
devres_release_all().
Thus, it can happen that, for instance, a class device or IRQ callback
still calls Device::drvdata(), which then runs concurrently to remove()
(which sets dev->driver_data to NULL and drops the driver's device
private data), before devres_release_all() started to tear down the
corresponding registration. This is because devres guarded registrations
can, as expected, access the corresponding Device<Bound> that defines
their scope.
In C it simply is the driver's responsibility to ensure that its device
private data is freed after e.g. an IRQ registration is unregistered.
Typically, C drivers achieve this by allocating their device private data
with e.g. devm_kzalloc() before doing anything else, i.e. before e.g.
registering an IRQ with devm_request_threaded_irq(), relying on the
reverse order cleanup of devres.
Technically, we could do something similar in Rust. However, the
resulting code would be pretty messy:
In Rust we have to differentiate between allocated but uninitialized
memory and initialized memory in the type system. Thus, we would need to
somehow keep track of whether the driver's device private data object
has been initialized (i.e. probe() was successful and returned a valid
initializer for this memory) and conditionally call the destructor of
the corresponding object when it is freed.
This is because we'd need to allocate and register the memory of the
driver's device private data *before* it is initialized by the
initializer returned by the driver's probe() callback, because the
driver could already register devres guarded registrations within
probe() outside of the driver's device private data initializer.
Luckily there is a much simpler solution: Instead of dropping the
driver's device private data at the end of remove(), we just drop it
after the device has been fully unbound, i.e. after all devres callbacks
have been processed.
For this, we introduce a new post_unbind() callback private to the
driver-core, i.e. the callback is neither exposed to drivers, nor to bus
abstractions.
This way, the driver-core code can simply continue to conditionally
allocate the memory for the driver's device private data when the
driver's initializer is returned from probe() - no change needed - and
drop it when the driver-core code receives the post_unbind() callback.
Closes: https://lore.kernel.org/all/DEZMS6Y4A7XE.XE7EUBT5SJFJ@kernel.org/
Fixes: 6f61a2637a ("rust: device: introduce Device::drvdata()")
Acked-by: Alice Ryhl <aliceryhl@google.com>
Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Acked-by: Igor Korotin <igor.korotin.linux@gmail.com>
Link: https://patch.msgid.link/20260107103511.570525-7-dakr@kernel.org
[ Remove #ifdef CONFIG_RUST, rename post_unbind() to post_unbind_rust().
- Danilo]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
The DriverLayout trait describes the layout of a specific driver
structure, such as `struct pci_driver` or `struct platform_driver`.
In a first step, this replaces the associated type RegType of the
RegistrationOps with the DriverLayout::DriverType associated type.
Acked-by: Alice Ryhl <aliceryhl@google.com>
Acked-by: Igor Korotin <igor.korotin.linux@gmail.com>
Link: https://patch.msgid.link/20260107103511.570525-4-dakr@kernel.org
[ Rename driver::Driver to driver::DriverLayout, as it represents the
layout of a driver structure rather than the driver structure itself.
- Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
We must not drop the device private data on shutdown(); none of the
registrations attached to devres that might access the device private
data are released before shutdown() is called.
Hence, freeing the device private data on shutdown() can cause UAF bugs.
Fixes: 57c5bd9aee ("rust: i2c: add basic I2C device and driver abstractions")
Acked-by: Alice Ryhl <aliceryhl@google.com>
Acked-by: Igor Korotin <igor.korotin.linux@gmail.com>
Link: https://patch.msgid.link/20260107103511.570525-2-dakr@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
On 32-bit ARM, you may encounter linker errors such as this one:
ld.lld: error: undefined symbol: _find_next_zero_bit
>>> referenced by rust_binder_main.43196037ba7bcee1-cgu.0
>>> drivers/android/binder/rust_binder_main.o:(<rust_binder_main::process::Process>::insert_or_update_handle) in archive vmlinux.a
>>> referenced by rust_binder_main.43196037ba7bcee1-cgu.0
>>> drivers/android/binder/rust_binder_main.o:(<rust_binder_main::process::Process>::insert_or_update_handle) in archive vmlinux.a
This error occurs because even though the functions are declared by
include/linux/find.h, the definition is #ifdef'd out on 32-bit ARM. This
is because arch/arm/include/asm/bitops.h contains:
#define find_first_zero_bit(p,sz) _find_first_zero_bit_le(p,sz)
#define find_next_zero_bit(p,sz,off) _find_next_zero_bit_le(p,sz,off)
#define find_first_bit(p,sz) _find_first_bit_le(p,sz)
#define find_next_bit(p,sz,off) _find_next_bit_le(p,sz,off)
And the underscore-prefixed function is conditional on #ifndef of the
non-underscore-prefixed name, but the declaration in find.h is *not*
conditional on that #ifndef.
To fix the linker error, we ensure that the symbols in question exist
when compiling Rust code. We do this by defining them in rust/helpers/
whenever the normal definition is #ifndef'd out.
Note that these helpers are somewhat unusual in that they do not have
the rust_helper_ prefix that most helpers have. Adding the rust_helper_
prefix does not compile, as 'bindings::_find_next_zero_bit()' will
result in a call to a symbol called _find_next_zero_bit as defined by
include/linux/find.h rather than a symbol with the rust_helper_ prefix.
This is because when a symbol is present in both include/ and
rust/helpers/, the one from include/ wins under the assumption that the
current configuration is one where that helper is unnecessary. This
heuristic fails for _find_next_zero_bit() because the header file always
declares it even if the symbol does not exist.
The functions still use the __rust_helper annotation. This lets the
wrapper function be inlined into Rust code even if full kernel LTO is
not used once the patch series for that feature lands.
Yury: arches are free to implement they own find_bit() functions. Most
rely on generic implementation, but arm32 and m86k - not; so they require
custom handling. Alice confirmed it fixes the build for both.
Cc: stable@vger.kernel.org
Fixes: 6cf93a9ed3 ("rust: add bindings for bitops.h")
Reported-by: Andreas Hindborg <a.hindborg@kernel.org>
Closes: https://rust-for-linux.zulipchat.com/#narrow/channel/x/topic/x/near/561677301
Tested-by: Andreas Hindborg <a.hindborg@kernel.org>
Reviewed-by: Dirk Behme <dirk.behme@de.bosch.com>
Signed-off-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Yury Norov (NVIDIA) <yury.norov@gmail.com>
Pull driver core fixes from Danilo Krummrich:
- Fix swapped example values for the `family` and `machine` attributes
in the sysfs SoC bus ABI documentation
- Fix Rust build and intra-doc issues when optional subsystems
(CONFIG_PCI, CONFIG_AUXILIARY_BUS, CONFIG_PRINTK) are disabled
- Fix typos and incorrect safety comments in Rust PCI, DMA, and
device ID documentation
* tag 'driver-core-6.19-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/driver-core/driver-core:
rust: device: Remove explicit import of CStrExt
rust: pci: fix typos in Bar struct's comments
rust: device: fix broken intra-doc links
rust: dma: fix broken intra-doc links
rust: driver: fix broken intra-doc links to example driver types
rust: device_id: replace incorrect word in safety documentation
rust: dma: remove incorrect safety documentation
docs: ABI: sysfs-devices-soc: Fix swapped sample values
Remove the explicit import of CStrExt. When CONFIG_PRINTK is disabled
this import causes a build error:
error: unused import: `crate::str::CStrExt`
--> rust/kernel/device.rs:17:5
|
17 | use crate::str::CStrExt as _;
| ^^^^^^^^^^^^^^^^^^^
|
= note: `-D unused-imports` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(unused_imports)]`
error: aborting due to 1 previous error
CStrExt is covered by prelude::* so the explicit import is redundant.
Signed-off-by: FUJITA Tomonori <fujita.tomonori@gmail.com>
Fixes: 3b83f5d5e7 ("rust: replace `CStr` with `core::ffi::CStr`")
Link: https://patch.msgid.link/20260106000320.2593800-1-fujita.tomonori@gmail.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
The `pci` module is conditional on CONFIG_PCI. When it's disabled, the
intra-doc link to `pci::Device` causes rustdoc warnings:
warning: unresolved link to `kernel::pci::Device`
--> rust/kernel/device.rs:163:22
|
163 | /// [`pci::Device`]: kernel::pci::Device
| ^^^^^^^^^^^^^^^^^^^ no item named `pci` in module `kernel`
|
= note: `#[warn(rustdoc::broken_intra_doc_links)]` on by default
Fix this by making the documentation conditional on CONFIG_PCI.
Fixes: d6e26c1ae4 ("device: rust: expand documentation for Device")
Signed-off-by: FUJITA Tomonori <fujita.tomonori@gmail.com>
Reviewed-by: Dirk Behme <dirk.behme@de.bosch.com>
Link: https://patch.msgid.link/20251231045728.1912024-2-fujita.tomonori@gmail.com
[ Keep the "such as" part indicating a list of examples; fix typos in
commit message. - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
The `pci` module is conditional on CONFIG_PCI. When it's disabled, the
intra-doc link to `pci::Device` causes rustdoc warnings:
warning: unresolved link to `::kernel::pci::Device`
--> rust/kernel/dma.rs:30:70
|
30 | /// where the underlying bus is DMA capable, such as [`pci::Device`](::kernel::pci::Device) or
| ^^^^^^^^^^^^^^^^^^^^^ no item named `pci` in module `kernel`
Fix this by making the documentation conditional on CONFIG_PCI.
Fixes: d06d5f66f5 ("rust: dma: implement `dma::Device` trait")
Signed-off-by: FUJITA Tomonori <fujita.tomonori@gmail.com>
Reviewed-by: Dirk Behme <dirk.behme@de.bosch.com>
Link: https://patch.msgid.link/20251231045728.1912024-1-fujita.tomonori@gmail.com
[ Keep the "such as" part indicating a list of examples; fix typos in
commit message. - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
The `auxiliary` and `pci` modules are conditional on
`CONFIG_AUXILIARY_BUS` and `CONFIG_PCI` respectively. When these are
disabled, the intra-doc links to `auxiliary::Driver` and `pci::Driver`
break, causing rustdoc warnings (or errors with `-D warnings`).
error: unresolved link to `kernel::auxiliary::Driver`
--> rust/kernel/driver.rs:82:28
|
82 | //! [`auxiliary::Driver`]: kernel::auxiliary::Driver
| ^^^^^^^^^^^^^^^^^^^^^^^^^ no item named `auxiliary` in module `kernel`
Fix this by making the documentation for these examples conditional on
the corresponding configuration options.
Fixes: 970a7c6878 ("driver: rust: expand documentation for driver infrastructure")
Signed-off-by: Alice Ryhl <aliceryhl@google.com>
Reported-by: FUJITA Tomonori <fujita.tomonori@gmail.com>
Closes: https://lore.kernel.org/rust-for-linux/20251209.151817.744108529426448097.fujita.tomonori@gmail.com/
Link: https://patch.msgid.link/20251227-driver-types-v1-1-1916154fbe5e@google.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Pull misc fixes from Andrew Morton:
"27 hotfixes. 12 are cc:stable, 18 are MM.
There's a patch series from Jiayuan Chen which fixes some
issues with KASAN and vmalloc. Apart from that it's the usual
shower of singletons - please see the respective changelogs
for details"
* tag 'mm-hotfixes-stable-2025-12-28-21-50' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (27 commits)
mm/ksm: fix pte_unmap_unlock of wrong address in break_ksm_pmd_entry
mm/page_owner: fix memory leak in page_owner_stack_fops->release()
mm/memremap: fix spurious large folio warning for FS-DAX
MAINTAINERS: notify the "Device Memory" community of memory hotplug changes
sparse: update MAINTAINERS info
mm/page_alloc: report 1 as zone_batchsize for !CONFIG_MMU
mm: consider non-anon swap cache folios in folio_expected_ref_count()
rust: maple_tree: rcu_read_lock() in destructor to silence lockdep
mm: memcg: fix unit conversion for K() macro in OOM log
mm: fixup pfnmap memory failure handling to use pgoff
tools/mm/page_owner_sort: fix timestamp comparison for stable sorting
selftests/mm: fix thread state check in uffd-unit-tests
kernel/kexec: fix IMA when allocation happens in CMA area
kernel/kexec: change the prototype of kimage_map_segment()
MAINTAINERS: add ABI headers to KHO and LIVE UPDATE
.mailmap: remove one of the entries for WangYuli
mm/damon/vaddr: fix missing pte_unmap_unlock in damos_va_migrate_pmd_entry()
MAINTAINERS: update one straggling entry for Bartosz Golaszewski
mm/page_alloc: change all pageblocks migrate type on coalescing
mm: leafops.h: correct kernel-doc function param. names
...
The safety documentation incorrectly refers to `RawDeviceId` when
transmuting to `RawType`. This fixes the documentation to correctly
indicate that implementers must ensure layout compatibility with
`RawType`, not `RawDeviceId`.
Fixes: 9b90864bb4 ("rust: implement `IdArray`, `IdTable` and `RawDeviceId`")
Signed-off-by: Yilin Chen <1479826151@qq.com>
Link: https://patch.msgid.link/tencent_C18DD5047749311142ED455779C7CCCF3A08@qq.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
When running the Rust maple tree kunit tests with lockdep, you may trigger
a warning that looks like this:
lib/maple_tree.c:780 suspicious rcu_dereference_check() usage!
other info that might help us debug this:
rcu_scheduler_active = 2, debug_locks = 1
no locks held by kunit_try_catch/344.
stack backtrace:
CPU: 3 UID: 0 PID: 344 Comm: kunit_try_catch Tainted: G N 6.19.0-rc1+ #2 NONE
Tainted: [N]=TEST
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x71/0x90
lockdep_rcu_suspicious+0x150/0x190
mas_start+0x104/0x150
mas_find+0x179/0x240
_RINvNtCs5QSdWC790r4_4core3ptr13drop_in_placeINtNtCs1cdwasc6FUb_6kernel10maple_tree9MapleTreeINtNtNtBL_5alloc4kbox3BoxlNtNtB1x_9allocator7KmallocEEECsgxAQYCfdR72_25doctests_kernel_generated+0xaf/0x130
rust_doctest_kernel_maple_tree_rs_0+0x600/0x6b0
? lock_release+0xeb/0x2a0
? kunit_try_catch_run+0x210/0x210
kunit_try_run_case+0x74/0x160
? kunit_try_catch_run+0x210/0x210
kunit_generic_run_threadfn_adapter+0x12/0x30
kthread+0x21c/0x230
? __do_trace_sched_kthread_stop_ret+0x40/0x40
ret_from_fork+0x16c/0x270
? __do_trace_sched_kthread_stop_ret+0x40/0x40
ret_from_fork_asm+0x11/0x20
</TASK>
This is because the destructor of maple tree calls mas_find() without
taking rcu_read_lock() or the spinlock. Doing that is actually ok in this
case since the destructor has exclusive access to the entire maple tree,
but it triggers a lockdep warning. To fix that, take the rcu read lock.
In the future, it's possible that memory reclaim could gain a feature
where it reallocates entries in maple trees even if no user-code is
touching it. If that feature is added, then this use of rcu read lock
would become load-bearing, so I did not make it conditional on lockdep.
We have to repeatedly take and release rcu because the destructor of T
might perform operations that sleep.
Link: https://lkml.kernel.org/r/20251217-maple-drop-rcu-v1-1-702af063573f@google.com
Fixes: da939ef4c4 ("rust: maple_tree: add MapleTree")
Signed-off-by: Alice Ryhl <aliceryhl@google.com>
Reported-by: Andreas Hindborg <a.hindborg@kernel.org>
Closes: https://rust-for-linux.zulipchat.com/#narrow/channel/x/topic/x/near/564215108
Reviewed-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Cc: Andrew Ballance <andrewjballance@gmail.com>
Cc: Björn Roy Baron <bjorn3_gh@protonmail.com>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Danilo Krummrich <dakr@kernel.org>
Cc: Liam Howlett <liam.howlett@oracle.com>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Miguel Ojeda <ojeda@kernel.org>
Cc: Trevor Gross <tmgross@umich.edu>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
