Christoffer reports that on some implementations, writing to
GICR_ISACTIVER0 (and similar GICD registers) can race badly with a guest
issuing a deactivation of that interrupt via the system register interface.
There are multiple reasons to this:
- this uses an early write-acknoledgement memory type (nGnRE), meaning
that the write may only have made it as far as some interconnect
by the time the store is considered "done"
- the GIC itself is allowed to buffer the write until it decides to
take it into account (as long as it is in finite time)
The effects are that the activation may not have taken effect by the time
the kernel enters the guest, forcing an immediate exit, or that a guest
deactivation occurs before the interrupt is active, doing nothing.
In order to guarantee that the write to the ISACTIVER register has taken
effect, read back from it, forcing the interconnect to propagate the write,
and the GIC to process the write before returning the read.
Reported-by: Christoffer Dall <christoffer.dall@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Christoffer Dall <christoffer.dall@arm.com>
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/all/20241106084418.3794612-1-maz@kernel.org
rzg2l_irqc_common_init() calls of_find_device_by_node(), but the
corresponding put_device() call is missing. This also gets reported by
make coccicheck.
Make use of the cleanup interfaces from cleanup.h to call into
__free_put_device(), which in turn calls into put_device when leaving
function rzg2l_irqc_common_init() and variable "dev" goes out of scope.
To prevent that the device is put on successful completion, assign NULL to
"dev" to prevent __free_put_device() from calling into put_device() within
the successful path.
"make coccicheck" will still complain about missing put_device() calls,
but those are false positives now.
Fixes: 3fed09559c ("irqchip: Add RZ/G2L IA55 Interrupt Controller driver")
Signed-off-by: Fabrizio Castro <fabrizio.castro.jz@renesas.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Link: https://lore.kernel.org/all/20241011172003.1242841-1-fabrizio.castro.jz@renesas.com
It is possible that an interrupt is disabled and masked at the same time.
When the interrupt is enabled again by enable_irq(), only plic_irq_enable()
is called, not plic_irq_unmask(). The interrupt remains masked and never
raises.
An example where interrupt is both disabled and masked is when
handle_fasteoi_irq() is the handler, and IRQS_ONESHOT is set. The interrupt
handler:
1. Mask the interrupt
2. Handle the interrupt
3. Check if interrupt is still enabled, and unmask it (see
cond_unmask_eoi_irq())
If another task disables the interrupt in the middle of the above steps,
the interrupt will not get unmasked, and will remain masked when it is
enabled in the future.
The problem is occasionally observed when PREEMPT_RT is enabled, because
PREEMPT_RT adds the IRQS_ONESHOT flag. But PREEMPT_RT only makes the problem
more likely to appear, the bug has been around since commit a1706a1c50
("irqchip/sifive-plic: Separate the enable and mask operations").
Fix it by unmasking interrupt in plic_irq_enable().
Fixes: a1706a1c50 ("irqchip/sifive-plic: Separate the enable and mask operations")
Signed-off-by: Nam Cao <namcao@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/all/20241003084152.2422969-1-namcao@linutronix.de
Controllers, supported by this driver, have two sets of registers:
* (main) interrupt registers control peripheral interrupt sources.
* device interrupt registers configure per-device (network interface)
interrupts and act as an extra stage before the main interrupt
registers.
In the driver unmask code, device trigger registers are used in the mask
calculation of the main interrupt sticky register, mixing two kinds of
registers.
Use the main interrupt trigger register instead.
Signed-off-by: Sergey Matsievskiy <matsievskiysv@gmail.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Link: https://lore.kernel.org/all/20240925184416.54204-2-matsievskiysv@gmail.com
Commit b5712bf89b ("irqchip/gic-v3-its: Provide MSI parent for
PCI/MSI[-X]") moves the functionality of irq-gic-v3-its-pci-msi.c into
irq-gic-v3-its-msi-parent.c, and drops the former file.
With that, the config option ARM_GIC_V3_ITS_PCI is obsolete, but the
definition of that config was not removed in the commit above.
Remove this obsolete config ARM_GIC_V3_ITS_PCI.
Signed-off-by: Lukas Bulwahn <lukas.bulwahn@redhat.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Link: https://lore.kernel.org/all/20240926125502.363364-1-lukas.bulwahn@redhat.com
Pull irq updates from Thomas Gleixner:
"Core:
- Remove a global lock in the affinity setting code
The lock protects a cpumask for intermediate results and the lock
causes a bottleneck on simultaneous start of multiple virtual
machines. Replace the lock and the static cpumask with a per CPU
cpumask which is nicely serialized by raw spinlock held when
executing this code.
- Provide support for giving a suffix to interrupt domain names.
That's required to support devices with subfunctions so that the
domain names are distinct even if they originate from the same
device node.
- The usual set of cleanups and enhancements all over the place
Drivers:
- Support for longarch AVEC interrupt chip
- Refurbishment of the Armada driver so it can be extended for new
variants.
- The usual set of cleanups and enhancements all over the place"
* tag 'irq-core-2024-09-16' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (73 commits)
genirq: Use cpumask_intersects()
genirq/cpuhotplug: Use cpumask_intersects()
irqchip/apple-aic: Only access system registers on SoCs which provide them
irqchip/apple-aic: Add a new "Global fast IPIs only" feature level
irqchip/apple-aic: Skip unnecessary enabling of use_fast_ipi
dt-bindings: apple,aic: Document A7-A11 compatibles
irqdomain: Use IS_ERR_OR_NULL() in irq_domain_trim_hierarchy()
genirq/msi: Use kmemdup_array() instead of kmemdup()
genirq/proc: Change the return value for set affinity permission error
genirq/proc: Use irq_move_pending() in show_irq_affinity()
genirq/proc: Correctly set file permissions for affinity control files
genirq: Get rid of global lock in irq_do_set_affinity()
genirq: Fix typo in struct comment
irqchip/loongarch-avec: Add AVEC irqchip support
irqchip/loongson-pch-msi: Prepare get_pch_msi_handle() for AVECINTC
irqchip/loongson-eiointc: Rename CPUHP_AP_IRQ_LOONGARCH_STARTING
LoongArch: Architectural preparation for AVEC irqchip
LoongArch: Move irqchip function prototypes to irq-loongson.h
irqchip/loongson-pch-msi: Switch to MSI parent domains
softirq: Remove unused 'action' parameter from action callback
...
Starting with the A11 (T8015) SoC, Apple began using arm64 sysregs for
fast IPIs. However, on A11, there is no such things as "Local" fast IPIs,
as the SYS_IMP_APL_IPI_RR_LOCAL_EL1 register does not seem to exist.
Add a new feature level, used by the compatible "apple,t8015-aic",
controlled by a static branch key named use_local_fast_ipi. When
use_fast_ipi is true and use_local_fast_ipi is false, fast IPIs are used
but all IPIs goes through the register SYS_IMP_APL_IPI_RR_GLOBAL_EL1, as
"global" IPIs.
Signed-off-by: Nick Chan <towinchenmi@gmail.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Sven Peter <sven@svenpeter.dev>
Link: https://lore.kernel.org/all/20240901034143.12731-4-towinchenmi@gmail.com
The GICv3 driver pokes GICv3 system registers in gic_prio_init() before
gic_cpu_sys_reg_init() ensures that GICv3 system registers have been
enabled by writing to ICC_SRE_EL1.SRE.
On arm64 this is benign as has_useable_gicv3_cpuif() runs earlier during
cpufeature detection, and this enables the GICv3 system registers.
On 32-bit arm when booting on an FVP using the boot-wrapper, the accesses
in gic_prio_init() end up being UNDEFINED and crashes the kernel during
boot.
This is a regression introduced by the addition of gic_prio_init().
Fix this by factoring out the SRE initialization into a new function and
calling it early in the three paths where SRE may not have been
initialized:
(1) gic_init_bases(), before the primary CPU pokes GICv3 sysregs in
gic_prio_init().
(2) gic_starting_cpu(), before secondary CPUs initialize GICv3 sysregs
in gic_cpu_init().
(3) gic_cpu_pm_notifier(), before CPUs re-initialize GICv3 sysregs in
gic_cpu_sys_reg_init().
Fixes: d447bf09a4 ("irqchip/gic-v3: Detect GICD_CTRL.DS and SCR_EL3.FIQ earlier")
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Marc Zyngier <maz@kernel.org>
Cc: stable@vger.kernel.org
The recently established lock ordering mandates that the per-VM
vmapp_lock is acquired before taking the per-VPE lock.
As it turns out, its_vpe_set_affinity() takes the VPE lock, and
then calls into its_send_vmovp(), which itself takes the vmapp
lock. Obviously, this is a lock order violation.
As its_send_vmovp() is only called from its_vpe_set_affinity(),
hoist the vmapp locking from the former into the latter, restoring
the expected order.
Fixes: f0eb154c39 ("irqchip/gic-v4: Substitute vmovp_lock for a per-VM lock")
Reported-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Link: https://lore.kernel.org/all/20240818171625.3030584-1-maz@kernel.org
The latest Linux RISC-V no longer boots on the Allwinner D1 platform
because the sun4i_timer driver fails to get an interrupt from PLIC due to
the recent conversion of the PLIC to a platform driver. Converting the
sun4i timer to a platform driver does not work either because the D1 does
not have a SBI timer available so early boot hangs. See the 'Closes:'
link for deeper analysis.
The real fix requires enabling the SBI time extension in the platform
firmware (OpenSBI) and convert sun4i_timer into platform driver.
Unfortunately, the real fix involves changing multiple places and can't be
achieved in a short duration and aside of that requires users to update
firmware.
As a work-around, retrofit PLIC probing such that the PLIC is probed early
only for the Allwinner D1 platform and probed as a regular platform driver
for rest of the RISC-V platforms. In the process, partially revert some of
the previous changes because the PLIC device pointer is not available in
all probing paths.
Fixes: e306a894bd ("irqchip/sifive-plic: Chain to parent IRQ after handlers are ready")
Fixes: 8ec99b0331 ("irqchip/sifive-plic: Convert PLIC driver into a platform driver")
Suggested-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Samuel Holland <samuel.holland@sifive.com>
Tested-by: Emil Renner Berthing <emil.renner.berthing@canonical.com>
Tested-by: Charlie Jenkins <charlie@rivosinc.com>
Reviewed-by: Samuel Holland <samuel.holland@sifive.com>
Reviewed-by: Charlie Jenkins <charlie@rivosinc.com>
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/all/20240820034850.3189912-1-apatel@ventanamicro.com
Closes: https://lore.kernel.org/lkml/20240814145642.344485-1-emil.renner.berthing@canonical.com/
The section 4.5.2 of the RISC-V AIA specification says that "any write
to a sourcecfg register of an APLIC might (or might not) cause the
corresponding interrupt-pending bit to be set to one if the rectified
input value is high (= 1) under the new source mode."
When the interrupt type is changed in the sourcecfg register, the APLIC
device might not set the corresponding pending bit, so the interrupt might
never become pending.
To handle sourcecfg register changes for level-triggered interrupts in MSI
mode, manually set the pending bit for retriggering interrupt so it gets
retriggered if it was already asserted.
Fixes: ca8df97fe6 ("irqchip/riscv-aplic: Add support for MSI-mode")
Signed-off-by: Yong-Xuan Wang <yongxuan.wang@sifive.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Vincent Chen <vincent.chen@sifive.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/all/20240809071049.2454-1-yongxuan.wang@sifive.com
The device tree property 'xlnx,kind-of-intr' is sanity checked that the
bitmask contains only set bits which are in the range of the number of
interrupts supported by the controller.
The check is done by shifting the mask right by the number of supported
interrupts and checking the result for zero.
The data type of the mask is u32 and the number of supported interrupts is
up to 32. In case of 32 interrupts the shift is out of bounds, resulting in
a mismatch warning. The out of bounds condition is also reported by UBSAN:
UBSAN: shift-out-of-bounds in irq-xilinx-intc.c:332:22
shift exponent 32 is too large for 32-bit type 'unsigned int'
Fix it by promoting the mask to u64 for the test.
Fixes: d50466c907 ("microblaze: intc: Refactor DT sanity check")
Signed-off-by: Radhey Shyam Pandey <radhey.shyam.pandey@amd.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/all/1723186944-3571957-1-git-send-email-radhey.shyam.pandey@amd.com
mpic_of_init() contains the last case where the open coded IPI support
condition needs to be replaced with mpic_is_ipi_available() to keep the
code consistent.
Signed-off-by: Marek Behún <kabel@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
On platforms where MPIC is not the top-level interrupt controller the
driver currently only supports handling of the per-CPU interrupts (the
first 29 interrupts). This is obvious from the code of
mpic_handle_cascade_irq(), which reads only one cause register.
Bound the number of available interrupts in the interrupt domain to 29 for
these platforms.
The corresponding device-trees refer only to per-CPU interrupts via MPIC,
the other interrupts are referred to via GIC.
Signed-off-by: Marek Behún <kabel@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Use MPIC_PER_CPU_IRQS_NR (29) bound instead of BITS_PER_LONG (32) when
iterating the bits of the per-CPU interrupt cause register, since there
are only 29 per-CPU interrupts. The top 3 bits are always zero anyway.
Signed-off-by: Marek Behún <kabel@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
The number of per-CPU interrupts is 29 (0 to 28). This is described by
the constant MPIC_MAX_PER_CPU_IRQS, set to 28 (the maximum per-CPU
interrupt).
Commit 0fa4ce746d ("irqchip/armada-370-xp: Re-enable per-CPU
interrupts at resume time") used the constant incorrectly in the
for-loop, it used the operator < instead of <=, causing it to iterate
only the first 28 interrupts (0 to 27), ignoring the last, 28th,
per-CPU interrupt.
To avoid this kind of confusions, fix this issue by renaming the constant
to MPIC_PER_CPU_IRQS_NR and set it to 29, the number of per-CPU IRQs.
Update its use in mpic_is_percpu_irq() accordingly.
Fixes: 0fa4ce746d ("irqchip/armada-370-xp: Re-enable per-CPU interrupts at resume time")
Signed-off-by: Marek Behún <kabel@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: <stable+noautosel@kernel.org> # The 29th interrupt is not used in any device-tree
Dynamically allocate the driver private structure. This concludes the
conversion of this driver to modern style.
Signed-off-by: Marek Behún <kabel@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
In continuation of converting the driver to modern style, drop the
global pointer to the driver private structure and instead pass it
around the functions and callbacks, wherever possible. (There are 3
cases where it is not possible: mpic_cascaded_starting_cpu() and the
syscore operations mpic_suspend() and mpic_resume()).
Signed-off-by: Marek Behún <kabel@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Put the MSI doorbell limits msi_doorbell_start, msi_doorbell_size and
msi_doorbell_mask into the driver private structure and get rid of the
corresponding functions.
Signed-off-by: Marek Behún <kabel@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
In preparation for converting the driver to modern style put all the
interrupt controller private static variables into driver private
structure.
Access to these variables changes as:
main_int_base mpic->base
per_cpu_int_base mpic->per_cpu
mpic_domain mpic->domain
parent_irq mpic->parent_irq
...
Signed-off-by: Marek Behún <kabel@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
For consistency with the rest of the driver, put the __init attribute
after the return type of the mpic_ipi_init() function.
Signed-off-by: Marek Behún <kabel@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Add the __init attribute to the mpic_msi_init() function. It is only
called from the device initializer, and so can be dropped after boot is
complete.
Signed-off-by: Marek Behún <kabel@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Drop the msi_doorbell_end() function and related constants, it is not
used anymore.
Signed-off-by: Marek Behún <kabel@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Drop IPI_DOORBELL_START since it is not used and rename IPI_DOORBELL_END
to IPI_DOORBELL_NR.
Signed-off-by: Marek Behún <kabel@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Pull irq fixes from Thomas Gleixner:
"A couple of fixes for interrupt chip drivers:
- Make sure to skip the clear register space in the MBIGEN driver
when calculating the node register index. Otherwise the clear
register is clobbered and the wrong node registers are accessed.
- Fix a signed/unsigned confusion in the loongarch CPU driver which
converts an error code to a huge "valid" interrupt number.
- Convert the mesion GPIO interrupt controller lock to a raw spinlock
so it works on RT.
- Add a missing static to a internal function in the pic32 EVIC
driver"
* tag 'irq-urgent-2024-08-04' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
irqchip/mbigen: Fix mbigen node address layout
irqchip/meson-gpio: Convert meson_gpio_irq_controller::lock to 'raw_spinlock_t'
irqchip/irq-pic32-evic: Add missing 'static' to internal function
irqchip/loongarch-cpu: Fix return value of lpic_gsi_to_irq()
The mbigen interrupt chip has its per node registers located in a
contiguous region of page sized chunks. The code maps them into virtual
address space as a contiguous region and determines the address of a node
by using the node ID as index.
mbigen chip
|-----------------|------------|--------------|
mgn_node_0 mgn_node_1 ... mgn_node_i
|--------------| |--------------| |----------------------|
[0x0000, 0x0x0FFF] [0x1000, 0x1FFF] [i*0x1000, (i+1)*0x1000 - 1]
This works correctly up to 10 nodes, but then fails because the 11th's
array slot is used for the MGN_CLEAR registers.
mbigen chip
|-----------|--------|--------|---------------|--------|
mgn_node_0 mgn_node_1 ... mgn_clear_register ... mgn_node_i
|-----------------|
[0xA000, 0xAFFF]
Skip the MGN_CLEAR register space when calculating the offset for node IDs
greater than or equal to ten.
Fixes: a6c2f87b88 ("irqchip/mbigen: Implement the mbigen irq chip operation functions")
Signed-off-by: Yipeng Zou <zouyipeng@huawei.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Link: https://lore.kernel.org/all/20240730014400.1751530-1-zouyipeng@huawei.com
