drivers: usb: xhci: prevent a theoretical race on non-coherent platforms

For platforms that have xHCI controllers attached over PCIe, and
non-coherent routes to main memory, a theoretical race exists between
posting new TRBs to a ring, and writing to the doorbell register.

In a contended system, write traffic from the CPU may be stalled before
the memory controller, whereas the CPU to Endpoint route is separate
and not likely to be contended. Similarly, the DMA route from the
endpoint to main memory may be separate and uncontended.

Therefore the xHCI can receive a doorbell write and find a stale view
of a transfer ring. In cases where only a single TRB is ping-ponged at
a time, this can cause the endpoint to not get polled at all.

Adding a readl() before the write forces a round-trip transaction
across PCIe, definitively serialising the CPU along the PCI
producer-consumer ordering rules.

Signed-off-by: Jonathan Bell <jonathan@raspberrypi.com>

drivers/usb/host/xhci-ring.c

@@ -566,6 +566,19 @@ void xhci_ring_ep_doorbell(struct xhci_hcd *xhci,
 
 	trace_xhci_ring_ep_doorbell(slot_id, DB_VALUE(ep_index, stream_id));
 
+	/*
+	 * For non-coherent systems with PCIe DMA (such as Pi 4, Pi 5) there
+	 * is a theoretical race between the TRB write and barrier, which
+	 * is reported complete as soon as the write leaves the CPU domain,
+	 * the doorbell write, which may be reported as complete by the RC
+	 * at some arbitrary point, and the visibility of new TRBs in system
+	 * RAM by the endpoint DMA engine.
+	 *
+	 * This read before the write positively serialises the CPU state
+	 * by incurring a round-trip across the link.
+	 */
+	readl(db_addr);
+
 	writel(DB_VALUE(ep_index, stream_id), db_addr);
 	/* flush the write */
 	readl(db_addr);