Tejun Heo
aa1b46dcdc
block: fix rq-qos breakage from skipping rq_qos_done_bio()
...
a647a524a4 ("block: don't call rq_qos_ops->done_bio if the bio isn't
tracked") made bio_endio() skip rq_qos_done_bio() if BIO_TRACKED is not set.
While this fixed a potential oops, it also broke blk-iocost by skipping the
done_bio callback for merged bios.
Before, whether a bio goes through rq_qos_throttle() or rq_qos_merge(),
rq_qos_done_bio() would be called on the bio on completion with BIO_TRACKED
distinguishing the former from the latter. rq_qos_done_bio() is not called
for bios which wenth through rq_qos_merge(). This royally confuses
blk-iocost as the merged bios never finish and are considered perpetually
in-flight.
One reliably reproducible failure mode is an intermediate cgroup geting
stuck active preventing its children from being activated due to the
leaf-only rule, leading to loss of control. The following is from
resctl-bench protection scenario which emulates isolating a web server like
workload from a memory bomb run on an iocost configuration which should
yield a reasonable level of protection.
# cat /sys/block/nvme2n1/device/model
Samsung SSD 970 PRO 512GB
# cat /sys/fs/cgroup/io.cost.model
259:0 ctrl=user model=linear rbps=834913556 rseqiops=93622 rrandiops=102913 wbps=618985353 wseqiops=72325 wrandiops=71025
# cat /sys/fs/cgroup/io.cost.qos
259:0 enable=1 ctrl=user rpct=95.00 rlat=18776 wpct=95.00 wlat=8897 min=60.00 max=100.00
# resctl-bench -m 29.6G -r out.json run protection::scenario=mem-hog,loops=1
...
Memory Hog Summary
==================
IO Latency: R p50=242u:336u/2.5m p90=794u:1.4m/7.5m p99=2.7m:8.0m/62.5m max=8.0m:36.4m/350m
W p50=221u:323u/1.5m p90=709u:1.2m/5.5m p99=1.5m:2.5m/9.5m max=6.9m:35.9m/350m
Isolation and Request Latency Impact Distributions:
min p01 p05 p10 p25 p50 p75 p90 p95 p99 max mean stdev
isol% 15.90 15.90 15.90 40.05 57.24 59.07 60.01 74.63 74.63 90.35 90.35 58.12 15.82
lat-imp% 0 0 0 0 0 4.55 14.68 15.54 233.5 548.1 548.1 53.88 143.6
Result: isol=58.12:15.82% lat_imp=53.88%:143.6 work_csv=100.0% missing=3.96%
The isolation result of 58.12% is close to what this device would show
without any IO control.
Fix it by introducing a new flag BIO_QOS_MERGED to mark merged bios and
calling rq_qos_done_bio() on them too. For consistency and clarity, rename
BIO_TRACKED to BIO_QOS_THROTTLED. The flag checks are moved into
rq_qos_done_bio() so that it's next to the code paths that set the flags.
With the patch applied, the above same benchmark shows:
# resctl-bench -m 29.6G -r out.json run protection::scenario=mem-hog,loops=1
...
Memory Hog Summary
==================
IO Latency: R p50=123u:84.4u/985u p90=322u:256u/2.5m p99=1.6m:1.4m/9.5m max=11.1m:36.0m/350m
W p50=429u:274u/995u p90=1.7m:1.3m/4.5m p99=3.4m:2.7m/11.5m max=7.9m:5.9m/26.5m
Isolation and Request Latency Impact Distributions:
min p01 p05 p10 p25 p50 p75 p90 p95 p99 max mean stdev
isol% 84.91 84.91 89.51 90.73 92.31 94.49 96.36 98.04 98.71 100.0 100.0 94.42 2.81
lat-imp% 0 0 0 0 0 2.81 5.73 11.11 13.92 17.53 22.61 4.10 4.68
Result: isol=94.42:2.81% lat_imp=4.10%:4.68 work_csv=58.34% missing=0%
Signed-off-by: Tejun Heo <tj@kernel.org >
Fixes: a647a524a4 ("block: don't call rq_qos_ops->done_bio if the bio isn't tracked")
Cc: stable@vger.kernel.org # v5.15+
Cc: Ming Lei <ming.lei@redhat.com >
Cc: Yu Kuai <yukuai3@huawei.com >
Reviewed-by: Ming Lei <ming.lei@redhat.com >
Link: https://lore.kernel.org/r/Yi7rdrzQEHjJLGKB@slm.duckdns.org
Signed-off-by: Jens Axboe <axboe@kernel.dk >
2022-03-14 14:23:13 -06:00
..
2021-11-25 18:30:52 +01:00
2021-12-09 16:42:27 -08:00
2021-12-17 12:37:18 -08:00
2021-11-26 16:19:19 +01:00
2022-01-05 12:09:06 +01:00
2022-01-20 13:46:20 +02:00
2021-11-23 10:29:05 +01:00
2021-11-26 16:48:59 +01:00
2021-10-27 11:07:59 +02:00
2021-12-03 10:58:13 -08:00
2022-01-03 17:05:02 +05:30
2021-12-31 14:35:40 +00:00
2022-01-14 16:02:28 +01:00
2021-11-28 14:02:41 -08:00
2021-12-16 22:16:41 +01:00
2021-12-17 12:26:14 +01:00
2021-12-12 17:12:18 +00:00
2021-11-02 16:36:28 -07:00
2021-12-16 13:21:12 +00:00
2022-01-08 14:42:01 -05:00
2021-11-29 13:02:00 +00:00
2022-01-12 17:02:27 -08:00
2021-10-05 13:15:35 +01:00
2022-01-06 16:22:52 -08:00
2021-11-17 14:52:44 +00:00
2021-12-31 13:31:34 +01:00
2021-10-21 20:02:42 +01:00
2021-11-22 08:41:56 -08:00
2022-01-09 23:30:13 +01:00
2021-10-14 23:06:53 +02:00
2021-10-14 23:06:53 +02:00
2021-10-14 23:06:53 +02:00
2021-10-14 23:06:53 +02:00
2021-11-15 10:13:36 +01:00
2022-01-17 08:07:57 +02:00
2021-10-02 17:49:11 +02:00
2021-12-03 18:24:21 -08:00
2022-01-06 08:37:03 -08:00
2022-01-11 12:17:45 -08:00
2022-01-22 08:33:34 +02:00
2022-01-13 09:05:29 -08:00
2021-12-24 14:06:45 +00:00
2022-01-07 17:14:30 +00:00
2022-01-17 05:49:30 +02:00
2021-11-16 10:56:54 +01:00
2022-01-20 08:52:53 +02:00
2022-01-25 18:30:15 +01:00
2022-01-12 15:57:59 -08:00
2022-01-22 08:33:34 +02:00
2021-10-29 22:34:31 -05:00
2021-11-23 15:10:26 +05:30
2021-10-18 13:11:06 +01:00
2021-10-07 16:51:57 +02:00
2021-10-26 21:01:48 -06:00
2021-12-30 17:27:44 -08:00
2021-10-01 16:52:48 -04:00
2021-11-01 21:17:39 -07:00
2021-12-22 13:56:19 +01:00
2021-11-06 13:30:34 -07:00
2021-11-06 14:08:17 -07:00
2022-03-07 06:42:33 -07:00
2021-12-11 09:09:45 +01:00
2022-01-15 08:47:31 -08:00
2022-01-15 08:47:31 -08:00
2022-03-14 14:23:13 -06:00
2022-02-11 10:02:41 -07:00
2021-10-21 10:49:32 -06:00
2021-10-18 06:17:02 -06:00
2022-03-08 19:39:38 -07:00
2021-12-22 23:38:29 -05:00
2022-03-08 19:39:38 -07:00
2021-10-18 06:17:02 -06:00
2021-10-10 22:16:02 -04:00
2022-01-06 12:27:03 +01:00
2021-11-09 10:02:49 -08:00
2021-12-29 17:54:40 -08:00
2021-11-01 19:59:46 -07:00
2022-01-19 01:21:24 +01:00
2021-12-16 14:57:10 -08:00
2021-12-16 14:57:10 -08:00
2021-12-29 20:03:05 -08:00
2022-01-19 01:21:46 +01:00
2021-10-05 17:07:41 -07:00
2021-10-24 13:42:28 +01:00
2021-11-12 10:19:09 -08:00
2021-12-10 15:56:13 -08:00
2021-10-18 06:17:36 -06:00
2021-11-17 10:36:15 -05:00
2021-12-03 10:58:13 -08:00
2021-12-08 16:49:42 +01:00
2022-01-11 09:14:37 -08:00
2021-12-08 11:19:20 +01:00
2021-11-06 13:30:39 -07:00
2021-12-09 16:42:28 -08:00
2021-12-09 16:42:28 -08:00
2021-11-06 14:08:17 -07:00
2021-12-21 15:09:46 -08:00
2021-11-27 13:52:22 +01:00
2021-11-09 10:02:49 -08:00
2022-01-22 08:33:36 +02:00
2021-12-30 17:44:07 +01:00
2021-11-02 19:31:28 +01:00
2021-12-14 12:30:26 +00:00
2022-01-15 08:47:31 -08:00
2021-11-06 13:30:38 -07:00
2021-11-09 10:02:52 -08:00
2021-12-23 22:35:00 +11:00
2022-01-15 16:30:33 +02:00
2021-12-18 08:04:53 -08:00
2022-01-22 08:33:36 +02:00
2021-12-10 17:10:55 -08:00
2022-01-20 08:52:55 +02:00
2021-12-18 08:04:53 -08:00
2021-12-16 22:22:16 +01:00
2021-11-18 09:36:39 +01:00
2021-11-11 09:35:13 +01:00
2021-11-17 14:44:29 +01:00
2021-12-17 21:42:17 +05:30
2021-10-18 12:31:48 +02:00
2022-01-22 08:33:34 +02:00
2021-12-23 16:55:20 +01:00
2021-12-10 12:51:28 +01:00
2022-01-18 08:51:51 +02:00
2022-01-20 08:52:53 +02:00
2022-01-20 08:52:53 +02:00
2021-10-05 16:33:05 +02:00
2021-12-01 00:06:43 +01:00
2021-10-19 12:46:24 +01:00
2021-11-03 11:20:45 +00:00
2022-01-20 11:30:15 +00:00
2022-01-08 14:42:01 -05:00
2022-01-22 08:33:35 +02:00
2021-12-31 08:57:45 -05:00
2022-01-05 19:46:32 -08:00
2022-01-15 08:47:31 -08:00
2021-11-01 13:44:26 -05:00
2021-10-22 14:13:53 +02:00
2021-10-18 07:49:39 -04:00
2021-10-25 15:34:41 -07:00
2022-01-22 08:33:38 +02:00
2022-01-17 09:40:29 +02:00
2021-12-09 14:09:36 -05:00
2022-01-22 08:33:38 +02:00
2022-01-07 13:43:18 +00:00
2022-01-21 21:36:28 +00:00
2021-10-22 09:50:55 +10:30
2021-12-15 14:04:27 +01:00
2022-01-24 14:16:46 +01:00
2021-10-21 14:19:00 -04:00
2021-11-09 10:02:49 -08:00
2021-11-09 10:02:49 -08:00
2022-01-15 16:30:29 +02:00
2022-01-20 08:52:54 +02:00
2022-01-10 09:49:13 +01:00
2021-10-18 07:49:38 -04:00
2021-11-18 15:05:56 -05:00
2021-12-16 14:07:07 +01:00
2022-01-04 13:15:33 -05:00
2022-01-15 16:30:29 +02:00
2022-01-15 16:30:29 +02:00
2021-12-26 15:02:06 -08:00
2021-12-20 18:01:09 +00:00
2021-12-13 11:44:47 +01:00
2021-11-28 21:53:04 +01:00
2021-12-07 20:44:58 -08:00
2021-12-07 20:44:58 -08:00
2021-12-24 10:24:30 -05:00
2021-12-06 16:05:11 -08:00
2022-01-22 08:33:35 +02:00
2021-11-09 10:02:49 -08:00
2021-12-21 15:09:46 -08:00
2021-12-17 16:13:14 +01:00
2021-10-18 12:31:48 +02:00
2021-11-11 22:45:40 +01:00
2021-12-17 09:06:15 +01:00
2021-12-10 20:47:38 +01:00
2021-11-06 13:30:36 -07:00
2021-12-16 10:59:02 -07:00
2022-01-12 15:46:11 -08:00
2021-12-20 09:03:05 +01:00
2021-12-20 13:53:26 +01:00
2021-11-20 10:35:54 -08:00
2021-10-05 06:54:16 -05:00
2021-10-05 06:54:16 -05:00
2022-01-04 12:17:35 +00:00
2021-10-15 11:25:18 +02:00
2021-10-28 13:34:57 +01:00
2021-10-28 21:11:34 +01:00
2021-10-26 10:13:31 +01:00
2021-12-01 00:15:07 +01:00
2021-12-04 10:56:23 +01:00
2021-11-09 10:02:51 -08:00
2022-01-15 20:37:06 +02:00
2021-12-09 16:42:29 -08:00
2021-12-09 16:42:26 -08:00
2021-10-05 15:51:35 +02:00
2022-01-20 10:41:01 +02:00
2021-12-21 10:34:39 +01:00
2021-11-06 13:30:43 -07:00
2022-01-04 14:36:06 +01:00
2022-01-22 08:33:36 +02:00
2021-10-18 07:49:39 -04:00
2022-01-20 10:41:01 +02:00
2022-01-07 10:44:44 -05:00
2022-01-22 09:40:01 +02:00
2022-01-07 10:44:44 -05:00
2021-09-27 16:16:42 +02:00
2022-01-14 15:17:17 +09:00
2022-01-20 08:52:53 +02:00
2021-11-09 10:02:49 -08:00
2021-10-30 16:37:28 +02:00
2022-01-29 08:52:27 +02:00
2021-12-06 13:46:24 -05:00
2022-01-16 23:34:43 +01:00
2021-09-30 12:46:44 -06:00
2022-01-05 11:22:17 +00:00
2021-11-26 18:31:46 +02:00
2021-10-04 11:47:24 +02:00
2022-01-11 12:36:47 +02:00
2022-01-15 20:37:06 +02:00
2021-11-06 13:30:42 -07:00
2021-11-06 13:30:42 -07:00
2022-01-15 16:30:30 +02:00
2022-01-15 20:37:06 +02:00
2021-12-17 17:17:14 +01:00
2021-09-29 10:54:50 +01:00
2021-11-06 13:30:41 -07:00
2022-01-22 08:33:34 +02:00
2022-01-15 16:30:27 +02:00
2022-01-30 09:56:58 +02:00
2022-01-30 09:56:58 +02:00
2022-01-15 16:30:29 +02:00
2021-12-05 10:28:57 +01:00
2021-11-11 22:45:40 +01:00
2022-01-17 07:32:51 +02:00
2022-01-22 08:33:36 +02:00
2021-12-06 16:06:02 -08:00
2021-12-16 22:22:20 +01:00
2022-01-20 10:12:53 +00:00
2022-01-09 23:35:16 +01:00
2022-01-09 23:30:13 +01:00
2022-01-07 09:22:19 +00:00
2021-10-05 13:07:03 +01:00
2021-12-06 16:06:02 -08:00
2021-11-02 12:31:23 -04:00
2022-01-25 20:16:03 +02:00
2022-01-25 20:16:03 +02:00
2022-01-13 09:30:48 -05:00
2021-11-06 13:30:42 -07:00
2021-12-09 07:02:22 -08:00
2021-10-20 19:16:00 +02:00
2021-10-20 19:16:01 +02:00
2021-10-27 08:03:19 +02:00
2021-10-13 15:33:17 +02:00
2021-09-30 21:24:07 -04:00
2022-01-12 10:14:09 -06:00
2021-10-14 15:59:04 -07:00
2022-01-20 12:55:26 -06:00
2021-11-16 19:07:53 -08:00
2022-01-15 16:30:31 +02:00
2021-11-11 09:34:35 -08:00
2021-09-27 09:27:29 -04:00
2022-01-15 16:30:28 +02:00
2022-01-18 06:40:47 +02:00
2022-01-12 12:37:02 -08:00
2022-01-13 16:50:50 -05:00
2022-02-02 07:49:59 -07:00
2022-01-16 08:08:11 +02:00
2021-09-27 17:00:21 +02:00
2022-01-16 08:08:11 +02:00
2022-01-02 18:48:47 +00:00
2021-12-09 15:41:09 -05:00
2022-01-20 08:52:52 +02:00
2022-01-18 12:09:47 +01:00
2022-01-15 16:30:28 +02:00
2021-12-09 13:23:02 -08:00
2021-12-16 10:37:13 +00:00
2022-01-26 18:57:09 -08:00
2021-10-14 13:29:18 +02:00
2022-01-22 08:33:36 +02:00
2022-01-03 21:24:34 -07:00
2021-11-09 10:02:49 -08:00
2021-10-04 16:33:15 +05:30
2022-01-12 19:59:05 +01:00
2021-10-27 20:49:32 +02:00
2022-01-12 19:59:05 +01:00
2021-12-23 22:35:00 +11:00
2022-01-22 08:33:35 +02:00
2021-11-02 12:52:17 +01:00
2022-01-11 11:26:57 -08:00
2021-11-01 20:05:19 -07:00
2022-01-22 08:33:36 +02:00
2022-01-22 08:33:37 +02:00
2022-01-08 12:43:57 -06:00
2022-01-14 16:02:28 +01:00
2022-01-18 12:09:57 +01:00
2022-01-30 09:56:58 +02:00
2021-12-17 16:59:47 +11:00
2021-11-26 11:38:20 -08:00
2022-01-06 12:18:08 +00:00
2021-10-06 11:27:41 -05:00
2021-11-18 09:36:39 +01:00
2021-10-07 16:51:57 +02:00
2022-01-24 14:45:02 +01:00
2021-11-09 10:02:49 -08:00
2022-01-07 00:25:25 +01:00
2021-12-07 15:14:12 +01:00
2021-12-07 16:24:44 -08:00
2021-11-30 17:25:20 -08:00
2021-11-30 17:25:20 -08:00
2022-01-22 08:33:37 +02:00
2021-11-15 13:27:13 +00:00
2021-10-15 09:49:55 -05:00
2021-11-19 12:23:45 +01:00
2021-12-20 11:02:38 +01:00
2021-12-21 10:22:19 +01:00
2021-10-18 07:49:39 -04:00
2021-11-10 09:05:11 -08:00
2021-10-18 17:20:50 +02:00
2021-12-04 10:56:23 +01:00
2022-01-22 08:33:37 +02:00
2022-01-22 08:33:37 +02:00
2022-01-22 08:33:37 +02:00
2021-11-09 10:02:49 -08:00
2022-02-08 06:55:49 -07:00
2021-12-22 09:21:43 +01:00
2022-01-23 17:35:27 +02:00
2021-10-18 20:22:03 -10:00
2021-12-06 13:46:24 -05:00
2021-12-10 06:38:26 -08:00
2022-01-22 08:33:37 +02:00
2021-12-31 13:42:30 +01:00
2021-12-30 13:23:44 +01:00
2021-12-20 16:53:44 +01:00
2022-01-22 08:33:38 +02:00
2021-11-03 14:09:26 -05:00
2021-11-09 10:11:53 -08:00
2021-12-02 11:44:56 -08:00
2022-01-27 08:45:13 -08:00
2021-11-16 13:10:34 +00:00
2022-01-06 12:26:13 +01:00
2022-01-19 11:15:19 +02:00
2022-01-06 12:26:13 +01:00
2021-11-09 10:02:49 -08:00
2021-10-26 14:58:45 +01:00
2021-10-30 16:37:28 +02:00
2022-01-22 08:33:37 +02:00
2021-12-07 15:14:12 +01:00
2021-10-30 16:37:28 +02:00
2021-12-09 16:42:27 -08:00
2021-11-30 17:25:20 -08:00
2022-01-22 08:33:37 +02:00
2022-01-22 08:33:35 +02:00
2021-12-10 14:06:03 +00:00
2021-10-18 12:28:52 -07:00
2021-11-30 11:57:58 +00:00
2021-11-18 18:40:08 +02:00
2021-11-03 11:41:25 -07:00
2022-01-25 18:34:08 +01:00
2022-01-15 16:30:28 +02:00
2022-01-22 08:33:38 +02:00
2021-12-20 18:01:09 +00:00
2022-01-11 15:38:59 -05:00
2022-01-15 16:30:30 +02:00
2022-01-30 09:56:58 +02:00
2021-10-18 06:17:02 -06:00
2021-12-07 11:36:30 -08:00
2022-01-10 08:13:52 -08:00
2021-12-01 00:06:43 +01:00
2021-12-03 15:58:27 +01:00
2021-12-07 16:36:17 -08:00
2022-01-09 00:18:47 +02:00
2022-01-13 16:23:04 -05:00
2021-11-01 20:05:19 -07:00
2022-01-08 12:43:58 -06:00
2021-11-26 16:27:42 +01:00
2021-11-25 18:35:23 +01:00
2021-11-26 16:27:42 +01:00
2021-11-26 16:27:42 +01:00
2021-11-26 16:27:42 +01:00
2021-12-13 12:42:08 +00:00
2022-01-12 15:46:11 -08:00
2021-10-12 11:41:39 -03:00
2021-12-30 12:13:04 +01:00
2021-10-18 16:02:30 -05:00
2022-01-14 18:50:54 -05:00
2021-11-11 13:09:33 +01:00
2021-09-30 12:46:44 -06:00
2021-11-01 05:26:48 -04:00
2021-12-20 18:47:37 -08:00
2021-11-01 04:30:34 -04:00
2022-01-14 18:50:52 -05:00
2022-01-15 16:30:29 +02:00
2022-01-15 16:30:25 +02:00
2021-10-18 07:49:40 -04:00
2021-12-09 10:49:56 -08:00
2021-12-06 22:33:39 +01:00
2021-10-20 06:19:03 -10:00
2022-01-07 09:22:19 +00:00
2021-12-23 11:21:53 +00:00
2022-01-08 00:28:41 -05:00
2021-10-19 23:44:30 +08:00
2021-11-08 16:55:32 -08:00
2021-11-08 16:55:32 -08:00
2021-11-08 16:55:32 -08:00