)]}'
{
  "commit": "6adb495d16d640a81ff44076a92642d8ebd33a58",
  "tree": "feb64bf33b46816cd39f71edb8cacdd1a987cc6b",
  "parents": [
    "6b77a338da8cc00ebcf6f7c7cf92f67362d3e8e5"
  ],
  "author": {
    "name": "David Howells",
    "email": "dhowells@redhat.com",
    "time": "Thu Apr 09 23:27:16 2026 +0100"
  },
  "committer": {
    "name": "David Howells",
    "email": "dhowells@redhat.com",
    "time": "Mon Apr 13 23:51:47 2026 +0100"
  },
  "message": "netfs: Combine prepare and issue ops and grab the buffers on request\n\nModify the way subrequests are generated in netfslib to try and simplify\nthe code.  The issue, primarily, is in writeback: the code has to create\nmultiple streams of write requests to disparate targets with different\nproperties (e.g. server and fscache), where not every folio needs to go to\nevery target (e.g. data just read from the server may only need writing to\nthe cache).\n\nThe current model in writeback, at least, is to go carefully through every\nfolio, preparing a subrequest for each stream when it was detected that\npart of the current folio needed to go to that stream, and repeating this\nwithin and across contiguous folios; then to issue subrequests as they\nbecome full or hit boundaries after first setting up the buffer.  However,\nthis is quite difficult to follow - and makes it tricky to handle\ndiscontiguous folios in a request.\n\nThis is changed such that netfs now accumulates buffers and attaches them\nto each stream when they become valid for that stream, then flushes the\nstream when a limit or a boundary is hit.  The issuing code in netfs then\nloops around creating and issuing subrequests without calling a separate\nprepare stage (though a function is provided to get an estimate of when\nflushing should occur).  The filesystem (or cache) then gets to take a\nslice of the master bvec chain as its I/O buffer for each subrequest,\nincluding discontiguities if it can support a sparse/vectored RPC (as Ceph\ncan).\n\nSimilar-ish changes also apply to buffered read and unbuffered read and\nwrite, though in each of those cases there is only a single contiguous\nstream.  Though for buffered read this consists of interwoven requests from\nmultiple sources (server or cache).\n\nTo this end, netfslib is changed in the following ways:\n\n (1) -\u003eprepare_xxx(), buffer selection and -\u003eissue_xxx() are now collapsed\n     together such that one -\u003eissue_xxx() call is made with the subrequest\n     defined to the maximum extent; the filesystem/cache then reduces the\n     length of the subrequest and calls back to netfslib to grab a slice of\n     the buffer, which may reduce the subrequest further if a maximum\n     segment limit is set.  The filesystem/cache then dispatches the\n     operation.\n\n (2) Retry buffer tracking is added to the netfs_io_request struct.  This\n     is then selected by the subrequest retry counter being non-zero.\n\n (3) The use of iov_iter is pushed down to the filesystem.  Netfslib now\n     provides the filesystem with a bvecq holding the buffer rather than an\n     iov_iter.  The bvecq can be duplicated and headers/trailers attached\n     to hold protocol and several bvecqs can be linked together to create a\n     compound operation.\n\n (4) If the -\u003eissue_xxx() functions terminate with -ENOMEM, a flag is set\n     on the request to abort further subrequest generation/retrying.\n\n (5) During writeback, netfslib now builds up an accumulation of buffered\n     data before issuing writes on each stream (one server, one cache).  It\n     asks each stream for an estimate of how much data to accumulate before\n     it next generates subrequests on the stream.  The filesystem or cache\n     is not required to use up all the data accumulated on a stream at that\n     time unless the end of the pagecache is hit.\n\n (6) During read-gaps, in which there are two gaps on either end of a dirty\n     streaming write page that need to be filled, a buffer is constructed\n     consisting of the two ends plus a sink page repeated to cover the\n     middle portion.  This is passed to the server as a single write.  For\n     something like Ceph, this should probably be done either as a\n     vectored/sparse read or as two separate reads (if different Ceph\n     objects are involved).\n\n (7) During unbuffered/DIO read/write, there is a single contiguous file\n     region to be read or written as a single stream.  The dispatching\n     function just creates subrequests and calls -\u003eissue_xxx() repeatedly\n     to eat through the bufferage.\n\n (8) At the start of buffered read, the entire set of folios allocated by\n     VM readahead is loaded into a bvecq chain, rather than trying to do it\n     piecemeal as-needed.  As the pages were already added and locked by\n     the VM, this is slightly more efficient than loading piecemeal as only\n     a single iteration of the xarray is required.\n\n (9) During buffered read, there is a single contiguous file region, to\n     read as a single stream - however, this stream may be stitched\n     together from subrequests to multiple sources.  Which sources are used\n     where is now determined by querying the cache to find the next couple\n     of extents in which it has data; netfslib uses this to direct the\n     subrequests towards the appropriate sources.\n\n     Each subrequest is given the maximum length in the current extent and\n     then -\u003eissue_read() is called.  The filesystem then limits the size\n     and slices off a piece of the buffer for that extent.\n\n(10) Cachefiles now provides an estimation function that indicates the\n     standard maxima for doing DIO (MAX_RW_COUNT and BIO_MAX_VECS).\n\nNote that sparse cachefiles still rely on the backing filesystem for\ncontent mapping.  That will need to be addressed in a future patch and is\nnot trivial to fix.\n\nSigned-off-by: David Howells \u003cdhowells@redhat.com\u003e\ncc: Paulo Alcantara \u003cpc@manguebit.org\u003e\ncc: Matthew Wilcox \u003cwilly@infradead.org\u003e\ncc: Christoph Hellwig \u003chch@infradead.org\u003e\ncc: netfs@lists.linux.dev\ncc: linux-fsdevel@vger.kernel.org\n",
  "tree_diff": [
    {
      "type": "modify",
      "old_id": "862164181baca1e88ecf8859c55868bb01525ce1",
      "old_mode": 33188,
      "old_path": "fs/9p/vfs_addr.c",
      "new_id": "3a03da512c6fcd4a8f4b6ccc5270a1ddc75665a6",
      "new_mode": 33188,
      "new_path": "fs/9p/vfs_addr.c"
    },
    {
      "type": "modify",
      "old_id": "b39044adb5e84fc219165a895aff54df829c3715",
      "old_mode": 33188,
      "old_path": "fs/afs/dir.c",
      "new_id": "9f362b6513b06a03d8a9fafef6fbb67bdbacc133",
      "new_mode": 33188,
      "new_path": "fs/afs/dir.c"
    },
    {
      "type": "modify",
      "old_id": "424e0c98d67fe7e34347a80ac9f9fa1e8b2ec0c3",
      "old_mode": 33188,
      "old_path": "fs/afs/file.c",
      "new_id": "0022706c3cd302a4db468b857fe32b6d7d45bfbe",
      "new_mode": 33188,
      "new_path": "fs/afs/file.c"
    },
    {
      "type": "modify",
      "old_id": "95494d5f2b8a9791c858fb7e85609534623a8bb6",
      "old_mode": 33188,
      "old_path": "fs/afs/fsclient.c",
      "new_id": "f59a9db4bb0ea734b026fc86b5c2b76ee49bf688",
      "new_mode": 33188,
      "new_path": "fs/afs/fsclient.c"
    },
    {
      "type": "modify",
      "old_id": "9bf5d2f1dbc4cc2e21193a10eb1350898914c1ca",
      "old_mode": 33188,
      "old_path": "fs/afs/internal.h",
      "new_id": "227ea40d5dc541e74443e0b0874eabda00d4dc3b",
      "new_mode": 33188,
      "new_path": "fs/afs/internal.h"
    },
    {
      "type": "modify",
      "old_id": "93ad86ff33453f58ce8051c573db13c3849bd8ed",
      "old_mode": 33188,
      "old_path": "fs/afs/write.c",
      "new_id": "2b9f8cdb2b050731d79862b4333433219a0846bf",
      "new_mode": 33188,
      "new_path": "fs/afs/write.c"
    },
    {
      "type": "modify",
      "old_id": "24fb562ebd33ace4aa13c3e77e1b7df8c521c579",
      "old_mode": 33188,
      "old_path": "fs/afs/yfsclient.c",
      "new_id": "ffd1d4c87290e62670d69c1462abf644d4b20a26",
      "new_mode": 33188,
      "new_path": "fs/afs/yfsclient.c"
    },
    {
      "type": "modify",
      "old_id": "427df09e55326bae30eb4af034286f7b7f309590",
      "old_mode": 33188,
      "old_path": "fs/cachefiles/io.c",
      "new_id": "64f23d36cf56670fac8c5263302122bc19eda1a0",
      "new_mode": 33188,
      "new_path": "fs/cachefiles/io.c"
    },
    {
      "type": "modify",
      "old_id": "3d64a316ca31d3b7c2bdb1a0c36ede3e06974c67",
      "old_mode": 33188,
      "old_path": "fs/ceph/Kconfig",
      "new_id": "aa6ccd7794d2338dfd27de16cd12ea2aa6c07490",
      "new_mode": 33188,
      "new_path": "fs/ceph/Kconfig"
    },
    {
      "type": "modify",
      "old_id": "2090fc78529cb4fc2ab9bef6aafe8529beae582e",
      "old_mode": 33188,
      "old_path": "fs/ceph/addr.c",
      "new_id": "098b958d1e543f31f130cacfbe8cecc9c0f3dd7c",
      "new_mode": 33188,
      "new_path": "fs/ceph/addr.c"
    },
    {
      "type": "modify",
      "old_id": "7701c037c3283f27e830a49483ce377c8a802538",
      "old_mode": 33188,
      "old_path": "fs/netfs/Kconfig",
      "new_id": "d0e7b0971fa3f0bf85c3b63e985cafbc05b3c4a1",
      "new_mode": 33188,
      "new_path": "fs/netfs/Kconfig"
    },
    {
      "type": "modify",
      "old_id": "0621e6870cbdb9033be1f7299c1d4b0773f10fa0",
      "old_mode": 33188,
      "old_path": "fs/netfs/Makefile",
      "new_id": "421dd0be413b3d92e6a5155951496de0cd618513",
      "new_mode": 33188,
      "new_path": "fs/netfs/Makefile"
    },
    {
      "type": "modify",
      "old_id": "c2d8c82ef39e93112a855f7cefe4cabe8ac7f117",
      "old_mode": 33188,
      "old_path": "fs/netfs/buffered_read.c",
      "new_id": "f024add71b70fddc8060446f4ef99b3ddd097223",
      "new_mode": 33188,
      "new_path": "fs/netfs/buffered_read.c"
    },
    {
      "type": "modify",
      "old_id": "e849db69383974ce0ba09dedaf6d0ba1db5d70de",
      "old_mode": 33188,
      "old_path": "fs/netfs/buffered_write.c",
      "new_id": "21a0b8f3e649021a1237017552f25e53d8337abf",
      "new_mode": 33188,
      "new_path": "fs/netfs/buffered_write.c"
    },
    {
      "type": "modify",
      "old_id": "05d09ba3d0d07ad126e56f7b715e76c317c2e484",
      "old_mode": 33188,
      "old_path": "fs/netfs/direct_read.c",
      "new_id": "83b5ec341b541d9b32514e1ebb2cfbf43f46772f",
      "new_mode": 33188,
      "new_path": "fs/netfs/direct_read.c"
    },
    {
      "type": "modify",
      "old_id": "a61c6d6fd17f7a78fc52175e12c242593f73bb44",
      "old_mode": 33188,
      "old_path": "fs/netfs/direct_write.c",
      "new_id": "7209d9e6c5971c5294bbcbd4bb1ff83e20099bb9",
      "new_mode": 33188,
      "new_path": "fs/netfs/direct_write.c"
    },
    {
      "type": "modify",
      "old_id": "fafa8c6bec57156f079c682c002485b6d0ba1782",
      "old_mode": 33188,
      "old_path": "fs/netfs/fscache_io.c",
      "new_id": "e4b7888fe75742f17a532bbfef398006d20fe9c9",
      "new_mode": 33188,
      "new_path": "fs/netfs/fscache_io.c"
    },
    {
      "type": "modify",
      "old_id": "ddae82f94ce009d458c71b894970c6494f335572",
      "old_mode": 33188,
      "old_path": "fs/netfs/internal.h",
      "new_id": "886077c2ab4405c64338cff047844e32d89dbb7a",
      "new_mode": 33188,
      "new_path": "fs/netfs/internal.h"
    },
    {
      "type": "modify",
      "old_id": "7969c0b1f9a962e2d6ed2c2cd782862797c28335",
      "old_mode": 33188,
      "old_path": "fs/netfs/iterator.c",
      "new_id": "69164e8b8e57a74cab1bb5b38cc28233e01e7a3d",
      "new_mode": 33188,
      "new_path": "fs/netfs/iterator.c"
    },
    {
      "type": "modify",
      "old_id": "b806848ee60fd3c9ba8096f48d61446dfe0fa246",
      "old_mode": 33188,
      "old_path": "fs/netfs/misc.c",
      "new_id": "2cbfeff03eac3a4fa0e40ad42635340a21f0218e",
      "new_mode": 33188,
      "new_path": "fs/netfs/misc.c"
    },
    {
      "type": "modify",
      "old_id": "7f5187c64ae95e48d2a6ea84e61a63dce46905b0",
      "old_mode": 33188,
      "old_path": "fs/netfs/objects.c",
      "new_id": "d4a95a462576f4f091eb1f8792b552a1fdef7e38",
      "new_mode": 33188,
      "new_path": "fs/netfs/objects.c"
    },
    {
      "type": "modify",
      "old_id": "6d49f9a6b1f0828ebe75d768b2a6a3c7c63fa65f",
      "old_mode": 33188,
      "old_path": "fs/netfs/read_collect.c",
      "new_id": "4288eff6252e079bc3ed1ff747ac7b401351d78b",
      "new_mode": 33188,
      "new_path": "fs/netfs/read_collect.c"
    },
    {
      "type": "modify",
      "old_id": "2eee35b5c759865d5b30c95806d5319fc71b4235",
      "old_mode": 33188,
      "old_path": "fs/netfs/read_pgpriv2.c",
      "new_id": "534ed7da700a7aef4c9e7843ab2e1d7c3dc25b46",
      "new_mode": 33188,
      "new_path": "fs/netfs/read_pgpriv2.c"
    },
    {
      "type": "modify",
      "old_id": "15fc6a3af5ed46daf87038da49417f52a1ddf2de",
      "old_mode": 33188,
      "old_path": "fs/netfs/read_retry.c",
      "new_id": "1f11320fb66a88acb16ee589daa43256c1a7e8de",
      "new_mode": 33188,
      "new_path": "fs/netfs/read_retry.c"
    },
    {
      "type": "modify",
      "old_id": "b386cae77eced11112b533416a70a1b1c90da261",
      "old_mode": 33188,
      "old_path": "fs/netfs/read_single.c",
      "new_id": "8237894c8fd84c1be125344bcd001f3ccb9edda3",
      "new_mode": 33188,
      "new_path": "fs/netfs/read_single.c"
    },
    {
      "type": "modify",
      "old_id": "9a24fcf20e15b9840a185c850f69113247774b3c",
      "old_mode": 33188,
      "old_path": "fs/netfs/write_collect.c",
      "new_id": "2b74b408b7adb85ae2c2477ec685b1dc9ccbb181",
      "new_mode": 33188,
      "new_path": "fs/netfs/write_collect.c"
    },
    {
      "type": "modify",
      "old_id": "eabe829de4fe894c46e528e50e827c2e160470e5",
      "old_mode": 33188,
      "old_path": "fs/netfs/write_issue.c",
      "new_id": "c350e22ad323804d573ab887564de917e46db058",
      "new_mode": 33188,
      "new_path": "fs/netfs/write_issue.c"
    },
    {
      "type": "modify",
      "old_id": "1d0091b88fc9f52e2b4b5cd02718e00efc7c95a2",
      "old_mode": 33188,
      "old_path": "fs/netfs/write_retry.c",
      "new_id": "bdcd277b86573dac67752e7fac935045079e7724",
      "new_mode": 33188,
      "new_path": "fs/netfs/write_retry.c"
    },
    {
      "type": "modify",
      "old_id": "6bb30543eff00fa550093d35c099d79802e99720",
      "old_mode": 33188,
      "old_path": "fs/nfs/Kconfig",
      "new_id": "e7862f35b72c8cdef43c6a120db8af5cbc304a34",
      "new_mode": 33188,
      "new_path": "fs/nfs/Kconfig"
    },
    {
      "type": "modify",
      "old_id": "9b7fdad4a92019bb5e9bd5c3823ec32ca4bb9fdf",
      "old_mode": 33188,
      "old_path": "fs/nfs/fscache.c",
      "new_id": "cf750faaec6afd28e8d728d01a3877b01e42842a",
      "new_mode": 33188,
      "new_path": "fs/nfs/fscache.c"
    },
    {
      "type": "modify",
      "old_id": "3990a901226401b3ca1151cf9b66e7f690773c2a",
      "old_mode": 33188,
      "old_path": "fs/smb/client/cifssmb.c",
      "new_id": "dc9120802edb08d70ddbf50a609d8f6a994e87b7",
      "new_mode": 33188,
      "new_path": "fs/smb/client/cifssmb.c"
    },
    {
      "type": "modify",
      "old_id": "a69e05f86d7e2f9b0c00f44c017ffde1b7b1ef0d",
      "old_mode": 33188,
      "old_path": "fs/smb/client/file.c",
      "new_id": "54556f6b27f515beac76f36ce244db7999612196",
      "new_mode": 33188,
      "new_path": "fs/smb/client/file.c"
    },
    {
      "type": "modify",
      "old_id": "89974e967327c947a747c6deb3e77e9bd6915642",
      "old_mode": 33188,
      "old_path": "fs/smb/client/smb2ops.c",
      "new_id": "b4edc5aa43f9e1e0974e30994743678b359ec8ca",
      "new_mode": 33188,
      "new_path": "fs/smb/client/smb2ops.c"
    },
    {
      "type": "modify",
      "old_id": "957aca2222b51f003c4dd2ce0cdcb1fb52103067",
      "old_mode": 33188,
      "old_path": "fs/smb/client/smb2pdu.c",
      "new_id": "3b93ba2d91c45a1a60f90e72755da1cf4143e5ae",
      "new_mode": 33188,
      "new_path": "fs/smb/client/smb2pdu.c"
    },
    {
      "type": "modify",
      "old_id": "05f8099047e1a423a12a1cea2eaec009d9dd050d",
      "old_mode": 33188,
      "old_path": "fs/smb/client/transport.c",
      "new_id": "dd1313736fcb9b36d9db917298c9f08393b50c53",
      "new_mode": 33188,
      "new_path": "fs/smb/client/transport.c"
    },
    {
      "type": "modify",
      "old_id": "e11784fd4f6d70ce4d59e1add9b6f36e98bc489c",
      "old_mode": 33188,
      "old_path": "include/linux/netfs.h",
      "new_id": "16b0cdf3eb91a892633a13dd3acd0d4dc35d9037",
      "new_mode": 33188,
      "new_path": "include/linux/netfs.h"
    },
    {
      "type": "modify",
      "old_id": "025a43f5e32c9bd762a29e74b51b857de738e5de",
      "old_mode": 33188,
      "old_path": "include/trace/events/netfs.h",
      "new_id": "9e0ffe84a03503d5531b257427001edbb6f1a7d0",
      "new_mode": 33188,
      "new_path": "include/trace/events/netfs.h"
    },
    {
      "type": "modify",
      "old_id": "f0dcf252af7e12328aef2d637c117ab9acee1766",
      "old_mode": 33188,
      "old_path": "net/9p/client.c",
      "new_id": "8d365c000553bc513bc5a53e0afdc30dfbf4a2a0",
      "new_mode": 33188,
      "new_path": "net/9p/client.c"
    }
  ]
}