I tried to tell pv to make large read()/write() sycalls (it is much more efficient to make large syscalls on e.g. networked file systems) using --buffer-size 16M.
However, I found that it had no effect; pv was making 512Ki-sized IO operations.
This is due to:
#define MAX_READ_AT_ONCE 524288 /* max to read() in one go */
#define MAX_WRITE_AT_ONCE 524288 /* max to write() in one go */
// ...
read(fd, buf,
count >
MAX_READ_AT_ONCE ? MAX_READ_AT_ONCE : count);
Why do these limits exisst?
Is there a point to allowing a --buffer-size 16M when the reads and writes are capped to small buffers anyway?
Could this restriction be lifted, removing the hardcoded limits, so that these syscalls can be as large as the --buffer size?
Thanks
I tried to tell
pvto make largeread()/write()sycalls (it is much more efficient to make large syscalls on e.g. networked file systems) using--buffer-size 16M.However, I found that it had no effect;
pvwas making512Ki-sized IO operations.This is due to:
Why do these limits exisst?
Is there a point to allowing a
--buffer-size 16Mwhen the reads and writes are capped to small buffers anyway?Could this restriction be lifted, removing the hardcoded limits, so that these syscalls can be as large as the
--buffersize?Thanks