0
0
mirror of https://github.com/python/cpython.git synced 2024-11-25 09:39:56 +01:00
cpython/Misc
morotti 6efd95c465
gh-117151: increase default buffer size of shutil.copyfileobj() to 256k. (GH-119783)
* gh-117151: increase default buffer size of shutil.copyfileobj() to 256k.

it was set to 16k in the 1990s.
it was raised to 64k in 2019. the discussion at the time mentioned another 5% improvement by raising to 128k and settled for a very conservative setting.

it's 2024 now, I think it should be revisited to match modern hardware. I am measuring 0-15% performance improvement when raising to 256k on various types of disk. there is no downside as far as I can tell.

this function is only intended for sequential copy of full files (or file like objects). it's the typical use case that benefits from larger operations.

for reference, I came across this function while trying to profile pip that is using it to copy files when installing python packages.

* add news

---------

Co-authored-by: rmorotti <romain.morotti@man.com>
2024-10-04 16:51:22 -07:00
..
NEWS.d gh-117151: increase default buffer size of shutil.copyfileobj() to 256k. (GH-119783) 2024-10-04 16:51:22 -07:00
rhel7
ACKS gh-124609: Fix _Py_ThreadId for Windows builds using MinGW (#124663) 2024-09-27 18:52:23 +00:00
externals.spdx.json gh-124448: Update Windows builds to use Tcl/Tk 8.6.15 (GH-124449) 2024-09-25 21:00:25 +00:00
HISTORY gh-123492: Remove unnecessary :func: parentheses (#123493) 2024-08-30 14:34:09 +03:00
indent.pro
platform_triplet.c
Porting
python-config.in
python-config.sh.in
python-embed.pc.in
python.man
python.pc.in
README gh-124457: Remove coverity from CPython repo (GH-124460) 2024-09-27 13:42:32 -07:00
README.AIX
README.valgrind
sbom.spdx.json gh-123748: Add conditional compilation rules for HACL SIMD256 and SIMD128 on macOS (#123989) 2024-09-16 12:23:05 +08:00
SpecialBuilds.txt
stable_abi.toml gh-124127: Make Py_REFCNT() opaque in limited C API 3.14 (#124128) 2024-09-24 08:42:58 +02:00
svnmap.txt
valgrind-python.supp
vgrindefs

Python Misc subdirectory
========================

This directory contains files that wouldn't fit in elsewhere.  Some
documents are only of historic importance.

Files found here
----------------

ACKS                    Acknowledgements
HISTORY                 News from previous releases -- oldest last
indent.pro              GNU indent profile approximating my C style
NEWS                    News for this release (for some meaning of "this")
Porting                 Mini-FAQ on porting to new platforms
python-config.in        Python script template for python-config
python.man              UNIX man page for the python interpreter
python.pc.in            Package configuration info template for pkg-config
README                  The file you're reading now
README.AIX              Information about using Python on AIX
README.valgrind         Information for Valgrind users, see valgrind-python.supp
SpecialBuilds.txt       Describes extra symbols you can set for debug builds
svnmap.txt              Map of old SVN revs and branches to hg changeset ids,
                        help history-digging
valgrind-python.supp    Valgrind suppression file, see README.valgrind
vgrindefs               Python configuration for vgrind (a generic pretty printer)