Joshua Herman 9e011e7c77 gh-82054: allow test runner to split test_asyncio to execute in parallel by sharding. (#103927) ... This runs test_asyncio sub-tests in parallel using sharding from Cinder. This suite is typically the longest-pole in runs because it is a test package with a lot of further sub-tests otherwise run serially. By breaking out the sub-tests as independent modules we can run a lot more in parallel. After porting we can see the direct impact on a multicore system. Without this change: Running make test is 5 min 26 seconds With this change: Running make test takes 3 min 39 seconds That'll vary based on system and parallelism. On a `-j 4` run similar to what CI and buildbot systems often do, it reduced the overall test suite completion latency by 10%. The drawbacks are that this implementation is hacky and due to the sorting of the tests it obscures when the asyncio tests occur and involves changing CPython test infrastructure but, the wall time saved it is worth it, especially in low-core count CI runs as it pulls a long tail. The win for productivity and reserved CI resource usage is significant. Future tests that deserve to be refactored into split up suites to benefit from are test_concurrent_futures and the way the _test_multiprocessing suite gets run for all start methods. As exposed by passing the -o flag to python -m test to get a list of the 10 longest running tests. --------- Co-authored-by: Carl Meyer <carl@oddbird.net> Co-authored-by: Gregory P. Smith <greg@krypto.org> [Google, LLC]		2023-04-29 18:26:24 -07:00
..
__phello__
asyncio	gh-83925: Make asyncio.subprocess communicate similar to non-asyncio (#18650)	2023-04-27 17:30:26 -07:00
collections	gh-103092: Isolate _collections (#103093)	2023-04-12 18:21:28 +05:30
concurrent	gh-94440: Fix issue of ProcessPoolExecutor shutdown hanging (#94468)	2023-03-15 21:44:52 -07:00
ctypes	Fix typo in _swappedbytes_ in ctypes comment (#102773)	2023-03-25 16:38:24 -07:00
curses	gh-60436: fix curses textbox backspace/del (#103783)	2023-04-26 22:54:07 +02:00
dbm
email	GH-103857: Deprecate utcnow and utcfromtimestamp (#103858)	2023-04-27 11:32:30 -06:00
encodings
ensurepip	gh-103872: Update bundled pip version to 23.1.2 (gh-103873)	2023-04-26 10:22:25 +00:00
html
http	GH-103857: Deprecate utcnow and utcfromtimestamp (#103858)	2023-04-27 11:32:30 -06:00
idlelib	Change 'dependant' to 'dependent' (#103745)	2023-04-24 09:40:30 -04:00
importlib	gh-87729: specialize LOAD_SUPER_ATTR_METHOD (#103809)	2023-04-25 17:45:51 +00:00
json
lib2to3
logging	gh-103357: Add logging.Formatter defaults support to logging.config fileConfig and dictConfig (GH-103359)	2023-04-12 08:35:56 +01:00
msilib
multiprocessing
pydoc_data	Python 3.12.0a7	2023-04-04 17:52:42 +02:00
re
site-packages
sqlite3	gh-102628: Fix sqlite3 CLI prompt in IDLE on Windows (#103945)	2023-04-27 21:22:26 +00:00
test	gh-82054: allow test runner to split test_asyncio to execute in parallel by sharding. (#103927)	2023-04-29 18:26:24 -07:00
tkinter	gh-103685: Fix tkinter.Menu.index() for Tk 8.7 (#103686)	2023-04-23 21:31:44 -04:00
tomllib
turtledemo
unittest	gh-62432: unittest runner: Exit code 5 if no tests were run (#102051)	2023-04-27 01:28:46 +00:00
urllib	gh-81403: Fix for CacheFTPHandler in urllib (#13951)	2023-04-22 21:41:23 -07:00
venv	gh-95299: Stop installing setuptools as a part of ensurepip and venv (#101039)	2023-04-17 23:43:34 -05:00
wsgiref
xml
xmlrpc
zipfile	gh-86094: Add support for Unicode Path Extra Field in ZipFile (gh-102566)	2023-04-05 20:54:48 +09:00
zoneinfo
__future__.py
__hello__.py
_aix_support.py
_collections_abc.py	gh-102721: Improve coverage of _collections_abc._CallableGenericAlias (#102722)	2023-03-16 14:47:30 +00:00
_compat_pickle.py
_compression.py
_markupbase.py
_osx_support.py
_py_abc.py
_pydecimal.py
_pyio.py
_pylong.py
_sitebuiltins.py
_strptime.py	GH-90750: Use datetime.fromisocalendar in _strptime (#103802)	2023-04-27 10:27:27 -06:00
_threading_local.py
_weakrefset.py
abc.py	gh-87864: Use correct function definition syntax in the docs (#103312)	2023-04-11 16:50:25 +03:00
aifc.py
antigravity.py
argparse.py
ast.py	gh-103285: Rewrite _splitlines_no_ff to improve performance (#103307)	2023-04-23 23:03:49 -06:00
base64.py	GH-102456: Fix docstring and getopt options for base64 (gh-102457)	2023-04-02 14:05:50 +09:00
bdb.py	gh-101517: fix line number propagation in code generated for except* (#103550)	2023-04-24 21:58:51 +01:00
bisect.py	GH-102833: Mention the key function in the docstrings (GH-103009)	2023-03-25 02:19:20 -05:00
bz2.py
calendar.py	gh-103636: issue warning for deprecated calendar constants (#103833)	2023-04-29 01:16:46 -06:00
cgi.py
cgitb.py
chunk.py
cmd.py
code.py	gh-102778: Add sys.last_exc, deprecate sys.last_type, sys.last_value,sys.last_traceback (#102779)	2023-03-18 11:47:11 +00:00
codecs.py
codeop.py
colorsys.py
compileall.py
configparser.py
contextlib.py	gh-103791: Make contextlib.suppress also act on exceptions within an ExceptionGroup (#103792)	2023-04-24 22:17:02 +00:00
contextvars.py
copy.py
copyreg.py
cProfile.py	gh-103935: Use io.open_code() when executing code in trace and profile modules (GH-103947)	2023-04-27 20:29:35 +00:00
crypt.py
csv.py	gh-67230: add quoting rules to csv module (GH-29469)	2023-04-12 15:32:30 -07:00
dataclasses.py	gh-103449: Fix a bug in dataclass docstring generation (#103454)	2023-04-17 17:33:22 -06:00
datetime.py	GH-103857: Deprecate utcnow and utcfromtimestamp (#103858)	2023-04-27 11:32:30 -06:00
decimal.py
difflib.py
dis.py	GH-99944: Make dis display the value of oparg of KW_NAMES (#103856)	2023-04-26 19:00:36 +01:00
doctest.py
enum.py	gh-103596: [Enum] do not shadow mixed-in methods/attributes (GH-103600)	2023-04-18 16:19:23 -07:00
filecmp.py
fileinput.py
fnmatch.py
fractions.py
ftplib.py
functools.py
genericpath.py
getopt.py
getpass.py
gettext.py
glob.py
graphlib.py
gzip.py
hashlib.py
heapq.py
hmac.py
imaplib.py
imghdr.py
inspect.py	gh-103556: [inspect.Signature] disallow pos-or-kw params without default after pos-only with default (#103557)	2023-04-21 23:45:10 -06:00
io.py
ipaddress.py	ipaddress: Remove non-existent ip_str param from docstring (#103461)	2023-04-14 00:44:10 -07:00
keyword.py
linecache.py
locale.py	gh-74940: Allow fallback to UTF-8 encoding on systems with no locales installed (GH-14925)	2023-04-26 15:33:09 +09:00
lzma.py
mailbox.py
mailcap.py
mimetypes.py
modulefinder.py
netrc.py
nntplib.py
ntpath.py	GH-103220: Fix ntpath.join() of partial UNC drive with trailing slash (GH-103221)	2023-04-11 17:26:45 +01:00
nturl2path.py
numbers.py
opcode.py	gh-87729: specialize LOAD_SUPER_ATTR_METHOD (#103809)	2023-04-25 17:45:51 +00:00
operator.py
optparse.py
os.py
pathlib.py	GH-78079: Fix UNC device path root normalization in pathlib (GH-102003)	2023-04-14 21:55:41 +01:00
pdb.py	gh-103578: Fix pdb reading code with non-utf8 encoding (#103581)	2023-04-25 23:04:51 -06:00
pickle.py	gh-102799: use exception instance instead of sys.exc_info() (#102885)	2023-03-31 11:23:02 +01:00
pickletools.py
pipes.py
pkgutil.py	gh-98040: Remove just the imp module (#98573)	2023-04-28 16:17:58 -07:00
platform.py	gh-102491: Remove IronPython version check in sys_version (#102492)	2023-03-19 15:52:47 -07:00
plistlib.py
poplib.py
posixpath.py
pprint.py
profile.py	gh-103935: Use io.open_code() when executing code in trace and profile modules (GH-103947)	2023-04-27 20:29:35 +00:00
pstats.py
pty.py
py_compile.py
pyclbr.py
pydoc.py	gh-98040: Remove just the imp module (#98573)	2023-04-28 16:17:58 -07:00
queue.py
quopri.py
random.py	Move binomialvariate() to a section for discrete distributions (GH-102955)	2023-03-23 12:10:12 -05:00
reprlib.py
rlcompleter.py
runpy.py	gh-98040: Remove just the imp module (#98573)	2023-04-28 16:17:58 -07:00
sched.py
secrets.py
selectors.py
shelve.py
shlex.py
shutil.py	gh-82814: fix shutil access error on WSL (#103790)	2023-04-25 00:45:38 +00:00
signal.py
site.py	gh-102038: Skip a sometimes unnecessary stat in site.py (#102039)	2023-04-02 15:47:31 -07:00
smtplib.py
sndhdr.py
socket.py
socketserver.py	gh-103673: Add missing ForkingUnixStreamServer and ForkingUnixDatagramServer socketservers (#103674)	2023-04-24 22:35:52 +00:00
sre_compile.py
sre_constants.py
sre_parse.py
ssl.py
stat.py
statistics.py
string.py
stringprep.py
struct.py
subprocess.py
sunau.py
symtable.py
sysconfig.py
tabnanny.py
tarfile.py	gh-102950: Implement PEP 706 – Filter for tarfile.extractall (#102953)	2023-04-24 10:58:06 +02:00
telnetlib.py
tempfile.py	gh-51574: Make tempfile.mkdtemp() always return absolute paths (#94612)	2023-04-25 16:05:59 +00:00
textwrap.py
this.py
threading.py
timeit.py
token.py	gh-102856: Initial implementation of PEP 701 (#102855)	2023-04-19 11:18:16 -05:00
tokenize.py
trace.py	gh-103935: Use io.open_code() when executing code in trace and profile modules (GH-103947)	2023-04-27 20:29:35 +00:00
traceback.py	gh-102778: Add sys.last_exc, deprecate sys.last_type, sys.last_value,sys.last_traceback (#102779)	2023-03-18 11:47:11 +00:00
tracemalloc.py
tty.py
turtle.py
types.py	gh-101688: Implement types.get_original_bases (#101827)	2023-04-23 20:24:30 +01:00
typing.py	GH-103699: Add __orig_bases__ to various typing classes (#103698)	2023-04-23 08:33:39 -06:00
uu.py
uuid.py	Update MAC_address wikipedia URL in comments (#29019)	2023-04-26 14:27:03 -06:00
warnings.py
wave.py
weakref.py
webbrowser.py	bpo-44844: Enable detection of Microsoft Edge browser in webbrowser module (GH-29908)	2023-04-04 17:00:03 +01:00
xdrlib.py
zipapp.py
zipimport.py