* Add _PyInitError functions:
* _PyInitError_Ok()
* _PyInitError_Error()
* _PyInitError_NoMemory()
* _PyInitError_Exit()
* _PyInitError_IsError()
* _PyInitError_IsExit()
* _PyInitError_Failed()
* frozenmain.c and _testembed.c now use functions rather than macros.
* Move _Py_INIT_xxx() macros to the internal API.
* Move _PyWstrList_INIT macro to the internal API.
Fix invalid function cast warnings with gcc 8
for method conventions different from METH_NOARGS, METH_O and
METH_VARARGS excluding Argument Clinic generated code.
Adds configure flags for msan and ubsan builds to make it easier to enable.
These also encode the detail that address sanitizer and memory sanitizer
should disable pymalloc.
Define MEMORY_SANITIZER when appropriate at build time and adds workarounds
to existing code to mark things as initialized where the sanitizer is otherwise unable to
determine that. This lets our build succeed under the memory sanitizer. not all tests
pass without sanitizer failures yet but we're in pretty good shape after this.
METH_NOARGS functions need only a single argument but they are cast
into a PyCFunction, which takes two arguments. This triggers an
invalid function cast warning in gcc8 due to the argument mismatch.
Fix this by adding a dummy unused argument.
* Fix multiple typos in code comments
* Add spacing in comments (test_logging.py, test_math.py)
* Fix spaces at the beginning of comments in test_logging.py
Parse more env vars in Py_Main():
* Add more options to _PyCoreConfig:
* faulthandler
* tracemalloc
* importtime
* Move code to parse environment variables from _Py_InitializeCore()
to Py_Main(). This change fixes a regression from Python 3.6:
PYTHONUNBUFFERED is now read before calling pymain_init_stdio().
* _PyFaulthandler_Init() and _PyTraceMalloc_Init() now take an
argument to decide if the module has to be enabled at startup.
* tracemalloc_start() is now responsible to check the maximum number
of frames.
Other changes:
* Cleanup Py_Main():
* Rename some pymain_xxx() subfunctions
* Add pymain_run_python() subfunction
* Cleanup Py_NewInterpreter()
* _PyInterpreterState_Enable() now reports failure
* init_hash_secret() now considers pyurandom() failure as an "user
error": don't fail with abort().
* pymain_optlist_append() and pymain_strdup() now sets err on memory
allocation failure.
* Don't use "Python runtime" anymore to parse command line options or
to get environment variables: pymain_init() is now a strict
separation.
* Use an error message rather than "crashing" directly with
Py_FatalError(). Limit the number of calls to Py_FatalError(). It
prepares the code to handle errors more nicely later.
* Warnings options (-W, PYTHONWARNINGS) and "XOptions" (-X) are now
only added to the sys module once Python core is properly
initialized.
* _PyMain is now the well identified owner of some important strings
like: warnings options, XOptions, and the "program name". The
program name string is now properly freed at exit.
pymain_free() is now responsible to free the "command" string.
* Rename most methods in Modules/main.c to use a "pymain_" prefix to
avoid conflits and ease debug.
* Replace _Py_CommandLineDetails_INIT with memset(0)
* Reorder a lot of code to fix the initialization ordering. For
example, initializing standard streams now comes before parsing
PYTHONWARNINGS.
* Py_Main() now handles errors when adding warnings options and
XOptions.
* Add _PyMem_GetDefaultRawAllocator() private function.
* Cleanup _PyMem_Initialize(): remove useless global constants: move
them into _PyMem_Initialize().
* Call _PyRuntime_Initialize() as soon as possible:
_PyRuntime_Initialize() now returns an error message on failure.
* Add _PyInitError structure and following macros:
* _Py_INIT_OK()
* _Py_INIT_ERR(msg)
* _Py_INIT_USER_ERR(msg): "user" error, don't abort() in that case
* _Py_INIT_FAILED(err)
kB (*kilo* byte) unit means 1000 bytes, whereas KiB ("kibibyte")
means 1024 bytes. KB was misused: replace kB or KB with KiB when
appropriate.
Same change for MB and GB which become MiB and GiB.
Change the output of Tools/iobench/iobench.py.
Round also the size of the documentation from 5.5 MB to 5 MiB.
Use the _PyTime_t type rather than double for the faulthandler
timeout in dump_traceback_later().
This change should fix the following Coverity warning:
CID 1420311: Incorrect expression (UNINTENDED_INTEGER_DIVISION)
Dividing integer expressions "9223372036854775807LL" and "1000LL",
and then converting the integer quotient to type "double". Any
remainder, or fractional part of the quotient, is ignored.
if ((timeout * 1e6) >= (double) PY_TIMEOUT_MAX) {
The warning comes from (double)PY_TIMEOUT_MAX with:
#define PY_TIMEOUT_MAX (PY_LLONG_MAX / 1000)
See PEP 539 for details.
Highlights of changes:
- Add Thread Specific Storage (TSS) API
- Document the Thread Local Storage (TLS) API as deprecated
- Update code that used TLS API to use TSS API
* bpo-30557: faulthandler now correctly filters and displays exception codes on Windows
* Adds test for non-fatal exceptions.
* Adds bpo number to comment.
* bpo-30125: Cleanup faulthandler.c
* Use size_t type for iterators
* Add { ... }
* bpo-30125: Fix faulthandler.disable() on Windows
On Windows, faulthandler.disable() now removes the exception handler
installed by faulthandler.enable().
* bpo-6532: Make the thread id an unsigned integer.
From C API side the type of results of PyThread_start_new_thread() and
PyThread_get_thread_ident(), the id parameter of
PyThreadState_SetAsyncExc(), and the thread_id field of PyThreadState
changed from "long" to "unsigned long".
* Restore a check in thread_get_ident().
Issue #23848, #26622:
* faulthandler now only logs fatal Windows exceptions.
* write error code as decimal, not as hexadecimal
* replace "Windows exception" with "Windows fatal exception"
Issue #23848: On Windows, faulthandler.enable() now also installs an exception
handler to dump the traceback of all Python threads on any Windows exception,
not only on UNIX signals (SIGSEGV, SIGFPE, SIGABRT).
Issue #26563:
* Add _PyGILState_GetInterpreterStateUnsafe() function: the single
PyInterpreterState used by this process' GILState implementation.
* Enhance _Py_DumpTracebackThreads() to retrieve the interpreter state from
autoInterpreterState in last resort. The function now accepts NULL for interp
and current_tstate parameters.
* test_faulthandler: fix a ResourceWarning when test is interrupted by CTRL+c
Issue #26154: Add a new private _PyThreadState_UncheckedGet() function which
gets the current thread state, but don't call Py_FatalError() if it is NULL.
Python 3.5.1 removed the _PyThreadState_Current symbol from the Python C API to
no more expose complex and private atomic types. Atomic types depends on the
compiler or can even depend on compiler options. The new function
_PyThreadState_UncheckedGet() allows to get the variable value without having
to care of the exact implementation of atomic types.
Changes:
* Replace direct usage of the _PyThreadState_Current variable with a call to
_PyThreadState_UncheckedGet().
* In pystate.c, replace direct usage of the _PyThreadState_Current variable
with the PyThreadState_GET() macro for readability.
* Document also PyThreadState_Get() in pystate.h
to its signal handlers.
Use also _Py_write_noraise() instead of write() to retry write() if it is
interrupted by a signal (fail with EINTR).
faulthandler.dump_traceback() also calls PyErr_CheckSignals() to call the
Python signal handler if a signal was received.
Issue #23654: Turn off ICC's tail call optimization for the stack_overflow
generator. ICC turns the recursive tail call into a loop.
Patch written by Matt Frank.
_read_null(), because _read_null() cannot be used on AIX. On AIX, reading from
NULL is allowed: the first page of memory is a mapped read-only on AIX.
_read_null() and _sigabrt() don't accept parameters.
- Use _testcapi.raise_signal() in test_signal
- close also os.pipe() file descriptors in some test_signal tests where they
were not closed properly
- Remove faulthandler._sigill() and faulthandler._sigbus(): reuse
_testcapi.raise_signal() in test_faulthandler
instead of creating temporary Unicode string objects
Add also more identifiers in pythonrun.c to avoid temporary Unicode string
objets for the interactive interpreter.
* Set the default value of all_threads arguments to True
* Py_FatalError() dumps all threads, instead of only the current thread
Dump only the current thread is not reliable. In some cases, Python is unable
to retrieve the state of the current thread and so is unable to dump the
traceback. faulthandler keeps a reference to the interpreter and so is always
able to dump the traceback of all threads.
faulthandler.enable(all_threads=True) dumps the tracebacks even if it is not
possible to get the state of the current thread
Create also the get_thread_state() subfunction to factorize the code.
* Write a new test to ensure that dump_tracebacks_later() still works if
it was already called and then cancelled before
* Don't use a variable to check the status of the thread, only rely on locks
* The thread only releases cancel_event if it was able to acquire it (if
the timer was interrupted)
* The main thread always hold this lock. It is only released when
faulthandler_thread() is interrupted until this thread exits, or at Python
exit.
The thread must not receive any signal. If the thread receives a signal,
sem_timedwait() is interrupted and returns EINTR, but in this case,
PyThread_acquire_lock_timed() retries sem_timedwait() and the main thread is
not aware of the signal. The problem is that some tests expect that the main
thread receives the signal, not faulthandler handler, which should be
invisible.
On Linux, the signal looks to be received by the main thread, whereas on
FreeBSD, it can be any thread.
* faulthandler_user() displays the tracebacks of all threads even if it is
unable to get the state of the current thread
* test_faulthandler: only release the GIL in test_gil_released() check
* create check_signum() subfunction
* faulthandler_cancel_dump_tracebacks_later() is responsible to set running
to zero (so we don't need the volatile keyword anymore)
* release locks if PyThread_start_new_thread() fails
assert(thread.running == 0) was wrong in a corner case
Always release the cancel join.
Fix also another corner case: _PyFaulthandler_Fini() called after setting
running variable to zero, but before releasing the join lock.
If the thread releases the join lock before the cancel lock, the thread may
sometimes still be alive at cancel_dump_tracebacks_later() exit. So the cancel
lock may be destroyed while the thread is still alive, whereas the thread will
try to release the cancel lock, which just crash.
Another minor fix: the thread doesn't release the cancel lock if it didn't
acquire it.
