mirrors/cpython
mirrors/cpython
0
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2024-11-21 21:09:37 +01:00
Code Issues Releases Activity

[3.13] bpo-34206: Improve docs and test coverage for pre-init functions (GH-8023) (#125092)

Browse Source 
- move the Py_Main documentation from the very high level API section
  to the initialization and finalization section
- make it clear that it encapsulates a full Py_Initialize/Finalize
  cycle of its own
- point out that exactly which settings will be read and applied
  correctly when Py_Main is called after a separate runtime
  initialization call is version dependent
- be explicit that Py_IsInitialized can be called prior to
  initialization
- actually test that Py_IsInitialized can be called prior to
  initialization
- flush stdout in the embedding tests that run code so it appears
  in the expected order when running with "-vv"
- make "-vv" on the subinterpreter embedding tests less spammy

---------

(cherry picked from commit 7c4b6a68f2)

Co-authored-by: Carol Willing <carolcode@willingconsulting.com>
This commit is contained in:
Alyssa Coghlan 2024-10-08 19:30:06 +10:00 committed by GitHub
parent 9e61aa9b38
commit 64391363ed
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
7 changed files with 198 additions and 77 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

160
Doc/c-api/init.rst
View File

@ -7,7 +7,8 @@
Initialization, Finalization, and Threads
*****************************************
See also :ref:`Python Initialization Configuration <init-config>`.
See :ref:`Python Initialization Configuration <init-config>` for details
on how to configure the interpreter prior to initialization.
.. _pre-init-safe:
@ -21,6 +22,15 @@ a few functions and the :ref:`global configuration variables
The following functions can be safely called before Python is initialized:
* Functions that initialize the interpreter:
* :c:func:`Py_Initialize`
* :c:func:`Py_InitializeEx`
* :c:func:`Py_InitializeFromConfig`
* :c:func:`Py_BytesMain`
* :c:func:`Py_Main`
* the runtime pre-initialization functions covered in :ref:`init-config`
* Configuration functions:
* :c:func:`PyImport_AppendInittab`
@ -32,6 +42,7 @@ The following functions can be safely called before Python is initialized:
* :c:func:`Py_SetProgramName`
* :c:func:`Py_SetPythonHome`
* :c:func:`PySys_ResetWarnOptions`
* the configuration functions covered in :ref:`init-config`
* Informative functions:
@ -43,10 +54,12 @@ The following functions can be safely called before Python is initialized:
* :c:func:`Py_GetCopyright`
* :c:func:`Py_GetPlatform`
* :c:func:`Py_GetVersion`
* :c:func:`Py_IsInitialized`
* Utilities:
* :c:func:`Py_DecodeLocale`
* the status reporting and utility functions covered in :ref:`init-config`
* Memory allocators:
@ -62,11 +75,13 @@ The following functions can be safely called before Python is initialized:
.. note::
The following functions **should not be called** before
:c:func:`Py_Initialize`: :c:func:`Py_EncodeLocale`, :c:func:`Py_GetPath`,
Despite their apparent similarity to some of the functions listed above,
the following functions **should not be called** before the interpreter has
been initialized: :c:func:`Py_EncodeLocale`, :c:func:`Py_GetPath`,
:c:func:`Py_GetPrefix`, :c:func:`Py_GetExecPrefix`,
:c:func:`Py_GetProgramFullPath`, :c:func:`Py_GetPythonHome`,
:c:func:`Py_GetProgramName` and :c:func:`PyEval_InitThreads`.
:c:func:`Py_GetProgramName`, :c:func:`PyEval_InitThreads`, and
:c:func:`Py_RunMain`.
.. _global-conf-vars:
@ -346,34 +361,42 @@ Initializing and finalizing the interpreter
this should be called before using any other Python/C API functions; see
:ref:`Before Python Initialization <pre-init-safe>` for the few exceptions.
This initializes
the table of loaded modules (``sys.modules``), and creates the fundamental
modules :mod:`builtins`, :mod:`__main__` and :mod:`sys`. It also initializes
the module search path (``sys.path``). It does not set ``sys.argv``; use
the new :c:type:`PyConfig` API of the :ref:`Python Initialization
Configuration <init-config>` for that. This is a no-op when called for a
second time
(without calling :c:func:`Py_FinalizeEx` first). There is no return value; it is a
fatal error if the initialization fails.
This initializes the table of loaded modules (``sys.modules``), and creates
the fundamental modules :mod:`builtins`, :mod:`__main__` and :mod:`sys`.
It also initializes the module search path (``sys.path``). It does not set
``sys.argv``; use the :ref:`Python Initialization Configuration <init-config>`
API for that. This is a no-op when called for a second time (without calling
:c:func:`Py_FinalizeEx` first). There is no return value; it is a fatal
error if the initialization fails.
Use the :c:func:`Py_InitializeFromConfig` function to customize the
Use :c:func:`Py_InitializeFromConfig` to customize the
:ref:`Python Initialization Configuration <init-config>`.
.. note::
On Windows, changes the console mode from ``O_TEXT`` to ``O_BINARY``, which will
also affect non-Python uses of the console using the C Runtime.
On Windows, changes the console mode from ``O_TEXT`` to ``O_BINARY``,
which will also affect non-Python uses of the console using the C Runtime.
.. c:function:: void Py_InitializeEx(int initsigs)
This function works like :c:func:`Py_Initialize` if *initsigs* is ``1``. If
*initsigs* is ``0``, it skips initialization registration of signal handlers, which
might be useful when Python is embedded.
*initsigs* is ``0``, it skips initialization registration of signal handlers,
which may be useful when CPython is embedded as part of a larger application.
Use the :c:func:`Py_InitializeFromConfig` function to customize the
Use :c:func:`Py_InitializeFromConfig` to customize the
:ref:`Python Initialization Configuration <init-config>`.
.. c:function:: PyStatus Py_InitializeFromConfig(const PyConfig *config)
Initialize Python from *config* configuration, as described in
:ref:`init-from-config`.
See the :ref:`init-config` section for details on pre-initializing the
interpreter, populating the runtime configuration structure, and querying
the returned status structure.
.. c:function:: int Py_IsInitialized()
Return true (nonzero) when the Python interpreter has been initialized, false
@ -430,12 +453,111 @@ Initializing and finalizing the interpreter
.. versionadded:: 3.6
.. c:function:: void Py_Finalize()
This is a backwards-compatible version of :c:func:`Py_FinalizeEx` that
disregards the return value.
.. c:function:: int Py_BytesMain(int argc, char **argv)
Similar to :c:func:`Py_Main` but *argv* is an array of bytes strings,
allowing the calling application to delegate the text decoding step to
the CPython runtime.
.. versionadded:: 3.8
.. c:function:: int Py_Main(int argc, wchar_t **argv)
The main program for the standard interpreter, encapsulating a full
initialization/finalization cycle, as well as additional
behaviour to implement reading configurations settings from the environment
and command line, and then executing ``__main__`` in accordance with
:ref:`using-on-cmdline`.
This is made available for programs which wish to support the full CPython
command line interface, rather than just embedding a Python runtime in a
larger application.
The *argc* and *argv* parameters are similar to those which are passed to a
C program's :c:func:`main` function, except that the *argv* entries are first
converted to ``wchar_t`` using :c:func:`Py_DecodeLocale`. It is also
important to note that the argument list entries may be modified to point to
strings other than those passed in (however, the contents of the strings
pointed to by the argument list are not modified).
The return value will be ``0`` if the interpreter exits normally (i.e.,
without an exception), ``1`` if the interpreter exits due to an exception,
or ``2`` if the argument list does not represent a valid Python command
line.
Note that if an otherwise unhandled :exc:`SystemExit` is raised, this
function will not return ``1``, but exit the process, as long as
``Py_InspectFlag`` is not set. If ``Py_InspectFlag`` is set, execution will
drop into the interactive Python prompt, at which point a second otherwise
unhandled :exc:`SystemExit` will still exit the process, while any other
means of exiting will set the return value as described above.
In terms of the CPython runtime configuration APIs documented in the
:ref:`runtime configuration <init-config>` section (and without accounting
for error handling), ``Py_Main`` is approximately equivalent to::
PyConfig config;
PyConfig_InitPythonConfig(&config);
PyConfig_SetArgv(&config, argc, argv);
Py_InitializeFromConfig(&config);
PyConfig_Clear(&config);
Py_RunMain();
In normal usage, an embedding application will call this function
*instead* of calling :c:func:`Py_Initialize`, :c:func:`Py_InitializeEx` or
:c:func:`Py_InitializeFromConfig` directly, and all settings will be applied
as described elsewhere in this documentation. If this function is instead
called *after* a preceding runtime initialization API call, then exactly
which environmental and command line configuration settings will be updated
is version dependent (as it depends on which settings correctly support
being modified after they have already been set once when the runtime was
first initialized).
.. c:function:: int Py_RunMain(void)
Executes the main module in a fully configured CPython runtime.
Executes the command (:c:member:`PyConfig.run_command`), the script
(:c:member:`PyConfig.run_filename`) or the module
(:c:member:`PyConfig.run_module`) specified on the command line or in the
configuration. If none of these values are set, runs the interactive Python
prompt (REPL) using the ``__main__`` module's global namespace.
If :c:member:`PyConfig.inspect` is not set (the default), the return value
will be ``0`` if the interpreter exits normally (that is, without raising
an exception), or ``1`` if the interpreter exits due to an exception. If an
otherwise unhandled :exc:`SystemExit` is raised, the function will immediately
exit the process instead of returning ``1``.
If :c:member:`PyConfig.inspect` is set (such as when the :option:`-i` option
is used), rather than returning when the interpreter exits, execution will
instead resume in an interactive Python prompt (REPL) using the ``__main__``
module's global namespace. If the interpreter exited with an exception, it
is immediately raised in the REPL session. The function return value is
then determined by the way the *REPL session* terminates: returning ``0``
if the session terminates without raising an unhandled exception, exiting
immediately for an unhandled :exc:`SystemExit`, and returning ``1`` for
any other unhandled exception.
This function always finalizes the Python interpreter regardless of whether
it returns a value or immediately exits the process due to an unhandled
:exc:`SystemExit` exception.
See :ref:`Python Configuration <init-python-config>` for an example of a
customized Python that always runs in isolated mode using
:c:func:`Py_RunMain`.
Process-wide parameters
=======================

29
Doc/c-api/init_config.rst
View File

@ -1348,14 +1348,13 @@ the :option:`-X` command line option.
The ``show_alloc_count`` field has been removed.
.. _init-from-config:
Initialization with PyConfig
============================
Function to initialize Python:
.. c:function:: PyStatus Py_InitializeFromConfig(const PyConfig *config)
Initialize Python from *config* configuration.
Initializing the interpreter from a populated configuration struct is handled
by calling :c:func:`Py_InitializeFromConfig`.
The caller is responsible to handle exceptions (error or exit) using
:c:func:`PyStatus_Exception` and :c:func:`Py_ExitStatusException`.
@ -1585,26 +1584,6 @@ The ``__PYVENV_LAUNCHER__`` environment variable is used to set
:c:member:`PyConfig.base_executable`.
Py_RunMain()
============
.. c:function:: int Py_RunMain(void)
Execute the command (:c:member:`PyConfig.run_command`), the script
(:c:member:`PyConfig.run_filename`) or the module
(:c:member:`PyConfig.run_module`) specified on the command line or in the
configuration.
By default and when if :option:`-i` option is used, run the REPL.
Finally, finalizes Python and returns an exit status that can be passed to
the ``exit()`` function.
See :ref:`Python Configuration <init-python-config>` for an example of
customized Python always running in isolated mode using
:c:func:`Py_RunMain`.
Py_GetArgcArgv()
================

24
Doc/c-api/veryhigh.rst
View File

@ -25,30 +25,6 @@ are only passed to these functions if it is certain that they were created by
the same library that the Python runtime is using.
.. c:function:: int Py_Main(int argc, wchar_t **argv)
The main program for the standard interpreter. This is made available for
programs which embed Python. The *argc* and *argv* parameters should be
prepared exactly as those which are passed to a C program's :c:func:`main`
function (converted to wchar_t according to the user's locale). It is
important to note that the argument list may be modified (but the contents of
the strings pointed to by the argument list are not). The return value will
be ``0`` if the interpreter exits normally (i.e., without an exception),
``1`` if the interpreter exits due to an exception, or ``2`` if the parameter
list does not represent a valid Python command line.
Note that if an otherwise unhandled :exc:`SystemExit` is raised, this
function will not return ``1``, but exit the process, as long as
:c:member:`PyConfig.inspect` is zero.
.. c:function:: int Py_BytesMain(int argc, char **argv)
Similar to :c:func:`Py_Main` but *argv* is an array of bytes strings.
.. versionadded:: 3.8
.. c:function:: int PyRun_AnyFile(FILE *fp, const char *filename)
This is a simplified interface to :c:func:`PyRun_AnyFileExFlags` below, leaving

11
Lib/test/test_embed.py
View File

@ -168,7 +168,8 @@ class EmbeddingTestsMixin:
# Parse the line from the loop. The first line is the main
# interpreter and the 3 afterward are subinterpreters.
interp = Interp(*match.groups())
if support.verbose > 1:
if support.verbose > 2:
# 5 lines per pass is super-spammy, so limit that to -vvv
print(interp)
self.assertTrue(interp.interp)
self.assertTrue(interp.tstate)
@ -279,6 +280,10 @@ class EmbeddingTests(EmbeddingTestsMixin, unittest.TestCase):
"""
env = dict(os.environ, PYTHONPATH=os.pathsep.join(sys.path))
out, err = self.run_embedded_interpreter("test_pre_initialization_api", env=env)
if support.verbose > 1:
print()
print(out)
print(err)
if MS_WINDOWS:
expected_path = self.test_exe
else:
@ -296,6 +301,10 @@ class EmbeddingTests(EmbeddingTestsMixin, unittest.TestCase):
env['PYTHONPATH'] = os.pathsep.join(sys.path)
out, err = self.run_embedded_interpreter(
"test_pre_initialization_sys_options", env=env)
if support.verbose > 1:
print()
print(out)
print(err)
expected_output = (
"sys.warnoptions: ['once', 'module', 'default']\n"
"sys._xoptions: {'not_an_option': '1', 'also_not_an_option': '2'}\n"

2
Misc/NEWS.d/next/C API/2018-06-30-21-48-16.bpo-34008.2Wjtm0.rst Normal file
View File

@ -0,0 +1,2 @@
Added ``Py_IsInitialized`` to the list of APIs that are safe to call before
the interpreter is initialized, and updated the embedding tests to cover it.

8
Misc/NEWS.d/next/Documentation/2018-07-04-20-35-25.bpo-34008.bqecIb.rst Normal file
View File

@ -0,0 +1,8 @@
The :c:func:`Py_Main` documentation moved from the "Very High Level API" section to the
"Initialization and Finalization" section.
Also make it explicit that we expect ``Py_Main`` to typically be called instead
of ``Py_Initialize`` rather than after it (since ``Py_Main`` makes its own
call to ``Py_Initialize``). Document that calling both is
supported but is version dependent on which settings
will be applied correctly.

41
Programs/_testembed.c
View File

@ -310,14 +310,36 @@ static int test_pre_initialization_api(void)
_Py_EMBED_PREINIT_CHECK("Checking Py_SetProgramName\n");
Py_SetProgramName(program);
_Py_EMBED_PREINIT_CHECK("Checking !Py_IsInitialized pre-initialization\n");
if (Py_IsInitialized()) {
fprintf(stderr, "Fatal error: initialized before initialization!\n");
return 1;
}
_Py_EMBED_PREINIT_CHECK("Initializing interpreter\n");
Py_Initialize();
_Py_EMBED_PREINIT_CHECK("Checking Py_IsInitialized post-initialization\n");
if (!Py_IsInitialized()) {
fprintf(stderr, "Fatal error: not initialized after initialization!\n");
return 1;
}
_Py_EMBED_PREINIT_CHECK("Check sys module contents\n");
PyRun_SimpleString("import sys; "
"print('sys.executable:', sys.executable)");
PyRun_SimpleString(
"import sys; "
"print('sys.executable:', sys.executable); "
"sys.stdout.flush(); "
);
_Py_EMBED_PREINIT_CHECK("Finalizing interpreter\n");
Py_Finalize();
_Py_EMBED_PREINIT_CHECK("Checking !Py_IsInitialized post-finalization\n");
if (Py_IsInitialized()) {
fprintf(stderr, "Fatal error: still initialized after finalization!\n");
return 1;
}
_Py_EMBED_PREINIT_CHECK("Freeing memory allocated by Py_DecodeLocale\n");
PyMem_RawFree(program);
return 0;
@ -363,12 +385,15 @@ static int test_pre_initialization_sys_options(void)
_Py_EMBED_PREINIT_CHECK("Initializing interpreter\n");
_testembed_Py_InitializeFromConfig();
_Py_EMBED_PREINIT_CHECK("Check sys module contents\n");
PyRun_SimpleString("import sys; "
"print('sys.warnoptions:', sys.warnoptions); "
"print('sys._xoptions:', sys._xoptions); "
"warnings = sys.modules['warnings']; "
"latest_filters = [f[0] for f in warnings.filters[:3]]; "
"print('warnings.filters[:3]:', latest_filters)");
PyRun_SimpleString(
"import sys; "
"print('sys.warnoptions:', sys.warnoptions); "
"print('sys._xoptions:', sys._xoptions); "
"warnings = sys.modules['warnings']; "
"latest_filters = [f[0] for f in warnings.filters[:3]]; "
"print('warnings.filters[:3]:', latest_filters); "
"sys.stdout.flush(); "
);
_Py_EMBED_PREINIT_CHECK("Finalizing interpreter\n");
Py_Finalize();