* ASCIIbetize directives
* Merge duplicate directives from 'conditions'
PR-URL: https://github.com/nodejs/node/pull/25931
Reviewed-By: João Reis <reis@janeasystems.com>
Adds a `repl.setupHistory()` instance method so that
programmatic REPLs can also write history to a file.
This change also refactors all of the history file
management to `lib/internal/repl/history.js`, cleaning
up and simplifying `lib/internal/repl.js`.
PR-URL: https://github.com/nodejs/node/pull/25895
Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com>
acorn and acorn-walk are now published as two different packages. Put
them both in subdirectories of `deps/acorn`.
Adapt the REPL's recoverable error detection to use the new API for
extending acorn parsers.
PR-URL: https://github.com/nodejs/node/pull/25844
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
For use in built-in modules that could benefit from `assert()` without
having to load the entire module (unless an AssertionError actually
occurs): lib/internal/assert.js.
PR-URL: https://github.com/nodejs/node/pull/25956
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Minwoo Jung <minwoo@nodesource.com>
Move lib/internal/assert.js to lib/internal/assert/assertion_error.js.
This is in preparation for making lib/internal/assert.js a tiny module
for use in Node.js built-ins so that we can use `assert()` without
having to load the entire ~1200 line `assert` module.
PR-URL: https://github.com/nodejs/node/pull/25956
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Minwoo Jung <minwoo@nodesource.com>
This patches changes the `safe_globals` internal module into a
script that gets run during bootstrap and saves JavaScript builtins
(primordials) into an object that is available for all other builtin
modules to access lexically later.
PR-URL: https://github.com/nodejs/node/pull/25816
Refs: https://github.com/nodejs/node/issues/18795
Reviewed-By: Bradley Farias <bradley.meck@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Gus Caplan <me@gus.host>
This patch moves most of the public C++ APIs into src/api/*.cc
so that it's easier to tell that we need to be careful about
the compatibility of these code.
Some APIs, like `node::LoadEnvironmet()`, `node::Start()` and
`node::Init()` still stay in `node.cc` because they are still
very specific to our use cases and do not work quite well yet
for embedders anyway - we could not even manage to write cctest for
them at the moment.
PR-URL: https://github.com/nodejs/node/pull/25541
Reviewed-By: Gus Caplan <me@gus.host>
This patch splits the execution mode selection from the environment
setup in `lib/internal/bootstrap/node.js`, and split the entry point
of different execution mode into main scripts under
`lib/internal/main`:
- `check_syntax.js`: used when `-c`/`--check` which only checks the
syntax of the input instead of executing it.
- `eval_stdin.js`: used when `-e` is passed without value and stdin
is not a TTY (e.g. something is piped).
- `eval_string`: used when `-e` is passed along with a string argument
- `inspect.js`: for `node inspect`/`node debug`
- `print_bash_completion.js`: for `--completion-bash`
- `print_help.js`: for `--help`
- `prof_process.js`: for `--prof-process`
- `repl.js`: for the REPL
- `run_main_module.js`: used when a main module is passed
- `run_third_party_main.js`: for the legacy `_third_party_main.js`
support
- `worker_thread.js`: for workers
This makes the entry points easier to navigate and paves the way
for customized v8 snapshots (that do not need to deserialize
execution mode setup) and better embedder APIs.
As an example, after this patch, for the most common case where
Node.js executes a user module as an entry point, it essentially
goes through:
- `lib/internal/per_context.js` to setup the v8 Context (which is
also run when setting up contexts for the `vm` module)
- `lib/internal/bootstrap/loaders.js` to set up internal binding
and builtin module loaders (that are separate from the loaders
accessible in the user land).
- `lib/internal/bootstrap/node.js`: to set up the rest of the
environment, including various globals and the process object
- `lib/internal/main/run_main_module.js`: which is selected from
C++ to prepare execution of the user module.
This patch also removes `NativeModuleLoader::CompileAndCall` and
exposes `NativeModuleLoader::LookupAndCompile` directly so that
we can handle syntax errors and runtime errors of bootstrap
scripts differently.
PR-URL: https://github.com/nodejs/node/pull/25667
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Before this commit, Node.js left it up to the system resolver or c-ares.
Leaving it to the system resolver introduces platform differences
because:
* some support IDNA 2008
* some only IDNA 2003 (glibc until 2.28), and
* some don't support IDNA at all (musl libc)
c-ares doesn't support IDNA either although curl does, by virtue of
linking against libidn2. Upgrading from libidn1 to libidn2 in order
to get proper IDNA 2008 support was the fix for curl's CVE-2016-8625.
libidn2 is not an option (incompatible license) but ICU has an IDNA API
and we already use that in one place. For non-ICU builds, we fall back
to the bundled punycode.js that also supports IDNA 2008.
Fixes: https://github.com/nodejs-private/security/issues/97
Fixes: https://github.com/nodejs/node/issues/25558
PR-URL: https://github.com/nodejs/node/pull/25679
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Tiancheng "Timothy" Gu <timothygu99@gmail.com>
Previously only simple escape sequences were handled
(i.e. \n, \t, r etc.). This commit adds escaping of other control
symbols in the range of 0x00 to 0x20.
Also, this replaces multiple find+replace calls with a single pass
replacer.
PR-URL: https://github.com/nodejs/node/pull/25626
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
- Instead of creating the console extensions eagerly during bootstrap
and storing them on an object, wrap the code into a function to be
called during `installAdditionalCommandLineAPI` only when the
extensions are actually needed, which may not even happen if the
user do not use the console in an inspector session, and does not
need to happen during bootstrap unconditionally.
- Simplify the console methods wrapping and move the `consoleFromVM`
storage to `internal/util/inspector.js`
PR-URL: https://github.com/nodejs/node/pull/25450
Reviewed-By: John-David Dalton <john.david.dalton@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Make node-report part of core runtime because:
1. When enabled, node-report significantly helps root cause various
types of problems, including support issues sent to the various repos
of the Node.js organization.
2. The requirement of explicitly adding the dependency to node-report
in user applications often represents a blocker to adoption.
Major deviation from the module version of the node-report is that the
report is generated in JSON format, as opposed to human readable text.
No new functionalities have been added, changes that are required for
melding it as a built-in capability has been affected on the module
version of node-report (https://github.com/nodejs/node-report)
Co-authored-by: Bidisha Pyne <bidipyne@in.ibm.com>
Co-authored-by: Howard Hellyer <hhellyer@uk.ibm.com>
Co-authored-by: Jeremiah Senkpiel <fishrock123@rocketmail.com>
Co-authored-by: Julian Alimin <dmastag@yahoo.com>
Co-authored-by: Lakshmi Swetha Gopireddy <lakshmigopireddy@in.ibm.com>
Co-authored-by: Manusaporn Treerungroj <m.treerungroj@gmail.com>
Co-authored-by: Michael Dawson <michael_dawson@ca.ibm.com>
Co-authored-by: Richard Chamberlain <richard_chamberlain@uk.ibm.com>
Co-authored-by: Richard Lau <riclau@uk.ibm.com>
Co-authored-by: Sam Roberts <vieuxtech@gmail.com>
Co-authored-by: Vipin Menon <vipinmv1@in.ibm.com>
PR-URL: https://github.com/nodejs/node/pull/22712
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Michael Dawson <Michael_Dawson@ca.ibm.com>
Reviewed-By: Vse Mozhet Byt <vsemozhetbyt@gmail.com>
This enables code loaded via the module system to be checked for
integrity to ensure the code loaded matches expectations.
PR-URL: https://github.com/nodejs/node/pull/23834
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Vladimir de Turckheim <vlad2t@hotmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
- Renamed `internal/process/write-coverage.js` to
`internal/coverage-gen/with_instrumentation.js`,
`internal/process/coverage.js` to
`internal/coverage-gen/with_profiler.js` to distinguish
the two better and added comments.
- Separate the coverage directory setup and the connection
setup, moves the directory setup into `node.js` and
closer to the exit hooks because that's where it's used.
- Moves the `process.reallyExit` overwrite and
`process.on('exit')` hooks setup into bootstrap/node.js
for clarity, and move them to a later stage of
bootstrap since they do not have to happen that early.
PR-URL: https://github.com/nodejs/node/pull/25398
Reviewed-By: Ben Coe <bencoe@gmail.com>
Move the C++ `process.emit` and `process.emitWarning` methods
from `node.cc` into into `node_process_events.cc`, and
reuse the implementation in other places that need to do
`process.emit` in C++.
PR-URL: https://github.com/nodejs/node/pull/25397
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Instead of in node_internals.h. Also move process property
accessors that are not reused into node_process_object.cc
and make them static.
PR-URL: https://github.com/nodejs/node/pull/25397
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Changes `SetupProcessObject` to `CreateProessObject` which creates
the process object from scratch and return it to `Environment::Start`
to be stored in the Environment object.
PR-URL: https://github.com/nodejs/node/pull/25397
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Remove lib/internal/test/unicode.js and associated test. When we added
the file and test, only comments in lib had non-ASCII characters. Now,
lib/internal/cli_table.js has non-ASCII characters. Tests that exercise
the `console.table()` therefore fulfill the need to test non-ASCII
characters in built-in modules.
PR-URL: https://github.com/nodejs/node/pull/25298
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
- Move `setupProcessStdio` which contains write access to
the process object into `bootstrap/node.js`
- Move `MessagePort`, `MessageChannel`, `ReadableWorkerStdio`,
and `WritableWorkerStdio` into `internal/worker/io.js`
- Move more worker-specific bootstrap code into
`internal/process/worker_thread_only` from `setupChild`
in `internal/worker.js`, and move the `process._fatalException`
overwrite into `bootstrap/node.js` for clarity.
PR-URL: https://github.com/nodejs/node/pull/25199
Reviewed-By: James M Snell <jasnell@gmail.com>
This patch:
- Moves `tryGetCwd`, `evalScript` and `fatalException` from
`bootstrap/node.js` into `process/execution.js` so that
they do have to be passed into the worker thread
setup function, instead the worker code can require them
when necessary.
- Moves `setUncaughtExceptionCaptureCallback` and
`hasUncaughtExceptionCaptureCallback` along with the two
global state `exceptionHandlerState` and
`shouldAbortOnUncaughtToggle` info `process.execution.js`
as those are only used by the fatalException and these
two accessors as one self-contained unit.
PR-URL: https://github.com/nodejs/node/pull/25199
Reviewed-By: James M Snell <jasnell@gmail.com>
Its confusing to call a js class with a handle a "Wrap", usually it's
the C++ handle that is called a Wrap (tcp_wrap, tls_wrap, ...). Its
derived from Socket, and makes a JS stream look like a Socket, so call
it that. Also, remove use of lib/_stream_wrap.js so it can be deprecated
some time.
PR-URL: https://github.com/nodejs/node/pull/25153
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
This commit makes multiple important changes:
1. A new key object API is introduced. The KeyObject class itself is
not exposed to users, instead, several new APIs can be used to
construct key objects: createSecretKey, createPrivateKey and
createPublicKey. The new API also allows to convert between
different key formats, and even though the API itself is not
compatible to the WebCrypto standard in any way, it makes
interoperability much simpler.
2. Key objects can be used instead of the raw key material in all
relevant crypto APIs.
3. The handling of asymmetric keys has been unified and greatly
improved. Node.js now fully supports both PEM-encoded and
DER-encoded public and private keys.
4. Conversions between buffers and strings have been moved to native
code for sensitive data such as symmetric keys due to security
considerations such as zeroing temporary buffers.
5. For compatibility with older versions of the crypto API, this
change allows to specify Buffers and strings as the "passphrase"
option when reading or writing an encoded key. Note that this
can result in unexpected behavior if the password contains a
null byte.
PR-URL: https://github.com/nodejs/node/pull/24234
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
This patch:
- Moves the process.nextTick and promise setup C++ code into
node_task_queue.cc which is exposed as
`internalBinding('task_queue')`
- Makes `lib/internal/process/promises.js` and
`lib/internal/process/next_tick.js` as side-effect-free
as possible
- Removes the bootstrapper object being passed into
`bootstrap/node.js`, let `next_tick.js` and `promises.js`
load whatever they need from `internalBinding('task_queue')`
instead.
- Rename `process._tickCallback` to `runNextTicks` internally
for clarity but still expose it as `process._tickCallback`.
PR-URL: https://github.com/nodejs/node/pull/25163
Refs: https://github.com/nodejs/node/issues/24961
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
Expose the POSIX credential accessors through
`internalBinding('credentials')` instead of setting them on the
process or bootstrapper object from C++ directly. Also moves
`SafeGetEnv` from `internalBinding('util')` to
`internalBinding('credentials')` since it's closely related to
the credentials.
In the JS land, instead of wrapping the bindings then writing
to the process object directly in main_thread_only.js, return
the wrapped functions back to bootstrap/node.js where they get
written to the process object conditionally for clarity.
Refs: https://github.com/nodejs/node/issues/24961
PR-URL: https://github.com/nodejs/node/pull/25066
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Instead of exposing all the NamedPropertyHandlerConfiguration()
parameters in node_internals, simply expose a CreateEnvVarProxy()
method that returns a Local<Value> that implements process.env,
and mark all the property handlers static in node_env_var.cc.
This makes the code more encapsulated.
PR-URL: https://github.com/nodejs/node/pull/25067
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
This patch moves the computation of version metadata from node.cc
into node_metadata{.h, .cc}, and creates a macro that can be
used to iterate over the available version keys (v8, uv, .etc).
This makes the code clearer as now we no longer need to add
all the headers in node.cc just to compute the versions, and
makes it easier to reuse the version definitions.
PR-URL: https://github.com/nodejs/node/pull/24774
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Move the wrapping of the inspector console in a separate file
for clarity. In addition, save the original console from the
VM explicitly via an exported property
`require('internal/console/inspector').consoleFromVM`
that `require('inspector').console` can alias to it later,
instead of hanging the original console onto `per_thread.js`
during bootstrap and counting on that `per_thread.js`
only gets evaluated once and gets cached.
PR-URL: https://github.com/nodejs/node/pull/24709
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Since we do not actually use the Console constructor to
instantiate the global console, move the two piece of
code into two different JS files for clarity, and make
console.js a mere re-export of the global console.
The hope is to make the global console, a namespace, more
web-compatible while keeping the Console constructor
available for backwards compatibility.
PR-URL: https://github.com/nodejs/node/pull/24709
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
This patch:
- Moves the C++ binding/addon related code out of
node_internals.h/node.cc and into dedicated files
node_binding.h/node_binding.cc, and only puts the code resued
by other files into the header.
- Introduce a node::binding namespace so that code exposed to
other files can be easily recognized.
PR-URL: https://github.com/nodejs/node/pull/24701
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
PR-URL: https://github.com/nodejs/node/pull/24505
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com>
Reviewed-By: Fedor Indutny <fedor.indutny@gmail.com>
Instead of putting the source code and the cache in v8::Objects,
put them in per-process std::maps. This has the following benefits:
- It's slightly lighter in weight compared to storing things on the
v8 heap. Also it may be slightly faster since the preivous v8::Object
is already in dictionary mode - though the difference is very small
given the number of native modules is limited.
- The source and code cache generation templates are now much simpler
since they just initialize static arrays and manipulate STL
constructs.
- The static native module data can be accessed independently of any
Environment or Isolate, and it's easy to look them up from the
C++'s side.
- It's now impossible to mutate the source code used to compile
native modules from the JS land since it's completely separate
from the v8 heap. We can still get the constant strings from
process.binding('natives') but that's all.
A few drive-by fixes:
- Remove DecorateErrorStack in LookupAndCompile - We don't need to
capture the exception to decorate when we encounter
errors during native module compilation, as those errors should be
syntax errors and v8 is able to decorate them well. We use
CompileFunctionInContext so there is no need to worry about
wrappers either.
- The code cache could be rejected when node is started with v8 flags.
Instead of aborting in that case, simply keep a record in the
native_module_without_cache set.
- Refactor js2c.py a bit, reduce code duplication and inline Render()
to make the one-byte/two-byte special treatment easier to read.
PR-URL: https://github.com/nodejs/node/pull/24384
Fixes: https://github.com/Remove
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Franziska Hinkelmann <franziska.hinkelmann@gmail.com>
Reviewed-By: Tiancheng "Timothy" Gu <timothygu99@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
This patch refactors out a part of NativeModule.prototype.compile
(in JS land) into a C++ NativeModule class, this enables a
couple of possibilities:
1. By moving the code to the C++ land, we have more opportunity
to specialize the compilation process of the native modules
(e.g. compilation options, code cache) that is orthogonal to
how user land modules are compiled
2. We can reuse the code to compile bootstrappers and context
fixers and enable them to be compiled with the code cache later,
since they are not loaded by NativeModule in the JS land their
caching must be done in C++.
3. Since there is no need to pass the static data to JS for
compilation anymore, this enables us to use
(std::map<std::string, const char*>) in the generated
node_code_cache.cc and node_javascript.cc later, and scope
every actual access to the source of native modules to a
std::map lookup instead of a lookup on a v8::Object in
dictionary mode.
This patch also refactor the code cache generator and tests
a bit and trace the `withCodeCache` and `withoutCodeCache`
in a Set instead of an Array, and makes sure that all the cachable
builtins are tested.
PR-URL: https://github.com/nodejs/node/pull/24221
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
llhttp is modern, written in human-readable TypeScript, verifiable, and
is very easy to maintain.
See: https://github.com/indutny/llhttp
PR-URL: https://github.com/nodejs/node/pull/24059
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: Ujjwal Sharma <usharma1998@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
This commit moves a few header files that are listed among the C++
source files for consistency.
PR-URL: https://github.com/nodejs/node/pull/24124
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Matheus Marchini <mat@mmarchini.me>
Instead of calling into C++ each time we need to check the value
of a command line option, cache the option map in a new
`internal/options` module for faster access to the values in JS land.
PR-URL: https://github.com/nodejs/node/pull/24091
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Move the following code into a new node_errors.cc file and
declare them in node_errors.h for clarity and make it possible
to include them with node_errors.h.
- AppendExceptionLine()
- DecorateErrorStack()
- FatalError()
- OnFatalError()
- PrintErrorString()
- FatalException()
- ReportException()
- FatalTryCatch
And move the following definitions (declared elsewhere than
node_errors.h) to node_errors.cc:
- Abort() (in util.h)
- Assert() (in util.h)
PR-URL: https://github.com/nodejs/node/pull/24058
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com>
Those categories are necessary to build addons that depends
on libcurl and libssh, the following were the missing symbols:
libcurl:
OCSP_cert_status_str
OCSP_check_validity
OCSP_basic_verify
OCSP_RESPONSE_free
OCSP_single_get0_status
OCSP_response_get1_basic
OCSP_BASICRESP_free
OCSP_crl_reason_str
OCSP_resp_count
OCSP_response_status
OCSP_response_status_str
OCSP_resp_get0
d2i_OCSP_RESPONSE
SSL_CTX_set_next_proto_select_cb
libssh:
EVP_ripemd160
EVP_cast5_cbc
Fixes: https://github.com/nodejs/node/issues/23293
PR-URL: https://github.com/nodejs/node/pull/23344
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
