Initial set of fs.promises APIs with documentation and one
benchmark.
PR-URL: https://github.com/nodejs/node/pull/18297
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Problem:
Node implements fs.readFile as:
- a call to stat, then
- a C++ -> libuv request to read the entire file using the stat size
Why is this bad?
The effect is to place on the libuv threadpool a potentially-large
read request, occupying the libuv thread until it completes.
While readFile certainly requires buffering the entire file contents,
it can partition the read into smaller buffers
(as is done on other read paths)
along the way to avoid threadpool exhaustion.
If the file is relatively large or stored on a slow medium, reading
the entire file in one shot seems particularly harmful,
and presents a possible DoS vector.
Solution:
Partition the read into multiple smaller requests.
Considerations:
1. Correctness
I don't think partitioning the read like this raises
any additional risk of read-write races on the FS.
If the application is concurrently readFile'ing and modifying the file,
it will already see funny behavior. Though libuv uses preadv where
available, this doesn't guarantee read atomicity in the presence of
concurrent writes.
2. Performance
Downside: Partitioning means that a single large readFile will
require into many "out and back" requests to libuv,
introducing overhead.
Upside: In between each "out and back", other work pending on the
threadpool can take a turn.
In short, although partitioning will slow down a large request,
it will lead to better throughput if the threadpool is handling
more than one type of request.
Fixes: https://github.com/nodejs/node/issues/17047
PR-URL: https://github.com/nodejs/node/pull/17054
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Tiancheng "Timothy" Gu <timothygu99@gmail.com>
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Also, alphabetize all types in type-parser.js
and fix some nits in type formats.
PR-URL: https://github.com/nodejs/node/pull/18444
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Install URL and URLSearchParams on the global object, like they can be
found in browser environments.
PR-URL: https://github.com/nodejs/node/pull/18281
Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Daijiro Wachi <daijiro.wachi@gmail.com>
Reviewed-By: Tiancheng "Timothy" Gu <timothygu99@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Evan Lucas <evanlucas@me.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
The `buffer` argument must be either Buffer or Uint8Array, string
values are not supported.
PR-URL: https://github.com/nodejs/node/pull/18022
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Weijia Wang <starkwang@126.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
This improves the api descriptions for fs.chown, fs.chmod, and fs.mkdir
along with their *Sync counterparts.
PR-URL: https://github.com/nodejs/node/pull/17679
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Jon Moss <me@jonathanmoss.me>
Reviewed-By: James M Snell <jasnell@gmail.com>
Causing the linter job to fail on CI right now.
PR-URL: https://github.com/nodejs/node/pull/17741
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/17692
Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Vse Mozhet Byt <vsemozhetbyt@gmail.com>
Also works on directories.
PR-URL: https://github.com/nodejs/node/pull/17702
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Vse Mozhet Byt <vsemozhetbyt@gmail.com>
Rewrite and edit somewhat confusing paragraph about inodes and AIX in
fs.md.
PR-URL: https://github.com/nodejs/node/pull/17519
Reviewed-By: Jon Moss <me@jonathanmoss.me>
Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
"On <operating system>" is used everywhere in this file except for one
use of "In <operating system>". Standardize to "On".
PR-URL: https://github.com/nodejs/node/pull/17463
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Jon Moss <me@jonathanmoss.me>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
PR-URL: https://github.com/nodejs/node/pull/17463
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Jon Moss <me@jonathanmoss.me>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
This change was suggested during the discussion of #17054.
PR-URL: https://github.com/nodejs/node/pull/17154
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Vse Mozhet Byt <vsemozhetbyt@gmail.com>
Mea culpa, somehow I managed to drop the documentation commit while
merging the pull request. This should have been included in commit
74023c072c ("fs: expose realpath(3) bindings") from this month.
PR-URL: https://github.com/nodejs/node/pull/17059
Refs: https://github.com/nodejs/node/pull/15776
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Evan Lucas <evanlucas@me.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
* Older experimental APIs have been removed.
[[`d731369b1d`](https://github.com/nodejs/node/commit/d731369b1d)]
[#14414](https://github.com/nodejs/node/pull/14414)
* **Errors**
* Improvements have been made to `buffer` module error messages.
* The assignment of static error codes to Node.js error continues:
* `buffer`
* `child_process`
* `console`
* `crypto`
* `dns`
* `events`
* `fs`
* `http`
* `inspector`
* `net`
* `path`
* `process`
* `querystring`
* `readline`
* `repl`
* `streams`
* `string_decoder`
* `timers`
* `tls`
* `url`
* `util`
* `v8`
* `zlib`
* **Child Processes**
* Errors are emitted on process nextTick.
* **Domains**
* The long-deprecated `.dispose()` method has been removed.
* **fs**
* The `fs.ReadStream` and `fs.WriteStream` classes now use `destroy()`.
* `fs` module callbacks are now invoked with an undefined context.
* **HTTP/1**
* A 400 Bad Request response will now be sent when parsing fails.
* Socket timeout will be set when the socket connects.
* A bug causing the request `'error'` event to fire twice was fixed.
* HTTP clients may now use generic `Duplex` streams in addition to `net.Socket`.
* **Intl**
* The deprecated `Intl.v8BreakIterator` has been removed.
* **Modules**
* The `require.resolve()` method now supports using custom lookup paths.
* **OS**
* The `os.EOL` property is now read-only.
* **Timers**
* `setTimeout()` will emit a warning if the timeout is larger that the maximum
32-bit unsigned integer.
PR-URL: https://github.com/nodejs/node/pull/15409
Fixes: https://github.com/nodejs/node/issues/14553
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Vse Mozhet Byt <vsemozhetbyt@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Nikolai Vavilov <vvnicholas@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Explains the expected behavior of previousStat in fs.watchFile() when
a watched file disappears and reappears.
PR-URL: https://github.com/nodejs/node/pull/16099
Fixes: https://github.com/nodejs/node/issues/15364
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Claudio Rodriguez <cjrodr@yahoo.com>
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
PR-URL: https://github.com/nodejs/node/pull/15990
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Specify that `NaN`, `Infinity`, and `-Infinity` are no longer valid
values for the `atime` and `mtime` arguments.
PR-URL: https://github.com/nodejs/node/pull/15680
Fixes: https://github.com/nodejs/node/issues/15453
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Each page of the API documentation should have links to other versions
of the same page. This will make it easier to switch between the current
"live" release at nodejs.org and LTS versions.
PR-URL: https://github.com/nodejs/node/pull/10958
Fixes: https://github.com/nodejs/node/issues/10726
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Evan Lucas <evanlucas@me.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Not knowing which APIs use libuv's threadpool can lead to surprising
performance problems. Document the APIs, and also document
UV_THREADPOOL_SIZE, which can be used to fix problems.
PR-URL: https://github.com/nodejs/node/pull/14995
Reviewed-By: Brian White <mscdex@mscdex.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
The documentation for `fs.createWriteStream()` references a
`defaultEncoding` as possible options property, but in reality
`encoding` property is expected and properly handled. This fix updates
the documentation to rename the `defaultEncoding` property to
`encoding`.
PR-URL: https://github.com/nodejs/node/pull/14867
Fixes: https://github.com/nodejs/node/issues/14611
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/14631
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
What happen to the file position after a read using a position null or
integer was not clear and you can assume that the cursor of the file
descriptor is updated even if position is an integer.
PR-URL: https://github.com/nodejs/node/pull/14631
Fixes: https://github.com/https://github.com/nodejs/node/issues/8397
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
fs.rmdir() on the file (not directory) results in different errors on
Windows to everything else
Fixes: https://github.com/nodejs/node/issues/8797
PR-URL: https://github.com/nodejs/node/pull/14323
Reviewed-By: Vse Mozhet Byt <vsemozhetbyt@gmail.com>
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
In preparation for stricter ESLint indentation checking, fix a few
issues in sample code.
PR-URL: https://github.com/nodejs/node/pull/13950
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Vse Mozhet Byt <vsemozhetbyt@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/13659
Fixes: https://github.com/nodejs/node/issues/12609
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com> i
PR-URL: https://github.com/nodejs/node/pull/13573
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
