Commit 9901b69c introduces a small regression where the trailing base64
padding is no longer written out when Cipher#final is called. Rectify
that.
Fixes #4837.
There are cases where a push() call would return true, even though
the thing being pushed was in fact way way larger than the high
water mark, simply because the 'needReadable' was already set, and
would not get unset until nextTick.
In some cases, this could lead to an infinite loop of pushing data
into the buffer, never getting to the 'readable' event which would
unset the needReadable flag.
Fix by splitting up the emitReadable function, so that it always
sets the flag on this tick, even if it defers until nextTick to
actually emit the event.
Also, if we're not ending or already in the process of reading, it
now calls read(0) if we're below the high water mark. Thus, the
highWaterMark value is the intended amount to buffer up to, and it
is smarter about hitting the target.
It seems like a good idea on the face of it, but lowWaterMarks are
actually not useful, and in practice should always be set to zero.
It would be worthwhile for writers if we actually did some kind of
writev() type of thing, but actually this just delays calling write()
and the overhead of doing a bunch of Buffer copies is not worth the
slight benefit of calling write() fewer times.
lib/path.js:
- throws a TypeError on the filter if the argument is not a string.
test/simple/test-path.js:
- removed the test to check if non-string types are filtered.
- added a test to check if path.join throws TypeError on arguments that
are not strings.
lib/http.js is using stream._handle.readStart/readStop to control
data-flow coming out from underlying stream. If this methods are not
present - data might be buffered regardless of whether it'll be read.
see #4657
Checks have been simplified and optimized for most-used cases.
Calling Buffer with another Buffer as the subject will now use the
SlowBuffer Copy method instead of the for loop.
No need to call for value coercion, just place the ternary inline.
This increases fs.WriteStream throughput dramatically by removing the
"higher default water marks" for fs.WriteStream.
Also includes a benchmark. Current performance is significantly higher
than v0.8 for strings at all tested levels except size=1. Buffer
performance is still lackluster.
Further improvement in the stream.Writable base class is required, but
this is a start.
Prior to v0.10, Node ignored ECONNRESET errors in many situations.
There *are* valid cases in which ECONNRESET should be ignored as a
normal part of the TCP dance, but in many others, it's a very relevant
signal that must be heeded with care.
Exacerbating this problem, if the OutgoingMessage does not have a
req.connection._handle, it assumes that it is in the process of
connecting, and thus buffers writes up in an array.
The problem happens when you reuse a socket between two requests, and it
is destroyed abruptly in between them. The writes will be buffered,
because the socket has no handle, but it's not ever going to GET a
handle, because it's not connecting, it's destroyed.
The proper fix is to treat ECONNRESET correctly. However, this is a
behavior/semantics change, and cannot land in a stable branch.
Fix #4775
Previously, we were only destroying sockets on end if their readable
side had already been ended. This causes a problem for non-readable
streams, since we don't expect to ever see an 'end' event from those.
Treat the lack of a 'readable' flag the same as if it was an ended
readable stream.
Fix #4751
This is causing the CryptoStreams to get into an awful state when
there is a tight loop calling connection.write(chunk) waiting for
a false return.
Because CryptoStreams use read(0) to cycle data, this was causing
the encrypted side to pull way too much data in from the cleartext
side, since the read(0) would make it always call _read.
The unfortunate side effect, fixed in the next patch, is that
CryptoStreams don't automatically cycle when the Socket drains.
Let ECONNRESET network errors bubble up so clients can detect them.
Commit c4454d2e suppressed and turned them into regular end-of-stream
events to fix the then-failing simple/test-regress-GH-1531 test. See
also issue #1571 for (scant) details.
It turns out that special handling is no longer necessary. Remove the
special casing and let the error bubble up naturally.
pummel/test-https-ci-reneg-attack and pummel/test-tls-ci-reneg-attack
are updated because they expected an EPIPE error code that is now an
ECONNRESET. Suppression of the ECONNRESET prevented the test from
detecting that the connection has been severed whereupon the next
write would fail with an EPIPE.
Fixes #1776.
Fix an exception that was raised when the WriteStream was closed
immediately after creating it:
TypeError: Cannot read property 'fd' of undefined
at WriteStream.close (fs.js:1537:18)
<snip>
Avoid the TypeError and make sure the file descriptor is closed.
Fixes #4745.
Otherwise sockets that are 'finish'ed won't be unpiped and `writing to
ended stream` error will arise.
This might sound unrealistic, but it happens in net.js. When
`socket.allowHalfOpen === false`, EOF will cause `.destroySoon()` call which
ends the writable side of net.Socket.
This is more backwards-compatible with stream1 streams like `fs.WriteStream`
which would allow a callback function to be passed in as the only argument.
Closes #4719.
In zlibBuffer(), don't wait for the garbage collector to reclaim the zlib memory
but release it manually. Reduces memory consumption by a factor of 10 or more
with some workloads.
Test case:
function f() {
require('zlib').deflate('xxx', g);
}
function g() {
setTimeout(f, 5);
}
f();
Observe RSS memory usage with and without this commit. After 10,000 iterations,
RSS stabilizes at ~35 MB with this commit. Without, RSS is over 300 MB and keeps
growing.
Cause: whenever the JS object heap hits the high-water mark, the V8 GC sweeps
it clean, then tries to grow it in order to avoid more sweeps in the near
future. Rule of thumb: the bigger the JS heap, the lazier the GC can be.
A side effect of a bigger heap is that objects now live longer. This is harmless
in general but it affects zlib context objects because those are tied to large
buffers that live outside the JS heap, on the order of 16K per context object.
Ergo, don't wait for the GC to reclaim the memory - it may take a long time.
Fixes #4172.
Move the implementation to C++ land. This is similar to commit 3f65916
but this time for the write() function and the Buffer(s, 'hex')
constructor.
Speeds up the benchmark below about 24x (2.6s vs 1:02m).
var s = 'f';
for (var i = 0; i < 26; ++i) s += s; // 64 MB
Buffer(s, 'hex');
Move the implementation to C++ land. The old JS implementation used
string concatenation, was dog slow and consumed copious amounts of
memory for large buffers. Example:
var buf = Buffer(0x1000000); // 16 MB
buf.toString('hex') // Used 3+ GB of memory.
The new implementation operates in O(n) time and space.
Fixes #4700.
Those values, if passed to the _read() cb, will not signal an EOF. Only
null or undefined will mark the end of data, and trigger the end event.
However, great care must be taken if you are returning an empty string
or buffer! There must be some other thing somewhere that will trigger
a read() call, because there will never be a readable event fired later.
This is in preparation for CryptoStreams being ported to streams2, where
it is safe to simply stop reading, because the crypto cycle process will
cause it to read(0) again at some future date.
Make lines ending \r\n emit one 'line' event, not two (where the second
one is an empty string).
This adds a new keypress name: 'return' (as in: 'carriage return').
Fixes #3305.
This commit breaks simple/test-stream2-stderr-sync. Need to figure out
a better way, or just accept that `(function W(){stream.write(b,W)})()`
is going to be noisy. People should really be using the `'drain'` event
for this use-case anyway.
This reverts commit 02f7d1bfd8.
Always defer the _write callback. The optimization here was only
relevant in some oddball edge cases that we don't actually care about.
Our benchmarks confirm that just always deferring the Socket._write cb
is perfectly fine to do, and in some cases, even slightly more
performant.
When a datagram socket hasn't been bound yet, node will defer `send()`
operations until binding has completed. Before this patch a `listening`
listener would be installed every time `send` was called. This triggered
an EventEmitter leak warning when more than 10 packets were sent in a
tight loop. Therefore switch to using a single `listening` listener, and
use an array to enqueue outbound packets.
The refactor in b43e544140 to use
stream.push() in Transform inadvertently caused it to immediately
consume all the written data, regardless of whether or not the readable
side was being consumed.
Only pull data through the _transform() process when the readable side
is being consumed.
Fix #4667
TCPWrap::Initialize() and PipeWrap::Initialize() should be called before
any data will be read from received socket. But, because of lazy
initialization of these bindings, Initialize() method isn't called.
Init bindings manually upon socket receiving.
See #4669
mainly to allow native addons to export single functions on
rather than being restricted to operating on an existing
object.
Init functions now receive exports as the first argument, like
before, but also the module object as the second argument, if they
support it.
Related to #4634
cc: @rvagg
Fix issue where SlowBuffers couldn't be passed as target to Buffer
copy().
Also included checks to see if Argument parameters are defined before
assigning their values. This offered ~3x's performance gain.
Backport of 16bbecc from master branch. Closes #4633.
Changed types of errors thrown to be more indicative of what the error
represents. Also removed a few unnecessary uses of the v8 fully
quantified typename.
Argument checks were simplified by setting all undefined/NaN or out of
bounds values equal to their defaults.
Also copy() tests had a flaw that each buffer had the same bit pattern at
the same offset. So even if the copy failed, the bit-by-bit comparison
would have still been true. This was fixed by filling each buffer with a
unique value before copy operations.
Fix issue where SlowBuffers couldn't be passed as target to Buffer
copy().
Also included checks to see if Argument parameters are defined before
assigning their values. This offered ~3x's performance gain.
This adds a proxy for bytesWritten to the tls.CryptoStream. This
change makes the connection object more similar between HTTP and
HTTPS requests in an effort to avoid confusion.
See issue #4650 for more background information.
We detect for non-string and non-buffer values in onread and
turn the stream into an "objectMode" stream.
If we are in "objectMode" mode then howMuchToRead will
always return 1, state.length will always have 1 appended
to it when there is a new item and fromList always takes
the first value from the list.
This means that for object streams, the n in read(n) is
ignored and read() will always return a single value
Fixed a bug with unpipe where the pipe would break because
the flowing state was not reset to false.
Fixed a bug with sync cb(null, null) in _read which would
forget to end the readable stream
This is similar to commit 2cbf458 but this time for 204 No Content
instead of 304 Not Modified responses.
When the user sends a 204 response with a Transfer-Encoding: chunked
header, suppress sending the zero chunk and force the connection to
close.
Force the connection to close when the response is a 304 Not Modified
and the user has set a "Transfer-Encoding: chunked" header.
RFC 2616 mandates that 304 responses MUST NOT have a body but node.js
used to send out a zero chunk anyway to accommodate clients that don't
have special handling for 304 responses.
It was pointed out that this might confuse reverse proxies to the point
of creating security liabilities, so suppress the zero chunk and force
the connection to close.
Profiling in clustered environments doesn't work out of the box.
By default, V8 writes the profile data of all processes to a single
v8.log.
Running that log file through a tick processor produces bogus numbers
because many events won't match up with the recorded memory mappings
and you end up with graphs where 80+% of ticks is unaccounted for.
Fixing the tick processor to deal with multi-process output is not very
useful because the processes may be running wildly disparate workloads.
That's why we fix up the command line arguments to include
a "--logfile=v8-%p.log" argument (where %p is expanded to the PID)
unless it already contains a --logfile argument.
Fixes #4617.
Keeping list of all sockets that were sent to child process causes memory
leak and thus unacceptable (see #4587). However `server.close()` should
still work properly.
This commit introduces two options:
* child.send(socket, { track: true }) - will send socket and track its status.
You should use it when you want to receive `close` event on sent sockets.
* child.send(socket) - will send socket without tracking it status. This
performs much better, because of smaller number of RTT between master and
child.
With both of these options `server.close()` will wait for all sent
sockets to get closed.
Keeping list of all sockets that were sent to child process causes memory
leak and thus unacceptable (see #4587). However `server.close()` should
still work properly.
This commit introduces two options:
* child.send(socket, { track: true }) - will send socket and track its status.
You should use it when you want `server.connections` to be a reliable
number, and receive `close` event on sent sockets.
* child.send(socket) - will send socket without tracking it status. This
performs much better, because of smaller number of RTT between master and
child.
With both of these options `server.close()` will wait for all sent
sockets to get closed.
Unfortunately, it's just too slow to do this in events.js. Users will
just have to live with not having events named __proto__ or toString.
This reverts commit b48e303af0.
Reject negative offsets in SlowBuffer::MakeFastBuffer(), it allows
the creation of buffers that point to arbitrary addresses.
Reported by Trevor Norris.
Problem 1: If stream.push() triggers a 'readable' event, and the user
calls `read(n)` with some n > the highWaterMark, then the push() will
return false (indicating that they should not push any more), but no
future 'readable' event is coming (because we're above the
highWaterMark).
Solution: return true from push() when needReadable is set.
Problem 2: A read(n) for n != 0, after the stream had encountered an
EOF, would not trigger the 'end' event if the EOF was pushed in
synchronously by the _read() function.
Solution: Check for ended in stream.read() and schedule an end event if
the length now equals 0.
Fix #4585
Improved assert check order of execution and added additional checks on
parameters to ensure no bad values make it through (e.g. negative offset
values).
Improvements:
* floating point operations are approx 4x's faster
* Now write quiet NaN's
* all read/write on floating point now done in C, so no more need for
lib/buffer_ieee754.js
* float values have more accurate min/max value checks
* add additional benchmarks for buffers read/write
* created benchmark/_bench_timer.js which is a simple library that
can be included into any benchmark and provides an intelligent tracker
for sync and async tests
* add benchmarks for DataView set methods
* add checks and tests to make sure offset is greater than 0
There was previously an assert() in there, but this part of the code is
so high-volume that the added cost made a measurable dent in http_simple.
Just checking inline is fine, though, and prevents a lot of potential
hazards.
Say that a stream's current read queue has 101 bytes in it, and the
underlying resource has ended (ie, reached EOF).
If you do something like this:
stream.read(100); // leave a byte behind
stream.read(0); // read(0) for some reason
then the read(0) will get 0 from the howMuchToRead function. Since the
stream was ended, this was incorrectly treating the 0 as a "there is no
more in the buffer", and emitting 'end' before that last byte was read.
Why have the read(0) in the first place? We do this in some cases to
trigger the last few bytes of a net socket (such as a child process's
stdio pipes). This was causing issues when piping a `git archive` job
to a file: the resulting tarball was incomplete, because it occasionally
was not getting the last chunk.
Fix the following exception:
http.js:974
this._httpMessage.emit('close');
^
TypeError: Cannot call method 'emit' of null
at Socket.onServerResponseClose (http.js:974:21)
at Socket.EventEmitter.emit (events.js:124:20)
at net.js:421:10
at process._tickCallback (node.js:386:13)
at process._makeCallback (node.js:304:15)
Fixes #4586.
This also slightly changes the semantics, in that a 'readable'
event may be triggered by the first write() call, even if a
user has not yet called read().
This happens because the Transform _write() handler is calling
read(0) to start the flow of data. Technically, the new behavior
is more 'correct', since it is more in line with the semantics
of the 'readable' event in other streams.
When switching into compatibility mode by setting `data` event listener,
`_read()` method will be called immediately. If method implementation
invokes callback in the same tick - all emitted `data` events will be
discarded, because `data` listener wasn't set yet.
Raise a TypeError when the argument to send() or sendto() is anything
but a Buffer.
Fixes the following assertion:
$ node -e 'require("dgram").createSocket("udp4").send("BAM")'
node: ../../src/udp_wrap.cc:220: static v8::Handle<v8::Value>
node::UDPWrap::DoSend(const v8::Arguments&, int): Assertion
`Buffer::HasInstance(args[0])' failed.
Aborted (core dumped)
Fixes #4496.
Allows for arbitrary path to executable spawned using `fork`. This
fixes some issues around running multiple versions of node with workers
and allows arbitrary IPC with compatible executables.
Fixes #3248.
In JS, the expression ".1" is a floating point number. Issue 4268 concerns the
REPL interpreting floating point numbers that lead with a "." as keywords. The
original bugfix worked for this specific case but not for the general case:
var x = [
.1,
.2,
.3
];
The attached change and test (`.1+.1` should be `.2`) fix the bug.
Closes #4513.
Don't give names of built-in libraries special treatment.
Changes the REPL's behavior from this:
> var path = 42
> path
A different "path" already exists globally
To this:
> var path = 42
> path
42
Fixes #4512.
Calling send() on an unbound socket forces an implicit bind to
a random port.
332fea5 made the 'listening' event asynchronous. Unfortunately,
it also introduced a bug where the implicit bind was tried more
than once if send() was called again before the first bind operation
completed.
Address that by keeping track of the bind status and making sure that
bind() is called only once.
Fixes #4499.
While it's true that error objects have a history of getting snake_case
properties attached by the host system, it's a point of confusion to
Node users that comes up a lot. It's still 'experimental', so best to
change this sooner rather than later.
This adds a process._fatalException method which is called into from
C++ in order to either emit the 'uncaughtException' method, or emit
'error' on the active domain.
The 'uncaughtException' event is an implementation detail that it would
be nice to deprecate one day, so exposing it as part of the domain
machinery is not ideal.
Fix #4375
Make `fs.createReadStream({ end: 42 })` work.
Before this commit, it worked only when used like this:
`fs.createReadStream({ start: 0, end: 42 })` - only when `start` was specified
by the caller.
Fixes #4423.
socket.readyState, .readable, and .writable behavior changed as
a result of the new streaming interfaces. Updated to be backwards
compatible with current API and adds regression test.
closes #4461
Fix a bug where calling .end() on a socket without calling .connect() first
throws a TypeError:
TypeError: Cannot read property 'shutdown' of undefined
at Socket.onSocketFinish (net.js:194:20)
at Socket.EventEmitter.emit (events.js:91:17)
at Socket.Writable.end (_stream_writable.js:281:10)
at Socket.end (net.js:352:31)
Fixes #4463.
Fixes #3226.
Consider a production server that uses a REPL to debug. Creating the instance
would wipe out the global cache of modules, and subsequent "require" calls in
the server would be reloaded from disk. The REPL should observe only, without
altering, its environment.
