nodejs/buffers.markdown at c0d8311f6b86ef35cdb0b233f3c3d2b83c4a3381

mirrors/nodejs

Fork 0

mirror of https://github.com/nodejs/node.git synced 2024-11-29 23:16:30 +01:00

Brian White b7c23ac3f5 Incorporate endianness into buffer.read* function names instead of passing in a boolean flag

2011-08-12 15:49:57 +02:00

15 KiB

Raw Blame History

Buffers

Pure Javascript is Unicode friendly but not nice to binary data. When dealing with TCP streams or the file system, it's necessary to handle octet streams. Node has several strategies for manipulating, creating, and consuming octet streams.

Raw data is stored in instances of the Buffer class. A Buffer is similar to an array of integers but corresponds to a raw memory allocation outside the V8 heap. A Buffer cannot be resized.

The Buffer object is global.

Converting between Buffers and JavaScript string objects requires an explicit encoding method. Here are the different string encodings;

'ascii' - for 7 bit ASCII data only. This encoding method is very fast, and will strip the high bit if set. Note that this encoding converts a null character ('\0' or '\u0000') into 0x20 (character code of a space). If you want to convert a null character into 0x00, you should use 'utf8'.
'utf8' - Multi byte encoded Unicode characters. Many web pages and other document formats use UTF-8.
'ucs2' - 2-bytes, little endian encoded Unicode characters. It can encode only BMP(Basic Multilingual Plane, U+0000 - U+FFFF).
'base64' - Base64 string encoding.
'binary' - A way of encoding raw binary data into strings by using only the first 8 bits of each character. This encoding method is deprecated and should be avoided in favor of Buffer objects where possible. This encoding will be removed in future versions of Node.
'hex' - Encode each byte as two hexidecimal characters.

new Buffer(size)

Allocates a new buffer of size octets.

new Buffer(array)

Allocates a new buffer using an array of octets.

new Buffer(str, encoding='utf8')

Allocates a new buffer containing the given str.

buffer.write(string, offset=0, length=buffer.length-offset, encoding='utf8')

Writes string to the buffer at offset using the given encoding. length is the number of bytes to write. Returns number of octets written. If buffer did not contain enough space to fit the entire string, it will write a partial amount of the string. The method will not write partial characters.

Example: write a utf8 string into a buffer, then print it

buf = new Buffer(256);
len = buf.write('\u00bd + \u00bc = \u00be', 0);
console.log(len + " bytes: " + buf.toString('utf8', 0, len));

The number of characters written (which may be different than the number of bytes written) is set in Buffer._charsWritten and will be overwritten the next time buf.write() is called.

buffer.toString(encoding, start=0, end=buffer.length)

Decodes and returns a string from buffer data encoded with encoding beginning at start and ending at end.

See buffer.write() example, above.

buffer[index]

Get and set the octet at index. The values refer to individual bytes, so the legal range is between 0x00 and 0xFF hex or 0 and 255.

Example: copy an ASCII string into a buffer, one byte at a time:

str = "node.js";
buf = new Buffer(str.length);

for (var i = 0; i < str.length ; i++) {
  buf[i] = str.charCodeAt(i);
}

console.log(buf);

// node.js

Buffer.isBuffer(obj)

Tests if obj is a Buffer.

Buffer.byteLength(string, encoding='utf8')

Gives the actual byte length of a string. This is not the same as String.prototype.length since that returns the number of characters in a string.

Example:

str = '\u00bd + \u00bc = \u00be';

console.log(str + ": " + str.length + " characters, " +
  Buffer.byteLength(str, 'utf8') + " bytes");

// ½ + ¼ = ¾: 9 characters, 12 bytes

buffer.length

The size of the buffer in bytes. Note that this is not necessarily the size of the contents. length refers to the amount of memory allocated for the buffer object. It does not change when the contents of the buffer are changed.

buf = new Buffer(1234);

console.log(buf.length);
buf.write("some string", "ascii", 0);
console.log(buf.length);

// 1234
// 1234

buffer.copy(targetBuffer, targetStart=0, sourceStart=0, sourceEnd=buffer.length)

Does a memcpy() between buffers.

Example: build two Buffers, then copy buf1 from byte 16 through byte 19 into buf2, starting at the 8th byte in buf2.

buf1 = new Buffer(26);
buf2 = new Buffer(26);

for (var i = 0 ; i < 26 ; i++) {
  buf1[i] = i + 97; // 97 is ASCII a
  buf2[i] = 33; // ASCII !
}

buf1.copy(buf2, 8, 16, 20);
console.log(buf2.toString('ascii', 0, 25));

// !!!!!!!!qrst!!!!!!!!!!!!!

buffer.slice(start, end=buffer.length)

Returns a new buffer which references the same memory as the old, but offset and cropped by the start and end indexes.

Modifying the new buffer slice will modify memory in the original buffer!

Example: build a Buffer with the ASCII alphabet, take a slice, then modify one byte from the original Buffer.

var buf1 = new Buffer(26);

for (var i = 0 ; i < 26 ; i++) {
  buf1[i] = i + 97; // 97 is ASCII a
}

var buf2 = buf1.slice(0, 3);
console.log(buf2.toString('ascii', 0, buf2.length));
buf1[0] = 33;
console.log(buf2.toString('ascii', 0, buf2.length));

// abc
// !bc