0
0
mirror of https://github.com/nodejs/node.git synced 2024-12-01 16:10:02 +01:00
nodejs/tools/icu/README.md

137 lines
4.4 KiB
Markdown
Raw Normal View History

# Notes about the `tools/icu` subdirectory
This directory contains tools, data, and information about the [ICU](http://icu-project.org)
(International Components for Unicode) integration. ICU is used to provide
internationalization functionality.
- `patches/` are one-off patches, actually entire source file replacements,
organized by ICU version number.
- `icu_small.json` controls the "small" (English only) ICU. It is input to
`icutrim.py`
- `icu-generic.gyp` is the build file used for most ICU builds within ICU.
<!-- have fun -->
- `icu-system.gyp` is an alternate build file used when `--with-intl=system-icu`
is invoked. It builds against the `pkg-config` located ICU.
- `iculslocs.cc` is source for the `iculslocs` utility, invoked by `icutrim.py`
as part of repackaging. Not used separately. See source for more details.
- `no-op.cc` — empty function to convince gyp to use a C++ compiler.
- `README.md` — you are here
- `shrink-icu-src.py` — this is used during upgrade (see guide below)
build, i18n: improve Intl build, add "--with-intl" The two main goals of this change are: - To make it easier to build the Intl option using ICU (particularly, using a newer ICU than v8/Chromium's version) - To enable a much smaller ICU build with only English support The goal here is to get node.js binaries built this way by default so that the Intl API can be used. Additional data can be added at execution time (see Readme and wiki) More details are at https://github.com/joyent/node/pull/7719 In particular, this change adds the "--with-intl=" configure option to provide more ways of building "Intl": - "full-icu" picks up an ICU from deps/icu - "small-icu" is similar, but builds only English - "system-icu" uses pkg-config to find an installed ICU - "none" does nothing (no Intl) For Windows builds, the "full-icu" or "small-icu" options are added to vcbuild.bat. Note that the existing "--with-icu-path" option is not removed from configure, but may not be used alongside the new option. Wiki changes have already been made on https://github.com/joyent/node/wiki/Installation and a new page created at https://github.com/joyent/node/wiki/Intl (marked as provisional until this change lands.) Summary of changes: * README.md : doc updates * .gitignore : added "deps/icu" as this is the location where ICU is unpacked to. * Makefile : added the tools/icu/* files to cpplint, but excluded a problematic file. * configure : added the "--with-intl" option mentioned above. Calculate at config time the list of ICU source files to use and data packaging options. * node.gyp : add the new files src/node_i18n.cc/.h as well as ICU linkage. * src/node.cc : add call into node::i18n::InitializeICUDirectory(icu_data_dir) as well as new --icu-data-dir option and NODE_ICU_DATA env variable to configure ICU data loading. This loading is only relevant in the "small" configuration. * src/node_i18n.cc : new source file for the above Initialize.. function, to setup ICU as needed. * tools/icu : new directory with some tools needed for this build. * tools/icu/icu-generic.gyp : new .gyp file that builds ICU in some new ways, both on unix/mac and windows. * tools/icu/icu-system.gyp : new .gyp file to build node against a pkg-config detected ICU. * tools/icu/icu_small.json : new config file for the "English-only" small build. * tools/icu/icutrim.py : new tool for trimming down ICU data. Reads the above .json file. * tools/icu/iculslocs.cc : new tool for repairing ICU data manifests after trim operation. * tools/icu/no-op.cc : dummy file to force .gyp into using a C++ linker. * vcbuild.bat : added small-icu and full-icu options, to call into configure. * Fixed toolset dependencies, see https://github.com/joyent/node/pull/7719#issuecomment-54641687 Note that because of a bug in gyp {CC,CXX}_host must also be set. Otherwise gcc/g++ will be used by default for part of the build. Reviewed-by: Trevor Norris <trev.norris@gmail.com> Reviewed-by: Fedor Indutny <fedor@indutny.com>
2014-09-05 07:03:24 +02:00
## How to upgrade ICU
- Make sure your Node.js workspace is clean (clean `git status`) should be
sufficient.
- Configure Node.js with the specific [ICU version](http://icu-project.org/download)
you want to upgrade to, for example:
```shell
./configure \
--with-intl=small-icu \
--with-icu-source=http://download.icu-project.org/files/icu4c/58.1/icu4c-58_1-src.tgz
make
```
> _Note_ in theory, the equivalent `vcbuild.bat` commands should work also,
> but the commands below are makefile-centric.
- If there are ICU version-specific changes needed, you may need to make changes
in `icu-generic.gyp` or add patch files to `tools/icu/patches`.
- Specifically, look for the lists in `sources!` in the `icu-generic.gyp` for
files to exclude.
- Verify the Node.js build works:
```shell
make test-ci
```
Also running
<!-- eslint-disable strict -->
```js
new Intl.DateTimeFormat('es', {month: 'long'}).format(new Date(9E8));
```
…Should return `January` not `enero`.
- Now, copy `deps/icu` over to `deps/icu-small`
```shell
python tools/icu/shrink-icu-src.py
```
- Now, do a clean rebuild of Node.js to test:
```shell
make -k distclean
./configure
make
```
- Test this newly default-generated Node.js
<!-- eslint-disable strict -->
```js
process.versions.icu;
new Intl.DateTimeFormat('es', {month: 'long'}).format(new Date(9E8));
```
(This should print your updated ICU version number, and also `January` again.)
You are ready to check in the updated `deps/small-icu`. This is a big commit,
so make this a separate commit from the smaller changes.
- Now, rebuild the Node.js license.
```shell
# clean up - remove deps/icu
make clean
tools/license-builder.sh
```
- Update the URL and hash for the full ICU file in `tools/icu/current_ver.dep`.
It should match the ICU URL used in the first step. When this is done, the
following should build with full ICU.
```shell
# clean up
rm -rf out deps/icu deps/icu4c*
./configure --with-intl=full-icu --download=all
make
make test-ci
```
- commit the change to `tools/icu/current_ver.dep` and `LICENSE` files.
- Note: To simplify review, I often will “pre-land” this patch, meaning that
I run the patch through `curl -L https://github.com/nodejs/node/pull/xxx.patch
| git am -3 --whitespace=fix` per the collaborators guide… and then push that
patched branch into my PR's branch. This reduces the whitespace changes that
show up in the PR, since the final land will eliminate those anyway.
-----
## Postscript about the tools
The files in this directory were written for the Node.js effort.
It was the intent of their author (Steven R. Loomis / srl295) to
merge them upstream into ICU, pending much discussion within the
ICU-TC.
build, i18n: improve Intl build, add "--with-intl" The two main goals of this change are: - To make it easier to build the Intl option using ICU (particularly, using a newer ICU than v8/Chromium's version) - To enable a much smaller ICU build with only English support The goal here is to get node.js binaries built this way by default so that the Intl API can be used. Additional data can be added at execution time (see Readme and wiki) More details are at https://github.com/joyent/node/pull/7719 In particular, this change adds the "--with-intl=" configure option to provide more ways of building "Intl": - "full-icu" picks up an ICU from deps/icu - "small-icu" is similar, but builds only English - "system-icu" uses pkg-config to find an installed ICU - "none" does nothing (no Intl) For Windows builds, the "full-icu" or "small-icu" options are added to vcbuild.bat. Note that the existing "--with-icu-path" option is not removed from configure, but may not be used alongside the new option. Wiki changes have already been made on https://github.com/joyent/node/wiki/Installation and a new page created at https://github.com/joyent/node/wiki/Intl (marked as provisional until this change lands.) Summary of changes: * README.md : doc updates * .gitignore : added "deps/icu" as this is the location where ICU is unpacked to. * Makefile : added the tools/icu/* files to cpplint, but excluded a problematic file. * configure : added the "--with-intl" option mentioned above. Calculate at config time the list of ICU source files to use and data packaging options. * node.gyp : add the new files src/node_i18n.cc/.h as well as ICU linkage. * src/node.cc : add call into node::i18n::InitializeICUDirectory(icu_data_dir) as well as new --icu-data-dir option and NODE_ICU_DATA env variable to configure ICU data loading. This loading is only relevant in the "small" configuration. * src/node_i18n.cc : new source file for the above Initialize.. function, to setup ICU as needed. * tools/icu : new directory with some tools needed for this build. * tools/icu/icu-generic.gyp : new .gyp file that builds ICU in some new ways, both on unix/mac and windows. * tools/icu/icu-system.gyp : new .gyp file to build node against a pkg-config detected ICU. * tools/icu/icu_small.json : new config file for the "English-only" small build. * tools/icu/icutrim.py : new tool for trimming down ICU data. Reads the above .json file. * tools/icu/iculslocs.cc : new tool for repairing ICU data manifests after trim operation. * tools/icu/no-op.cc : dummy file to force .gyp into using a C++ linker. * vcbuild.bat : added small-icu and full-icu options, to call into configure. * Fixed toolset dependencies, see https://github.com/joyent/node/pull/7719#issuecomment-54641687 Note that because of a bug in gyp {CC,CXX}_host must also be set. Otherwise gcc/g++ will be used by default for part of the build. Reviewed-by: Trevor Norris <trev.norris@gmail.com> Reviewed-by: Fedor Indutny <fedor@indutny.com>
2014-09-05 07:03:24 +02:00
`icu_small.json` is somewhat node-specific as it specifies a "small ICU"
configuration file for the `icutrim.py` script. `icutrim.py` and
`iculslocs.cpp` may themselves be superseded by components built into
ICU in the future. As of this writing, however, the tools are separate
entities within Node.js, although theyare being scrutinized by interested
members of the ICU-TC. The “upstream” ICU bugs are given below.
build, i18n: improve Intl build, add "--with-intl" The two main goals of this change are: - To make it easier to build the Intl option using ICU (particularly, using a newer ICU than v8/Chromium's version) - To enable a much smaller ICU build with only English support The goal here is to get node.js binaries built this way by default so that the Intl API can be used. Additional data can be added at execution time (see Readme and wiki) More details are at https://github.com/joyent/node/pull/7719 In particular, this change adds the "--with-intl=" configure option to provide more ways of building "Intl": - "full-icu" picks up an ICU from deps/icu - "small-icu" is similar, but builds only English - "system-icu" uses pkg-config to find an installed ICU - "none" does nothing (no Intl) For Windows builds, the "full-icu" or "small-icu" options are added to vcbuild.bat. Note that the existing "--with-icu-path" option is not removed from configure, but may not be used alongside the new option. Wiki changes have already been made on https://github.com/joyent/node/wiki/Installation and a new page created at https://github.com/joyent/node/wiki/Intl (marked as provisional until this change lands.) Summary of changes: * README.md : doc updates * .gitignore : added "deps/icu" as this is the location where ICU is unpacked to. * Makefile : added the tools/icu/* files to cpplint, but excluded a problematic file. * configure : added the "--with-intl" option mentioned above. Calculate at config time the list of ICU source files to use and data packaging options. * node.gyp : add the new files src/node_i18n.cc/.h as well as ICU linkage. * src/node.cc : add call into node::i18n::InitializeICUDirectory(icu_data_dir) as well as new --icu-data-dir option and NODE_ICU_DATA env variable to configure ICU data loading. This loading is only relevant in the "small" configuration. * src/node_i18n.cc : new source file for the above Initialize.. function, to setup ICU as needed. * tools/icu : new directory with some tools needed for this build. * tools/icu/icu-generic.gyp : new .gyp file that builds ICU in some new ways, both on unix/mac and windows. * tools/icu/icu-system.gyp : new .gyp file to build node against a pkg-config detected ICU. * tools/icu/icu_small.json : new config file for the "English-only" small build. * tools/icu/icutrim.py : new tool for trimming down ICU data. Reads the above .json file. * tools/icu/iculslocs.cc : new tool for repairing ICU data manifests after trim operation. * tools/icu/no-op.cc : dummy file to force .gyp into using a C++ linker. * vcbuild.bat : added small-icu and full-icu options, to call into configure. * Fixed toolset dependencies, see https://github.com/joyent/node/pull/7719#issuecomment-54641687 Note that because of a bug in gyp {CC,CXX}_host must also be set. Otherwise gcc/g++ will be used by default for part of the build. Reviewed-by: Trevor Norris <trev.norris@gmail.com> Reviewed-by: Fedor Indutny <fedor@indutny.com>
2014-09-05 07:03:24 +02:00
* [#10919](http://bugs.icu-project.org/trac/ticket/10919)
(experimental branch - may copy any source patches here)
* [#10922](http://bugs.icu-project.org/trac/ticket/10922)
(data packaging improvements)
* [#10923](http://bugs.icu-project.org/trac/ticket/10923)
(rewrite data building in python)